Scientific Method and the Design of Research

Abstract

This chapter commences with an overview of the term “scientific method”. The term is not only used by scientists to refer to a systematic procedure for carrying out a scientific investigation. It is also called upon to justify the privileged status of their findings and profession. After reflections as to “scientific method” in general, the second half of the chapter develops a practical template or structure for designing one’s own specific investigation/research project.

Appendix: Checklists to Assist You in the Preparation of Proposals (and Dissertations)

I humbly suggest that the four checklists below can be of value to you in planning and writing your research proposal. Even though there are differences between the structure and content of a proposal and a dissertation—one being prepared more or less prior to your investigations, the other being prepared, or at least finalized, after you have completed your empirical work and analyses—the major content of these checklists is also relevant for planning and writing your dissertation. Nevertheless, for the sake of precision and clarity, the presentation below will specifically refer to a proposal.

• 1. Checklist structured in accord with a traditional template for a proposal

Title page

Question	Needs work	OK as is
Have you checked the rules about what should be entered here?
Does the title clearly and concisely encompass the focus of your proposal?
Have you considered how your title can be shortened, clarified, be more precise?

Acknowledgements

Question	Needs work	OK as is
Do you clearly acknowledge help of people who have
(a) Supervised you or given you advice underway?
(b) Read earlier drafts and provided you with feedback?
(c) Helped you to gain access to laboratory facilities, an organisation, data, etc.?
(d) Inspired you, perhaps by providing you with new insights or by challenging your ideas/arguments/methods—even if this inspiration was a result of informal discussions?

Table of contents

Question	Needs work	OK as is
Do you have a well-structured table of contents indicating chapters, sections/sub-sections, appendices etc. and the corresponding pages numbers?
Do you want to have a separate list of tables, figures etc. and do you have a well-structured numbering system for referring to them?

Abstract (Whether you should present an abstract depends on the traditions at your institution).

Question	Needs work	OK as is
Do you provide a clear and concise overview of the proposal including the major goals being addressed, why these are important and interesting and how you will address the resultant research questions/hypotheses?

Introduction

Question	Needs work	OK as is
1. Statement of the Focus and Purpose of the Study
Do the opening sentences grab our attention and give a clear indication of the focus of the study and its broader significance?
Is there a clear and concise statement of purpose indicating why you want to do the study and what you intend to accomplish—of the central, controlling idea in the study? Is this statement of goals/purposes concretized in the form of clearly stated objectives and projected results or outputs?
2. Research Questions/Hypotheses
Do you provide a clear and concise statement of your Research Questions and/or Hypotheses?
Do you avoid constructing too many questions/hypotheses?
Is there a clear and concise statement of the significance of these research questions/hypotheses for: (a) the scientific community, (b) those directly affected by your research (particularly relevant in the social sciences), and (c) society?
Do you argue that the proposed research is ‘original’—that it will contribute new knowledge to the field?
Do you briefly discuss how your study has evolved from previous research (e.g. your own earlier research, that of your advisor, and in particular, from your review of the literature)?
3. Theory and Method/Methodology
Do you briefly present the theoretical approach or perspective you will be using and justify the choice?
Do you provide a brief overview of your research strategy, design, and the methods you will employ to answer your research questions/test your hypotheses?
Do you provide information as to the feasibility of the study with respect to time and resources—and your competences and skills?
Is there a clear and concise statement of how your research design and process will enable you to meet potential challenges to validity and demonstrate the credibility of your results?

Conceptual Framework and Literature Review

Question	Needs work	OK as is
Do you provide a clear review of the main ideas, existing theory and research related to your field of investigation and its potential significance?
Do you provide reflection on your choice of theoretical approach based on both the literature, advice from experts in your field and your own experience (including prior investigations/pilot studies) and competences?
Are you selective in your choice of literature? Have you established priorities as to carrying out your literature search (overviews, journal articles, books etc. including emphasis on recent rather than older references)?
Do you provide the literature review in such a manner that it clearly supports your formulation of your Research Questions/Hypotheses and demonstrates why your research is important? This may include reference to
(a) A lack of relevant research on a topic, or gaps in existing knowledge,
(b) The existence of competing perspectives on a topic,
(c) The value of replicating research carried out on a topic by others,
(d) Benchmarks that enable comparing your results with other findings.
Do you provide comments on the literature you review in relation to the formulation of your research questions/hypotheses? Are you reflective and critical in your presentation and not simply descriptive where you essentially just provide a concise abstract?
Is there a clear structure to your Conceptual Framework and the Review—do you organise these according to themes or important concepts and principles?
Do you graphically or otherwise demonstrate the interconnections between the themes/concepts?
Do you provide summaries of and reflections on the individual publications you review and a conclusion to your overall literature review?

Methodology/Research Methods

Question	Needs work	OK as is
Do you discuss and justify your research strategy?
Do you formulate your research questions so they can directly serve as the focal point of your proposal or do you formulate them as hypotheses that can be tested?
Do you establish the interconnections between these questions/hypotheses and your overall objectives and specific aims, as well as the significance of your study?
Do you discuss the appropriateness of your research designs—including why you do not propose to use other relevant designs?
Do you discuss the appropriateness of your specific data collection methods—including the choice of technologies and experimental designs as well as why you do not propose to use other relevant methods?
Do you discuss the appropriateness of the data analysis methods you propose to use—including why you do not propose to use other relevant analytical methods?
Do you discuss the feasibility of your methods given restrictions as to time, budgets, your own skills, available support etc.?
Do you reflect on the external validity—on whether the data and the analyses will permit you to generalise your results—to populations, to theory or to methodology?
Do you clearly present the delimitations and limitations of your proposed investigation?
Do you reflect on your own role as researcher—including how your own perspectives and values have played a role in your choice of topic and objectives, methods, design of experiments, analysis and documentation?
Do you discuss what precautions you will take against procedural bias, including possible bias resulting from your own background, worldviews, and values?
Do you discuss the possible ethical aspects of your proposed study and how these may have affected your research design?

Presentation of Results: Note that aside from the checklists below with respect to Appendices and References, the remainder of Checklist 1 is primarily relevant for dissertations and not for proposals since it refers to your results—to work that has been completed!

Question	Needs work	OK as is
Are your results clear and concise? Are they comprehensible: (a) to your peers—in particular to the members of your evaluation committee, (b) to others who may be affected by/interested in your results but who may lack special knowledge or skills?
Are the concepts, hypotheses and theories that you have developed clearly identified and justified?
Do you discuss the actual development of your research and the accompanying learning process whereby the concepts, hypotheses and theories that you refer to have evolved (in contrast to giving the impression that everything was done in a completely well-structured and planned manner)?
Do you make good use of tables, figures etc., including those derived from secondary data sources?
Have you asked yourself what “story” you want a table, figure etc. to convey and do you try to relate that story to the reader?
Do you provide a rigorous justification of your results? Do you “validate” your findings—do you demonstrate their credibility/transferability/authenticity/confirmability/reliability, etc.? Do you for example refer to
(a) Theory you may have employed to design your investigation?
(b) Evidence provided by others, including the literature you refer to and secondary data sources?
(c) Your own empirical results—including salient aspects of tables, graphs or other forms of analysis you present?
(d) Re. the social sciences: ‘Rich, thick descriptions’ resulting from your observations in the field—including the use of representative quotes (as well as dissenting quotes) from interviews? Comments from participants as to the accuracy/credibility of your findings?
Do you present not just the results but also reflect on their implications and relate these to the research questions/hypotheses that have driven your research?
Do you also present anomalous, discrepant and unexpected data that appear to run counter to your major hypotheses and results and do you discuss how such data can influence and be interpreted with respect to your findings?
Do you provide clear, concise discussions of the usefulness of your results, including their possible influence on future work in this area?

Conclusions

Question	Needs work	OK as is
Do you provide strong, clear statements of your findings—including the concepts, hypotheses, and theories you have developed?
Do you clearly underline the significance of what you have done—both for the scientific community and for society?
Do you discuss the implications of your acceptance/rejection of hypotheses for theory development and future research?
Do you propose areas of further research suggested by your findings?
Do you, with the benefit of hindsight, draw attention to any limitations of your research and its results?
Have you written your conclusions in a manner that demonstrates that you have a good command of research methodology?

Appendices

Question	Needs work	OK as is
Do you place whatever is not essential to the proposal/dissertation in an appendix?
Primarily for the social sciences: Do you include (in separate appendices): questionnaires, interview guides, coding frames, letters sent to sample members …?

References

Question	Needs work	OK as is
Do you provide a logical and consistent way of referencing, and a logical and consistent list of all references cited in the proposal/dissertation—and of only those references?

Other obligations

Question	Needs work	OK as is
Have you supplied a copy or a summary of the dissertation to those you promised a copy? To others you feel indebted to?
Primarily for the Social Sciences
Have you carefully respected the confidentiality of all information that was provided to you under the explicit or implicit assumption of its being kept confidential? If necessary, provide transcripts that are to be kept confidential in an appendix that is only made available to the external examiners upon their request
Have you respected the explicit or implicit assumption by research participants that they would remain anonymous?

• 2. Checklist with respect to Planning and Writing

Planning

Question	Needs work	OK as is
Do you develop a provisional table of contents, including annotated titles of chapters/major sections, together with the approximate number of pages for each?
(And when you are underway with your research project: Do you regularly (say once every month) update your provisional table of contents?)
Do you develop a provisional time plan—including the time to be spent on major activities such as literature search, pilot studies/experiments/observations, analyses of data, discussions with peers and advisors, writing, etc.)?
(And when you are underway with your research project: Do you regularly (say once every month) update your provisional time plan? And do you also keep a rough track of the actual time you have spent on such activities in the past period so that you can compare this to your earlier estimates and be more accurate in your planning?)
Do you include in your time plan considerable “slack” to take into account unexpected events and unforeseen difficulties—as well as the need to re-write and “re-polish” your “gem”?
Do you keep a “diary” of your activities and progress that you can use to modify current plans as well as to document your research process?

Writing

Question	Needs work	OK as is
Have you edited and revised drafts so that your writing is clear, concise, consistent, coherent, and credible?
In particular, do you continually check that you are avoiding “sloppiness”—including grammatical and spelling errors and the indiscriminate use of capital letters; inconsistency in the use of terminology; repeating yourself? Do you make use of the facility in your word-processing software for checking spelling and grammar?
Do you continually ask yourself whether others, especially the external examiners, will readily follow your arguments, presentations etc.? Do you try to look at your writing “through their eyes”—even though you do not know who they are?
Do you structure the dissertation in an easy-to-follow and systematic way, with a clear and logical structure (chapter and section/sub-section headings, line spacing, typography, footnotes/endnotes, references, quotes, tables, models and schematics, figures, use of 1st or 3rd person, gender or gender neutrality …)?
Do you start each chapter with a brief introduction to the particular issues that are being examined?
Do you start each major section with a “topic sentence” that clearly tells the reader what the main message of the section is—and then follow that topic sentence with supporting detail?
At the conclusion of each chapter and section, do you make clear what your results have shown—and draw out any links that may be made with preceding or following chapters/sections so as to provide continuity?
Do you build ‘bridges’ between your chapters and major sections such that there is a logical ‘flow’ in the presentation?
Do you clearly explain each conclusion that you draw from your research in a manner that a scientifically inclined reader could understand and accept?
Are you persuasive? Do you provide convincing arguments for why your research is important, and why the findings and results are plausible/reliable and significant?
Primarily for the social sciences:
Have you treated confidential material in a proper manner?

• 3. Checklist with respect to your Oral Defence/Viva voce

Structure, Visual Aids & Presentation

Question	Needs work	OK as is
Have you designed your presentation to take into account that a listener’s concentration curve results in best memory retention at the start and end of your presentation?
Does your use of visual aids (e.g. PowerPoint) permit you to gather your thoughts while the reviewer’s attention is focused on the screen?
Do you make sure that your use of slides in fact assists you and does not disturb the flow of your presentation?
(a) Do you make sure that your slides are not ‘packed’, that they are easy to read and are not so long that they divert attention from your message?
(b) Do you avoid cutting and pasting sections of your dissertation which can lead to poor slides and poor presentations?
(c) Do you make only the minimum number of slides required so that you avoid racing through your presentation and are able to make thoughtful pauses? Rule of thumb: maximum 2 slides/minute; use Arial as font (easier to see from distance)
(d) Do you avoid putting things on a slide that will not be covered in your presentation?
(e) Do you minimize the use of upper case letters that are more difficult to read?
Do you rehearse and pre-test your presentation so that your oral language as well as your ‘body language’ contribute to the credibility of your presentation and are well-suited to the locality where you will make your presentation—and so that you can have good ‘eye-contact’ with the examiners and the audience?
Do you avoid looking at the screen instead of focusing on your examiners and audience?
Do you avoid reading from the screen?
Do you provide some good ‘stories’ to liven up your presentation?
Are you positive and enthusiastic yet calm? And do you speak clearly and firmly?
Have you prepared yourself for likely questions?
Have you familiarized yourself with publications that have appeared after you submitted your thesis?
Do you have extra slides ready to deal with questions you can already anticipate?

• 4. 4. Checklist of precautions (relevant for both proposals and dissertations)

Question	Needs work	OK as is
Do you avoid the use of “normative” statements (employing terms such as “ought”, “should”) unless these are backed up by strong evidence?
Do you avoid subjective statements unless you provide good arguments for them (e.g. when you present your own views as to the importance of your research)?
Do you avoid the use of words such as “clearly”, “obviously”, etc. unless your arguments really provide credence to such terms?
Are you aware of your possible biases and preconceptions?
Do you avoid speaking of “proof” since in empirical/inductive research one is not able to prove one’s hypotheses (only to support them via data and analyses)?
In case you justify the significance of your proposal/results by referring to a gap in existing knowledge, have you been very careful with your review of the literature?
Do you avoid being too self-critical? Do you avoid being self-laudatory? Do you avoid praising the significance of your proposed investigation/your results?
Do you avoid providing too many citations and quotes?
Do you avoid giving the impression of a “linear” process, where everything will proceed (has proceeded in the case of a dissertation) according to your plans?
Do you wait to finalize your Introduction/Abstract until you are pretty much finished with your writing?
Do you avoid sloppiness?
The following precautions are primarily relevant for your dissertation and viva
Do you avoid presenting too many analyses and results? Do you present only those findings that relate to your questions/hypotheses so that the thread of your argument is not lost?
Do you avoid presenting all your data so as not to burden the reader and to avoid giving the impression that your findings are unstructured? Parsimony is a virtue in science.
Do you avoid giving the impression that everything has proceeded according to your original proposal? In other words, are you honest and reflective as to your process—including how you have had to modify your ideas, research questions/hypotheses, perspectives and research design as you have gained new insights underway?
Have you remembered that, in principle, your dissertation serves three major purposes:
(a) To demonstrate that you have a sufficient command of research methodology and of skill in applying this methodology so that your thesis can be accepted—and that you can be accepted as a member of the scientific community
(b) To contribute to scientific knowledge in your field
(c) To contribute to those affected by your research—including yourself as you develop your competences, the scientific community, and society in general