Skip to main content
SpringerLink
Log in

Experimental Design

  • Chapter
  • First Online:
Human Subject Research for Engineers

Part of the book series: SpringerBriefs in Applied Sciences and Technology ((BRIEFSAPPLSCIENCES))

Abstract

This chapter starts with explaining the difference between an experiment and a quasi-experiment. Next, between-subjects and within-subject research designs are compared, and criteria about the choice for either design are discussed. The importance of a control group is highlighted, and techniques for participant assignment to groups are presented. Validity threats are described, including sample representativeness, demand characteristics, experimenter expectancy bias, causation versus correlation, and attrition. We explain the notion of statistical reliability and discuss self-reported measures and associated pitfalls such as social desirability and response style.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Alimena, B. S. (1962). A method of determining unbiased distribution in the Latin square. Psychometrika, 27, 315–317. https://doi.org/10.1007/BF02289627

  • Bhatt, A. (2010). Evolution of clinical research: A history before and beyond James Lind. Perspectives in Clinical Research, 1, 6–10.

    Google Scholar 

  • Biglan, A., Ary, D., & Wagenaar, A. C. (2000). The value of interrupted time-series experiments for community intervention research. Prevention Science, 1, 31–49. https://doi.org/10.1023/A:1010024016308

  • Birnbaum, M. H. (1999). How to show that 9 > 221: Collect judgments in a between-subjects design. Psychological Methods, 4, 243–249. https://doi.org/10.1037/1082-989X.4.3.243

  • Borman, G. D. (2002). Experiments for educational evaluation and improvement. Peabody Journal of Education, 77, 7–27. https://doi.org/10.1207/S15327930PJE7704_2

  • Buchanan, T. (2000). Potential of the Internet for personality research. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 121–140). San Diego, CA: Academic Press.

    Chapter  Google Scholar 

  • Burtless, G. (2002). Randomized field trials for policy evaluation: Why not in education? In F. Mosteller & R. Boruch (Eds.), Evidence matters: Randomized trials in education research (pp. 179–197). Washington, DC: Brookings Institute.

    Google Scholar 

  • Campbell, G., & Geller, S. (1980). Balanced Latin squares (Mimeoseries No. 80-26). West Lafayette, IN: Department of Statistics, Purdue University.

    Google Scholar 

  • Centers for Disease Control and Prevention. (2010). How tobacco smoke causes disease: The biology and behavioral basis for smoking-attributable disease: A report of the surgeon general. Centers for Disease Control and Prevention (US).

    Google Scholar 

  • Charness, G., Gneezy, U., & Kuhn, M. A. (2012). Experimental methods: Between-subject and within-subject design. Journal of Economic Behavior & Organization, 81, 1–8. https://doi.org/10.1016/j.jebo.2011.08.009

  • Colbourn, C. J., Dinitz, J. H., & Wanless, I. M. (1996). Latin squares. In C. J. Colbourn & J. H. Dinitz (Eds.), Handbook of combinatorial designs (pp. 135–152). Boca Raton, FL: CRC Press, Taylor & Francis Group.

    Chapter  Google Scholar 

  • Connors, A. F., Speroff, T., Dawson, N. V., Thomas, C., Harrell, F. E., Wagner, D., et al. (1996). The effectiveness of right heart catheterization in the initial care of critically III patients. JAMA, 276, 889–897. https://doi.org/10.1001/jama.1996.03540110043030

  • De Craen, A. J., Kaptchuk, T. J., Tijssen, J. G., & Kleijnen, J. (1999). Placebos and placebo effects in medicine: Historical overview. Journal of the Royal Society of Medicine, 92, 511–515.

    Google Scholar 

  • De Winter, J. C. F., & Dodou, D. (2011). Predicting academic performance in engineering using high school exam scores. International Journal of Engineering Education, 27, 1343–1351.

    Google Scholar 

  • Dodou, D., & De Winter, J. C. F. (2014). Social desirability is the same in offline, online, and paper surveys: A meta-analysis. Computers in Human Behavior, 36, 487–495. https://doi.org/10.1016/j.chb.2014.04.005

  • Doll, R., Peto, R., Boreham, J., & Sutherland, I. (2004). Mortality in relation to smoking: 50 years’ observations on male British doctors. BMJ, 328, 1519. https://doi.org/10.1136/bmj.38142.554479.AE

  • Ederer, F. (1975). Patient bias, investigator bias and the double-masked procedure in clinical trials. The American Journal of Medicine, 58, 295–299. https://doi.org/10.1016/0002-9343(75)90594-X

  • Eysenck, H. J. (1939). The validity of judgments as a function of the number of judges. Journal of Experimental Psychology, 25, 650–654. https://doi.org/10.1037/h0058754

  • Eysenck, S. B., Eysenck, H. J., & Barrett, P. (1985). A revised version of the psychoticism scale. Personality and Individual Differences, 6, 21–29. https://doi.org/10.1016/0191-8869(85)90026-1

  • Feynman, R. P. (1974). Cargo cult science. Some remarks on science, pseudoscience, and learning how to not fool yourself. Caltech’s 1974 commencement address. http://calteches.library.caltech.edu/51/2/CargoCult.pdf

  • Fischhoff, B., Slovic, P., & Lichtenstein, S. (1979). Subjective sensitivity analysis. Organizational Behavior and Human Performance, 23, 339–359. https://doi.org/10.1016/0030-5073(79)90002-3

  • Freedman, N. D., Leitzmann, M. F., Hollenbeck, A. R., Schatzkin, A., & Abnet, C. C. (2008). Cigarette smoking and subsequent risk of lung cancer in men and women: Analysis of a prospective cohort study. The Lancet Oncology, 9, 649–656. https://doi.org/10.1016/S1470-2045(08)70154-2

  • Friedman, L. M., Furberg, C. D., DeMets, D. L., Reboussin, D. M., & Granger, C. B. (2015). Fundamentals of clinical trials (5th ed.). Springer International Publishing.

    Google Scholar 

  • Glasziou, P. (2016, February 16). Still no evidence for homeopathy [blog]. The BMJ blogs. http://blogs.bmj.com/bmj/2016/02/16/paul-glasziou-still-no-evidence-for-homeopathy/

  • Gordon, K. (1924). Group judgments in the field of lifted weights. Journal of Experimental Psychology, 7, 398–400. https://doi.org/10.1037/h0074666

  • Greenwald, A. G. (1976). Within-subjects designs: To use or not to use? Psychological Bulletin, 83, 314–320. https://doi.org/10.1037/0033-2909.83.2.314

  • Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., et al. (2008). GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ, 366, 924–926. https://doi.org/10.1136/bmj.39489.470347.AD

  • Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33, 61–83. https://doi.org/10.1017/S0140525X0999152X

  • Henry, G. T. (1990). Practical sampling. Newbury Park, CA: Sage Publications.

    Book  Google Scholar 

  • Hiatt, K. L., Braithwaite, M. G., Crowley, J. S., Rash, C. E., Van de Pol, C., Ranchino, D. J., et al. (2001). The effect of a monocular helmet-mounted display on aircrew health: A cohort study of Apache AH MK1 pilots (Initial Report No. USAARL-2002-04). Fort Rucker, AL: U.S. Army Aeromedical Research Laboratory.

    Google Scholar 

  • Hill, A. B. (1965). The environment and disease: Association or causation? Proceedings of the Royal Society of Medicine, 58, 295–300.

    Google Scholar 

  • Hoorens, V., & Harris, P. (1998). Distortions in reports of health behaviors: The time span effect and illusory superiority. Psychology and Health, 13, 451–466. https://doi.org/10.1080/08870449808407303

  • Ioannidis, J. P. (2013). Mega-trials for blockbusters. JAMA, 309, 239–240. https://doi.org/10.1001/jama.2012.168095

  • Jackson, D. N., & Messick, S. (1958). Content and style in personality assessment. Psychological Bulletin, 55, 243–252. https://doi.org/10.1037/h0045996

  • Jensen, A. R. (2006). Clocking the mind: Mental chronometry and individual differences. Amsterdam: Elsevier.

    Google Scholar 

  • Johnson, R. C., McClearn, G. E., Yuen, S., Nagoshi, C. T., Ahern, F. M., & Cole, R. E. (1985). Galton’s data a century later. American Psychologist, 40, 875–892. https://doi.org/10.1037/0003-066X.40.8.875

  • Kaasinen, V., Aalto, S., Någren, K., & Rinne, J. O. (2004). Expectation of caffeine induces dopaminergic responses in humans. European Journal of Neuroscience, 19, 2352–2356. https://doi.org/10.1111/j.1460-9568.2004.03310.x

  • Kang, M., Ragan, B. G., & Park, J. H. (2008). Issues in outcomes research: An overview of randomization techniques for clinical trials. Journal of Athletic Training, 43, 215–221.

    Article  Google Scholar 

  • Keren, G. (1993). Between-or within-subjects design: A methodological dilemma. In G. Keren & C. Lewis (Eds.), A handbook for data analysis in the behaviorial sciences: Methodological issues (pp. 257–272). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Key, T. J., Allen, N. E., Spencer, E. A., & Travis, R. C. (2002). The effect of diet on risk of cancer. The Lancet, 360, 861–868. https://doi.org/10.1016/S0140-6736(02)09958-0

  • Kim, B. G., & Kim, T. (2010). A program for making completely balanced Latin square designs employing a systemic method. Revista Colombiana de Ciencias Pecuarias, 23, 277–282.

    Google Scholar 

  • Kirsch, I., Deacon, B. J., Huedo-Medina, T. B., Scoboria, A., Moore, T. J., & Johnson, B. T. (2008). Initial severity and antidepressant benefits: A meta-analysis of data submitted to the Food and Drug Administration. PLOS Medicine, 5, e45. https://doi.org/10.1371/journal.pmed.0050045

  • Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Jr., Bahník, Š., Bernstein, M. J., et al. (2014). Investigating variation in replicability: A “many labs” replication project. Social Psychology, 45, 142–152. https://doi.org/10.1027/1864-9335/a000178

  • Krosnick, J. A., & Presser, S. (2010). Question and questionnaire design. In J. D. Wright & P. V. Marsden (Eds.), Handbook of survey research (2nd ed., pp. 263–313). West Yorkshire, England: Emerald Group.

    Google Scholar 

  • Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77, 1121–1134. https://doi.org/10.1037/0022-3514.77.6.1121

  • Liu, Y., & Salvendy, G. (2009). Effects of measurement errors on psychometric measurements in ergonomics studies: Implications for correlations, ANOVA, linear regression, factor analysis, and linear discriminant analysis. Ergonomics, 52, 499–511. https://doi.org/10.1080/00140130802392999

  • Loftus, E. F., & Palmer, J. C. (1974). Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behavior, 13, 585–589. https://doi.org/10.1016/S0022-5371(74)80011-3

  • Mars, F., Deroo, M., & Charron, C. (2014). Driver adaptation to haptic shared control of the steering wheel. In Proceedings of the 2014 IEEE International Conference on Systems, Man and Cybernetics (pp. 1505–1509). https://doi.org/10.1109/SMC.2014.6974129

  • McGuire, L., & Smith, N. (2000). Cycling safety: Injury prevention in Oxford cyclists. Injury Prevention, 6, 285–287. https://doi.org/10.1136/ip.6.4.285

  • Mikalsen, A., Bertelsen, B., & Flaten, M. (2001). Effects of caffeine, caffeine-associated stimuli, and caffeine-related information on physiological and psychological arousal. Psychopharmacology, 157, 373–380. https://doi.org/10.1007/s002130100841

  • National Health and Medical Research Council. (1999). A guide to the development, implementation and evaluation of clinical practice guidelines. Canberra: National Health and Medical Research Council.

    Google Scholar 

  • Oken, B. S. (2008). Placebo effects: Clinical aspects and neurobiology. Brain, 131, 2812–2823. https://doi.org/10.1093/brain/awn116

  • Peto, R., Darby, S., Deo, H., Silcocks, P., Whitley, E., & Doll, R. (2000). Smoking, smoking cessation, and lung cancer in the UK since 1950: Combination of national statistics with two case-control studies. BMJ, 321, 323–329. https://doi.org/10.1136/bmj.321.7257.323

  • Pocock, S. J., & Simon, R. (1975). Sequential treatment assignment with balancing for prognostic factors in the controlled clinical trial. Biometrics, 31, 103–115. https://doi.org/10.2307/2529712

  • Pozdêna, R. F. (1905). Versuche über Blondlots „Emission pesante”. Annalen der Physik, 322, 104–131. https://doi.org/10.1002/andp.19053220606

  • Rosenthal, R., Kohn, P., Greenfield, P. M., & Carota, N. (1966). Data desirability, experimenter expectancy, and the results of psychological research. Journal of Personality and Social Psychology, 3, 20–27. https://doi.org/10.1037/h0022604

  • Rushton, J. P., Brainerd, C. J., & Pressley, M. (1983). Behavioral development and construct validity: The principle of aggregation. Psychological Bulletin, 94, 18–38. https://doi.org/10.1037/0033-2909.94.1.18

  • Schmidt, F. L., & Hunter, J. E. (1999). Theory testing and measurement error. Intelligence, 27, 183–198. https://doi.org/10.1016/S0160-2896(99)00024-0

  • Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93–105. https://doi.org/10.1037/0003-066X.54.2.93

  • Shibuya, K., Inoue, M., & Lopez, A. D. (2005). Statistical modeling and projections of lung cancer mortality in 4 industrialized countries. International Journal of Cancer, 117, 476–485. https://doi.org/10.1002/ijc.21078

  • Sinha, R., Cross, A. J., Graubard, B. I., Leitzmann, M. F., & Schatzkin, A. (2009). Meat intake and mortality: A prospective study of over half a million people. Archives of Internal Medicine, 169, 562–571. https://doi.org/10.1001/archinternmed.2009.6

  • Stoney, C. M., & Johnson, L. L. (2012). Design of clinical studies and trials. In J. I. Gallin & F. P. Ognibene (Eds.), Principles and practice of clinical research (pp. 225–242). Academic Press.

    Google Scholar 

  • Taves, D. R. (1974). Minimization: A new method of assigning patients to treatment and control groups. Clinical Pharmacology and Therapeutics, 15, 443–453. https://doi.org/10.1002/cpt1974155443

  • Thompson, B., Diamond, K. E., McWilliam, R., Snyder, P., & Snyder, S. W. (2005). Evaluating the quality of evidence from correlational research for evidence-based practice. Exceptional Children, 71, 181–194. https://doi.org/10.1177/001440290507100204

  • Thompson, R. S., Rivara, F. P., & Thompson, D. C. (1989). A case-control study of the effectiveness of bicycle safety helmets. The New England Journal of Medicine, 320, 1361–1367. https://doi.org/10.1056/NEJM198905253202101

  • Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124–1131. https://doi.org/10.1126/science.185.4157.1124

  • Underwood, B. J. (1949). Experimental psychology: An introduction. East Norwalk, CT: Appleton-Century-Crofts.

    Book  Google Scholar 

  • U.S. Department of Health and Human Services (2014). The health consequences of smoking—50 years of progress: A report of the Surgeon General. Atlanta: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health.

    Google Scholar 

  • U.S. Preventive Services Task Force. (1996). Guide to clinical preventive services (2nd ed.). Baltimore: Williams and Wilkins.

    Google Scholar 

  • Van der Geest, J. (2009a). LATSQ. (randomized) Latin Square. MATLAB script. http://www.mathworks.com/matlabcentral/fileexchange/12315-latsq/content/latsq.m

  • Van der Geest, J. (2009b). BALLATSQ—Balanced Latin Square. MATLAB script. https://nl.mathworks.com/matlabcentral/fileexchange/9996-ballatsq/content/ballatsq.m

  • Van Leeuwen, P. M., Happee, R., & De Winter, J. C. F. (2014). Vertical field of view restriction in driver training: A simulator-based evaluation. Transportation Research Part F: Traffic Psychology and Behaviour, 24, 169–182. https://doi.org/10.1016/j.trf.2014.04.010

  • Van Vaerenbergh, Y., & Thomas, T. D. (2013). Response styles in survey research: A literature review of antecedents, consequences, and remedies. International Journal of Public Opinion Research, 25, 195–217. https://doi.org/10.1093/ijpor/eds021

  • Wai, J., Lubinski, D., & Benbow, C. P. (2009). Spatial ability for STEM domains: Aligning over 50 years of cumulative psychological knowledge solidifies its importance. Journal of Educational Psychology, 101, 817–835. https://doi.org/10.1037/a0016127

  • Wang, C., Lu, L., & Lu, J. (2015). Statistical analysis of bicyclists’ injury severity at unsignalized intersections. Traffic Injury Prevention, 16, 507–512. https://doi.org/10.1080/15389588.2014.969802

  • Williams, E. J. (1949). Experimental designs balanced for the estimation of residual effects of treatments. Australian Journal of Scientific Research, 2, 149–168. https://doi.org/10.1071/CH9490149

  • Wood, R. W. (1904). The n-rays. Nature, 70, 530–531. https://doi.org/10.1038/070530a0

  • Zanotto, D., Rosati, G., Spagnol, S., Stegall, P., & Agrawal, S. K. (2013). Effects of complementary auditory feedback in robot-assisted lower extremity motor adaptation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 21, 775–786. https://doi.org/10.1109/TNSRE.2013.2242902

Download references

Author information

Authors and Affiliations

  1. Department of BioMechanical Engineering, Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, Delft, The Netherlands

    Joost C. F. de Winter

  2. Department of BioMechanical Engineering, Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, Delft, The Netherlands

    Dimitra Dodou

Authors
  1. Joost C. F. de Winter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dimitra Dodou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joost C. F. de Winter .

Rights and permissions

Reprints and permissions

Copyright information

© 2017 The Author(s)

About this chapter

Cite this chapter

de Winter, J.C.F., Dodou, D. (2017). Experimental Design. In: Human Subject Research for Engineers . SpringerBriefs in Applied Sciences and Technology. Springer, Cham. https://doi.org/10.1007/978-3-319-56964-2_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-56964-2_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-56963-5

  • Online ISBN: 978-3-319-56964-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation