Quality & Quantity

, Volume 45, Issue 1, pp 21–42 | Cite as

The use of quasi-experiments in the social sciences: a content analysis

  • Marie-Claire E. Aussems
  • Anne Boomsma
  • Tom A. B. Snijders


This article examines the use of various research designs in the social sciences as well as the choices that are made when a quasi-experimental design is used. A content analysis was carried out on articles published in 18 social science journals with various impact factors. The presence of quasi-experimental studies was investigated as well as choices in the design and analysis stage. It was found that quasi-experimental designs are not very often used in the inspected journals, and when they are applied they are not very well designed and analyzed. These findings suggest that the literature on how to deal with selection bias has not yet found its way to the practice of the applied researcher.


Quasi-experiments Social science Selection bias Research designs Content analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Angrist J.D., Imbens G.W., Rubin D.B.: Identification of causal effects using instrumental variables. J. Am. Stat. Assoc. 91, 444–455 (1996)CrossRefGoogle Scholar
  2. APA: Publication Manual of the American Psychological Association, 5th edn. APA, Washington (2001)Google Scholar
  3. Berk R.A.: Regression Analysis: A Constructive Critique. Sage, Thousand Oaks (2003)Google Scholar
  4. Campbell D.T.: Prospective: artifact and control. In: Rosenthal, R., Rosnow, R.L. (eds) Artifact in Behavioral Research, pp. 351–382. Academic Press, New York (1969)Google Scholar
  5. Campbell D.T., Erlebacher A.: How regression artifacts in quasi-experimental evaluations can mistakenly make compensatory education look harmful. In: Hellmuth, J. (eds) Disadvantaged Child: Compensatory Education, a National Debate, vol. 3, pp. 185–210. Brunner/Mazel, New York (1970)Google Scholar
  6. Cochran W.G.: The planning of observational studies in human populations. J. R. Stat. Soc. Ser. A-G 128, 134–155 (1965)Google Scholar
  7. Cochran W.G.: Planning and Analysis of Observational Studies. Wiley, Toronto (1983)CrossRefGoogle Scholar
  8. Coleman J.S., Campbell E.Q., Hobson C.J., McPartland J., Mood A.M., Weinfeld F.D., York R.L.: Equality of Educational Opportunity. Government Printing Office, Washington (1966)Google Scholar
  9. Heckman J.J.: Sample selection bias as a specification error. Econometrica 47, 153–161 (1979)CrossRefGoogle Scholar
  10. Holland P.W.: Statistics and causal inference (with discussion). J. Am. Stat. Assoc. 81, 945–960 (1986)CrossRefGoogle Scholar
  11. Holland P.W.: Comment: it’s very clear. J. Am. Stat. Assoc. 84, 875–877 (1989)CrossRefGoogle Scholar
  12. Mosteller, F.: Preliminary report for Group D. In: Report of the Harvard Faculty Seminar on the Equal Educational Opportunity Report (1967)Google Scholar
  13. Nichols R.C.: Schools and the disadvantaged. Science 154, 1312–1314 (1966)CrossRefGoogle Scholar
  14. Rosenbaum P.R.: Observational Studies, 2nd edn. Springer, New York (2002)Google Scholar
  15. Rosenbaum P.R., Rubin D.B.: The central role of the propensity score in observational studies for causal effects. Biometrika 76, 41–55 (1983)CrossRefGoogle Scholar
  16. Rosenbaum P.R., Rubin D.B.: Reducing bias in observational studies using subclassification on the propensity score. J. Am. Stat. Assoc. 79, 516–524 (1984)CrossRefGoogle Scholar
  17. Shadish W.R., Cook T.D., Campbell D.T.: Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin, Boston (2002)Google Scholar
  18. Westinghouse Learning Corporation: The Impact of Head Start: An Evaluation of the Effects of Head Start on Childrens Cognitive and Affective Development. Ohio University, Athens (1969)Google Scholar
  19. Wilkinson L.: The Task Force on Statistical Inference: Guidelines and explanations: statistical methods in psychology journals. Am. Psychol. 54, 594–604 (1999)CrossRefGoogle Scholar
  20. Winship C., Mare R.D.: Models for sample selection bias. Annu. Rev. Sociol. 18, 327–350 (1992)CrossRefGoogle Scholar
  21. Winship C., Morgan S.L.: The estimation of causal effects from observational studies. Annu. Rev. Sociol. 25, 659–704 (1999)CrossRefGoogle Scholar
  22. Winship C., Morgan S.L.: Counterfactuals for Causal Inference. Cambridge University Press, New York (2007)Google Scholar
  23. Winship C., Sobel M.: Causal inference in sociological studies. In: Hardy, M., Bryman, A. (eds) Handbook of Data Analysis, pp. 481–503. Sage, Thousand Oaks (2004)Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2009

Authors and Affiliations

  • Marie-Claire E. Aussems
    • 1
  • Anne Boomsma
    • 2
  • Tom A. B. Snijders
    • 2
    • 3
  1. 1.Department of Social Research Methodology, Faculty of Social SciencesVU UniversityAmsterdamThe Netherlands
  2. 2.University of GroningenGroningenThe Netherlands
  3. 3.University of OxfordOxfordUK

Personalised recommendations