Introduction to Causal Inference Approaches

  • Elizabeth A. StuartEmail author
  • Sarah Naeger
Reference work entry
Part of the Health Services Research book series (HEALTHSR)


Many questions in health services research require causal estimates of the effects of policies or programs on a health outcome. Although randomized experiments are seen as the gold standard for estimating causal effects, randomization is often unfeasible and/or impractical or will not answer the question of interest. In those cases, rigorous nonexperimental study designs can be used, as highlighted in this chapter. The chapter first takes care to carefully define the causal effects of interest and stresses the importance of careful study design. Overviews of four common nonexperimental study designs are then provided: instrumental variables, regression discontinuity, interrupted time series (and the related approach of difference in differences), and propensity score matching. An emphasis is on applications of these methods in health services research and the assumptions underlying each approach. The chapter concludes with open topics and suggestions for the conduct of studies aiming to estimate causal effects in health services research.


  1. Andersson K, Petzold MG, Sonesson C, Lonnroth K, Carlsten A. Do policy changes in the pharmaceutical reimbursement schedule affect drug expenditures? Interrupted time series analysis of cost, volume, and cost per volume trends in Sweden 1986–2002. Health Policy. 2006;79:231–43.PubMedCrossRefGoogle Scholar
  2. Angrist JD, Imbens GW. Two-stage least squares estimation of average causal effects in models with variable treatment intensity. J Am Stat Assoc. 1995;90(430):431–42. Scholar
  3. Angrist JD, Imbens GW. Identification of causal effects using instrumental variables. J Am Stat Assoc. 1996;91:444–55.CrossRefGoogle Scholar
  4. Baicker K, Finkelstein A. The effects of Medicaid coverage – learning from the Oregon experiment. N Engl J Med. 2011;365(8):683–5.PubMedPubMedCentralCrossRefGoogle Scholar
  5. Bao Y, Duan N, Fox SA. Is some provider advice on smoking cessation better than no advice? An instrumental variable analysis of the 2001 National Health Iinterview Survey. Health Serv Res. 2006;41(6):2114–35.PubMedPubMedCentralCrossRefGoogle Scholar
  6. Berger ML, Mamdani M, Atkins D, Johnson ML. Good research practices for comparative effectiveness research: defining, reporting and interpreting nonrandomized studies of treatment effects using secondary data sources: the ISPOR good research practices for retrospective database analysis task force report – part I. Value Health. 2009;12(8):1044–52.PubMedCrossRefGoogle Scholar
  7. Bound J, Jaeger DA, Baker RM. Problems with instrumental variables estimation when the correlation between the instruments and the endogenous explanatory variable is weak. J Am Stat Assoc. 1995;90(430):443–50.Google Scholar
  8. Campbell SM, Reeves D, Kontopantelis E, Sibbald B, Roland M. Effects of pay for performance on the quality of primary care in England. N Engl J Med. 2009;361(4):368–78. Scholar
  9. Carneiro P, Heckman JJ, Vytlacil EJ. Estimating marginal returns to education. Am Econ Rev. 2011;101(6):2754–81.PubMedPubMedCentralCrossRefGoogle Scholar
  10. Cole SR, Frangakis CE. The consistency statement in causal inference: a definition or an assumption? Epidemiology. 2009;20(1):3–5.PubMedCrossRefGoogle Scholar
  11. Cook TD, Shadish WR, Wong VC. Three conditions under which experiments and observational studies produce comparable causal estimates: new findings from within-study comparisons. J Policy Anal Manage. 2008;27(4):724–50. Scholar
  12. Crawford MJ, Thana L, Methuen C, Ghosh P, Stanley SV, Ross J, Gordon F, et al. Impact of screening for risk of suicide: randomized controlled trial. Br J Psychiatry. 2011;198(5):379–84.PubMedCrossRefGoogle Scholar
  13. De Melo-Martín I, Sondhi D, Crystal RG. When ethics constrains clinical research: trial design of control arms in “greater than minimal risk” pediatric trials. Hum Gene Ther. 2011;22(9):1121–7.PubMedPubMedCentralCrossRefGoogle Scholar
  14. Dowd BE. Separated at birth: statisticians, social scientists, and causality in health services research. Health Serv Res. 2011;46(2):397–420.PubMedPubMedCentralCrossRefGoogle Scholar
  15. Durbin J. Testing for serial correlation in least-squares regression when some of the Regressors are lagged dependent variables. Econometrica. 1970;38(3):410–21.CrossRefGoogle Scholar
  16. Escarce JJ, Flood AB. Introduction to special section: causality in health services research. Health Serv Res. 2011;46(2):394–6. Scholar
  17. Finkelstein EA, Fiebelkorn IC, Wang G. State-level estimates of annual medical expenditures attributable to obesity*. Obes Res. 2004;12(1):18–24. Scholar
  18. Fisher R. The arrangement of field experiments. Journal of Ministry of Agriculture. 1926;33:500–13.Google Scholar
  19. Frangakis CE, Rubin DB. Principal stratification in causal inference. Biometrics. 2002;58(1):21–9.PubMedPubMedCentralCrossRefGoogle Scholar
  20. Frangakis CE, Rubin DB, An MW, MacKenzie E. Principal stratification designs to estimate input data missing due to death. Biometrics. 2007;63(3):641–9.PubMedCrossRefGoogle Scholar
  21. Gluud LL. Bias in clinical intervention research. Am J Epidemiol. 2006;163(6):493–501. Scholar
  22. Goldberger A. Selection bias in evaluating treatment effects: some formal illustrations. In: Modelling and evaluating treatment effects in econometrics, Advances in econometrics. Bingley: Emerald Group Publishing Limited; 2008. p. 1–31.Google Scholar
  23. Greenland S. An introduction to instrumental variables for epidemiologists. Int J Epidemiol. 2000;29(4):722–9.PubMedCrossRefGoogle Scholar
  24. Greenland S. Epidemiologic measures and policy formulation: lessons from potential outcomes. Emerging Themes in Epidemiology. 2005;2(1):5.PubMedPubMedCentralCrossRefGoogle Scholar
  25. Hacker K, Penfold R, Zhang F, Soumerai SB. Impact of electronic health record transition on behavioral health screening in a large pediatric practice. Psychiatr Serv. 2012;63(3):256–61.PubMedCrossRefGoogle Scholar
  26. Holland PW. Statistics and causal inference. J Am Stat Assoc. 1986;81(396):945–60.CrossRefGoogle Scholar
  27. Hughes JR. Ethical concerns about non-active conditions in smoking cessation trials and methods to decrease such concerns. Drug Alcohol Depend. 2009;100(3):187–93.PubMedCrossRefGoogle Scholar
  28. Imai K, Keele L, Yamamoto T. Identification, inference and sensitivity analysis for causal mediation effects. Stat Sci. 2010;25(1):51–71.CrossRefGoogle Scholar
  29. Imai K, King G, Stuart EA. Misunderstandings between experimentalists and observationalists about causal inference. J R Stat Soc Ser A Stat Soc. 2008;171(2):481–502.CrossRefGoogle Scholar
  30. Imbens GW, Lemieux T. Regression discontinuity designs: a guide to practice. J Econ. 2008;142(2):615–35.CrossRefGoogle Scholar
  31. Lee BK, Lessler J, Stuart EA. Improving propensity score weighting using machine learning. Stat Med. 2010;29(3):337–46.PubMedPubMedCentralGoogle Scholar
  32. Lehman DR, Wortman CB, Williams AF. Long-term effects of losing a spouse or child in a motor vehicle crash. J Pers Soc Psychol. 1987;52(1):218–31.PubMedCrossRefGoogle Scholar
  33. Linden A, Adams JL. Evaluating disease management programme effectiveness: an introduction to instrumental variables. J Eval Clin Pract. 2006;12(2):148–54. Scholar
  34. Linden A, Adams JL, Roberts N. Evaluating disease management programme effectiveness: an introduction to the regression discontinuity design. J Eval Clin Pract. 2006;12(2):124–31.PubMedCrossRefGoogle Scholar
  35. Linden A, Adams JL. Using propensity score-based weighting in the evaluation of health management programme effectiveness. J Eval Clin Pract. 2010;16(1):175–9.PubMedCrossRefGoogle Scholar
  36. Little RJ, Rubin DB. Causal effects in clinical and epidemiological studies via potential outcomes: concepts and analytical approaches. Annu Rev Public Health. 2000;21:121–45. Scholar
  37. Liu W, Kuramoto SK, Stuart EA. An introduction to sensitivity analysis for unobserved confounding in non-experimental prevention research. Prev Sci. 2013;14(6):570–80. PMCID:3800481.PubMedPubMedCentralCrossRefGoogle Scholar
  38. Long SK, Coughlin T, King J. How well does medicaid work in improving access to care? Health Serv Res. 2005;40(1):36–58. Scholar
  39. Ludwig J, Miller DL. Does head start improve children’s life chances? Evidence from a regression discontinuity design. Institute for the Study of Labor (IZA). 2006. Retrieved from
  40. Marasinghe JP, Amarasinghe AAW. Noncompliance in randomized controlled trials [4]. CMAJ. 2007;176(12):1735.PubMedPubMedCentralCrossRefGoogle Scholar
  41. Marcus SM, Stuart EA, Wang P, Shadish WR, Steiner PM. Estimating the causal effect of randomization versus treatment preference in a doubly randomized preference trial. Psychol Methods. 2012;17(2):244–54.PubMedPubMedCentralCrossRefGoogle Scholar
  42. McClellan M, McNeil BJ, Newhouse JP. Does more intensive treatment of acute myocardial infarction in the elderly reduce mortality? Analysis using instrumental variables. JAMA. 1994;272:859–66.PubMedCrossRefGoogle Scholar
  43. Mills TC. Time series techniques for economists. Cambridge: Cambridge University Press; 1990.Google Scholar
  44. Mullins CD, Abdulhalim AM, Lavallee DC. Continuous patient engagement in comparative effectiveness research. JAMA J Am Med Assoc. 2012;307(15):1587–8.CrossRefGoogle Scholar
  45. Neyman J. On the application of probability theory to agricultural experiments. Essay on principles. Stat Sci. 1923;5(4):465–80.CrossRefGoogle Scholar
  46. Neyman J. On the two different aspects of the representative method: the method of stratified sampling and the method of purposive selection. J R Stat Soc. 1934;97:558–606.CrossRefGoogle Scholar
  47. Oliver S, Armes DG, Gyte G. Public involvement in setting a national research agenda: a mixed methods evaluation. Patient Patient-Cent Outcomes Res. 2009;2(3):179–90.CrossRefGoogle Scholar
  48. O’Malley AJ. Commentary on Bryan Dowd’s paper “Separated at birth: statisticians, social scientists, and causality in health services research”. Health Serv Res. 2011;46(2):430–6.PubMedCrossRefGoogle Scholar
  49. O’Malley AJ, Frank RG, Kaddis A, Rothenberg BM, McNeil BJ. Impact of alternative interventions on changes in generic dispensing rates. Health Serv Res. 2006;415(5):1876–94.CrossRefGoogle Scholar
  50. Pearl J. Statistics and causality: Separated to reunite – commentary on Bryan Dowd’s “Separated at birth”. Health Serv Res. 2011;46(2):421–9.PubMedPubMedCentralCrossRefGoogle Scholar
  51. Peduzzi P, Wittes J, Detre K, Holford T. Analysis as-randomized and the problem of non-adherence: an example from the veterans affairs randomized trial of coronary artery bypass surgery. Stat Med. 1993;12(13):1185–95. Scholar
  52. Rosenbaum PR. Choice as an alternative to control in observational studies. Stat Sci. 1999;14(3):259–304.CrossRefGoogle Scholar
  53. Rosenbaum PR. Observational study. In: Everitt B, Howell D, editors. Encyclopedia of statistics in behavioral science. Chichester: Wiley; 2005a.Google Scholar
  54. Rosenbaum PR. Sensitivity analysis in observational studies. In: Everitt BS, Howell DC, editors. Encyclopedia of statistics in behavioral science, vol. 4. Chichester: Wiley; 2005b. p. 1809–14.Google Scholar
  55. Rosenbaum PR. Design of observational studies, Springer series in statistics. New York: Springer; 2010.CrossRefGoogle Scholar
  56. Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41–55.CrossRefGoogle Scholar
  57. Rosenberg L. Comparative effectiveness research: making it work for those we serve. J Behav Health Serv Res. 2009;36(3):283–4.PubMedCrossRefGoogle Scholar
  58. Rothwell PM. External validity of randomised controlled trials? To whom do the results of this trial apply?? Lancet. 2005;365(9453):82–93.PubMedPubMedCentralCrossRefGoogle Scholar
  59. Rubin DB. The design versus the analysis of observational studies for causal effects: parallels with the design of randomized trials. Stat Med. 2007;26(1):20–36.PubMedCrossRefGoogle Scholar
  60. Schneeweiss S. Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol Drug Saf. 2006;15(5):291–303.PubMedCrossRefGoogle Scholar
  61. Schneeweiss S, Rassen JA, Glynn RJ, Avorn J, Mogun H, Brookhart MA. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology. 2009;20(4):512–22.PubMedPubMedCentralCrossRefGoogle Scholar
  62. Shadish WR, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalized causal inference. 2nd ed. Belmont: Wadsworth Publishing; 2002.Google Scholar
  63. Steiner PM, Cook TD, Shadish WR, Clark MH. The importance of covariate selection in controlling for selection bias in observational studies. Psychol Methods. 2010;15(3):250–67.PubMedCrossRefGoogle Scholar
  64. Stuart EA. Matching methods for causal inference: a review and a look forward. Stat Sci. 2010;25(1):1–21.PubMedPubMedCentralCrossRefGoogle Scholar
  65. Thistlethwaite DL, Campbell DT. Regression-discontinuity analysis: an alternative to the ex post facto experiment. J Educ Psychol. 1960;51(6):309–17.CrossRefGoogle Scholar
  66. Trochim W. Research design for program evaluation; the regression-discontinuity design. Beverly Hills: Sage; 1984.Google Scholar
  67. Wagenaar AC, Maldonado-Molina MM, Wagenaar BH. Effects of alcohol tax increases on alcohol-related disease mortality in Alaska: time-series analysis from 1976 to 2004. Am J Public Health. 2009;99(8):1464–70.PubMedPubMedCentralCrossRefGoogle Scholar
  68. Werner RM, Konetzka RT, Stuart EA, Norton EC, Polsky D, Park J. Impact of public reporting on quality of Postacute care. Health Serv Res. 2009;44(4):1169–87. Scholar
  69. Wong VC, Steiner PM, Cook TD. Analyzing regression-discontinuity designs with multiple assignment variables: a comparative study of four estimation methods. J Educ Behav Stat. 2012; Scholar
  70. Zaslavsky AM, Ayanian JZ, Zaborski LB. The validity of race and ethnicity in enrollment data for medicare beneficiaries. Health Serv Res. 2012;47(3 Part 2):1300–21.PubMedPubMedCentralCrossRefGoogle Scholar
  71. Zimmerman M, Chelminski I, Posternak MA. Generalizability of antidepressant efficacy trials: differences between depressed psychiatric outpatients who would or would not qualify for an efficacy trial. Am J Psychiatr. 2005;162(7):1370–2.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Mental HealthJohns Hopkins Bloomberg School of Public HealthBaltimoreUSA
  2. 2.Behavioral Health Research and PolicyIBM Watson HealthBethesdaUSA

Personalised recommendations