Current Epidemiology Reports

, Volume 6, Issue 3, pp 373–379 | Cite as

Identifying Heterogeneous Treatment Effects of Drug Policy in Quasi-experimental Settings

  • Aaron N. WinnEmail author
  • Matthew L. Maciejewski
  • Stacie B. Dusetzina
Pharmacoepidemiology (U Haug, Section Editor)
Part of the following topical collections:
  1. Topical Collection on Pharmacoepidemiology


Purpose of Review

We sought to describe the difference-in-differences study design and how they can be applied to identify the average treatment effect. We then extend this approach to identify heterogeneity in treatment effects based on (1) an individuals’ baseline risk of an event using risk scores, (2) the outcome distribution using quantile regression, and (3) prior trajectories of outcomes using group-based trajectory models.

Recent Findings

The methods for the identification of heterogeneous treatment effect have developed in ways that can provide researchers and policymakers a more nuanced understanding of treatment effects.


Recent analytic advances found in other fields should be adopted and tested by pharmacoepidemiology and drug policy researcher to better understand the effects of new policies and interventions.


Heterogeneity Heterogeneous treatment effects Policy analysis Difference-in-difference Quantile regression Risk score Group-based trajectory model Propensity score 


Compliance With Ethical Standards

Conflict of Interest

Aaron N. Winn is supported by the National Center for Research Resources, the National Center for Advancing Translational Sciences, and the office of the Director, National Institutes of Health, through Grant Number KL2TR001438. Matthew L. Maciejewski reports grants from VA HSR&D (RCS 10–391), grants from VA HSR&D (CIN 13–410), grants from VA HSR&D (CRE 12–306), and grants from NIDA (R01 DA040056), outside the submitted work. Dr. Maciejewski owns Amgen stock due to his spouse’s employment. Stacie B. Dusetzina declares no potential conflicts of interest.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.


Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance

  1. 1.
    Kravitz RL, Duan N, Braslow J. Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. Milbank Q. 2004;82(4):661–87.Google Scholar
  2. 2.
    Rothwell PM, Warlow CP. Prediction of benefit from carotid endarterectomy in individual patients: a risk-modelling study. European Carotid Surgery Trialists’ Collaborative Group. Lancet. 1999;353(9170):2105–10.Google Scholar
  3. 3.
    •• Le Cook B, Manning WG. Thinking beyond the mean: a practical guide for using quantile regression methods for health services research. Shanghai Arch Psychiatry. 2013;25(1):55–9 Excellent article that describes how to use quantile regression. Google Scholar
  4. 4.
    Haviland A, Nagin DS, Rosenbaum PR, Tremblay RE. Combining group-based trajectory modeling and propensity score matching for causal inferences in nonexperimental longitudinal data. Dev Psychol. 2008;44(2):422–36.Google Scholar
  5. 5.
    •• Haviland A, Nagin DS, Rosenbaum PR. Combining propensity score matching and group-based trajectory analysis in an observational study. Psychol Methods. 2007;12(3):247–67 First article that lays out how to combine group based trajectory models to identify heterogeneous treatment effects. Google Scholar
  6. 6.
    Winn AN, Fergestrom NM, Neuner JM. Using group-based trajectory models and propensity score weighting to detect heterogeneous treatment effects: the case study of generic hormonal therapy for women with breast cancer. Med Care. 2019;57(1):85–93.Google Scholar
  7. 7.
    Angrist J, Pischke J. Mostly harmless econometrics: an empiricist’s companion. Princeton, NJ: Princeton University Press; 2008 2008.Google Scholar
  8. 8.
    Bertrand M, Duflo E, Mullainathan S. How much should we trust difference-in-differences estimates? Q J Econ. 2004;119(1):249–75.Google Scholar
  9. 9.
    •• Daw JR, Hatfield LA. Matching and regression to the mean in difference-in-differences analysis. Health Serv Res. 2018;53(6):4138–56 Provides an excellent overview on the use of difference-in-difference design and how concerns about how to appropriate adjust for non-parallel trends. Google Scholar
  10. 10.
    Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther. 2002;27(4):299–309.Google Scholar
  11. 11.
    •• Ryan AM, Kontopantelis E, Linden A, Burgess JF, Jr. Now trending: coping with non-parallel trends in difference-in-differences analysis. Statistical methods in medical research. 2018:962280218814570. Provides an excellent overview on the use of difference-in-difference design and if a researcher can use matching to overcome non-parallel trends. Google Scholar
  12. 12.
    Daw JR, Hatfield LA. Matching in difference-in-differences: between a rock and a hard place. Health Serv Res. 2018;53(6):4111–7.Google Scholar
  13. 13.
    • Ryan AM, Burgess JF Jr, Dimick JB. Why we should not be indifferent to specification choices for difference-in-differences. Health Serv Res. 2015;50(4):1211–35 Details approaches to ensure that difference-in-difference design do not suffer from bias and fixes to common problems. Google Scholar
  14. 14.
    •• Norton EC, Dowd BE, Maciejewski ML. Marginal effects-quantifying the effect of changes in risk factors in logistic regression models. JAMA. 2019;321(13):1304–5 Provides guidance on how to use understand interaction terms in non-linear models. Google Scholar
  15. 15.
    Ai C, Norton EC. Interaction terms in logit and probit models. Econ Lett. 2003;80(1):123–9.Google Scholar
  16. 16.
    •• Kent DM, Nelson J, Dahabreh IJ, Rothwell PM, Altman DG, Hayward RA. Risk and treatment effect heterogeneity: re-analysis of individual participant data from 32 large clinical trials. Int J Epidemiol. 2016;45(6):2075–88 Superb article that details how important baseline risk in when understanding the harms and benefits of treatments. Google Scholar
  17. 17.
    Hayward RA, Kent DM, Vijan S, Hofer TP. Reporting clinical trial results to inform providers, payers, and consumers. Health Aff (Millwood). 2005;24(6):1571–81.Google Scholar
  18. 18.
    Kent DM, Hayward RA, Griffith JL, Vijan S, Beshansky JR, Califf RM, et al. An independently derived and validated predictive model for selecting patients with myocardial infarction who are likely to benefit from tissue plasminogen activator compared with streptokinase. Am J Med. 2002;113(2):104–11.Google Scholar
  19. 19.
    •• Lipkovich I, Dmitrienko A. B. R. D’ Agostino S. Tutorial in biostatistics: data-driven subgroup identification and analysis in clinical trials. Stat Med. 2017;36(1):136–96 Excellent guide to researcher that would like to identify subgroups using dava-driven approaches. Google Scholar
  20. 20.
    Green KM, Stuart EA. Examining moderation analyses in propensity score methods: application to depression and substance use. J Consult Clin Psychol. 2014;82(5):773–83.Google Scholar
  21. 21.
    Chin AL, Bentley JP, Pollom EL. Impact of state parity laws on copayments for and adherence to oral endocrine therapy for breast cancer. Cancer. 2018;125(3):374–81.Google Scholar
  22. 22.
    Dusetzina SB, Huskamp HA, Winn AN, Basch E, Keating NL. Out-of-pocket and health care spending changes for patients using orally administered anticancer therapy after adoption of state parity laws. JAMA Oncol. 2018;4(6):e173598.Google Scholar
  23. 23.
    Heckman J, Smith JC, Clements N. Making the most out of programme evaluations and social experiments: accounting for heterogeneity in programme impacts. Rev Econ Stud. 1997;64(4):487–535.Google Scholar
  24. 24.
    Sergio F. Efficient semiparametric estimation of quantile treatment effects. Econometrica. 2007;75(1):259–76.Google Scholar
  25. 25.
    Nagin DS, Odgers CL. Group-based trajectory modeling in clinical research. Annu Rev Clin Psychol. 2010;6:109–38.Google Scholar
  26. 26.
    Austin PC, Stuart EA. Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies. Stat Med. 2015;34(28):3661–79.Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Aaron N. Winn
    • 1
    Email author
  • Matthew L. Maciejewski
    • 2
    • 3
  • Stacie B. Dusetzina
    • 4
    • 5
  1. 1.Medical College of Wisconsin, School of PharmacyMilwaukeeUSA
  2. 2.Durham Center of Innovation to Accelerate Discovery and Practice TransformationDurham Veterans Affairs Health Care SystemDurhamUSA
  3. 3.Department of Population Health SciencesDuke UniversityDurhamUSA
  4. 4.Department of Health PolicyVanderbilt University School of MedicineNashvilleUSA
  5. 5.Vanderbilt-Ingram Cancer CenterNashvilleUSA

Personalised recommendations