Abstract
Background
Propensity score methods are techniques commonly employed in observational research to account for confounding when estimating the effects of treatments and exposures. These methods have been increasingly employed in the acute care surgery literature in an attempt to infer causality; however, the adequacy of reporting and the appropriateness of statistical analyses when using propensity score matching remain unclear.
Objectives
The goal of this systematic review is to assess the adequacy of reporting of propensity score methods, with an emphasis on propensity score matching (to assess balance and the use of appropriate statistical tests), in acute care surgery (ACS) studies and to provide suggestions for improvement for junior investigators.
Methods
We searched three databases, and other relevant literature (from January 2005 to June 2015) to identify observational studies within the ACS literature using propensity score methods (PROSPERO No: CRD42016036432). Two reviewers extracted data and assessed the quality of the studies retrieved by reviewing the adequacy of both overall reporting and of the propensity score matching methods used.
Results
A total of 49/71 (69%) of studies adequately reported propensity score methods overall. Matching was the most common propensity score method used in 46/71 (65%) studies, with 36/46 (78%) studies reporting matching methods adequately. Only 19/46 (41%) of matching studies reported the balance of baseline characteristics between treated and untreated subjects while 6/46 (13%) used correct statistical methods to assess balance. There were 35/46 (76%) of matching studies that explicitly used statistical methods appropriate for the analysis of matched data when estimating the treatment effect and its statistical significance.
Conclusion
We have proposed reporting guidelines for the use of propensity score methods in the acute care surgery literature. This is to help investigators improve the adequacy of reporting and statistical analyses when using observational data to estimate effects of treatments and exposures.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivar Behav Res. 2011;46(3):399–424.
McMurry TL, Hu Y, Blackstone EH, Kozower BD. Propensity scores: methods, considerations, and applications in the. Journal of Thoracic Cardiovascular Surgery. J Thorac Cardiovasc Surg. 2015;150(1):14–9.
Luo Z, Gardiner JC, Bradley CJ. Applying propensity score methods in medical research: pitfalls and prospects. Med Care Res Rev. 2010;67(5):528–54.
Austin PC. The relative ability of different propensity score methods to balance measured covariates between treated and untreated subjects in observational studies. Med Decis Mak. 2009;29(6):661–77.
Austin PC. A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003. Stat Med. 2008;27(12):2037–49.
Austin PC. Primer on statistical interpretation or methods report card on propensity-score matching in the cardiology literature from 2004 to 2006: a systematic review. Circ Cardiovasc Qual Outcomes. 2008;1(1):62–7.
Austin PC. Propensity-score matching in the cardiovascular surgery literature from 2004 to 2006: a systematic review and suggestions for improvement. J Thorac Cardiovasc Surg. 2007;134(5):1128–35.
Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions. New York: Wiley; 2011.
Panic N, Leoncini E, de Belvis G, Ricciardi W, Boccia S. Evaluation of the endorsement of the preferred reporting items for systematic reviews and meta-analysis (PRISMA) statement on the quality of published systematic review and meta-analyses. PLoS One. 2013;8(12):e83138.
Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.
Gayat E, Pirracchio R, Resche-Rigon M, Mebazaa A, Mary JY, Porcher R. Propensity scores in intensive care and anaesthesiology literature: a systematic review. Intensive Care Med. 2010;36(12):1993–2003.
Inoue J, Shiraishi A, Yoshiyuki A, Haruta K, Matsui H, Otomo Y. Resuscitative endovascular balloon occlusion of the aorta might be dangerous in patients with severe torso trauma: a propensity score analysis. J Trauma Acute Care Surg. 2016;80(4):559–67.
Austin PC, Grootendorst P, Anderson GM. A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study. Stat Med. 2007;26(4):734–53.
Austin PC. Optimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studies. Pharm Stat. 2011;10(2):150–61.
Austin PC. A comparison of 12 algorithms for matching on the propensity score. Stat Med. 2014;33(6):1057–69.
Austin PC. Some methods of propensity-score matching had superior performance to others: results of an empirical investigation and Monte Carlo simulations. Biom J. 2009;51(1):171–84.
Austin PC. The performance of different propensity score methods for estimating marginal odds ratios. Stat Med. 2007;26(16):3078–94.
Austin PC. The use of propensity score methods with survival or time-to-event outcomes: reporting measures of effect similar to those used in randomized experiments. Stat Med. 2014;33(7):1242–58.
Morgan SL, Todd JL. A diagnostic routine for the detection of consequential heterogeneity of causal effects. Sociol Methodol. 2008;38:231–81.
Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Stat Med. 2009;28(25):3083–107.
Mitra R, Reiter JP. A comparison of two methods of estimating propensity scores after multiple imputation. Stat Methods Med Res. 2016;25(1):188–204.
Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41–55.
Rassen JA, Shelat AA, Myers J, Glynn RJ, Rothman KJ, Schneeweiss S. One-to-many propensity score matching in cohort studies. Pharmacoepidemiol Drug Saf. 2012;21(Suppl 2):69–80.
Austin PC, Mamdani MM, Stukel TA, Anderson GM, Tu JV. The use of the propensity score for estimating treatment effects: administrative versus clinical data. Stat Med. 2005;24(10):1563–78.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Victoria McCredie, Peter Austin and Tanya L. Zakrison declare that they have no conflict of interest.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Zakrison, T.L., Austin, P.C. & McCredie, V.A. A systematic review of propensity score methods in the acute care surgery literature: avoiding the pitfalls and proposing a set of reporting guidelines. Eur J Trauma Emerg Surg 44, 385–395 (2018). https://doi.org/10.1007/s00068-017-0786-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00068-017-0786-6