Journal of Experimental Criminology

, Volume 14, Issue 2, pp 141–154 | Cite as

Time series robustness checks to test the effects of the 1996 Australian firearm law on cause-specific mortality

  • Benjamin UkertEmail author
  • Elena Andreyeva
  • Charles C. Branas



Many studies utilize time series methods to identify causal effects without accounting for an underlying time trend. We show that accounting for trends changes the conclusions in the study of Chapman et al. (JAMA, 316(3), 291–299, 2016), who evaluated the impact of the Australian firearm law in 1996. We also introduce a new empirical method that tests whether their empirical strategy can actually identify a causal effect that is also useful for panel analyses.


We use national data from the Australian Bureau of Statistics, assembled in annual counts of: total firearm deaths, firearm suicides, and firearm homicides. These data are used in an independent re-analysis of the impact of the 1996 Australian firearm law that accounts for underlying stochastic trends. We then estimate a series of artificially created interruptions using interrupted times series analysis in a time frame before 1996, to test for changes in the slope of mortality across several years prior to the actual regulatory changes. This tests whether the empirical model produces effects in years other than the year of the intervention, thereby testing if the results can simply be replicated at random using other interruption years.


Controlling for stochastic trends produces less statistical evidence of the impact of the firearm law on firearm mortality than previously reported by Chapman et al. (JAMA, 316(3), 291–299, 2016). Introducing artificial interruptions in 1990 through 1995 produces statistically significant decreases in all firearm-related mortality measures well above the expected type 1 error. Overall, 19 out of the 36 artificial interruption models we tested were found to be statistically significant, suggesting that the empirical model can be implemented in multiple non-intervention years with results similar to the true 1996 interruption year.


Current evidence showing decreases in firearm mortality after the 1996 Australian national firearm law relies on an empirical model that may have limited ability to identify the true effects of the law.


Australia Firearm regulation Interrupted time series Methods 


  1. Abadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program. Journal of the American Statistical Association, 105(490), 493–505.CrossRefGoogle Scholar
  2. Andreyeva, E., & Ukert, B. (2017). Do firearm regulations work? Evidence from the Australian National Firearms Agreement.Google Scholar
  3. Bernal, J. L., Cummins, S., & Gasparrini, A. (2017). Interrupted time series regression for the evaluation of public health interventions: A tutorial. International Journal of Epidemiology, 46(1), 348–355.Google Scholar
  4. Branas, C. C., Kastanaki, A. E., Michalodimitrakis, M., Tzougas, J., Kranioti, E. F., Theodorakis, P. N., Carr, B. G., & Wiebe, D. J. (2015). The impact of economic austerity and prosperity events on suicide in Greece: A 30-year interrupted time-series analysis. British Medical Journal Open, 5(1), 1–11.Google Scholar
  5. Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasi-experimental designs for research. Skokie, IL: Rand McNally.Google Scholar
  6. Carmichael, F., & Ward, R. (2001). Male unemployment and crime in England and Wales. Economics Letters, 73(1), 111–115.CrossRefGoogle Scholar
  7. Chapman, S., Alpers, P., Agho, K., & Jones, M. (2006). Australia’s 1996 gun law reforms: Faster falls in firearm deaths, firearm suicides, and a decade without mass shootings. Injury Prevention, 12(6), 365–372.CrossRefGoogle Scholar
  8. Chapman, S., Alpers, P., & Jones, M. (2016). Association between gun law reforms and intentional firearm deaths in Australia, 1979–2013. JAMA, 316(3), 291–299.CrossRefGoogle Scholar
  9. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston, MA: Houghton Mifflin.Google Scholar
  10. D’Alessio, S. J., & Stolzenberg, L. (1995). The impact of sentencing guidelines on jail incarceration in Minnesota. Criminology, 33(2), 283–302.CrossRefGoogle Scholar
  11. Dugan, L. (2010). Estimating effects over time for single and multiple units. In A. Piquero & D. Weisburd (Eds.), Handbook of quantitative criminology (pp. 741–763). New York, NY: Springer.CrossRefGoogle Scholar
  12. Enders, W. (2014). Applied econometric time series (4th ed.). New York, NY: Wiley.Google Scholar
  13. Gagné, M., Robitaille, Y., Hamel, D., & St-Laurent, D. (2010). Firearms regulation and declining rates of male suicide in Quebec. Injury Prevention, ip-2009.Google Scholar
  14. Greene, W. H. (2003). Econometric analysis. Delhi, India: Pearson Education.Google Scholar
  15. Hansen, B. E. (2001). The new econometrics of structural change: Dating breaks in U.S. labor productivity. The Journal of Economic Perspectives, 15, 117–128.CrossRefGoogle Scholar
  16. Humphreys, D. K., Gasparrini, A., & Wiebe, D. J. (2017). Evaluating the impact of Florida’s “stand your ground” self-defense law on homicide and suicide by firearm: An interrupted time series study. JAMA Internal Medicine, 177(1), 44–50.CrossRefGoogle Scholar
  17. Klieve, H., Barnes, M., & De Leo, D. (2009). Controlling firearms use in Australia: Has the 1996 gun law reform produced the decrease in rates of suicide with this method? Social Psychiatry and Psychiatric Epidemiology, 44(4), 285.CrossRefGoogle Scholar
  18. Laliotis, I., Ioannidis, J. P., & Stavropoulou, C. (2016). Total and cause-specific mortality before and after the onset of the Greek economic crisis: An interrupted time-series analysis. The Lancet Public Health, 1(2), e56–e65.CrossRefGoogle Scholar
  19. Lee, D. Y., & Holoviak, S. J. (2006). Unemployment and crime: An empirical investigation. Applied Economics Letters, 13(12), 805–810.CrossRefGoogle Scholar
  20. Lee, W.-S., & Suardi, S. (2010). The Australian firearms buyback and its effect on gun deaths. Contemporary Economic Policy, 28(1), 65–79.CrossRefGoogle Scholar
  21. McCleary, R., McDowall, D., & Bartos, B. J. (2017). Design and analysis of time series experiments. New York, NY: Oxford University Press.CrossRefGoogle Scholar
  22. Messner, S. F., Raffalovich, L. E., & McMillan, R. (2001). Economic deprivation and changes in homicide arrest rates for white and black youths, 1967–1998: A national time-series analysis. Criminology, 39(3), 591–614.CrossRefGoogle Scholar
  23. Pridemore, W. A., Chamlin, M. B., & Cochran, J. K. (2007). An interrupted time-series analysis of Durkheim’s social deregulation thesis: The case of the Russian Federation. Justice Quarterly, 24(2), 271–290.CrossRefGoogle Scholar
  24. Pridemore, W. A., Chamlin, M. B., & Trahan, A. (2008). A test of competing hypotheses about homicide following terrorist attacks: An interrupted time series analysis of September 11 and Oklahoma City. Journal of Quantitative Criminology, 24(4), 381–396.CrossRefGoogle Scholar
  25. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. New York, NY: Houghton Mifflin.Google Scholar
  26. Spittal, M. J., Pirkis, J., Miller, M., & Studdert, D. M. (2012). Declines in the lethality of suicide attempts explain the decline in suicide deaths in Australia. PLoS One, 7(9), e44565.CrossRefGoogle Scholar
  27. Tang, C. F., & Lean, H. H. (2007). Will inflation increase crime rate? New evidence from bounds and modified Wald tests. Global Crime, 8(4), 311–323.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2017

Authors and Affiliations

  1. 1.Wharton, Leonard Davis Institute of Health Economics, and Perelman School of MedicineUniversity of PennsylvaniaPhiladelphiaUSA
  2. 2.Gelman Professor and Chair, Department of EpidemiologyColumbia University, Mailman School of Public HealthNew YorkUSA

Personalised recommendations