The Relative Incident Rate Ratio Effect Size for Count-Based Impact Evaluations: When an Odds Ratio is Not an Odds Ratio

Abstract

Area-based prevention studies often produce results that can be represented in a 2-by-2 table of counts. For example, a table may show the crime counts during a 12-month period prior to the intervention compared to a 12-month period during the intervention for a treatment and control area or areas. Studies of this type have used either Cohen’s d or the odds ratio as an effect size index. The former is unsuitable and the latter is a misnomer when used on data of this type. Based on the quasi-Poisson regression model, an incident rate ratio and relative incident rate ratio effect size and associated overdispersion parameter are developed and advocated as the preferred effect size for count-based outcomes in impact evaluations and meta-analyses of such studies.

This is a preview of subscription content, access via your institution.

References

  1. Agresti A (2002) Categorical data analysis, 2nd edn. Wiley, New York

    Google Scholar 

  2. Al-Khasawneh MF (2010) Estimating the negative binomial dispersion parameter. Asian J Math Stat 3:1–15

    Article  Google Scholar 

  3. Ariel B, Farrar WA, Sutherland A (2015) The effect of police body-worn cameras on use of force and citizens’ complaints against the police: a randomized controlled trial. J Quant Criminol 31(3):509–535

    Article  Google Scholar 

  4. Ariel B, Sutherland A, Henstock D, Young J, Drover P, Sykes J, Megicks S, Henderson R (2016) Report: Increases in police use of force in the presence of body-worn cameras are driven by officer discretion: a protocol-based subgroup analysis of ten randomized experiments. J Exp Criminol 12(3):453–463

    Article  Google Scholar 

  5. Berk R, MacDonald JM (2008) Overdispersion and Poisson regression. J Quant Criminol 24(3):269–284

    Article  Google Scholar 

  6. Braga AA, Weisburd DL (2012) The effects of “pulling levers” focused deterrence strategies on crime. Campbell Syst Rev 8(1):1–90

    Google Scholar 

  7. Braga AA, Papachristos AV, Hureau DM (2014) The effects of hot spots policing on crime: an updated systematic review and meta-analysis. Justice Q 31(4):633–663

    Article  Google Scholar 

  8. Braga AA, Sousa WH, Coldren JR Jr, Rodriguez D (2018) The effects of body-worn cameras on police activity and police–citizen encounters: randomized controlled trial. J Crim Law Criminol 108:511

    Google Scholar 

  9. Farrington DP, Welsh BC (2006) Improved street lighting. In: Farrington DP, Welsh BC (eds) Preventing crime. Springer, New York, pp 209–224

    Google Scholar 

  10. Farrington DP, Welsh B (2013) Measuring effect size in meta-analysis, with special reference to area-based crime prevention programmes and the effects of closed circuit television on crime. In: Kuhn A, Schwarzenegger C, Margot P, Donatsch A, Aebi M, Jositsch D (eds) Criminology, criminal policy and criminal law from an international perspective. Stampfli, Berne, pp 75–89

    Google Scholar 

  11. Farrington DP, Gill M, Waples SJ, Argomaniz J (2007) The effects of closed-circuit television on crime: meta-analysis of an English national quasi-experimental multi-site evaluation. J Exp Criminol 3(1):21–38

    Article  Google Scholar 

  12. Gardner W, Mulvey EP, Shaw EC (1995) Regression analyses of counts and rates: Poisson, overdispersed Poisson, and negative binomial models. Psychol Bull 118(3):392

    Article  Google Scholar 

  13. Gu K, Ng HKT, Tang ML, Schucany WR (2008) Testing the ratio of two Poisson rates. Biometr J J Math Methods Biosci 50(2):283–298

    Google Scholar 

  14. Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M, Welch V (2019) Cochrane handbook for systematic reviews of interventions (version 6.0; updated July 2019 ed.). Cochrane. www.training.cochrane.org/handbook. Accessed 22 Apr 2020

  15. Lum C, Koper C, Wilsond DB, Stoltz M, Goodier M, Eggins E, Higginson A, Mazerolle L (2020) Body-worn cameras’ effects on police officers and citizen behavior: a systematic review. Campbell Syst Rev 16(3):e1112. https://doi.org/10.1002/cl2.1112

    Article  Google Scholar 

  16. Ng HKT, Tang M-L (2005) Testing the equality of two Poisson means using the rate ratio. Stat Med 24(6):955–965

    Article  Google Scholar 

  17. Osgood DW (2000) Poisson-based regression analysis of aggregate crime rates. J Quant Criminol 16(1):21–43

    Article  Google Scholar 

  18. Rodríguez G (2013) Models for count data with overdispersion. Add. WWS 509 Notes, 672

  19. Ver Hoef JM, Boveng PL (2007) Quasi-Poisson versus negative binomial regression: how should we model overdispersed count data? Ecology 88(11):2766–2772

    Article  Google Scholar 

  20. Weisburd D, Green L (1995) Policing drug hot spots: the Jersey City drug market analysis experiment. Justice Q 12(4):711–735

    Article  Google Scholar 

  21. Welsh BC, Farrington DP (2009) Public area CCTV and crime prevention: an updated systematic review and meta-analysis. Justice Q 26(4):716–745

    Article  Google Scholar 

  22. Wooldridge JM (2010) Econometric analysis of cross section and panel data. MIT Press, Cambridge

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to David B. Wilson.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1: R Script Demonstrating that Cohen’s d is Affected by Level of Aggregation

figurea

Appendix 2: R Script Demonstrating that Eqs. (7), (9), (15), and (14) Reproduced Results from a Quasi-Poisson Model

figureb

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wilson, D.B. The Relative Incident Rate Ratio Effect Size for Count-Based Impact Evaluations: When an Odds Ratio is Not an Odds Ratio. J Quant Criminol (2021). https://doi.org/10.1007/s10940-021-09494-w

Download citation

Keywords

  • Effect size
  • Incident rate ratio
  • Poisson
  • Counts
  • Meta-analysis
  • Cohen’s d
  • Odds ratio