Advertisement

Meta-analysis

  • David B. Wilson
Chapter

Abstract

Meta-analysis is a statistical method of synthesizing quantitative results across studies examining a common research question. This chapter discusses the logic and methods of meta-analysis with specific application to the field of criminology and criminal justice. A key feature of meta-analysis is the effect size, which encodes the study findings on a common index, such as the standardized mean difference, correlation coefficient, or odds-ratio. Analysis of these effect sizes considers not only the central tendency of effects across studies but also the relationship of study features to variability in effects. Both fixed- and random-effects models are discussed, as are the important issue of publication selection bias. Meta-analysis applies social science methodology and statistical methods to the task of taking stock of the evidence in an area, providing a robust foundation for future research and theorizing.

Keywords

Criminal Justice Ordinary Little Square Regression Unstandardized Regression Coefficient Multivariate Research Schmidt Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. Becker BJ (2009) Model-based meta-analysis, 2nd edn, Russell Sage Foundation, New YorkGoogle Scholar
  2. Berk R (2007) Statistical inference and meta-analysis. J Exp Criminol 3:247–270, doi10.1007/s11292-007-9036-y, URL http://dx.doi.org/10.1007/s11292-007-9036-y Google Scholar
  3. Borenstein M (2009) Effect sizes for studies with continuous outcome data, 2nd edn. Russell Sage Foundation, New YorkGoogle Scholar
  4. Burnett WL (1996) Treating postincarcerated offenders with Moral Reconation TherapyTM: A one-year recidivism study. PhD thesis, University of PhoenixGoogle Scholar
  5. Bushway SD, Sweeten G, Wilson DB (2006) Size matters: standard errors in the application of null hypothesis significance testing in criminology and criminal justice. J Exp Criminol 2:1–22, doi: 10.1007/s11292-005-5129-7, URL http://dx.doi.org/10.1007/s11292-005-5129-7
  6. Cheung MWL, Chan W (2005) Meta-analytic structural equation modeling: A two-stage approach. Psychol Methods 10:40–64. doi: 10.1037/1082-989X.10.1.40, URL http://search.ebscohost.com/login.aspx?direct=true/&db=pdh/&AN=met-10-1-40/&site=ehost-live Google Scholar
  7. Cohen J, Cohen P (1983) Applied multiple regression/correlation analysis for the behavioral sciences, 2nd edn. L. Erlbaum Associates, Hillsdale, NJGoogle Scholar
  8. Cooper HM (1998) Synthesizing research: A guide for literature reviews. Sage, Thousand Oaks, CAGoogle Scholar
  9. Cooper HM, DeNeve K, Charlton K (1997) Finding the missing science: The fate of studies submitted for review by a human subjects committee. Psychol Methods 2:447–452. doi: 10.1037/1082-989X.2.4.447, URL http://search.ebscohost.com/login.aspx?direct=true/&db=pdh/&AN=met-2-4-447/&site=ehost-live
  10. Cox DR (1970) The analysis of binary data. Methuen, LondonGoogle Scholar
  11. DerSimonian R, Laird N (1986) Meta-analysis in clinical trials. Control Clin Trials 7:177–88. doi: 3802833, URL http://www.ncbi.nlm.nih.gov/pubmed/3802833 Google Scholar
  12. Dickersin K (2005) Publication bias: recognizing the problem, understanding its origins, and preventing harm. Wiley, Chichester, pp 11–33Google Scholar
  13. Egger M, Smith GD, Schneider M, Minder C (1997) Bias in meta-analysis detected by a simple, graphical test. BMJ 315:629–634, URL http://www.bmj.com/cgi/content/abstract/315/7109/629
  14. Field AP (2001) Meta-analysis of correlation coefficients: A monte carlo comparison of fixed- and random-effects methods. Psychol Methods 6:161–180. doi: 10.1037/1082-989X.6.2.161, URL http://search.ebscohost.com/login.aspx?direct=true/&db=pdh/&AN=met-6-2-161/&site=ehost-live
  15. Fleiss JL (1994) Measures of effect size for categorical data. Russell Sage Foundation, New York, pp 245–260Google Scholar
  16. Fleiss JL, Berlin JA (2009) Measures of effect size for categorical data, 2nd edn. Russell Sage Foundation, New YorkGoogle Scholar
  17. Friedman L (2000) Estimators of random effects variance components in meta-analysis. J Educ Behav Stat 25:1–12. doi: 10.3102/10769986025001001, URL http://jeb.sagepub.com/cgi/content/abstract/25/1/1 Google Scholar
  18. Frost JJ, Forrest JD (1995) Understanding the impact of effective teenage pregnancy prevention programs. Fam Plann Perspect 27:188–195, URL http://mutex.gmu.edu:2112/stable/2136274
  19. Furlow CF, Beretvas SN (2005) Meta-analytic methods of pooling correlation matrices for structural equation modeling under different patterns of missing data. Psychol Methods 10:227–254, doi: 10.1037/1082-989X.10.2.227, URL http://search.ebscohost.com/login.aspx?direct=true/&db=pdh/&AN=met-10-2-227/&site=ehost-live
  20. Gerber AS, Malhotra N (2008) Publication bias in empirical sociological research: Do arbitrary significance levels distort published results? Sociol Methods Res 37:3–30, doi: 10.1177/0049124108318973, URL URL http://smr.sagepub.com/cgi/content/abstract/37/1/3 Google Scholar
  21. Glass GV (1976) Primary, secondary, and meta-analysis research. Educ Res 5:3–8Google Scholar
  22. Gleser LJ, Olkin I (1994) Stochastically dependent effect sizes. Russell Sage Foundation, New York, pp 339–356Google Scholar
  23. Hasselblad V, Hedges (1995) Meta-analysis of screening and diagnostic tests. Psychol Bull 117:167–178. doi: 10.1037/0033-2909.117.1.167, URL http://search.ebscohost.com/login.aspx?direct=true/&db=pdh/&AN=bul-117-1-167/&site=ehost-live
  24. Hedges LV (1981) Distribution theory for Glass’s estimator of effect size and related estimators. J Educ Behav Stat 6:107–128, doi: 10.3102/10769986006002107, URL http://jeb.sagepub.com/cgi/content/abstract/6/2/107
  25. Hedges LV, Olkin I (1985) Statistical methods for meta-analysis. Academic, Orlando, FLGoogle Scholar
  26. Higgins JPT, Thompson SG, Deeks JJ, Altman DG (2003) Measuring inconsistency in meta-analyses. BMJ 327:557–560. doi: 10.1136/bmj.327.7414.557, URL http://www.bmj.com Google Scholar
  27. Hunt MM (1997) How science takes stock: The story of meta-analysis. Russell Sage Foundation, New YorkGoogle Scholar
  28. Hunter JE, Schmidt FL (1990) Methods of meta-analysis: Correcting error and bias in research findings. Sage Publications, Newbury ParkGoogle Scholar
  29. Hunter JE, Schmidt FL (2004) Methods of meta-analysis: Correcting error and bias in research findings, 2nd edn. Sage, Thousand Oaks, CAGoogle Scholar
  30. Johnson G, Hunter RM (1995) Evaluation of the specialized drug offender program. In: Ross RR, Ross B (eds) Thinking straight. Cognitive Center, Ottawa, ON, pp 215–234Google Scholar
  31. Kalaian HA, Raudenbush SW (1996) A multivariate mixed linear model for meta-analysis. Psychol Methods 1:227–235. doi: 10.1037/1082-989X.1.3.227, URL http://search.ebscohost.com/login.aspx?direct=true/&db=pdh/&AN=met-1-3-227/&site=ehost-live
  32. Kuhns JB, Wilson DB, Maguire ER, Ainsworth SA, Clodfelter TA (2008) A meta-analysis of marijuana, cocaine, and opiate toxicology study findings among homicide victims. Addiction 104:1122–1131CrossRefGoogle Scholar
  33. Lipsey MW (1995) What do we learn from 400 research studies on the effectiveness of treatment with juvenile delinquents? Wiley, New York, pp 63–78Google Scholar
  34. Lipsey MW (2007) Unjustified inferences about meta-analysis. J Exp Criminol 3:271–279. doi: 10.1007/s11292-007-9037-x, URL http://dx.doi.org/10.1007/s11292-007-9037-x Google Scholar
  35. Lipsey MW, Cullen FT (2007) The effectiveness of correctional rehabilitation: A review of systematic reviews. Annu Rev Law Soc Sci 3:297–320, doi: 10.1146/annurev.lawsocsci.3.081806.112833, URL http://mutex.gmu.edu:2078/doi/full/10.1146/annurev.lawsocsci.%3.081806.112833 Google Scholar
  36. Lipsey MW, Derzon JH (1998) Predictors of violent or serious delinquency in adolescence and early adulthood: A synthesis of longitudinal research. Sage Publications, Thousand Oaks, CA, pp 86–105Google Scholar
  37. Lipsey MW, Wilson DB (1993) The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. Am Psychol 48:1181–1209. doi: 10.1037/0003-066X.48.12.1181, URL http://search.ebscohost.com/login.aspx?direct=true/&db=pdh/&AN=amp-48-12-1181/&site=ehost-live Google Scholar
  38. Lipsey MW, Wilson DB (1998) Effective intervention for serious juvenile offenders: A synthesis of research. Sage Publications, Thousand Oaks, CA, pp 313–345Google Scholar
  39. Lipsey MW, Wilson DB (2001a) Practical meta-analysis. Applied social research methods series. Sage Publications, Thousand Oaks, CAGoogle Scholar
  40. Lipsey MW, Wilson DB (2001b) The way in which intervention studies have “personality” and why it is important to meta-analysis. Eval Health Prof 24:236–254. doi: 10.1177/016327870102400302, URL http://ehp.sagepub.com/cgi/content/abstract/24/3/236
  41. Lipsey MW, Crosse S, Dunkle J, Pollard J, Stobart G (1985) Evaluation: The state of the art and the sorry state of the science. New Dir Program Eval 27:7–28CrossRefGoogle Scholar
  42. Little GL and Robinson KD (1989) Treating drunk drivers with Moral Reconation therapy: A one-year recidivism report. Psychol Rep 64:960-962Google Scholar
  43. Little GL, Robinson KD Burnette KD (1991) Treating drug offenders with Moral Reconation therapy: A three-year recidivism report. Psychol Rep 69:1151–1154Google Scholar
  44. Little GL, Robinson KD, Burnette KD (1994) Treating offenders with cognitive-behavioral therapy: 5-year recidivism outcome data on MRT. Cogn Behav Treat Rev 3:1-3Google Scholar
  45. Overton RC (1998) A comparison of fixed-effects and mixed (random-effects) models for meta-analysis tests of moderator variable effects. Psychol Methods 3:354–379. doi: 10.1037/1082-989X.3.3.354, URL http://search.ebscohost.com/login.aspx?direct=true/&db=pdh/&AN=met-3-3-354/&site=ehost-live Google Scholar
  46. Porporino FJ, Robinson D (1995) An evaluation of the Reasoning and Rehabilitation program with Canadian federal offenders. In: Ross RR, Ross B (eds) Thinking straight. Cognitive Centre, Ottawa, ON, pp 155–191Google Scholar
  47. Porporino FJ , Fabiano EA, Robinson D (1991) Focusing on successful reintegration: Cognitive skills training for offenders, r19. Ottawa, Canada: Research and Statistics Branch, The Correctional Service of CanadaGoogle Scholar
  48. Pratt TC, Cullen FT (2000) The empirical status of Gottfredson and Hirschi’s General Theory of crime: A meta-analysis. Criminology 38:931–964, doi: 10.1111/j.1745-9125.2000.tb00911.x, URL http://mutex.gmu.edu:2167/doi/abs/10.1111/j.1745-9125.2000.tb%00911.x
  49. Raudenbush SW (1994) Random effects models. Russell Sage Foundation, New York, pp 301–322Google Scholar
  50. Raudenbush SW (2009) Statistically analyzing effect sizes: Random effects models, 2nd edn. Russell Sage Foundation, New YorkGoogle Scholar
  51. Robinson D (1995) The impact of cognitive skills training on postrelease recidivism among Canadian federal offenders. Correctional Research and Development, The Correctional Service of Canada, Ottawa, ONGoogle Scholar
  52. Rosenthal R (1991) Meta-analytic procedures for social research, Rev. ed edn. Applied social research methods series. Sage Publications, Newbury ParkGoogle Scholar
  53. Ross RR, Fabiano EA, Ewles CD (1988) Reasoning and Rehabilitation. Int J Offender Ther Comp Criminol 32:29-36CrossRefGoogle Scholar
  54. Schmidt F, Le H, Oh IS (2009) Correcting for the distorting effects of study artifacts in meta-analysis, 2nd edn. Russell Sage Foundation, New YorkGoogle Scholar
  55. Schulze R (2004) Meta-analysis: A comparison of approaches. Hogrefe & Huber, TorontoGoogle Scholar
  56. Shadish WR (2007) A world without meta-analysis. J Exp Criminol 3:281–291. doi: 10.1007/s11292-007-9034-0, URL http://dx.doi.org/10.1007/s11292-007-9034-0 Google Scholar
  57. Shadish WR, Haddock CK (2009) Combining estimates of effect size, 2nd edn. Russell Sage Foundation, New YorkGoogle Scholar
  58. Sánchez-Meca J, Marín-Martínez F, Chacón-Moscoso S (2003) Effect-size indices for dichotomized outcomes in meta-analysis. Psychol Methods 8:448–467, URL jsmeca@um.es Google Scholar
  59. Stern JM, Simes RJ (1997) Publication bias: Evidence of delayed publication in a cohort study of clinical research projects. BMJ 315:640–645Google Scholar
  60. Sterne JAC, Becker BJ, Egger M (2005) The funnel plot. Wiley, Chichester, pp 75–99Google Scholar
  61. Viechtbauer W (2005) Bias and efficiency of meta-analytic variance estimators in the random-effects model. J Educ Behav Stat 30:261–293. doi: 10.3102/10769986030003261, URL http://jeb.sagepub.com/cgi/content/abstract/30/3/261 Google Scholar
  62. Weisburd D, Lum CM, Yang SM (2003) When can we conclude that treatments or programs “don’t work”? Ann Am Acad Pol Soc Sci 587:31–48, doi: 10.1177/0002716202250782, URL http://ann.sagepub.com/cgi/content/abstract/587/1/31
  63. Wilson DB (2009) Missing a critical piece of the pie: Simple document search strategies inadequate for systematic reviews. J Exp Criminol. doi: 10.1007/s11292-009-9085-5Google Scholar
  64. Wilson DB, Bouffard LA, Mackenzie DL (2005) A quantitative review of structured, group-oriented, cognitive-behavioral programs for offenders. Crim Justice Behav 32:172–204. doi: 10.1177/0093854804272889, URL http://cjb.sagepub.com/cgi/content/abstract/32/2/172

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • David B. Wilson
    • 1
  1. 1.Administration of Justice DepartmentGeorge Mason UniversityManassasUSA

Personalised recommendations