Advertisement

Step 3 of EBP: Part 3—Meta-analysis and Systematic Reviews, Aggregating Research Results

  • James W. Drisko
  • Melissa D. Grady
Chapter
Part of the Essential Clinical Social Work Series book series (ECSWS)

Abstract

Step 3 of the EBP process centers on evaluating the quality and relevance of research results. This chapter examines the aggregation and evaluation of multiple research studies using a method called the systematic review. Systematic reviews locate and combine the results of multiple studies using clearly defined and transparently reported standards. The statistical results of studies are aggregated and compared using a set of techniques called meta-analysis. As terminology has evolved in this area, the shared and distinct features of systematic reviews and meta-analyses are carefully defined and distinguished. The standards for Cochrane Collaboration quality reviews are identified. A Cochrane systematic review is analyzed in detail to show the strength and limitations of such publications.

References

  1. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.) (DSM-5). Washington, DC: Author.Google Scholar
  2. Amos, T., Stein, D. J., & Ipser, J. (2014). Pharmacological interventions for preventing post-traumatic stress disorder (PTSD). Cochrane Database of Systematic Reviews, 2014(7), CD006239.  https://doi.org/10.1002/14651858.CD006239.pub2 CrossRefGoogle Scholar
  3. Bisson, J., Roberts, N., Andrew, M., Cooper, R., & Lewis, C. (2013). Psychological therapies for chronic post-traumatic stress disorder (PTSD) in adults. Cochrane Database of Systematic Reviews, 2013(12), CD003388.  https://doi.org/10.1002/14651858.CD003388.pub4 CrossRefGoogle Scholar
  4. Carey, B. (2008, January 17). Antidepressant studies unpublished. The New York Times. Retrieved from https://www.nytimes.com/2008/01/17/health/17depress.html
  5. Carroll, H., Toumpakari, Z., Johnson, L., & Betts, J. (2017). The perceived feasibility of methods to reduce publication bias. PLoS One, 12(10), e0186472.  https://doi.org/10.1371/journal.pone.0186472 CrossRefPubMedPubMedCentralGoogle Scholar
  6. Chester, R., Smith, T., Sweeting, D., Dixon, J., Wood, S., & Song, F. (2008). The relative timing of VMO and VL in the aetiology of anterior knee pain: a systematic review and meta-analysis. British Medical College Musculoskeletal Disorders, 9(1), 64.  https://doi.org/10.1186/1471-2474-9-64 CrossRefGoogle Scholar
  7. Chittaranjan, A. (2015). Understanding relative risk, odds ratio, and related terms: As simple as it can get. The Journal of Clinical Psychiatry, 76(7), e857–ee86.CrossRefGoogle Scholar
  8. Christensen, P., & Kristiansen, I. (2006). Number-needed-to-treat (NNT): Needs treatment with care. Basic & Clinical Pharmacology & Toxicology, 99(1), 12–16.CrossRefGoogle Scholar
  9. Cochrane Methods (Group): Qualitative and implementation. (2017). Core library of qualitative synthesis methodology [online bibliography]. Retrieved from: http://methods.cochrane.org/qi/core-library-qualitative-synthesis-methodology
  10. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York: Routledge Academic.Google Scholar
  11. Deng, C.Q. (2012, February 12). How to interpret odds ratios that are smaller than 1? Retrieved from https://onbiostatistics.blogspot.com/2012/02/how-to-interpret-odds-ratios-that-are.html
  12. Dickersin, K. (1990). The existence of publication bias and risk factors for its occurrence. Journal of the American Medical Association, 263, 1385–1389.CrossRefGoogle Scholar
  13. Dixon-Woods, M., Bonas, S., Booth, A., Jones, D., Miller, T., Sutton, J., et al. (2006). How can systematic reviews incorporate qualitative research? A critical perspective. Qualitative Research, 6(1), 27–44.CrossRefGoogle Scholar
  14. Dixon-Woods, M., Cavers, D., Agarwal, S., Annandale, E., Arthur, A., Harvey, J., et al. (2006). Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Medical Research Methodology, 6. (electronic journal). Retrieved from http://www.biomedcentral.com/1471-2288/6/3
  15. Drisko, J., & Simmons, B. (2012). The evidence base for psychodynamic psychotherapy. Smith College Studies in Social Work, 82(4), 374–400.CrossRefGoogle Scholar
  16. Ellis, P. (2010). The essential guide to effect sizes: An introduction to statistical power, meta-analysis and the interpretation of research results. New York: Cambridge University Press.CrossRefGoogle Scholar
  17. European Patients’ Academy. (2015). Statistics in clinical trials: Key concepts. Retrieved from https://www.eupati.eu/clinical-development-and-trials/statistics-clinical-trials-key-concepts/
  18. Gillies, D., Taylor, F., Gray, C., O’Brien, L., & D’Abrew, N. (2012). Psychological therapies for the treatment of post-traumatic stress disorder in children and adolescents. Cochrane Database of Systematic Reviews, 2012(12), CD006726.  https://doi.org/10.1002/14651858.CD006726.pub2 CrossRefGoogle Scholar
  19. Higgins J., & Green, S. (Eds.). (2011). Cochrane handbook for systematic reviews of interventions, version 5.1. Available from https://training.cochrane.org/handbook
  20. Hopewell, S., Loudon, K., Clarke, M. J., Oxman, A., & Dickersin, K. (2009). Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database of Systematic Reviews, 2009(1), MR000006.  https://doi.org/10.1002/14651858.MR000006.pub3 CrossRefGoogle Scholar
  21. Lenth, R. (2008). Java applets for power and sample size. Retrieved from http://www.stat.uiowa.edu/~rlenth/Power/
  22. Litschge, C., Vaughn, M., & McCrea, C. (2010). The empirical status of treatments for children and youth with conduct problems: An overview of meta-analytic studies. Research on Social Work Practice, 20(1), 21–35.CrossRefGoogle Scholar
  23. Littell, J., Corcoran, J., & Pillai, V. (2008). Systematic reviews and meta-analysis. New York: Oxford University Press.CrossRefGoogle Scholar
  24. McGauran, N., Wiesler, B., Kreis, J., Schüler, Y.-B., Kölsch, H., & Kaiser, T. (2010). Reporting bias in medical research: A narrative review. Trials, 11, 37.CrossRefGoogle Scholar
  25. McQuay, H., & Moore, A. (1997). Using numerical results from systematic reviews in clinical practice. Annals of Internal Medicine, 126, 712–720. Retrieved from http://www.medicine.ox.ac.uk/bandolier/booth/painpag/NNTstuff/numeric.htm
  26. Mendes, D., Alves, C., & Batel-Marques, F. (2017). Number needed to treat (NNT) in clinical literature: An appraisal. BMC Medicine, 15, 112. Retrieved from  https://doi.org/10.1186/s12916-017-0875-8 CrossRefGoogle Scholar
  27. Moher, D., Cook, D., Eastwood, S., Olkin, I., Rennie, D., & Stroup, F. (1999). Improving the quality of reports of meta-analyses of randomized controlled trails: The Quorom group statement. Lancet, 354(9193), 1896–1900.CrossRefGoogle Scholar
  28. Noblit, G., & Hare, R. (1988). Meta-ethnography: Synthesizing qualitative studies. Newbury Park, CA: Sage.CrossRefGoogle Scholar
  29. O’Brien, S., & Yi, Q. L. (2016). How do I interpret a confidence interval? Transfusion, 56(7), 1680–1683.CrossRefGoogle Scholar
  30. Oxford University Centre for Evidence-based Medicine. (2016, May). The Oxford levels of evidence 2.1. Retrieved from https://www.cebm.net/2016/05/ocebm-levels-of-evidence/
  31. Pratt, J., Rhine, J., Smith, B., Stuart, C., & Greenwood, J. (1940). Extra-sensory perception after sixty years. New York: Henry Holt.CrossRefGoogle Scholar
  32. Rose, S., Bisson, J., Churchill, R., & Wessely, S. (2002). Psychological debriefing for preventing post traumatic stress disorder (PTSD). Cochrane Database of Systematic Reviews, 2002(2), CD000560.  https://doi.org/10.1002/14651858.CD000560 CrossRefGoogle Scholar
  33. Sandelowski, M., & Barroso, J. (2007). Handbook for synthesizing qualitative research. New York: Springer Publishing.Google Scholar
  34. Scargle, J. (2000). Publication bias: The “file-drawer problem” in scientific inference. Journal of Scientific Exploration, 14(2), 94–106.Google Scholar
  35. Shea, B., Grimshaw, J., Wells, G., Boers, M., Andersson, N., Hamel, C., et al. (2007). Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews. BMC Medical Research Methodology, 7, 10.  https://doi.org/10.1186/1471-2288-7-10 CrossRefPubMedPubMedCentralGoogle Scholar
  36. Sistrom, C., & Garvan, C. (2004). Proportions, odds, and risk. Radiology, 230(1), 12–19.  https://doi.org/10.1148/radiol.2301031028 CrossRefPubMedGoogle Scholar
  37. Smith, M., Glass, G., & Miller, T. (1980). The benefits of psychotherapy. Baltimore, MD: Johns Hopkins University Press.Google Scholar
  38. Sridharan, L., & Greenland, P. (2009). Editorial policies and publication bias: The importance of negative studies (editorial commentary). Archives of Internal Medicine, 169, 1022–1023.CrossRefGoogle Scholar
  39. Szumilas, M. (2010). Explaining odds ratios. Journal of the Canadian Academy of Child and Adolescent Psychiatry, 19(3), 227–229.PubMedPubMedCentralGoogle Scholar
  40. Thombs, B., & Jewett, L. (2009). Letter: Analyzing effectiveness of long-term psychodynamic psychotherapy. Journal of the American Medical Association, 301(9), 930.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • James W. Drisko
    • 1
  • Melissa D. Grady
    • 2
  1. 1.School for Social WorkSmith CollegeNorthamptonUSA
  2. 2.School of Social ServiceCatholic University of AmericaWashington, DCUSA

Personalised recommendations