Systematic Review and Meta-analysis in Clinical Practice

  • Srdjan Saso
  • Sukhmeet S. Panesar
  • Weiming Siow
  • Thanos Athanasiou


Systematic review and meta-analysis provide a means to comprehensively analyse and objectively summarise and synthesize primary research. Prior to commencing, it is important to frame a specific question in a systematic review. Although such a review is ideal in many situations, it might not be possible to perform a meta-analysis due to the heterogeneity of summary statistics or nature of study design. Furthermore, it needs to be understood that performing a meta-analysis is a time-consuming, ordered process that needs to be well-planned in order to yield valid results.

The quality of the individual studies incorporated into the meta-analysis must be assessed. In addition, for clinicians interpreting a meta-analysis, its quality must also be assessed. The inclusion of negative trials and small studies might involve the inclusion of studies with suboptimal methodological quality leading to the inclusion of bias inherent in individual studies into the meta-analysis. As ‘garbage-in’ transliterates to ‘garbage-out’, this would lead to aberrant results. Publication bias and other forms of bias must be expected and accounted for via the utilisation of appropriate review methodology and statistical compensation in order to ensure the inclusion of the whole gamut of positive and negative trials available in a field of study. Meta-analysis in surgery warrant special attention as a greater degree of heterogeneity is expected when compared to meta-analysis of medical treatments. Generally, we note that the meta-analytic technique has limitations and detail should be paid to these when basing clinical decisions on the results of a meta-analysis. Nevertheless, well-conducted meta-analyses have the ability to inform and alter clinical practice.


Systematic Review Publication Bias Methodological Quality Individual Patient Data Narrative Review 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1.  1.
    Panesar SS, Thakrar R, Athanasiou T, Sheikh A. Comparison of reports of RCTs and systematic reviews in surgical journals: literature review. J R Soc Med. 2006;99(9):470-472.CrossRefPubMedGoogle Scholar
  2. 2.
    Royal College of Surgeons of England. Implementing the EWTD: College response. Available online at Last accessed on April 1, 2009.
  3. 3.
    Davidoff F, Haynes B, Sackett D, Smith R. Evidence based medicine: a new journal to help doctors identify the information they need. BMJ. 1995;310:1085-1086.PubMedGoogle Scholar
  4. 4.
    Williams CJ. The pitfalls of narrative reviews in clinical medicine. Ann Oncol. 1998;9(6):601-605.CrossRefPubMedGoogle Scholar
  5. 5.
    Sauerland S, Seiler CM. Role of systematic reviews and meta-analysis in evidence-based medicine. World J Surg. 2005;29(5):582-587.CrossRefPubMedGoogle Scholar
  6. 6.
    Glass GV. Primary, secondary and meta-analysis of research. Educ Res. 1976;5:3-8.Google Scholar
  7. 7.
    Ng TT, McGory ML, Ko CY, et al. Meta-analysis in surgery: methods and limitations. Arch Surg. 2006;141(11):1125-1130.CrossRefPubMedGoogle Scholar
  8. 8.
    Berman NG, Parker RA. Meta-analysis: neither quick nor easy. BMC Med Res Methodol. 2002;9(2):10.CrossRefGoogle Scholar
  9. 9.
    Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis of observational studies in epidemiology (MOOSE) group. JAMA. 2000;283(15):2008-2012.CrossRefPubMedGoogle Scholar
  10. 10.
    Thompson SG, Pocock SJ. Can meta-analyses be trusted? Lancet. 1991;338(8775):1127-1130.CrossRefPubMedGoogle Scholar
  11. 11.
    Olkin I. Meta-analysis: reconciling the results of independent studies. Stat Med. 1995;14(5–7):457-472.CrossRefPubMedGoogle Scholar
  12. 12.
    Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Control Clin Trials. 1996;17(1):1-12.CrossRefPubMedGoogle Scholar
  13. 13.
    Moher D, Jadad AR, Nichol G, et al. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials. 1995;16(1):62-73.CrossRefPubMedGoogle Scholar
  14. 14.
    The Cochrane Collaboration. Measures of relative effect: the risk ratio and odds ratio. In: The Cochrane Handbook for Review Manager 5. p. 111-113.Google Scholar
  15. 15.
    Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med. 1998;318(26):1728-1733.CrossRefGoogle Scholar
  16. 16.
    Berlin JA, Laird NM, Sacks HS, et al. A comparison of statistical methods for combining event rates from clinical trials. Stat Med. 1989;8(2):141-151.CrossRefPubMedGoogle Scholar
  17. 17.
    DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177-188.CrossRefPubMedGoogle Scholar
  18. 18.
    The Cochrane Collaboration. Diversity and heterogeneity: Identifying statistical heterogeneity. The Cochrane Collaboration open learning material 2002 cited; Available from:
  19. 19.
    Mantel N, Hanezsel W. Statistical aspects of the analysis of data from retrospective studies of disease. J Natl Cancer Inst. 1959;22(4):719-748.PubMedGoogle Scholar
  20. 20.
    Greenland S, Robins JM. Estimation of a common effect parameter from sparse follow-up data. Biometrics. 1985;41(1):55-68.CrossRefPubMedGoogle Scholar
  21. 21.
    Greenland S. Randomization, statistics, and causal inference. Epidemiology. 1990;1(6):421-429.CrossRefPubMedGoogle Scholar
  22. 22.
    Fleiss JL. The statistical basis of meta-analysis. Stat Meth Med Res. 1993;2(2):121-145.CrossRefGoogle Scholar
  23. 23.
    Sankey SS et al. An assessment of the use of the continuity correction for sparse data in meta-analysis. Commun Stat Simul Comput. 1996;25(4):1031-1056.CrossRefGoogle Scholar
  24. 24.
    Bailey KR. Inter-study differences: how should they influence the interpretation and analysis of results? Stat Med. 1987;6(3):351-360.CrossRefPubMedGoogle Scholar
  25. 25.
    Huedo-Medina TB, Sanchez-Meca J, Marin-Martinez F, et al. Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychol Meth. 2006;11:193-206.CrossRefGoogle Scholar
  26. 26.
    Higgins JP, Thompson SG, Deeks JJ, et al. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557-560.CrossRefPubMedGoogle Scholar
  27. 27.
    Egger M, Smith GD, Phillips AN. Meta-analysis: principles and procedures. BMJ. 1997;315(7121):1533-1537.PubMedGoogle Scholar
  28. 28.
    Easterbrook PJ, Berlin JA, Gopalan R, et al. Publication bias in clinical research. Lancet. 1991;337(8746):867-872.CrossRefPubMedGoogle Scholar
  29. 29.
    Davey Smith G, Egger M, et al. Meta-analysis. Beyond the grand mean? BMJ. 1997;315(7122):1610-1614.PubMedGoogle Scholar
  30. 30.
    Efron B, Morris C. Stein’s paradox in statistics. Sci Am. 1977;236:119-127.CrossRefGoogle Scholar
  31. 31.
    Egger M, Smith GD, Schneider M. Systematic reviews of observational studies. In: Egger M, Smith GD, Altman D, eds. Systematic Reviews in Healthcare. London: British Medical Association; 2001.CrossRefGoogle Scholar
  32. 32.
    Krahn J, Sauerland S, Rixen D, Gregor S, Bouillon B, Neugebauer EA. Applying evidence-based surgery in daily clinical routine: a feasibility study. Arch Orthop Trauma Surg. 2006;126(2):88-92.CrossRefPubMedGoogle Scholar
  33. 33.
    Ramsay CR, Grant AM, Wallace SA, et al. Statistical assessment of the learning curves of health technologies. Health Technol Assess. 2001;5(12):1-79.PubMedGoogle Scholar
  34. 34.
    Freemantle N, Cleland J, Young P, et al. beta Blockade after myocardial infarction: systematic review and metaregression analysis. BMJ. 1999;318(7200):1730-1737.PubMedGoogle Scholar
  35. 35.
    Lau J, Ioannidis JP, Schmid CH. Summing up evidence: one answer is not always enough. Lancet. 1998;351:123-127.CrossRefPubMedGoogle Scholar
  36. 36.
    Shea B, Dube C, Moher D. Assessing the quality of reports of systematic reviews: QUOROM statement compared to other tools. In: Egger M, Smith G, Altman D, eds. Systematic Reviews in Health Care: Meta-Analysis in Context. London: BMJ Publishing; 2001:122-129.CrossRefGoogle Scholar
  37. 37.
    Oxman AD. Checklists for review articles. BMJ. 1994;309(6955):648-651.PubMedGoogle Scholar
  38. 38.
    Moher D, Cook DJ, Eastwood S, et al. Improving the quality of reports of meta-analyses of randomised controlled trials: The QUOROM statement. Quality of Reporting of Meta-analyses. Lancet. 1999;354(9193):1896-1900.CrossRefPubMedGoogle Scholar
  39. 39.
    Petticrew M. Why certain systematic reviews reach uncertain conclusions. BMJ. 2003;326(7392):756-758.CrossRefPubMedGoogle Scholar
  40. 40.
    Jüni P, Altman D, Egger M. Assessing the quality of controlled clinical trials. In: Egger M, Davey Smith G, Altman D, eds. Systematic Reviews in Health Care: Meta-Analysis in context. London: BMJ Books; 2001.Google Scholar
  41. 41.
    Campbell DT. Factors relevant to the validity of experiments in social settings. Psychol Bull. 1957;54(4):297-312.CrossRefPubMedGoogle Scholar
  42. 42.
    Moher D, Jones A, Lepage L, CONSORT Group (Consolitdated Standards for Reporting of Trials). Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA. 2001;285(15):1992-1995.CrossRefPubMedGoogle Scholar
  43. 43.
    Gueyffier F, Bulpitt C, Boissel JP, et al. Antihypertensive drugs in very old people: a subgroup meta-analysis of randomised controlled trials. INDANA Group. Lancet. 1999;353(9155):793-796.CrossRefPubMedGoogle Scholar
  44. 44.
    Shea B, Boers M, Grimshaw JM, et al. Does updating improve the methodological and reporting quality of systematic reviews? BMC Med Res Methodol. 2006;6:27.CrossRefPubMedGoogle Scholar
  45. 45.
    Gluud LL. Bias in clinical intervention research. Am J Epidemiol. 2006;163(6):493-501.CrossRefPubMedGoogle Scholar
  46. 46.
    Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.0.0 edn. Oxford: The Cochrane Collaboration; 2008.Google Scholar
  47. 47.
    Sterne JA, Egger M, Smith GD. Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis. BMJ. 2001;323(7304):101-105.CrossRefPubMedGoogle Scholar
  48. 48.
    Hopewell S, McDonald S, Clarke M, et al. Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev. 2007;18(2):MR000010.Google Scholar
  49. 49.
    Ezzo J. Should journals devote space to trials with no results? J Altern Complement Med. 2003;9(5):611-612.CrossRefPubMedGoogle Scholar
  50. 50.
    Krleza-Jerić K, Chan AW, Dickersin K, et al. Principles for international registration of protocol information and results from human trials of health related interventions: Ottawa statement (part 1). BMJ. 2005;330(7497):956-958.CrossRefPubMedGoogle Scholar
  51. 51.
    DeAngelis CD, Drazen JM, Frizelle FA, et al. International Committee of Medical Journal Editors. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. JAMA. 2004;292(11):1363-1364.CrossRefPubMedGoogle Scholar
  52. 52.
    McCray AT. Better access to information about clinical trials. Ann Intern Med. 2000;133(8):609-614.PubMedGoogle Scholar
  53. 53.
    Egger M, Smith GD. Bias in location and selection of studies. BMJ. 1998;316(7124):61-66.PubMedGoogle Scholar
  54. 54.
    Stuck AE, Rubenstein LZ, Wieland D. Bias in meta-analysis detected by a simple, graphical test. Asymmetry detected in funnel plot was probably due to true heterogeneity. BMJ. 1998;316(7129):469.PubMedGoogle Scholar
  55. 55.
    Duval S, Tweedie R. A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis. J Am Stat Assoc. 2000;95:89-98.CrossRefGoogle Scholar
  56. 56.
    Ioannidis JP, Trikalinos TA. The appropriateness of asymmetry tests for publication bias in meta-analyses: a large survey. CMAJ. 2007;176(8):1091-1096.CrossRefPubMedGoogle Scholar
  57. 57.
    Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994;50(4):1088-1101.CrossRefPubMedGoogle Scholar
  58. 58.
    Egger M, Davey Smith G, et al. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629-634.PubMedGoogle Scholar
  59. 59.
    Sutton A, et al. Publication bias. In: Sutton A, ed. Methods for Meta-Analysis in Medical Research. Chichester, UK: Wiley; 2000.Google Scholar
  60. 60.
    Blettner M, Sauerbrei W, Schlehofer B, et al. Traditional reviews, meta-analyses and pooled analyses in epidemiology. Int J Epidemiol. 1999;28(1):1-9.CrossRefPubMedGoogle Scholar
  61. 61.
    Salanti G, Higgins J, Ades AE, et al. Evaluation of networks of randomized trials. Stat Methods Med Res. 2008;17:279-301.CrossRefPubMedGoogle Scholar
  62. 62.
    Devereaux PJ, Bhandari M, Clarke M, et al. Need for expertise based randomised controlled trials. BMJ. 2005;330:88.CrossRefPubMedGoogle Scholar
  63. 63.
    Bednarska E, Bryant D, Devereaux PJ, et al. Orthopaedic surgeons prefer to participate in expertise-based randomized trials. Clin Orthop Relat Res. 2008;466(7):1734-1744.CrossRefPubMedGoogle Scholar
  64. 64.
    Sprague S, Matta JM, Bhandari M. Multicenter collaboration in observational research: Improving generalizability and efficiency. J Bone Joint Surg Am. 2009;91(Suppl 3):80-86.CrossRefPubMedGoogle Scholar
  65. 65.
    von Elm E, Altman DG, Egger M, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370(9596):1453-1457.CrossRefGoogle Scholar
  66. 66.
    Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56(2):455-463.CrossRefPubMedGoogle Scholar

Copyright information

© Springer London 2011

Authors and Affiliations

  • Srdjan Saso
    • 1
  • Sukhmeet S. Panesar
  • Weiming Siow
  • Thanos Athanasiou
  1. 1.Institute of Reproductive & Developmental BiologyImperial College LondonLondonUK

Personalised recommendations