Skip to main content
SpringerLink
Log in

Barriers to Evidence Synthesis

  • Chapter
  • First Online:
Evidence Synthesis in Healthcare

  • 1305 Accesses

Abstract

In the hierarchy of research designs, randomised controlled trials and meta-analyses of randomised controlled trials are considered to be the highest level of evidence. They have been established as essential areas of research since their introduction into clinical sciences. Research in the interventional disciplines such as surgery, rely mostly on observational studies. Therefore, the quality and quantity of randomised trials with regards to interventions remain limited. Researchers in these disciplines face various obstacles during building, assessment or implementation of evidence. This chapter aims to provide a critical overview of the obstacles to randomised trials and meta-analyses. It also proposes solution to these problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000;342:1887-1892.

    Article  CAS  PubMed  Google Scholar 

  2. MRC_Investigation. Streptomycin treatment of pulmonary tuberculosis: a Medical Research Council investigation. BMJ. 1948;2:769-782.

    Article  Google Scholar 

  3. Egger M, Smith GD, Phillips AN. Meta-analysis: principles and procedures. BMJ. 1997;315:1533-1537.

    CAS  PubMed  Google Scholar 

  4. Huque MF. Experiences with meta-analysis in NDA submissions. Proc Biopharm Sect Am Stat Assoc. 1988;2:28-33.

    Google Scholar 

  5. Sacks H, Chalmers TC, Smith H Jr. Randomized versus historical controls for clinical trials. Am J Med. 1982;72:233-240.

    Article  CAS  PubMed  Google Scholar 

  6. Farrokhyar F, Karanicolas PJ, Thoma A, et al. Randomized controlled trials of surgical interventions. Ann Surg. 2010;251:409-416.

    Article  PubMed  Google Scholar 

  7. Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ. 1996;312:1215-1218.

    CAS  PubMed  Google Scholar 

  8. Solomon MJ, McLeod RS. Should we be performing more randomized controlled trials evaluating surgical operations? Surgery. 1995;118:459-467.

    Article  CAS  PubMed  Google Scholar 

  9. Shaw LW, Chalmers TC. Ethics in cooperative clinical trials. Ann NY Acad Sci. 1970;169:487-495.

    Article  CAS  PubMed  Google Scholar 

  10. Weijer C, Shapiro SH, Cranley Glass K. For and against: clinical equipoise and not the uncertainty principle is the moral underpinning of the randomised controlled trial. BMJ. 2000;321:756-758.

    Article  CAS  PubMed  Google Scholar 

  11. McCulloch P, Taylor I, Sasako M, Lovett B, Griffin D. Randomised trials in surgery: problems and possible solutions. BMJ. 2002;324:1448-1451.

    Article  PubMed  Google Scholar 

  12. Barkun JS, Barkun AN, Sampalis JS, et al. Randomised controlled trial of laparoscopic versus mini cholecystectomy. The McGill Gallstone Treatment Group. Lancet. 1992;340:1116-1119.

    Article  CAS  PubMed  Google Scholar 

  13. McMahon AJ, Russell IT, Baxter JN, et al. Laparoscopic versus minilaparotomy cholecystectomy: a randomised trial. Lancet. 1994;343:135-138.

    Article  CAS  PubMed  Google Scholar 

  14. Neugebauer E, Troidl H, Kum CK, Eypasch E, Miserez M, Paul A. The E.A.E.S. Consensus development conferences on laparoscopic cholecystectomy, appendectomy, and hernia repair. Consensus statements – September 1994. The Educational Committee of the European Association for Endoscopic Surgery. Surg Endosc. 1995;9:550-563.

    CAS  PubMed  Google Scholar 

  15. Ahmed K, Khan MS, Vats A, et al. Current status of robotic assisted pelvic surgery and future developments. Int J Surg. 2009;7:431-440.

    Article  PubMed  Google Scholar 

  16. Nix J, Smith A, Kurpad R, Nielsen ME, Wallen EM, Pruthi RS. Prospective randomized controlled trial of robotic versus open radical cystectomy for bladder cancer: perioperative and pathologic results. Eur Urol. 2010;57:196-201.

    Article  PubMed  Google Scholar 

  17. Ouriel K. The PIVOTAL study: a randomized comparison of endovascular repair versus surveillance in patients with smaller abdominal aortic aneurysms. J Vasc Surg. 2009;49:266-269.

    Article  PubMed  Google Scholar 

  18. Solomon MJ, Laxamana A, Devore L, McLeod RS. Randomized controlled trials in surgery. Surgery. 1994;115:707-712.

    CAS  PubMed  Google Scholar 

  19. Solomon MJ, McLeod RS. Surgery and the randomised controlled trial: past, present and future. Med J Aust. 1998;169:380-383.

    CAS  PubMed  Google Scholar 

  20. Hall JC, Mills B, Nguyen H, Hall JL. Methodologic standards in surgical trials. Surgery. 1996;119:466-472.

    Article  CAS  PubMed  Google Scholar 

  21. Diamond GA, Forrester JS. Clinical trials and statistical verdicts: probable grounds for appeal. Ann Intern Med. 1983;98:385-394.

    CAS  PubMed  Google Scholar 

  22. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet. 1991;337:867-872.

    Article  CAS  PubMed  Google Scholar 

  23. Hochman JS, Sleeper LA, Godfrey E, et al. Should we emergently revascularize Occluded Coronaries for cardiogenic shock: an international randomized trial of emergency PTCA/CABG-trial design. The SHOCK Trial Study Group. Am Heart J. 1999;137:313-321.

    Article  CAS  PubMed  Google Scholar 

  24. Hochman JS, Sleeper LA, White HD, et al. One-year survival following early revascularization for cardiogenic shock. JAMA. 2001;285:190-192.

    Article  CAS  PubMed  Google Scholar 

  25. van der Linden W. Pitfalls in randomized surgical trials. Surgery. 1980;87:258-262.

    PubMed  Google Scholar 

  26. Chalmers TC, Celano P, Sacks HS, Smith H Jr. Bias in treatment assignment in controlled clinical trials. N Engl J Med. 1983;309:1358-1361.

    Article  CAS  PubMed  Google Scholar 

  27. McCulloch P. Developing appropriate methodology for the study of surgical techniques. J R Soc Med. 2009;102:51-55.

    Article  PubMed  Google Scholar 

  28. Parikh D, Johnson M, Chagla L, Lowe D, McCulloch P. D2 gastrectomy: lessons from a prospective audit of the learning curve. Br J Surg. 1996;83:1595-1599.

    Article  CAS  PubMed  Google Scholar 

  29. Testori A, Bartolomei M, Grana C, et al. Sentinel node localization in primary melanoma: learning curve and results. Melanoma Res. 1999;9:587-593.

    Article  CAS  PubMed  Google Scholar 

  30. Bonenkamp JJ, Songun I, Hermans J, et al. Randomised comparison of morbidity after D1 and D2 dissection for gastric cancer in 996 Dutch patients. Lancet. 1995;345:745-748.

    Article  CAS  PubMed  Google Scholar 

  31. Bonenkamp JJ, Hermans J, Sasako M, et al. Extended lymph-node dissection for gastric cancer. N Engl J Med. 1999;340:908-914.

    Article  CAS  PubMed  Google Scholar 

  32. McCulloch P. D1 versus D2 dissection for gastric cancer. Lancet. 1995;345:1516-1517. author reply 1517–1518.

    CAS  PubMed  Google Scholar 

  33. Sauerland S, Seiler CM. Role of systematic reviews and meta-analysis in evidence-based medicine. World J Surg. 2005;29:582-587.

    Article  PubMed  Google Scholar 

  34. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315:629-634.

    CAS  PubMed  Google Scholar 

  35. Slim K, Raspado O, Brugere C, Lanay-Savary MV, Chipponi J. Failure of a meta-analysis on the role of elective surgery for left colonic diverticulitis in young patients. Int J Colorectal Dis. 2008;23:665-667.

    Article  CAS  PubMed  Google Scholar 

  36. Hernandez AV, Walker E, Ioannidis JP, Kattan MW. Challenges in meta-analysis of randomized clinical trials for rare harmful cardiovascular events: the case of rosiglitazone. Am Heart J. 2008;156:23-30.

    Article  CAS  PubMed  Google Scholar 

  37. Borzak S, Ridker PM. Discordance between meta-analyses and large-scale randomized, controlled trials. Examples from the management of acute myocardial infarction. Ann Intern Med. 1995;123:873-877.

    CAS  PubMed  Google Scholar 

  38. Cappelleri JC, Ioannidis JP, Schmid CH, et al. Large trials vs meta-analysis of smaller trials: how do their results compare? JAMA. 1996;276:1332-1338.

    Article  CAS  PubMed  Google Scholar 

  39. LeLorier J, Gregoire G, Benhaddad A, Lapierre J, Derderian F. Discrepancies between meta-analyses and subsequent large randomized, controlled trials. N Engl J Med. 1997;337:536-542.

    Article  CAS  PubMed  Google Scholar 

  40. Bailar JC 3rd. The promise and problems of meta-analysis. N Engl J Med. 1997;337:559-561.

    Article  PubMed  Google Scholar 

  41. Lau J, Ioannidis JP, Schmid CH. Summing up evidence: one answer is not always enough. Lancet. 1998;351:123-127.

    Article  CAS  PubMed  Google Scholar 

  42. Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC. Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med. 1992;327:248-254.

    Article  CAS  PubMed  Google Scholar 

  43. Huston P, Naylor CD. Health services research: reporting on studies using secondary data sources. CMAJ. 1996;155:1697-1709.

    CAS  PubMed  Google Scholar 

  44. Wachter KW. Disturbed by meta-analysis? Science. 1988;241:1407-1408.

    Article  CAS  PubMed  Google Scholar 

  45. Abrams KR, Gillies CL, Lambert PC. Meta-analysis of heterogeneously reported trials assessing change from baseline. Stat Med. 2005;24:3823-3844.

    Article  PubMed  Google Scholar 

  46. Ioannidis JP, Patsopoulos NA, Evangelou E. Uncertainty in heterogeneity estimates in meta-analyses. BMJ. 2007;335:914-916.

    Article  PubMed  Google Scholar 

  47. Davey Smith G, Egger M, Phillips AN. Meta-analysis. Beyond the grand mean? BMJ. 1997;315:1610-1614.

    CAS  PubMed  Google Scholar 

  48. Rosenberg MS. The file-drawer problem revisited: a general weighted method for calculating fail-safe numbers in meta-analysis. Evolution. 2005;59:464-468.

    PubMed  Google Scholar 

  49. Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L. Comparison of two methods to detect publication bias in meta-analysis. JAMA. 2006;295:676-680.

    Article  CAS  PubMed  Google Scholar 

  50. Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996;17:1-12.

    Article  CAS  PubMed  Google Scholar 

  51. Oxman AD. Checklists for review articles. BMJ. 1994;309:648-651.

    CAS  PubMed  Google Scholar 

  52. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet. 1999;354:1896-1900.

    Article  CAS  PubMed  Google Scholar 

  53. Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA. 2000;283:2008-2012.

    Article  CAS  PubMed  Google Scholar 

  54. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.

    Article  PubMed  Google Scholar 

  55. Juni P, Witschi A, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999;282:1054-1060.

    Article  CAS  PubMed  Google Scholar 

  56. Huedo-Medina TB, Sanchez-Meca J, Marin-Martinez F, Botella J. Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychol Methods. 2006;11:193-206.

    Article  PubMed  Google Scholar 

  57. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21:1539-1558.

    Article  PubMed  Google Scholar 

  58. Bailey KR. Inter-study differences: how should they influence the interpretation and analysis of results? Stat Med. 1987;6:351-360.

    Article  CAS  PubMed  Google Scholar 

  59. Sevdalis N, Jacklin R. Interaction effects and subgroup analyses in clinical trials: more than meets the eye? J Eval Clin Pract. 2008;14:919-922.

    PubMed  Google Scholar 

  60. Cook DI, Gebski VJ, Keech AC. Subgroup analysis in clinical trials. Med J Aust. 2004;180:289-291.

    PubMed  Google Scholar 

  61. Mayer EK, Bottle A, Rao C, Darzi AW, Athanasiou T. Funnel plots and their emerging application in surgery. Ann Surg. 2009;249:376-383.

    Article  PubMed  Google Scholar 

  62. Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56:455-463.

    Article  CAS  PubMed  Google Scholar 

  63. Gray SM. Knowledge management: a core skill for surgeons who manage. Surg Clin North Am. 2006;86:17-39. vii-viii.

    Article  PubMed  Google Scholar 

  64. Howes N, Chagla L, Thorpe M, McCulloch P. Surgical practice is evidence based. Br J Surg. 1997;84:1220-1223.

    Article  CAS  PubMed  Google Scholar 

  65. Kingston R, Barry M, Tierney S, Drumm J, Grace P. Treatment of surgical patients is evidence-based. Eur J Surg. 2001;167:324-330.

    Article  CAS  PubMed  Google Scholar 

  66. Meakins JL. Evidence-based surgery. Surg Clin North Am. 2006;86:1-16. vii.

    Article  PubMed  Google Scholar 

  67. Ergina PL, Cook JA, Blazeby JM, et al. Challenges in evaluating surgical innovation. Lancet. 2009;374:1097-1104.

    Article  PubMed  Google Scholar 

  68. Knight T, Brice A. Librarians, surgeons, and knowledge. Surg Clin North Am. 2006;86:71-90. viii–ix.

    Article  PubMed  Google Scholar 

  69. Glasziou P. Managing the evidence flood. Surg Clin North Am. 2006;86:193-199. xi.

    Article  PubMed  Google Scholar 

  70. Lee MJ. Evidence-based surgery: creating the culture. Surg Clin North Am. 2006;86:91-100. ix.

    Article  PubMed  Google Scholar 

  71. McCulloch P, Badenoch D. Finding and appraising evidence. Surg Clin North Am. 2006;86:41-57. viii.

    Article  PubMed  Google Scholar 

  72. Ahmed K, Ashrafian H. Life-long learning for physicians. Science. 2009;326:227.

    Article  CAS  PubMed  Google Scholar 

  73. Ahmed K, Jawad M, Dasgupta P, Darzi A, Athanasiou T, Khan MS. Assessment and maintenance of competence in urology. Nat Rev Urol. 2010;7:403-413.

    Article  PubMed  Google Scholar 

  74. Arora S, Sevdalis N, Suliman I, Athanasiou T, Kneebone R, Darzi A. What makes a competent surgeon?: experts’ and trainees’ perceptions of the roles of a surgeon. Am J Surg. 2009;198:726-732.

    Article  PubMed  Google Scholar 

  75. Satava RM, Gallagher AG, Pellegrini CA. Surgical competence and surgical proficiency: definitions, taxonomy, and metrics. J Am Coll Surg. 2003;196:933-937.

    Article  PubMed  Google Scholar 

  76. Rothwell PM. Factors that can affect the external validity of randomised controlled trials. PLoS Clin Trials. 2006;1:e9.

    Article  PubMed  Google Scholar 

  77. Marshall JC. Surgical decision-making: integrating evidence, inference, and experience. Surg Clin North Am. 2006;86:201-215. xii.

    Article  PubMed  Google Scholar 

  78. Kraemer HC, Wilson GT, Fairburn CG, Agras WS. Mediators and moderators of treatment effects in randomized clinical trials. Arch Gen Psychiatry. 2002;59:877-883.

    Article  PubMed  Google Scholar 

  79. Sevdalis N, McCulloch P. Teaching evidence-based decision-making. Surg Clin North Am. 2006;86:59-70. viii.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Surgery and Cancer, Imperial College London, St Mary’s Hospital Campus, London, UK

    Kamran Ahmed

Authors
  1. Kamran Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nick Sevdalis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ara Darzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thanos Athanasiou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kamran Ahmed .

Editor information

Editors and Affiliations

  1. St. Mary' Hospital London, Dept. Biosurgery & Surgical Technology, Imperial College London, London, W2 1NY, United Kingdom

    Ara Darzi

  2. St. Mary' Hospital London, Dept. Biosurgery & Surgical Technology, Imperial College London, London, W2 1NY, United Kingdom

    Thanos Athanasiou

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer London

About this chapter

Cite this chapter

Ahmed, K., Sevdalis, N., Darzi, A., Athanasiou, T. (2011). Barriers to Evidence Synthesis. In: Darzi, A., Athanasiou, T. (eds) Evidence Synthesis in Healthcare. Springer, London. https://doi.org/10.1007/978-0-85729-206-3_2

Download citation

  • DOI: https://doi.org/10.1007/978-0-85729-206-3_2

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-0-85729-175-2

  • Online ISBN: 978-0-85729-206-3

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation