Skip to main content
SpringerLink
Log in

From Qualitative Reviews to Umbrella Reviews

  • Chapter
  • First Online:
Umbrella Reviews

Abstract

The volume of health-care literature is growing at an increasing rate, with a huge amount of studies difficult to process. Therefore, we need tools or techniques to synthesize the information to help us in clinical decision-making. In fact, the available body of evidence ranges from single studies to umbrella reviews. In this scenario, evidence-based clinical decision-making requires knowing what type of evidence to use in every situation. However, a prerequisite for optimal decision-making is a greater understanding by professionals of the different techniques used to analyse their strengths, limitations and utilities. The purpose of this chapter is to take a journey from qualitative reviews to umbrella reviews. We start the tour on a fundamental point: term definitions, showing the variability among different authors. We go on to describe the differences, advantages, disadvantages and uses of different types of evidence, from individual studies to the ‘more specific methods’ for knowledge synthesis, both qualitative and quantitative syntheses (systematic reviews, meta-analysis, network meta-analysis). Finally, in the last part of our journey, we compare the strengths and weaknesses of different evidence synthesis methods from the more traditional or specific to the more general or broader reviews (umbrella reviews, overviews of reviews, meta-epidemiologic reviews). Systematic reviews are at the top of the evidence pyramid. However, the number of systematic reviews published is increasing at a high rate, and decision-makers need to evaluate more evidence to answer their questions. Systematic reviews of existing systematic reviews, known as umbrella reviews, provide an overall examination of the body of information that is available for a given topic. Despite the limitations and weaknesses of tools to appraise and synthesize evidence, systematic reviews and umbrella reviews, including overviews of reviews and meta-epidemiological studies, continue to be the best tool for an approximation to the truth, in evidence-based terms.

What information consumes is rather obvious: it consumes the attention of its recipients.

Herbert Simon (1916–2001) – Economic Sciences Nobel Prize 1978

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    EMBASE search on April 22, 2015 (filter: systematic AND ([systematic review]/lim OR [meta analysis]/lim) AND [2015–2015]/py).

References

  1. Cornell JE, Laine C. The science and art of deduction: complex systematic overviews. Ann Intern Med. 2008;148:786–8.

    Article  PubMed  Google Scholar 

  2. Klassen TP, Jadad AR, Moher D. Guides for reading and interpreting systematic reviews. Arch Pediatr Adolesc Med. 1998;152:700–4.

    CAS  PubMed  Google Scholar 

  3. Hartling L, Vandermeer B, Fernandes RM. Systematic reviews, overviews of reviews and comparative effectiveness reviews: a discussion of approaches to knowledge synthesis. Evid Based Child Health. 2014;9:486–94.

    Article  PubMed  Google Scholar 

  4. Pubmed. National Center for Biotechnology Information, U.S. National Library of Medicine 8600 Rockville Pike, Bethesda, MD, 20894 USA.

    Google Scholar 

  5. Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26:91–108.

    Article  PubMed  Google Scholar 

  6. Grimshaw J. A knowledge synthesis chapter. Canadian Institutes of Health Research. 2010. Available from URL: http://www.cihr-irsc.gc.ca/e/documents/knowledge_synthesis_chapter_e.pdf. Cited 25 Apr 2015.

  7. Biondi-Zoccai G, Lotrionte M, Landoni G, et al. The rough guide to systematic reviews and meta-analyses. HSR Proc Intensive Care Cardiovas Anesth. 2011;3:161–73.

    CAS  Google Scholar 

  8. Whittemore R, Chao A, Jang M, et al. Methods for knowledge synthesis: an overview. Heart Lung. 2014;43:453–61.

    Article  PubMed  Google Scholar 

  9. Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from URL: www.cochrane-handbook.org.

  10. Murad MH, Montori VM, Ioannidis JPA, et al. How to read a systematic review and meta-analysis and apply the results to patient care users’ guides to the medical literature. JAMA. 2014;312:171–9.

    Article  CAS  PubMed  Google Scholar 

  11. Noyes J, Popay J, Pearson A, et al. Qualitative research and Cochrane reviews. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from URL: www.cochrane-handbook.org.

  12. Salanti G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Syn Meth. 2012;3:80–97.

    Article  Google Scholar 

  13. Cipriani A, Higgins JPT, Geddes JR, et al. Conceptual and technical challenges in network meta-analysis. Ann Intern Med. 2013;159:130–7.

    Article  PubMed  Google Scholar 

  14. Caldwell DM, Ades AE, Higgins JP. Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ. 2005;331:897–900.

    Article  PubMed Central  PubMed  Google Scholar 

  15. Salanti G, Becker L, Cadwell D, et al. Evolution of Cochrane intervention reviews and overviews of reviews to better accommodate comparisons among multiple interventions. Report from a meeting of the Cochrane Comparing Multiple Interventions Methods Groups, Milan: March 2011. Available from URL: http://cmimg.cochrane.org/Milan-report. Cited 25 Apr 2015.

  16. Glasziou PP, Chalmers I, Green S, et al. Intervention synthesis: a missing link between a systematic review and practical treatment(s). PLoS Med. 2014;11(8), e1001690. doi:10.1371/journal.pmed.1001690.

    Article  PubMed Central  PubMed  Google Scholar 

  17. Ioannidis JP. Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. CMAJ. 2009;181:488–93.

    Article  PubMed Central  PubMed  Google Scholar 

  18. Joanna Briggs Institute Reviewers’ Manual: 2014 edition. The Joanna Briggs Institute. ISBN 978-1-920684-11-2. Available from URL: http://joannabriggs.org/assets/docs/sumari/reviewersmanual-2014.pdf. Cited 25 Apr 2015.

  19. Bae JM. Meta-epidemiology. Epidemiol Health. 2014;36, e2014019. doi:10.4178/epih/e2014019.

    Article  PubMed Central  PubMed  Google Scholar 

  20. Greenhalgh T, Robert G, Macfarlane F, et al. Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review. Soc Sci Med. 2005;61:417–30.

    Article  PubMed  Google Scholar 

  21. Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Social Res Method. 2005;8:19–31.

    Article  Google Scholar 

  22. Colquhoum HL, Levac D, O’Brien KK, et al. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol. 2014;67:1291–4.

    Article  Google Scholar 

  23. Biondi-Zoccai G, Landoni G, Modena MG. A journey into clinical evidence: from case reports to mixed treatment comparisons. HSR Proc Intensive Care Cardiovas Anesth. 2011;3:93–6.

    CAS  Google Scholar 

  24. Ioannidis JP. Evolution and translation of research findings: from bench to where? PLoS Clin Trials. 2006;1(7), e36.

    Article  PubMed Central  PubMed  Google Scholar 

  25. Ebrahim S, Sohani ZN, Montoya L, et al. Reanalyses of randomized clinical trial data. JAMA. 2014;312:1024–32.

    Article  CAS  PubMed  Google Scholar 

  26. Clarke M, Hopewell S, Chalmers I. Reports of clinical trials should begin and end with up-to-date systematic reviews of other relevant evidence: a status report. J R Soc Med. 2007;100:187–90.

    Article  PubMed Central  PubMed  Google Scholar 

  27. Pieper D, Antoine SL, Morfield JC, et al. Methodological approaches in conducting overviews: current state in HTA agencies. Res Syn Meth. 2014;5:187–99.

    Article  Google Scholar 

  28. Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9), e1000326. doi:10.1371/journal.pmed.1000326.

    Article  PubMed Central  PubMed  Google Scholar 

  29. Beller EM, Chen JK, Wang UL, et al. Are systematic reviews up-to-date at the time of publication? Syst Rev. 2013;2:36. doi:10.1186/2046-4053-2-36.

    Article  PubMed Central  PubMed  Google Scholar 

  30. Knottnerus JA, Turgwell P. Knowledge synthesis to improve practice requires up-to-date definitions, methods, and techniques. J Clin Epidemiol. 2014;67:1289–90.

    Article  PubMed  Google Scholar 

  31. Glasziou PP, Shepperd S, Brassey J. Can we rely on the best trial? A comparison of individual trials and systematic reviews. BMC Med Res Methodol. 2010;10:23. doi:10.1186/1471-2288-10-23.

    Article  PubMed Central  PubMed  Google Scholar 

  32. Boeije HR, van Wesel F, Alisic E. Making a difference: towards a method for weighing the evidence in a qualitative synthesis. J Eval Clin Pract. 2011;17:657–63. doi:10.1111/j.1365-2753.2011.01674.x.

    Article  PubMed  Google Scholar 

  33. Nordmann AJ, Kasenda B, Briel M. Meta-analyses: what they can and cannot do. Swiss Med Wkly. 2012;142:w13518.

    PubMed  Google Scholar 

  34. Thomson D, Russell K, Becker L, et al. The evolution of a new publication 867 type: steps and challenged of producing overviews of reviews. Res Syn Meth. 2010;1:198–211.

    Article  Google Scholar 

  35. Biondi-Zoccai G, Peruzzi M, Frati G, et al. Which do you like better…a bowl of Cheerios or a Big Mac? Pros and cons of meta-analyses in endovascular research. J Endovasc Ther. 2013;20:145–8.

    Article  PubMed  Google Scholar 

  36. Donegan S, Williamson P, D’Alessandro U, et al. Assessing key assumptions of network meta-analysis: a review of methods. Res Syn Meth. 2013;4:291–323.

    Article  Google Scholar 

  37. Jansen JP, Naci H. Is network meta-analysis as valid as standard pairwise meta-analysis? It depends on the distribution of effect modifiers. BMC Med. 2013;11:159. doi:10.1186/1741-7015-11-159.

    Article  PubMed Central  PubMed  Google Scholar 

  38. Nikolakopoulou A, Chaimani A, Veroniki AA, et al. Characteristics of networks of interventions: a description of a database of 186 published networks. PLoS One. 2014;9(1), e86754. doi:10.1371/journal.pone.0086754.

    Article  PubMed Central  PubMed  Google Scholar 

  39. Lee AW. Review of mixed treatment comparisons in published systematic reviews shows marked increase since 2009. J Clin Epidemiol. 2014;67:138–43.

    Article  PubMed  Google Scholar 

  40. Mills EJ, Kanters S, Thorlund K, et al. The effects of excluding treatments from network meta-analyses: survey. BMJ. 2013;347:f5195. doi:10.1136/bmj.f5195.

    Article  PubMed Central  PubMed  Google Scholar 

  41. Ortega A, Fraga MD, Alegre-del-Rey EJ, et al. A checklist for critical appraisal of indirect comparisons. Int J Clin Pract. 2014;68:1181–9. doi:10.1111/ijcp.12487.

    Article  CAS  PubMed  Google Scholar 

  42. Mills EJ, Ioannidis JP, Thourlund K, et al. How to use and article reporting a multiple treatment comparison meta-analysis. JAMA. 2012;308:1246–53.

    Article  CAS  PubMed  Google Scholar 

  43. Jansen JP, Fleurence R, Devine B, et al. Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1. Value Health. 2011;14:417–28.

    Article  PubMed  Google Scholar 

  44. Jansen JP, Trikalinos T, Cappelleri JC, et al. Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision making: an ISPOR-AMCP-NCP Good Practice Task Force report. Value Health. 2014;17:157–73.

    Article  PubMed  Google Scholar 

  45. Yang CC, Hsinchun C, Kay H. Visualization of large category map for Internet browsing. Decis Support Syst. 2003;35:89–102.

    Article  Google Scholar 

  46. DiCenso A, Bayley L, Haynes RB. Accessing pre-appraised evidence: fine-tuning the 5S model into a 6S model. Evid Based Nurs. 2009;12:99–101.

    Article  PubMed  Google Scholar 

  47. Hartling L, Chisholm A, Thomson D, et al. A descriptive analysis of overviews of reviews published between 2000 and 2011. PLoS One. 2012;7(11), e49667. doi:10.1371/journal.pone.0049667.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  48. Bartolucci AA, Hillegass WB. Overview, strengths, and limitations of systematic reviews and meta-analyses. In: Chiappelli F, Caldeira Brant XM, Neagos N, Oludawara OO, Ramchandani MH, editors. Evidence-based practice: toward optimizing clinical outcomes. Berlin: Springer; 2010.

    Google Scholar 

  49. Noble JH. Meta-analysis: methods, strengths, weaknesses, and political uses. J Lab Clin Med. 2006;147:7–20.

    Article  PubMed  Google Scholar 

  50. Smith V, Devane D, Begley CM, et al. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. BMC Med Res Methodol. 2011;11:15.

    Google Scholar 

  51. Naylor CD. Meta-analysis and the meta-epidemiology of clinical research. BMJ. 1997;315:617.

    Google Scholar 

  52. Sterne JA, Jüni P, Schulz KF, et al. Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research. Stat Med. 2002;21:1513–24.

    Article  PubMed  Google Scholar 

  53. Trinquart L, Dechartres A, Ravaud P. Meta-epidemiology, meta-meta-epidemiology or network meta-epidemiology? Int J Epidemiol. 2013;42:1131–3.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Pharmacy Services, Clínica Universidad de Navarra, Pio XII, 36, Pamplona, 31008, Spain

    Ana Ortega PhD

  2. Pharmacy Department, Hospital Universitario y Politécnico La Fe, Valencia, Spain

    Eduardo Lopez-Briz

  3. Pharmacy Services, Hospital General La Mancha Centro, Alcázar de San Juan, Spain

    María Dolores Fraga-Fuentes

Authors
  1. Ana Ortega PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eduardo Lopez-Briz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. María Dolores Fraga-Fuentes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ana Ortega PhD .

Editor information

Editors and Affiliations

  1. Dept of Medico-Surgical Sci, Sapienza University of Rome, Latina, Italy

    Giuseppe Biondi-Zoccai

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Ortega, A., Lopez-Briz, E., Fraga-Fuentes, M.D. (2016). From Qualitative Reviews to Umbrella Reviews. In: Biondi-Zoccai, G. (eds) Umbrella Reviews. Springer, Cham. https://doi.org/10.1007/978-3-319-25655-9_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-25655-9_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-25653-5

  • Online ISBN: 978-3-319-25655-9

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation