Abstracting Evidence

  • Luca Testa
  • Mario Bollati


Retrieval and collection of accurate and detailed data is an obviously crucial aspect of any umbrella review, overview of reviews, or meta-epidemiologic study. Yet there is limited evidence guiding this key reviewing step, and many reviewers overlook its importance and ensuing need for accurate planning and undertaking. Nonetheless, the available evidence and expert opinion are coherently supporting a set of best practices to ensure the validity, thoroughness, and usability of retrieved data. In particular, data abstraction should be performed by two or more independent reviewers, on formally developed and piloted report forms. The utmost transparency should be sought, for instance, storing reviewing details in online data repositories for scrutiny or subsequent use. Finally, the risk of duplication when conducting an umbrella review or overview of reviews (e.g., considering twice the same trial results because of being reported by two separate systematic reviews) should be minimized, unless this is one of the meta-epidemiologic goals of the reviewing and research synthesis effort.


Data extraction Data collection Systematic search Risk of bias Precision Sensitivity 


  1. 1.
  2. 2.
    Guyatt G, Rennie D, Meade MO, Cook DJ. Users’ guide to the medical literature. A manual for evidence-based clinical practice. 2nd ed. New York: McGraw-Hill Professional; 2008.Google Scholar
  3. 3.
    Biondi-Zoccai G, editor. Network meta-analysis: evidence synthesis with mixed treatment comparison. Hauppauge: Nova; 2014.Google Scholar
  4. 4.
    Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions. Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. 2011. Available from: Last accessed 28 June 2018.
  5. 5.
    The Joanna Briggs Institute. The Joanna Briggs Institute Reviewers’ manual. Methodology for JBI umbrella reviews. Adelaide: The University of Adelaide; 2014.Google Scholar
  6. 6.
    Li L, Tian J, Tian H, Sun R, Liu Y, Yang K. Quality and transparency of overviews of systematic reviews. J Evid Based Med. 2012;5:166–73.CrossRefPubMedGoogle Scholar
  7. 7.
    Systematic Review Data Repository (SRDR). Available at: Last accessed 28 June 2018.
  8. 8.
    Li T, Vedula SS, Hadar N, Parkin C, Lau J, Dickersin K. Innovations in data collection, management, and archiving for systematic reviews. Ann Intern Med. 2015;162:287–94.CrossRefPubMedGoogle Scholar
  9. 9.
    Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991;44:1271–8.CrossRefPubMedGoogle Scholar
  10. 10.
    Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10.CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Diekemper RL, Ireland BK, Merz LR. Development of the documentation and appraisal review tool for systematic reviews. World J Meta Anal. 2015;3:142–50.CrossRefGoogle Scholar
  12. 12.
    Pieper D, Buechter RB, Li L, Prediger B, Eikermann M. Systematic review found AMSTAR, but not r(evised)-AMSTAR, to have good measurement properties. J Clin Epidemiol. 2015;68:574–83.CrossRefPubMedGoogle Scholar
  13. 13.
    La Torre G, Backhaus I, Mannocci A. Rating for narrative reviews: concept and development of the International Narrative Systematic Assessment tool. Senses Sci. 2015;2:31–5.Google Scholar
  14. 14.
    Stang A. Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses. Eur J Epidemiol. 2010;25:603–5.CrossRefGoogle Scholar
  15. 15.
    Jørgensen AW, Maric KL, Tendal B, Faurschou A, Gøtzsche PC. Industry-supported meta-analyses compared with meta-analyses with non-profit or no support: differences in methodological quality and conclusions. BMC Med Res Methodol. 2008;8:60.CrossRefPubMedPubMedCentralGoogle Scholar
  16. 16.
    Centre for Reviews and Dissemination. Systematic reviews: CRD’s guidance for undertaking reviews in healthcare. York: University of York; 2009.Google Scholar
  17. 17.
    Eden J, Levit L, Berg A, Morton S. Finding what works in health care. Standards for systematic reviews. Washington, DC: The National Academies Press; 2011.Google Scholar
  18. 18.
    Agency for Healthcare Research and Quality. Methods guide for effectiveness and comparative effectiveness reviews. Rockville: Agency for Healthcare Research and Quality (US); 2008.Google Scholar
  19. 19.
    Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP. Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol. 2006;59:697–703.CrossRefPubMedGoogle Scholar
  20. 20.
    Horton J, Vandermeer B, Hartling L, Tjosvold L, Klassen TP, Buscemi N. Systematic review data extraction: cross-sectional study showed that experience did not increase accuracy. J Clin Epidemiol. 2010;63:289–98.CrossRefPubMedGoogle Scholar
  21. 21.
    Bachmann LM, Coray R, Estermann P, Ter Riet G. Identifying diagnostic studies in MEDLINE: reducing the number needed to read. J Am Med Inform Assoc. 2002;9:653–8.CrossRefPubMedPubMedCentralGoogle Scholar
  22. 22.
    Elamin MB, Flynn DN, Bassler D, Briel M, Alonso-Coello P, Karanicolas PJ, Guyatt GH, Malaga G, Furukawa TA, Kunz R, Schünemann H, Murad MH, Barbui C, Cipriani A, Montori VM. Choice of data extraction tools for systematic reviews depends on resources and review complexity. J Clin Epidemiol. 2009;62:506–10.CrossRefPubMedGoogle Scholar
  23. 23.
    Ip S, Hadar N, Keefe S, Parkin C, Iovin R, Balk EM, Lau J. A web-based archive of systematic review data. Syst Rev. 2012;1:15.CrossRefPubMedPubMedCentralGoogle Scholar
  24. 24.
    Doctor evidence. Available at: Last accessed 28 June 2018.
  25. 25.
    Berlin JA. Does blinding of readers affect the results of meta-analyses? University of Pennsylvania Meta-analysis Blinding Study Group. Lancet. 1997;350:185–6.CrossRefPubMedGoogle Scholar
  26. 26.
    Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996;17:1–12.CrossRefPubMedGoogle Scholar
  27. 27.
    Gøtzsche PC, Hróbjartsson A, Maric K, Tendal B. Data extraction errors in meta-analyses that use standardized mean differences. JAMA. 2007;298:430–7.PubMedGoogle Scholar
  28. 28.
    Jonnalagadda SR, Goyal P, Huffman MD. Automating data extraction in systematic reviews: a systematic review. Syst Rev. 2015;4:78.CrossRefPubMedPubMedCentralGoogle Scholar
  29. 29.
    Marshall IJ, Kuiper J, Wallace BC. Automating risk of bias assessment for clinical trials. IEEE J Biomed Health Inform. 2015;19:1406–12.CrossRefPubMedGoogle Scholar
  30. 30.
    Tramèr MR, Reynolds DJ, Moore RA, McQuay HJ. Impact of covert duplicate publication on meta-analysis: a case study. BMJ. 1997;315:635–40.CrossRefPubMedPubMedCentralGoogle Scholar
  31. 31.
    Caldwell DM, Welton NJ, Ades AE. Mixed treatment comparison analysis provides internally coherent treatment effect estimates based on overviews of reviews and can reveal inconsistency. J Clin Epidemiol. 2010;63:875–82.CrossRefPubMedGoogle Scholar
  32. 32.
    Biondi-Zoccai GG, Lotrionte M, Abbate A, Testa L, Remigi E, Burzotta F, Valgimigli M, Romagnoli E, Crea F, Agostoni P. Compliance with QUOROM and quality of reporting of overlapping meta-analyses on the role of acetylcysteine in the prevention of contrast associated nephropathy: case study. BMJ. 2006;332:202–9.CrossRefPubMedPubMedCentralGoogle Scholar
  33. 33.
    Nowbar AN, Mielewczik M, Karavassilis M, Dehbi HM, Shun-Shin MJ, Jones S, Howard JP, Cole GD, Francis DP, DAMASCENE Writing Group. Discrepancies in autologous bone marrow stem cell trials and enhancement of ejection fraction (DAMASCENE): weighted regression and meta-analysis. BMJ. 2014;348:g2688.CrossRefPubMedPubMedCentralGoogle Scholar
  34. 34.
    Peruzzi M, De Falco E, Abbate A, Biondi-Zoccai G, Chimenti I, Lotrionte M, Benedetto U, Delewi R, Marullo AG, Frati G. State of the art on the evidence base in cardiac regenerative therapy: overview of 41 systematic reviews. Biomed Res Int. 2015;2015:613782.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Luca Testa
    • 1
  • Mario Bollati
    • 1
  1. 1.Department of CardiologyIRCCS Pol. S. Donato, S. Donato MilaneseMilanItaly

Personalised recommendations