Opportunities for Computer Support for Systematic Reviewing - A Gap Analysis

  • Linh HoangEmail author
  • Jodi Schneider
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10766)


Systematic review is a type of literature review designed to synthesize all available evidence on a given question. Systematic reviews require significant time and effort, which has led to the continuing development of computer support. This paper seeks to identify the gaps and opportunities for computer support. By interviewing experienced systematic reviewers from diverse fields, we identify the technical problems and challenges reviewers face in conducting a systematic review and their current uses of computer support. We propose potential research directions for how computer support could help to speed the systematic review process while retaining or improving review quality.


Systematic review Meta-analysis Gap analysis Interview study 



We would like to show our gratitude to all of the interview participants for sharing their experiences and also the pearls of wisdom that allowed us to complete this study. We would also like to thank our colleagues Lori Kendall and Peter Darch for discussions about qualitative research methodologies; Susan Lafferty who provided expertise that greatly assisted in the IRB process; and Katrina Felon for comments that greatly improved the manuscript. Research reported in this publication was supported in part by the National Library of Medicine of the National Institutes of Health under grant number R01LM010817, “Text Mining Pipeline to Accelerate Systematic Reviews in Evidence-based Medicine”. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.


  1. 1.
    Borah, R., Brown, A.W., Capers, P.L., Kaiser, K.A.: Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open 7(2), e012545 (2017)CrossRefGoogle Scholar
  2. 2.
    Allen, I.E., Olkin, I.: Estimating time to conduct a meta-analysis from number of citations retrieved. J. Am. Med. Assoc. 282(7), 634–635 (1999)CrossRefGoogle Scholar
  3. 3.
    Tsafnat, G., Glasziou, P., Choong, M.K., Dunn, A., Galgani, F., Coiera, E.: Systematic review automation technologies. Syst. Rev. 3(1), 74–88 (2014)CrossRefGoogle Scholar
  4. 4.
    Turner, T., Green, S., Tovey, D., McDonald, S., Soares-Weiser, K., Pestridge, C., Elliott, J.: Producing Cochrane systematic reviews—a qualitative study of current approaches and opportunities for innovation and improvement. Syst. Rev. 6(1), 147–157 (2017)CrossRefGoogle Scholar
  5. 5.
    Thomas, J., Noel-Storr, A., Elliott, J.: Human and machine effort in project transform: how intersecting technologies will help us to identify studies reliably, efficiently and at scale. Cochrane Methods Suppl. 1, 37–41 (2015)Google Scholar
  6. 6.
    Wright, R.W., Brand, R.A., Dunn, W., Spindler, K.P.: How to write a systematic review? Clin. Orthop. Relat. Res. 455, 23–29 (2007)CrossRefGoogle Scholar
  7. 7.
    Anderson, N.K., Jayaratne, Y.S.: Methodological challenges when performing a systematic review. Eur. J. Orthod. 37(3), 248–250 (2015)CrossRefGoogle Scholar
  8. 8.
    Grant, M.J., Booth, A.: A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inf. Lib. J. 26(2), 91–108 (2009)CrossRefGoogle Scholar
  9. 9.
    Bartels, E.M.: How to perform a systematic search. Best Pract. Res. Clin. Rheumatol. 27(2), 295–306 (2013)CrossRefGoogle Scholar
  10. 10.
    Ross-White, A., Godfrey, C.: Is there an optimum number needed to retrieve to justify inclusion of a database in a systematic review search? Health Inf. Libr. J. 34(3), 217–224 (2017)CrossRefGoogle Scholar
  11. 11.
    Lorenzetti, D.L., Ghali, W.A.: Reference management software for systematic reviews and meta-analyses: an exploration of usage and usability. BMC Med. Res. Methodol. 13(1), 141–145 (2013)CrossRefGoogle Scholar
  12. 12.
    DiCicco-Bloom, B., Crabtree, B.F.: The qualitative research interview. Med. Educ. 40(4), 314–321 (2006)CrossRefGoogle Scholar
  13. 13.
    Braun, V., Clarke, V.: Using thematic analysis in psychology. Qual. Res. Psychol. 3(2), 77–101 (2006)CrossRefGoogle Scholar
  14. 14.
    Svenonius, E.: The Intellectual Foundation of Information Organization. MIT Press, Cambridge (2000)Google Scholar
  15. 15.
    Sutton, S.: Encyclopedia of Library and Information Sciences. Taylor and Francis, Abingdon (2009)Google Scholar
  16. 16.
    Marshall, I.J., Kuiper, J., Wallace, B.C.: RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials. J. Am. Med. Inform. Assoc. 23(1), 193–201 (2015)CrossRefGoogle Scholar
  17. 17.
    Thomas, J.M.: Diffusion of innovation in systematic review methodology: why is study selection not yet assisted by automation? OA Evid. Based Med. 1(2), 12–17 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of Illinois at Urbana-ChampaignChampaignUSA

Personalised recommendations