Abstract
High-quality systematic reviews start with good design and careful planning. It is not possible to eliminate bias, but by prespecifying methods in a protocol, the risk of bias can be minimised. A publicly available protocol also provides transparency in the process. Systematic reviewing is an iterative process, so subsequent deviations and changes from what was planned may be inevitable but should be recorded and justified at the stage of review when they occur. Such transparency in conduct and reporting enables those using systematic review findings to judge the quality of a review and assess for themselves the potential impact of any changes from the initial protocol. In this chapter we discuss the value of systematic review protocol registration and focus on PROSPERO, an open register designed specifically for prospective registration of systematic reviews. Examples from PROSPERO are used to illustrate considerations specific to systematic review of diagnostic test accuracy studies.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Centre for reviews and dissemination. University of York. 2017. https://www.york.ac.uk/crd/. Accessed 28 June 2018.
Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291:2457–65.
Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan A-W, Cronin E, et al. Systematic review of the empirical evidence of study publication Bias and outcome reporting Bias. PLoS One. 2008;3:e3081.
Silagy CA, Middleton P, Hopewell S. Publishing protocols of systematic reviews: comparing what was done to what was planned. JAMA. 2002;287:2831–4.
Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4:e78.
Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of reporting of meta-analyses. Lancet (Lond Engl). 1999;354:1896–900.
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100.
Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.
Joanna Briggs Institute. The Joanna Briggs Institute. 2017. http://joannabriggs.org/. Accessed 28 June 2018.
The Campbell Collaboration. Campbell collaboration: better evidence for a better world. 2017. https://www.campbellcollaboration.org/. Accessed 28 June 2018.
The Cochrane Collaboration. Cochrane. 2017. http://www.cochrane.org/. Accessed 28 June 2018.
Moher D, Stewart L, Shekelle P. Establishing a new journal for systematic review products. Syst Rev. 2012;1:1.
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1–9.
Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration & explanation. BMJ. 2015;349:g7647.
Smidt N, Rutjes AWS, Van der Windt D, Ostelo R, Bossuyt PM, Reitsma JB, et al. The quality of diagnostic accuracy studies since the STARD statement: has it improved? Neurology. 2006;67:792–7.
Noel-Storr AH, McCleery JM, Richard E, Ritchie CW, Flicker L, Cullum SJ, et al. Reporting standards for studies of diagnostic test accuracy in dementia: the STARDdem initiative. Neurology. 2014;83:364–73.
Kirkham JJ, Altman DG, Williamson PR. Bias due to changes in specified outcomes during the systematic review process. PLoS One. 2010;5:e9810.
Page MJ, McKenzie JE, Kirkham J, Dwan K, Kramer S, Green S, et al. Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database Syst Rev. 2014;(10):MR000035. https://doi.org/10.1002/14651858.MR000035.pub2. Accessed 28 June 2018.
Beller EM, Glasziou PP, Altman DG, Hopewell S, Bastian H, Chalmers I, et al. PRISMA for abstracts: reporting systematic reviews in journal and conference abstracts. PLoS Med. 2013;10:e1001419.
Tricco AC, Pham B, Brehaut J, Tetroe J, Cappelli M, Hopewell S, et al. An international survey indicated that unpublished systematic reviews exist. J Clin Epidemiol. 2009;62:617–23.e5.
Dwan K, Altman DG, Clarke M, Gamble C, Higgins JP, Sterne JA, et al. Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials. PLoS Med. 2014;11:e1001666.
Dwan K, Gamble C, Williamson PR, Kirkham JJ. Systematic review of the empirical evidence of study publication bias and outcome reporting bias—an updated review. PLoS One. 2013;8:e66844.
Tricco AC, Cogo E, Page MJ, Polisena J, Booth A, Dwan K, et al. A third of systematic reviews changed or did not specify the primary outcome: a PROSPERO register study. J Clin Epidemiol. 2016;79:46–54.
Ioannidis JPA, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383:166–75.
Moher D. The problem of duplicate systematic reviews. BMJ. 2013;347:f5040.
Moher D, Booth A, Stewart L. How to reduce unnecessary duplication: use PROSPERO. BJOG. 2014;121:784–6.
Siontis KC, Hernandez-Boussard T, Ioannidis JPA. Overlapping meta-analyses on the same topic: survey of published studies. BMJ. 2013;347:f4501.
Centre for Reviews and Dissemination. PROSPERO International prospective register of systematic reviews. 2017. https://www.crd.york.ac.uk/PROSPERO/. Accessed 28 June 2018.
The PME. Best practice in systematic reviews: the importance of protocols and registration. PLoS Med. 2011;8:e1001009.
Viergever RF, Ghersi D. The quality of registration of clinical trials. PLoS One. 2011;6:e14701.
Centre for Open Science. Open science framework. 2017. https://osf.io/jsznk/register/565fb3678c5e4a66b5582f67. Accessed 28 June 2018.
Booth A, Clarke M, Ghersi D, Moher D, Petticrew M, Stewart L. An international registry of systematic-review protocols. The Lancet. 2011;377:108–9.
Booth A, Clarke M, Ghersi D, Moher D, Petticrew M, Stewart L. Establishing a minimum dataset for prospective registration of systematic reviews: an international consultation. PLoS One. 2011;6:e27319.
Booth A. PROSPERO’s progress and activities 2012/13. Syst Rev. 2013;2:111.
Booth A, Clarke M, Dooley G, Ghersi D, Moher D, Petticrew M, et al. PROSPERO at one year: an evaluation of its utility. Syst Rev. 2013;2:4.
Booth A, Clarke M, Dooley G, Ghersi D, Moher D, Petticrew M, et al. The nuts and bolts of PROSPERO: an international prospective register of systematic reviews. Syst Rev. 2012;1:2.
Zhelev Z, Garside R, Hyde C. A qualitative study into the difficulties experienced by healthcare decision makers when reading a Cochrane diagnostic test accuracy review. Syst Rev. 2013;2:32.
Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 2017;7:e012545.
Pennant M, Wisniewski S, Hyde C, Davenport C, Deeks JJ, Cochrane Diagnostic Test Accuracy Editorial T, editors. A tool to improve efficiency and quality in the production of protocols for Cochrane Reviews of Diagnostic Test Accuracy. 19th Cochrane Colloquium; 2011; Madrid, Spain.
van Enst WA, Scholten RJ, Whiting P, Zwinderman AH, Hooft L. Meta-epidemiologic analysis indicates that MEDLINE searches are sufficient for diagnostic test accuracy systematic reviews. J Clin Epidemiol. 2014;67:1192–9.
Rice DB, Kloda LA, Levis B, Qi B, Kingsland E, Thombs BD. Are MEDLINE searches sufficient for systematic reviews and meta-analyses of the diagnostic accuracy of depression screening tools? A review of meta-analyses. J Psychosom Res. 2016;87:7–13.
Preston L, Carroll C, Gardois P, Paisley S, Kaltenthaler E. Improving search efficiency for systematic reviews of diagnostic test accuracy: an exploratory study to assess the viability of limiting to MEDLINE, EMBASE and reference checking. Syst Rev. 2015;4:82.
Leeflang MM, Deeks JJ, Takwoingi Y, Macaskill P. Cochrane diagnostic test accuracy reviews. Syst Rev. 2013;2:82.
Glanville J, Cikalo M, Crawford F, Dozier M, McIntosh H. Handsearching did not yield additional unique FDG-PET diagnostic test accuracy studies compared with electronic searches: a preliminary investigation. Res Synth Methods. 2012;3:202–13.
Agency for Healthcare Research and Quality. Systematic reviews data register (SRDR). 2017. https://srdr.ahrq.gov/. Accessed 28 June 2018.
Barbic D, Chenkin J, Cho D, Jelic T. Point-of-care ultrasonography for the diagnosis of abscess in patients presenting with skin and soft tissue infections to the emergency department. PROSPERO 2015 CRD42015017115. www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42015017115. Accessed 28 June 2018.
Smith T, Daniell A, Geere J, Toms A, Hing C. The diagnostic accuracy of MRI for rotator cuff tears: a systematic review and meta-analysis. PROSPERO. 2011;CRD42011001283. www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42011001283. Accessed 28 June 2018.
Centre for Reviews and Dissemination. Clinical tests. In: Systematic reviews: CRD’s guidance for undertaking reviews in health care. York: University of York; 2009. www.york.ac.uk/inst/crd/index_guidance.htm. Accessed 28 June 2018.
Deeks JJ. Systematic reviews of evaluations of diagnostic and screening tests. BMJ. 2001;323:157–62.
Deeks JJ, Bossuyt PM, Gatsonis C, editors. Cochrane handbook for systematic reviews of diagnostic test accuracy Version 1.0.0. The Cochrane Collaboration; 2013. srdta.cochrane.org. Accessed 28 June 2018.
Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155:529–36.
The Joanna Briggs Institute critical appraisal tools for use in JBI systematic reviews: checklist for diagnostic test accuracy studies. 2016. http://joannabriggs.org/research/critical-appraisal-tools.html. Accessed 28 June 2018.
Riemsma R, Al M, Deshpande S, Ramos IC, Armstrong N, Lee Y-C, et al. A systematic review and economic evaluation of SeHCAT (Tauroselcholic [75Selenium] acid) for the investigation of bile acid malabsorption (BAM) and measurement of bile acid pool loss. PROSPERO. 2012:CRD42012001911. www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42012001911. Accessed 28 June 2018.
Smith TO, Daniell H, Geere J-A, Toms AP, Hing CB. The diagnostic accuracy of MRI for the detection of partial- and full-thickness rotator cuff tears in adults. Magn Reson Imaging. 2012;30:336–46.
van Enst W, Ochodo E, Scholten RJ, Hooft L, Leeflang MM. Investigation of publication bias in meta-analyses of diagnostic test accuracy: a meta-epidemiological study. BMC Med Res Methodol. 2014;14:70.
Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994;50:1088–101.
Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol. 2005;58:882–93.
Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315:629–34.
Acknowledgements
The authors would like to thank the Centre for Reviews and Dissemination, the producers of PROSPERO, for permission to base this chapter on information provided on the register website. We are also grateful to Dr. Nick Meader for his advice and peer comments on the draft.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Booth, A., Jones-Diette, J. (2018). Registering the Review. In: Biondi-Zoccai, G. (eds) Diagnostic Meta-Analysis. Springer, Cham. https://doi.org/10.1007/978-3-319-78966-8_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-78966-8_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-78965-1
Online ISBN: 978-3-319-78966-8
eBook Packages: MedicineMedicine (R0)