Advertisement

Registering the Review

  • Alison Booth
  • Julie Jones-Diette
Chapter

Abstract

High-quality systematic reviews start with good design and careful planning. It is not possible to eliminate bias, but by prespecifying methods in a protocol, the risk of bias can be minimised. A publicly available protocol also provides transparency in the process. Systematic reviewing is an iterative process, so subsequent deviations and changes from what was planned may be inevitable but should be recorded and justified at the stage of review when they occur. Such transparency in conduct and reporting enables those using systematic review findings to judge the quality of a review and assess for themselves the potential impact of any changes from the initial protocol. In this chapter we discuss the value of systematic review protocol registration and focus on PROSPERO, an open register designed specifically for prospective registration of systematic reviews. Examples from PROSPERO are used to illustrate considerations specific to systematic review of diagnostic test accuracy studies.

Keywords

Protocol Registration Bias Transparency 

Notes

Acknowledgements

The authors would like to thank the Centre for Reviews and Dissemination, the producers of PROSPERO, for permission to base this chapter on information provided on the register website. We are also grateful to Dr. Nick Meader for his advice and peer comments on the draft.

References

  1. 1.
    Centre for reviews and dissemination. University of York. 2017. https://www.york.ac.uk/crd/. Accessed 28 June 2018.
  2. 2.
    Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291:2457–65.CrossRefPubMedGoogle Scholar
  3. 3.
    Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan A-W, Cronin E, et al. Systematic review of the empirical evidence of study publication Bias and outcome reporting Bias. PLoS One. 2008;3:e3081.CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    Silagy CA, Middleton P, Hopewell S. Publishing protocols of systematic reviews: comparing what was done to what was planned. JAMA. 2002;287:2831–4.CrossRefPubMedGoogle Scholar
  5. 5.
    Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4:e78.CrossRefPubMedPubMedCentralGoogle Scholar
  6. 6.
    Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of reporting of meta-analyses. Lancet (Lond Engl). 1999;354:1896–900.CrossRefGoogle Scholar
  7. 7.
    Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100.CrossRefPubMedPubMedCentralGoogle Scholar
  8. 8.
    Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    Joanna Briggs Institute. The Joanna Briggs Institute. 2017. http://joannabriggs.org/. Accessed 28 June 2018.
  10. 10.
    The Campbell Collaboration. Campbell collaboration: better evidence for a better world. 2017. https://www.campbellcollaboration.org/. Accessed 28 June 2018.
  11. 11.
    The Cochrane Collaboration. Cochrane. 2017. http://www.cochrane.org/. Accessed 28 June 2018.
  12. 12.
    Moher D, Stewart L, Shekelle P. Establishing a new journal for systematic review products. Syst Rev. 2012;1:1.CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1–9.CrossRefPubMedPubMedCentralGoogle Scholar
  14. 14.
    Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration & explanation. BMJ. 2015;349:g7647.CrossRefGoogle Scholar
  15. 15.
    Smidt N, Rutjes AWS, Van der Windt D, Ostelo R, Bossuyt PM, Reitsma JB, et al. The quality of diagnostic accuracy studies since the STARD statement: has it improved? Neurology. 2006;67:792–7.CrossRefPubMedGoogle Scholar
  16. 16.
    Noel-Storr AH, McCleery JM, Richard E, Ritchie CW, Flicker L, Cullum SJ, et al. Reporting standards for studies of diagnostic test accuracy in dementia: the STARDdem initiative. Neurology. 2014;83:364–73.CrossRefPubMedPubMedCentralGoogle Scholar
  17. 17.
    Kirkham JJ, Altman DG, Williamson PR. Bias due to changes in specified outcomes during the systematic review process. PLoS One. 2010;5:e9810.CrossRefPubMedPubMedCentralGoogle Scholar
  18. 18.
    Page MJ, McKenzie JE, Kirkham J, Dwan K, Kramer S, Green S, et al. Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database Syst Rev. 2014;(10):MR000035.  https://doi.org/10.1002/14651858.MR000035.pub2. Accessed 28 June 2018.
  19. 19.
    Beller EM, Glasziou PP, Altman DG, Hopewell S, Bastian H, Chalmers I, et al. PRISMA for abstracts: reporting systematic reviews in journal and conference abstracts. PLoS Med. 2013;10:e1001419.CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    Tricco AC, Pham B, Brehaut J, Tetroe J, Cappelli M, Hopewell S, et al. An international survey indicated that unpublished systematic reviews exist. J Clin Epidemiol. 2009;62:617–23.e5.CrossRefPubMedGoogle Scholar
  21. 21.
    Dwan K, Altman DG, Clarke M, Gamble C, Higgins JP, Sterne JA, et al. Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials. PLoS Med. 2014;11:e1001666.CrossRefPubMedPubMedCentralGoogle Scholar
  22. 22.
    Dwan K, Gamble C, Williamson PR, Kirkham JJ. Systematic review of the empirical evidence of study publication bias and outcome reporting bias—an updated review. PLoS One. 2013;8:e66844.CrossRefPubMedPubMedCentralGoogle Scholar
  23. 23.
    Tricco AC, Cogo E, Page MJ, Polisena J, Booth A, Dwan K, et al. A third of systematic reviews changed or did not specify the primary outcome: a PROSPERO register study. J Clin Epidemiol. 2016;79:46–54.CrossRefPubMedGoogle Scholar
  24. 24.
    Ioannidis JPA, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383:166–75.CrossRefPubMedPubMedCentralGoogle Scholar
  25. 25.
    Moher D. The problem of duplicate systematic reviews. BMJ. 2013;347:f5040.CrossRefPubMedGoogle Scholar
  26. 26.
    Moher D, Booth A, Stewart L. How to reduce unnecessary duplication: use PROSPERO. BJOG. 2014;121:784–6.CrossRefPubMedGoogle Scholar
  27. 27.
    Siontis KC, Hernandez-Boussard T, Ioannidis JPA. Overlapping meta-analyses on the same topic: survey of published studies. BMJ. 2013;347:f4501.CrossRefPubMedPubMedCentralGoogle Scholar
  28. 28.
    Centre for Reviews and Dissemination. PROSPERO International prospective register of systematic reviews. 2017. https://www.crd.york.ac.uk/PROSPERO/. Accessed 28 June 2018.
  29. 29.
    The PME. Best practice in systematic reviews: the importance of protocols and registration. PLoS Med. 2011;8:e1001009.CrossRefGoogle Scholar
  30. 30.
    Viergever RF, Ghersi D. The quality of registration of clinical trials. PLoS One. 2011;6:e14701.CrossRefPubMedPubMedCentralGoogle Scholar
  31. 31.
    Centre for Open Science. Open science framework. 2017. https://osf.io/jsznk/register/565fb3678c5e4a66b5582f67. Accessed 28 June 2018.
  32. 32.
    Booth A, Clarke M, Ghersi D, Moher D, Petticrew M, Stewart L. An international registry of systematic-review protocols. The Lancet. 2011;377:108–9.CrossRefGoogle Scholar
  33. 33.
    Booth A, Clarke M, Ghersi D, Moher D, Petticrew M, Stewart L. Establishing a minimum dataset for prospective registration of systematic reviews: an international consultation. PLoS One. 2011;6:e27319.CrossRefPubMedPubMedCentralGoogle Scholar
  34. 34.
    Booth A. PROSPERO’s progress and activities 2012/13. Syst Rev. 2013;2:111.CrossRefPubMedPubMedCentralGoogle Scholar
  35. 35.
    Booth A, Clarke M, Dooley G, Ghersi D, Moher D, Petticrew M, et al. PROSPERO at one year: an evaluation of its utility. Syst Rev. 2013;2:4.CrossRefPubMedPubMedCentralGoogle Scholar
  36. 36.
    Booth A, Clarke M, Dooley G, Ghersi D, Moher D, Petticrew M, et al. The nuts and bolts of PROSPERO: an international prospective register of systematic reviews. Syst Rev. 2012;1:2.CrossRefPubMedPubMedCentralGoogle Scholar
  37. 37.
    Zhelev Z, Garside R, Hyde C. A qualitative study into the difficulties experienced by healthcare decision makers when reading a Cochrane diagnostic test accuracy review. Syst Rev. 2013;2:32.CrossRefPubMedPubMedCentralGoogle Scholar
  38. 38.
    Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 2017;7:e012545.CrossRefPubMedPubMedCentralGoogle Scholar
  39. 39.
    Pennant M, Wisniewski S, Hyde C, Davenport C, Deeks JJ, Cochrane Diagnostic Test Accuracy Editorial T, editors. A tool to improve efficiency and quality in the production of protocols for Cochrane Reviews of Diagnostic Test Accuracy. 19th Cochrane Colloquium; 2011; Madrid, Spain.Google Scholar
  40. 40.
    van Enst WA, Scholten RJ, Whiting P, Zwinderman AH, Hooft L. Meta-epidemiologic analysis indicates that MEDLINE searches are sufficient for diagnostic test accuracy systematic reviews. J Clin Epidemiol. 2014;67:1192–9.CrossRefPubMedGoogle Scholar
  41. 41.
    Rice DB, Kloda LA, Levis B, Qi B, Kingsland E, Thombs BD. Are MEDLINE searches sufficient for systematic reviews and meta-analyses of the diagnostic accuracy of depression screening tools? A review of meta-analyses. J Psychosom Res. 2016;87:7–13.CrossRefPubMedGoogle Scholar
  42. 42.
    Preston L, Carroll C, Gardois P, Paisley S, Kaltenthaler E. Improving search efficiency for systematic reviews of diagnostic test accuracy: an exploratory study to assess the viability of limiting to MEDLINE, EMBASE and reference checking. Syst Rev. 2015;4:82.CrossRefPubMedPubMedCentralGoogle Scholar
  43. 43.
    Leeflang MM, Deeks JJ, Takwoingi Y, Macaskill P. Cochrane diagnostic test accuracy reviews. Syst Rev. 2013;2:82.CrossRefPubMedPubMedCentralGoogle Scholar
  44. 44.
    Glanville J, Cikalo M, Crawford F, Dozier M, McIntosh H. Handsearching did not yield additional unique FDG-PET diagnostic test accuracy studies compared with electronic searches: a preliminary investigation. Res Synth Methods. 2012;3:202–13.CrossRefPubMedGoogle Scholar
  45. 45.
    Agency for Healthcare Research and Quality. Systematic reviews data register (SRDR). 2017. https://srdr.ahrq.gov/. Accessed 28 June 2018.
  46. 46.
    Barbic D, Chenkin J, Cho D, Jelic T. Point-of-care ultrasonography for the diagnosis of abscess in patients presenting with skin and soft tissue infections to the emergency department. PROSPERO 2015 CRD42015017115. www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42015017115. Accessed 28 June 2018.
  47. 47.
    Smith T, Daniell A, Geere J, Toms A, Hing C. The diagnostic accuracy of MRI for rotator cuff tears: a systematic review and meta-analysis. PROSPERO. 2011;CRD42011001283. www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42011001283. Accessed 28 June 2018.
  48. 48.
    Centre for Reviews and Dissemination. Clinical tests. In: Systematic reviews: CRD’s guidance for undertaking reviews in health care. York: University of York; 2009. www.york.ac.uk/inst/crd/index_guidance.htm. Accessed 28 June 2018.
  49. 49.
    Deeks JJ. Systematic reviews of evaluations of diagnostic and screening tests. BMJ. 2001;323:157–62.CrossRefPubMedPubMedCentralGoogle Scholar
  50. 50.
    Deeks JJ, Bossuyt PM, Gatsonis C, editors. Cochrane handbook for systematic reviews of diagnostic test accuracy Version 1.0.0. The Cochrane Collaboration; 2013. srdta.cochrane.org. Accessed 28 June 2018.Google Scholar
  51. 51.
    Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155:529–36.CrossRefGoogle Scholar
  52. 52.
    The Joanna Briggs Institute critical appraisal tools for use in JBI systematic reviews: checklist for diagnostic test accuracy studies. 2016. http://joannabriggs.org/research/critical-appraisal-tools.html. Accessed 28 June 2018.
  53. 53.
    Riemsma R, Al M, Deshpande S, Ramos IC, Armstrong N, Lee Y-C, et al. A systematic review and economic evaluation of SeHCAT (Tauroselcholic [75Selenium] acid) for the investigation of bile acid malabsorption (BAM) and measurement of bile acid pool loss. PROSPERO. 2012:CRD42012001911. www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42012001911. Accessed 28 June 2018.
  54. 54.
    Smith TO, Daniell H, Geere J-A, Toms AP, Hing CB. The diagnostic accuracy of MRI for the detection of partial- and full-thickness rotator cuff tears in adults. Magn Reson Imaging. 2012;30:336–46.CrossRefPubMedGoogle Scholar
  55. 55.
    van Enst W, Ochodo E, Scholten RJ, Hooft L, Leeflang MM. Investigation of publication bias in meta-analyses of diagnostic test accuracy: a meta-epidemiological study. BMC Med Res Methodol. 2014;14:70.CrossRefPubMedPubMedCentralGoogle Scholar
  56. 56.
    Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994;50:1088–101.CrossRefPubMedGoogle Scholar
  57. 57.
    Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol. 2005;58:882–93.CrossRefPubMedGoogle Scholar
  58. 58.
    Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315:629–34.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Health SciencesUniversity of YorkYorkUK
  2. 2.Centre for Reviews and DisseminationUniversity of YorkYorkUK

Personalised recommendations