Replicate systematic review and meta-analyses on robotic surgery: a quality appraisal and overlap investigation

  • Jin Ji
  • Han Zhang
  • Da Xu
  • Tianyi Zhang
  • Depei Kong
  • Guang’an Xiao
  • Zhi Cao
  • Fubo Wang
  • Xu Gao
  • Ying-Hao SunEmail author



The number of publications of systematic reviews and meta-analyses (MAs) on robotic surgery have been increasing, including many investigating the same topic. Their quality and extent of overlap remains unclear. We assessed the quality of the MAs in this area and investigated the extent of their overlap.


Relevant studies were identified by searching the MEDLINE, EMBASE, and Cochrane Library databases up to August 1, 2017. Reporting and methodological quality levels were assessed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and Assessment of Multiple Systematic Reviews (AMSTAR) checklists. A thorough investigation of the extent of overlap was performed.


In total, 90 MAs in 5 surgical subspecialties were included after full-text review. The mean reporting and methodological quality scores were 22.5 (83.2%) and 7.6 (69.2%), respectively. Authors from university-affiliated institutions and the presence of statistician or epidemiologist coauthors were associated with better-reporting quality scores. The topics with the most overlapping MAs (all ≥ 6) were robot-assisted thyroidectomy, prostatectomy, gastrectomy, colectomy, and fundoplication. 36 (40%) of the included MAs cited previous MAs on the same topic. Among the 7 MAs comparing robot-assisted radical prostatectomy to the open procedure, most (6/7) drew the same conclusion. 50 to 86% of MAs on this topic included the same trials as primary studies.


Conducting multiple overlapping MAs with identical conclusions on the same topic that are of suboptimal quality may be a waste of resource and effort. Authors from university-affiliated institutes and experts in epidemiology and statistics are more likely to conduct MAs that have better quality. More guidelines and registries are needed to avoid overlapping MAs.


Robotic surgery Meta-analysis Reporting quality Methodological quality Overlapping study 



Preferred Reporting Items for Systematic Reviews and Meta-Analyses


Assessment of Multiple Systematic Reviews




Da Vinci surgical system


Randomized controlled trial



The authors acknowledge the contributions of Drs. Lin Zhao, Ya-Sheng Zhu, Zhe-Xu Cao, and Chen Ye from the Department of Urology, Changhai Hospital, Naval Medical University (Second Military Medical University) during literature search and full-text retrieval process.

Authors’ contributions

JJ, HZ, and DX wrote the manuscript. JJ, DX, DPK, GAX, ZC, and FBW acquired and analyzed the data, and HZ, TYZ, and XG interpreted the data. XG and YHS designed and supervised the study. JJ, HZ, and DX participated in assessing the quality of the included reviews. All authors contributed to the conception and design of the study and read and approved the final manuscript.


This study received no external funding.

Compliance with ethical standards


Drs. Jin Ji, Han Zhang, Da Xu, Tianyi Zhang, Depei Kong, Guang’an Xiao, Zhi Cao, Fubo Wang, Xu Gao, and Ying-Hao Sun have no conflicts of interest or financial ties to disclose.

Ethical approval

Not applicable for this study because of the lack of human and animal participants.

Informed consent

All authors approved the final manuscript and agree to publish it.

Supplementary material

464_2019_6780_MOESM1_ESM.docx (27 kb)
Supplementary material 1 (DOCX 26 kb)
464_2019_6780_MOESM2_ESM.docx (20 kb)
Supplementary material 2 (DOCX 20 kb)


  1. 1.
    Intuitive Surgical, Inc. Annual Reports 2015. Accessed 1 Aug 2018
  2. 2.
    Sacks HS, Berrier J, Reitman D, Ancona-Berk VA, Chalmers TC (1987) Meta-analyses of randomized controlled trials. N Engl J Med 316(8):450–455CrossRefGoogle Scholar
  3. 3.
    Cook DJ (1997) Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med 126(5):376CrossRefGoogle Scholar
  4. 4.
    Jadad AR, Cook DJ, Browman GP (1997) A guide to interpreting discordant systematic reviews. CMAJ 156(10):1411–1416Google Scholar
  5. 5.
    Siontis KC, Hernandez-Boussard T, Ioannidis JP (2013) Overlapping meta-analyses on the same topic: survey of published studies. BMJ 347:f4501CrossRefGoogle Scholar
  6. 6.
    Bastian H, Glasziou P, Chalmers I (2010) Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med 7(9):e1000326CrossRefGoogle Scholar
  7. 7.
    Tan A, Ashrafian H, Scott AJ, Mason SE, Harling L, Athanasiou T, Darzi A (2016) Robotic surgery: disruptive innovation or unfulfilled promise? A systematic review and meta-analysis of the first 30 years. Surg Endosc 30(10):4330–4352CrossRefGoogle Scholar
  8. 8.
    Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 6(7):e1000100CrossRefGoogle Scholar
  9. 9.
    Jüni P, Egger M (2009) PRISMAtic reporting of systematic reviews and meta-analyses. Lancet 374(9697):1221–1223CrossRefGoogle Scholar
  10. 10.
    Moher D (1994) Statistical power, sample size, and their reporting in randomized controlled trials. J Am Med Assoc 272(2):122CrossRefGoogle Scholar
  11. 11.
    Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM (2007) Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 7:10CrossRefGoogle Scholar
  12. 12.
    Higgins JPT, Green S, Cochrane Collaboration (2008) Cochrane handbook for systematic reviews of interventions. Wiley-Blackwell, Chichester, p 649CrossRefGoogle Scholar
  13. 13.
    Zhang H, Han J, Zhu YB, Lau WY, Schwartz ME, Xie GQ, Dai SY, Shen YN, Wu MC, Shen F et al (2016) Reporting and methodological qualities of published surgical meta-analyses. J Clin Epidemiol 70:4–16CrossRefGoogle Scholar
  14. 14.
    Thomson Reuters. 2016 Journal Citation Reports, Thomson Reuters. Accessed 1 Aug 2018
  15. 15.
    The Thomson Reuters Links Open Access Journal Title List. Accessed 1 Aug 2018
  16. 16.
    Booth A, Clarke M, Ghersi D, Moher D, Petticrew M, Stewart L (2011) An international registry of systematic-review protocols. Lancet 377(9760):108–109CrossRefGoogle Scholar
  17. 17.
    Tunis AS, McInnes MD, Hanna R, Esmail K (2013) Association of study quality with completeness of reporting: have completeness of reporting and quality of systematic reviews and meta-analyses in major radiology journals changed since publication of the PRISMA statement? Radiology 269(2):413–426CrossRefGoogle Scholar
  18. 18.
    Adie S, Ma D, Harris IA, Naylor JM, Craig JC (2015) Quality of conduct and reporting of meta-analyses of surgical interventions. Ann Surg 261(4):685–694CrossRefGoogle Scholar
  19. 19.
    Gagnier JJ, Kellam PJ (2013) Reporting and methodological quality of systematic reviews in the orthopaedic literature. J Bone Jt Surg Am 95(11):e771–e777CrossRefGoogle Scholar
  20. 20.
    van den Berg T, Heymans MW, Leone SS, Vergouw D, Hayden JA, Verhagen AP, de Vet HC (2013) Overview of data-synthesis in systematic reviews of studies on outcome prediction models. BMC Med Res Methodol 13:42CrossRefGoogle Scholar
  21. 21.
    Lakens D, Hilgard J, Staaks J (2016) On the reproducibility of meta-analyses: six practical recommendations. BMC Psychol 4(1):24CrossRefGoogle Scholar
  22. 22.
    Wuchty S, Jones BF, Uzzi B (2007) The increasing dominance of teams in production of knowledge. Science 316(5827):1036–1039CrossRefGoogle Scholar
  23. 23.
    Faggion CM Jr (2015) Critical appraisal of AMSTAR: challenges, limitations, and potential solutions from the perspective of an assessor. BMC Med Res Methodol 15:63CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Jin Ji
    • 1
  • Han Zhang
    • 2
  • Da Xu
    • 3
  • Tianyi Zhang
    • 4
  • Depei Kong
    • 1
  • Guang’an Xiao
    • 1
  • Zhi Cao
    • 1
  • Fubo Wang
    • 1
  • Xu Gao
    • 1
  • Ying-Hao Sun
    • 1
    Email author
  1. 1.Department of Urology, Changhai HospitalNaval Medical University (Second Military Medical University)ShanghaiChina
  2. 2.Eastern Hepatobiliary Surgery HospitalNaval Medical University (Second Military Medical University)ShanghaiChina
  3. 3.Department of Urology, Eastern Hepatobiliary Surgery HospitalNaval Medical University (Second Military Medical University)ShanghaiChina
  4. 4.Department of Clinical Epidemiology and StatisticsNaval Medical University (Second Military Medical University)ShanghaiChina

Personalised recommendations