Linking surgical skills to postoperative outcomes: a Delphi study on the robot-assisted radical prostatectomy

  • A. J. W. BeulensEmail author
  • W. M. Brinkman
  • H. G. Van der Poel
  • A. N. Vis
  • J. P. van Basten
  • R. P. Meijer
  • C. J. Wijburg
  • A. J. M. Hendrikx
  • J. J. G. van Merriënboer
  • C. Wagner
Original Article



To develop an assessment instrument for the evaluation of surgical videos to elucidate the association between surgical skills and postoperative outcomes after a robot-assisted radical prostatectomy (RARP).


A Delphi study consisting of two consecutive online surveys and a consensus group meeting.


Urology departments of general, teaching and university hospitals in the Netherlands.


All Dutch urologists with a specialization in RARP.


Of 18 invited experts, 12 (67%) participated in the first online survey. In the second round, 9 of the 18 invited experts participated (50%). The Delphi meeting was attended by 5 of the 18 (27%) invited experts. The panel identified seven surgical steps with a possible association to postoperative outcomes. The experts also expected an association between adverse postoperative outcomes and the frequency of camera removals, the number of stitches placed, the amount of bleeding, and the extent of coagulation. These factors were incorporated into an assessment instrument.


Experts in the field of RARP achieved consensus on 7 surgical steps and 4 aspects of the RARP procedure that may be related to adverse postoperative outcomes. The resulting assessment instrument will be tested in future research to determine its validity.


Delphi RARP Video analysis Prostatectomy Surgical skills Postoperative outcomes 



The authors thank H.P. Beerlage, A.E. Boeken Kruger, J.B.W. Rietbergen, J.P.M. Sedelaar, P.C. Weijerman, M.B. Busstra, O.S. Klaver, and B.C. Knipscheer for their participation during the Delphi process.

Compliance with ethical standards

Conflict of interest

Drs. Beulens, Dr. Brinkman, Dr. van der Poel, Dr. Vis, Dr. van Basten, Dr. Meijer, Dr. Wijburg, Dr. Hendrikx, Prof. van Merrienboer,, and Prof. Wagner have no conflicts of interest or financial ties to disclose.


  1. 1.
    Barocas DA, Alvarez J, Resnick MJ, Koyama T, Hoffman KE, Tyson MD, Conwill R, McCollum D, Cooperberg MR, Goodman M, Greenfield S, Hamilton AS, Hashibe M, Kaplan SH, Paddock LE, Stroup AM, Wu X-C, Penson DF (2017) Association between radiation therapy, surgery, or observation for localized prostate cancer and patient-reported outcomes after 3 years. JAMA 317(11):1126CrossRefGoogle Scholar
  2. 2.
    Donovan JL, Hamdy FC, Lane JA, Mason M, Metcalfe C, Walsh E, Blazeby JM, Peters TJ, Holding P, Bonnington S, Lennon T, Bradshaw L, Cooper D, Herbert P, Howson J, Jones A, Lyons N, Salter E, Thompson P et al (2016) Patient-reported outcomes after monitoring, surgery, or radiotherapy for prostate cancer. N Engl J Med 375(15):1425–1437CrossRefGoogle Scholar
  3. 3.
    Lardas M, Liew M, van den Bergh RC, De Santis M, Bellmunt J, Van den Broeck T, Cornford P, Cumberbatch MG, Fossati N, Gross T, Henry AM, Bolla M, Briers E, Joniau S, Lam TB, Mason MD, Mottet N, van der Poel HG, Rouvière O et al (2017) Quality of life outcomes after primary treatment for clinically localised prostate cancer: a systematic review. Eur Urol 72(6):869–885CrossRefGoogle Scholar
  4. 4.
    Maike HJ, Cathelijne MP, Ziedses des Plantes DM, Somford JA, van Erkelens RG, de Vries KH, Jozette JC, Bart P (2011) Wijsman MBB en J van L. Hoe vaak incontinentie na radicale prostatectomie? Ned Tijdschr Geneeskd 162:D2294Google Scholar
  5. 5.
    Wilson LC, Gilling PJ. Post-prostatectomy urinary incontinence: a review of surgical treatment options. BJU Int 107:7–10Google Scholar
  6. 6.
    Goluboff ET, Saidi JA, Mazer S, Bagiella E, Heitjan DF, Benson MC, Olsson CA (1998) Urinary continence after radical prostatectomy: the Columbia experience. J Urol 159(4):1276–1280CrossRefGoogle Scholar
  7. 7.
    Tollefson MK, Gettman MT, Karnes RJ, Frank I (2011) Administrative data sets are inaccurate for assessing functional outcomes after radical prostatectomy. J Urol 185(5):1686–1690CrossRefGoogle Scholar
  8. 8.
    Salonia A, Burnett AL, Graefen M, Hatzimouratidis K, Montorsi F, Mulhall JP, Stief C (2012) Prevention and management of postprostatectomy sexual dysfunctions. Part 1: choosing the right patient at the right time for the right surgery. Eur Urol 62(2):261–272CrossRefGoogle Scholar
  9. 9.
    Vickers A, Savage C, Bianco F, Mulhall J, Guillonneau B, Cronin A, Scardino P (2011) Cancer control and functional outcomes after radical prostatectomy as markers of surgical quality: analysis of heterogeneity between surgeons at a single cancer center. Eur Urol 59(3):317–322CrossRefGoogle Scholar
  10. 10.
    Cathcart P, Sridhara A, Ramachandran N, Briggs T, Nathan S, Kelly J (2015) Achieving quality assurance of prostate cancer surgery during reorganisation of cancer services. Eur Urol 68(1):22–29CrossRefGoogle Scholar
  11. 11.
    Gershman B, Meier SK, Jeffery MM, Moreira DM, Tollefson MK, Kim SP, Karnes RJ, Shah ND (2017) Redefining and contextualizing the hospital volume-outcome relationship for robot-assisted radical prostatectomy: implications for centralization of care. J Urol 198(1):92–99CrossRefGoogle Scholar
  12. 12.
    Birkmeyer JD, Finks JF, O’reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJO (2013) Surgical skill and complication rates after bariatric surgery for the Michigan Bariatric Surgery Collaborative. N Engl J Med [Internet] 369:1434–1442CrossRefGoogle Scholar
  13. 13.
    Goldenberg MG, Lee JY, Kwong JC, Grantcharov TP, Costello A (2018) Implementing assessments of robotic-assisted technical skill in urologic education: a systematic review and synthesis of the validity evidence. BJU IntGoogle Scholar
  14. 14.
    Goldenberg MG, Goldenberg L, Grantcharov TP (2017) Surgeon performance predicts early continence after robot-assisted radical prostatectomy. J Endourol 31(9):858–863CrossRefGoogle Scholar
  15. 15.
    Hussein AA, Ghani KR, Peabody J, Sarle R, Abaza R, Eun D, Hu J, Fumo M, Lane B, Montgomery JS, Hinata N, Rooney D, Comstock B, Chan HK, Mane SS, Mohler JL, Wilding G, Miller D, Guru KA (2017) Development and validation of an objective scoring tool for robot-assisted radical prostatectomy: prostatectomy assessment and competency evaluation. J Urol 197(5):1237–1244CrossRefGoogle Scholar
  16. 16.
    Morris C, Hoogenes J, Shayegan B, Matsumoto ED (2017) Towards development and validation of an intraoperative assessment tool for robot-assisted radical prostatectomy training: results of a Delphi study. Int Braz J Urol 43(4):661–670CrossRefGoogle Scholar
  17. 17.
    Stolzenburg J-U, Schwaibold H, Bhanot SM, Rabenalt R, Do M, Truss M, Ho K, Anderson C (2005) Modular surgical training for endoscopic extraperitoneal radical prostatectomy. BJU Int 96(7):1022–1027CrossRefGoogle Scholar
  18. 18.
    Gillessen S, Attard G, Beer TM, Beltran H, Bossi A, Bristow R, Carver B, Castellano D, Chung BH, Clarke N, Daugaard G, Davis ID, de Bono J, Borges dos Reis R, Drake CG, Eeles R, Efstathiou E, Evans CP, Fanti S et al (2018) Management of patients with advanced prostate cancer: the report of the advanced prostate cancer consensus conference APCCC 2017. Eur Urol 73(2):178–211CrossRefGoogle Scholar
  19. 19.
    van den Bos W, Muller BG, Ahmed H, Bangma CH, Barret E, Crouzet S, Eggener SE, Gill IS, Joniau S, Kovacs G, Pahernik S, de la Rosette JJ, Rouvière O, Salomon G, Ward JF, Scardino PT (2014) Focal therapy in prostate cancer: international multidisciplinary consensus on trial design. Eur Urol 65(6):1078–1083CrossRefGoogle Scholar
  20. 20.
    Lutkenhaus LJ, van Os RM, Bel A, Hulshof MCCM (2016) Clinical results of conformal versus intensity-modulated radiotherapy using a focal simultaneous boost for muscle-invasive bladder cancer in elderly or medically unfit patients. Radiat Oncol 11(1):45CrossRefGoogle Scholar
  21. 21.
    MacLennan S, Bekema HJ, Williamson PR, Campbell MK, Stewart F, MacLennan SJ, N’Dow JM, Lam TB (2015) A core outcome set for localised prostate cancer effectiveness trials: protocol for a systematic review of the literature and stakeholder involvement through interviews and a Delphi survey. Trials 16(1):76CrossRefGoogle Scholar
  22. 22.
    Fitch K, Bernstein SJJ, Aguilar MDD, Burnand B, LaCalle JRR, Lazaro P, Van Het Loo M, Mcdonnell J, Vader JPP, Kahan JPP, Loo M (2001) The RAND/UCLA appropriateness Method User’ s Manual [Internet]. TransformationGoogle Scholar
  23. 23.
    Kong H, West S (2013) WMA declaration of Helsinki—ethical principles for scientific requirements and research protocols 29–32Google Scholar
  24. 24.
    Goh AC, Goldfarb DW, Sander JC, Miles BJ, Dunkin BJ (2012) Global evaluative assessment of robotic skills: validation of a clinical assessment tool to measure robotic surgical skills. J Urol 187(1):247–252CrossRefGoogle Scholar
  25. 25.
    Husslein H, Shirreff L, Shore EM, Lefebvre GG, Grantcharov TP (2015) The generic error rating tool: a novel approach to assessment of performance and surgical education in gynecologic laparoscopy. J Surg Educ 72(6):1259–1265CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  • A. J. W. Beulens
    • 1
    • 2
    • 12
    Email author
  • W. M. Brinkman
    • 3
  • H. G. Van der Poel
    • 4
  • A. N. Vis
    • 5
  • J. P. van Basten
    • 6
  • R. P. Meijer
    • 7
  • C. J. Wijburg
    • 8
  • A. J. M. Hendrikx
    • 9
  • J. J. G. van Merriënboer
    • 10
  • C. Wagner
    • 1
    • 11
  1. 1.Netherlands Institute for Health Services Research (NIVEL)UtrechtThe Netherlands
  2. 2.Department of UrologyCatharina HospitalEindhovenThe Netherlands
  3. 3.Department of UrologyErasmus Medical CentreRotterdamThe Netherlands
  4. 4.Department of UrologyDutch Cancer Institute-Antoni van Leeuwenhoek HospitalAmsterdamThe Netherlands
  5. 5.Department of UrologyVU Medical CentreAmsterdamThe Netherlands
  6. 6.Department of UrologyCanisius Wilhelmina HospitalNijmegenThe Netherlands
  7. 7.Department of Oncological UrologyUniversity Medical Centre UtrechtUtrechtThe Netherlands
  8. 8.Department of UrologyRijnstate HospitalArnhemThe Netherlands
  9. 9.Department of Urology, Urologist NPCatharina HospitalEindhovenThe Netherlands
  10. 10.School of Health Professions EducationMaastricht UniversityMaastrichtThe Netherlands
  11. 11.VU Medical Centre, Amsterdam Public Health Research InstituteAmsterdamThe Netherlands
  12. 12.Catharina Hospital EindhovenEindhovenThe Netherlands

Personalised recommendations