Operator experience determines performance in a simulated computer-based brain tumor resection task

  • Terrell Holloway
  • Zachary S. Lorsch
  • Michael A. Chary
  • Stanislaw Sobotka
  • Maximillian M. Moore
  • Anthony B. Costa
  • Rolando F. Del Maestro
  • Joshua BedersonEmail author
Original Article



Develop measures to differentiate between experienced and inexperienced neurosurgeons in a virtual reality brain surgery simulator environment.


Medical students (\(n=71\)) and neurosurgery residents (\(n=12\)) completed four simulated Glioblastoma multiforme resections. Simulated surgeries took place over four days with intermittent spacing in between (average time between surgeries of 4.77 \(\pm \) 0.73 days). The volume of tumor removed (cc), volume of healthy brain removed (cc), and instrument path length (mm) were recorded. Additionally, surgical effectiveness (% tumor removed divided by % healthy brain removed) and efficiency (% tumor removed divided by instrument movement in mm) were calculated. Performance was compared (1) between groups, and (2) for each participant over time to assess the learning curve. In addition, the effect of real-time instruction (“coaching”) was assessed with a randomly selected group of medical students.


Neurosurgery residents removed less healthy brain, were more effective in removing tumor and sparing healthy brain tissue, required less instrument movement, and were more efficient in removing tumor tissue than medical students. Medical students approached the resident level of performance over serial sessions. Coached medical students showed more conservative surgical behavior, removing both less tumor and less healthy brain. In sum, neurosurgery residents removed more tumor, removed less healthy brain, and required less instrument movement than medical students. Coaching modified medical student performance.


Virtual Reality brain surgery can differentiate operators based on both recent and long-term experience and may be useful in the acquisition and assessment of neurosurgical skills. Coaching alters the learning curve of naïve inexperienced individuals.


Simulator Virtual reality Training NeuroTouch Coaching Glioblastoma multiforme tumor 



We would like to thank the NRC for logistical support, Steven Philemond for coaching participants in the study, and Dr. Errol Gordon and Dr. E. J. Fernandez for their assistance with experimental design, the medical students and neurosurgery residents of the Icahn School of Medicine at Mount Sinai for their participation in our study, and all those who contributed to the study or manuscript preparation. Zachary Lorsch and Michael Chary are supported by the Medical Scientist Training Program of the Icahn School of Medicine at Mount Sinai (NIH Grant # T32 GM 007280).

Conflict of interest

Terrell Holloway, Zachary Lorsch, Michael Chary, Stanislaw Sobotka, Maximilian Moore, Anthony Costa, and Rolando Del Maestro report no Conflicts of Interest. As stated in the disclosures, Dr. Joshua Bederson, is part of the Neurotouch Consortium advisory board without compensation.

Ethical standard All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent Informed consent was obtained from all individual participants included in the study. The institutional review board (IRB) of Mount Sinai Hospital determined that this study was exempt from IRB review.

Supplementary material

Video demonstration of simulated GBM resection using NeuroTouch surgical simulator. Outline of tools used %during virtual GBM resection; suction utilized by the aspirator in right hand (RH) and cauterization utilized by the %bipolar in left hand (LH)


  1. 1.
    Aarabi B, Hesdorffer DC, Ahn ES, Aresco C, Scalea TM, Eisenberg HM (2006) Outcome following decompressive craniectomy for malignant swelling due to severe head injury. J Neurosurg 104:469–479. doi: 10.3171/jns.2006.104.4.469 CrossRefPubMedGoogle Scholar
  2. 2.
    Robison RA, Liu CY, Apuzzo MLJ (2011) Man, mind, and machine: the past and future of virtual reality simulation in neurologic surgery. World Neurosurg 76:419–430. doi: 10.1016/j.wneu.2011.07.008 CrossRefPubMedGoogle Scholar
  3. 3.
    Schmitt PJ, Agarwal N, Prestigiacomo CJ (2012) From planes to brains: parallels between military development of virtual reality environments and virtual neurological surgery. World Neurosurg 78:214–219. doi: 10.1016/j.wneu.2012.06.014 CrossRefPubMedGoogle Scholar
  4. 4.
    Choudhury N, Gélinas-Phaneuf N, Delorme S, Del Maestro R (2012) Fundamentals of neurosurgery: virtual reality tasks for training and evaluation of technical skills. World Neurosurg 9–19. doi: 10.1016/j.wneu.2012.08.022
  5. 5.
    Kockro RA (2013) Neurosurgery simulators-beyond the experiment. World Neurosurg 80:e101–e102. doi: 10.1016/j.wneu.2013.02.017 CrossRefPubMedGoogle Scholar
  6. 6.
    Aoun SG, Bendok BR, Rahme RJ, Dacey RG, Batjer HH (2013) Standardizing the evaluation of scientific and academic performance in neurosurgery-critical review of the “h” Index and its variants. World Neurosurg 80:e85–e90. doi: 10.1016/j.wneu.2012.01.052 CrossRefPubMedGoogle Scholar
  7. 7.
    Munz Y, Almoudaris AM, Moorthy K, Dosis A, Liddle AD, Darzi AW (2007) Curriculum-based solo virtual reality training for laparoscopic intracorporeal knot tying: objective assessment of the transfer of skill from virtual reality to reality. Am J Surg 193:774–783. doi: 10.1016/j.amjsurg.2007.01.022 CrossRefPubMedGoogle Scholar
  8. 8.
    Carter FJ, Schijven MP, Aggarwal R, Grantcharov T, Francis NK, Hanna GB, Jakimowicz JJ (2005) Consensus guidelines for validation of virtual reality surgical simulators. Surg Endosc 19:1523–1532. doi: 10.1007/s00464-005-0384-2 CrossRefPubMedGoogle Scholar
  9. 9.
    Seixas-Mikelus SA, Kesavadas T, Srimathveeravalli G, Chandrasekhar R, Wilding GE, Guru KA (2010) Face validation of a novel robotic surgical simulator. Urology 76:357–360. doi: 10.1016/j.urology.2009.11.069 CrossRefPubMedGoogle Scholar
  10. 10.
    Kenney PA, Wszolek MF, Gould JJ, Libertino JA, Moinzadeh A (2009) Face, content, and construct validity of dV-trainer, a novel virtual reality simulator for robotic surgery. Urology 73:1288–1292. doi: 10.1016/j.urology.2008.12.044 CrossRefPubMedGoogle Scholar
  11. 11.
    Lee JY, Mucksavage P, Kerbl DC, Hunyh VB, Etafy M, McDougall EM (2012) Validation study of a virtual reality robotic simulator-role as an assessment tool? J Urol 187:998–1002. doi: 10.1016/j.juro.2011.10.160 CrossRefPubMedGoogle Scholar
  12. 12.
    Gélinas-Phaneuf N, Choudhury N, Al-Habib AR, Cabral A, Nadeau E, Mora V, Pazos V, Debergue P, DiRaddo R, Del Maestro RF (2013) Assessing performance in brain tumor resection using a novel virtual reality simulator. Int J Comput Assist Radiol Surg. doi: 10.1007/s11548-013-0905-8
  13. 13.
    Azarnoush H, Alzhrani G, Winkler-Schwartz A, Alotaibi F, Gelinas-Phaneuf N, Pazos V, Choudhury N, Fares J, DiRaddo R, Del Maestro RF (2014) Neurosurgical virtual reality simulation metrics to assess psychomotor skills during brain tumor resection. Int J Comput Assist Radiol Surg. doi: 10.1007/s11548-014-1091-z
  14. 14.
    Paschold M, Schröder M, Kauff DW, Gorbauch T, Herzer M, Lang H, Kneist W (2011) Virtual reality laparoscopy: which potential trainee starts with a higher proficiency level? Int J Comput Assist Radiol Surg 6:653–662. doi: 10.1007/s11548-010-0542-4 CrossRefPubMedGoogle Scholar
  15. 15.
    Selvander M, Åsman P (2012) Virtual reality cataract surgery training: learning curves and concurrent validity. Acta Ophthalmol 90:412–417. doi: 10.1111/j.1755-3768.2010.02028.x CrossRefPubMedGoogle Scholar
  16. 16.
    Iwata N, Fujiwara M, Kodera Y, Tanaka C, Ohashi N, Nakayama G, Koike M, Nakao A (2011) Construct validity of the LapVR virtual-reality surgical simulator. Surg Endosc 25:423–428. doi: 10.1007/s00464-010-1184-x CrossRefPubMedGoogle Scholar
  17. 17.
    Ohgaki H, Kleihues P (2005) Population-based studies on incidence, survival rates, and genetic alterations in astrocytic and oligodendroglial gliomas. J Neuropathol Exp Neurol 64:479–489PubMedGoogle Scholar
  18. 18.
    Gélinas-Phaneuf N, Del Maestro RF (2013) Surgical expertise in neurosurgery: integrating theory into practice. Neurosurgery 73(Suppl 1):30–38. doi: 10.1227/NEU.0000000000000115 CrossRefPubMedGoogle Scholar
  19. 19.
    Delorme S, Laroche D, DiRaddo R, Del Maestro RF (2012) NeuroTouch: a physics-based virtual simulator for cranial microneurosurgery training. Neurosurgery 71:32–42. doi: 10.1227/NEU.0b013e318249c744 CrossRefPubMedGoogle Scholar
  20. 20.
    Cole SJ, Mackenzie H, Ha J, Hanna GB, Miskovic D (2014) Randomized controlled trial on the effect of coaching in simulated laparoscopic training. Surg Endosc 28:979–986. doi: 10.1007/s00464-013-3265-0
  21. 21.
    Teitelbaum EN, Soper NJ, Santos BF, Rooney DM, Patel P, Nagle AP, Hungness ES (2014) A simulator-based resident curriculum for laparoscopic common bile duct exploration. Surgery 156:880–887, 890–893. doi: 10.1016/j.surg.2014.06.020
  22. 22.
    Rosseau G, Bailes J, del Maestro R, Choudhury N, Comas O, Debergue P, De Luca G, Hovdebo J, Jiang D, Laroche D, Neubauer A, Pazos V, Thibault F, DiRaddo R (2013) The development of a virtual simulator for training neurosurgeons to perform and perfect endoscopic endonasal transsphenoidal surgery. Neurosurgery 73(Suppl 1):85–93. doi: 10.1227/NEU.0000000000000112 CrossRefPubMedGoogle Scholar
  23. 23.
    Neubauer A, Wolfsberger S (2013) Virtual endoscopy in neurosurgery: a review. Neurosurgery 72(Suppl 1):97–106. doi: 10.1227/NEU.0b013e31827393c9 CrossRefPubMedGoogle Scholar
  24. 24.
    Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA, Ramel S, Smith CD, Arvidsson D (2007) Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 193:797–804. doi: 10.1016/j.amjsurg.2006.06.050 CrossRefPubMedGoogle Scholar
  25. 25.
    Badurdeen S, Abdul-Samad O, Story G, Wilson C, Down S, Harris A (2010) Nintendo Wii video-gaming ability predicts laparoscopic skill. Surg Endosc 24:1824–1828. doi: 10.1007/s00464-009-0862-z CrossRefPubMedGoogle Scholar
  26. 26.
    Grantcharov TP, Bardram L, Funch-Jensen P, Rosenberg J (2003) Impact of hand dominance, gender, and experience with computer games on performance in virtual reality laparoscopy. Surg Endosc 17:1082–1085. doi: 10.1007/s00464-002-9176-0 CrossRefPubMedGoogle Scholar
  27. 27.
    Kolozsvari NO, Andalib A, Kaneva P, Cao J, Vassiliou MC, Fried GM, Feldman LS (2011) Sex is not everything: the role of gender in early performance of a fundamental laparoscopic skill. Surg Endosc 25:1037–1042. doi: 10.1007/s00464-010-1311-8 CrossRefPubMedGoogle Scholar
  28. 28.
    Schlickum MK, Hedman L, Enochsson L, Kjellin A, Fellander-Tsai L (2009) Systematic video game training in surgical novices improves performance in virtual reality endoscopic surgical simulators: a prospective randomized study. World J Surg 33:2360–2367. doi: 10.1007/s00268-009-0151-y CrossRefPubMedGoogle Scholar
  29. 29.
    Boyle E, Kennedy A-M, Traynor O, Hill ADK (2011) Training surgical skills using nonsurgical tasks-can Nintendo \(\text{ Wii }^{\text{ TM }}\) improve surgical performance? J Surg Educ 68:148–154. doi: 10.1016/j.jsurg.2010.11.005 CrossRefPubMedGoogle Scholar

Copyright information

© CARS 2015

Authors and Affiliations

  • Terrell Holloway
    • 1
  • Zachary S. Lorsch
    • 1
  • Michael A. Chary
    • 1
  • Stanislaw Sobotka
    • 1
  • Maximillian M. Moore
    • 1
  • Anthony B. Costa
    • 1
  • Rolando F. Del Maestro
    • 2
  • Joshua Bederson
    • 1
    • 3
    Email author
  1. 1.Department of NeurosurgeryIcahn School of Medicine at Mount SinaiNew YorkUSA
  2. 2.Department of Neurology and Neurosurgery, Brain Tumor Research Centre, Montreal Neurological Institute and HospitalMcGill UniversityMontrealCanada
  3. 3.Department of NeurosurgeryMount Sinai Medical CenterNew YorkUSA

Personalised recommendations