Patient-Specific Virtual Reality Simulation for Minimally Invasive Neurosurgery

  • Ralf A. KockroEmail author
  • Luis Serra
Part of the Comprehensive Healthcare Simulation book series (CHS)


The Dextroscope is a surgical planning system that takes advantage of some of the key aspects of virtual reality to maximize the potential of multimodal patient data: two-handed input in 3D and stereoscopic volume-rendering visualization of 3D data. 3D patient models are manipulated in 3D by the surgeon in an easy, intuitive, and flexible manner, without the limitations imposed by mouse and keyboard. It features additionally comprehensive software tools for data segmentation, simulated bone drilling, and simulation of intraoperative views. The Dextrobeam was introduced for use as an educational tool for students and practitioners of anatomy and surgery. It was used in a number of international medical courses and teaching departments.

In order to take full advantage in the operating room of the 3D models created by the Dextroscope, the DEX-Ray augmented reality navigation system was developed. It featured a handheld tracked probe incorporating a video camera, which overlaid 3D virtual patient information over a video stream, to achieve unambiguous intraoperative image guidance.

The Dextroscope was applied mainly to neurosurgery, but it was also used in other specialties presenting a surgical challenge – an anatomical or structural complexity that requires planning of the surgical (or interventional) approach, for example, ENT, orthopedic, trauma, craniofacial, and liver surgery.


3D human–computer interaction Virtual reality Neurosurgery Surgical planning software Medical education Dextroscope Dextrobeam DEX-Ray Surgery planning Augmented reality surgery navigation 3D interactions Stereoscopic display 



The Dextroscope, Dextrobeam, DEX-Ray, and DextroVision were developed by Volume Interactions Pte Ltd, a company member of the Bracco Group (Milan, Italy). Volume Interactions was based in Singapore and was a spin-off from the Kent Ridge Digital Labs (now Institute for Infocom Research) in Singapore.

The Dextroscope and Dextrobeam products received USA FDA 510(K)-Class II (2002) clearance, CE Marking-Class I (2002), China SFDA Registration-Class II (2004) and Taiwan Registration-type P (Radiology) (2007). DEX-Ray and DextroVision were research prototypes.

Supplementary material

Video 13.1

Interactions in the Dextroscope


  1. 1.
    Alaraj A, Lemole MG, Finkle JH, Yudkowsky R, Wallace A, Luciano C, et al. Virtual reality training in neurosurgery: review of current status and future applications. Surg Neurol Int [Internet]. 2011 Apr 28.; [cited 2015 Jan 4];2. Available from:
  2. 2.
    Alaraj A, Tobin MK, Birk DM, Charbel FT. Simulation in neurosurgery and neurosurgical procedures. In: MD AIL, MD SDJ, MD ADS, MD AJS, editors. The comprehensive textbook of healthcare simulation [Internet]. New York: Springer; 2013 [cited 2015 Jan 4]. p. 415–23. Available from: Scholar
  3. 3.
    Malone HR, Syed ON, Downes MS, DʼAmbrosio AL, Quest DO, Kaiser MG. Simulation in neurosurgery: a review of computer-based simulation environments and their surgical applications. Neurosurgery. 2010 Oct;67(4):1105–16.CrossRefGoogle Scholar
  4. 4.
    O’Toole R, Playter R, Krummel T, Blank W, Cornelius N, Roberts W, et al. Assessing skill and learning in surgeons and medical students using a force feedback surgical simulator. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI’98 [Internet]. Berlin, Heidelberg: Springer; 1998 [cited 2017 Jun 5]. p. 899–909. Available from: Scholar
  5. 5.
    Hill JW, Holst PA, Jensen JF, Goldman J, Gorfu Y, Ploeger DW. Telepresence interface with applications to microsurgery and surgical simulation. Stud Health Technol Inform. 1998;50:96–102.PubMedGoogle Scholar
  6. 6.
    Delorme S, Laroche D, DiRaddo R, Del Maestro RF. NeuroTouch. Neurosurgery. 2012 Sep;71:ons32–42.Google Scholar
  7. 7.
    Poston T, Serra L. Dextrous virtual work. Commun ACM. 1996 May;39(5):37–45.CrossRefGoogle Scholar
  8. 8.
    Poston T, Serra L, Lawton W, Chua BC. Interactive tube finding on a virtual workbench. In: Second International Symposium on Medical Robotics and Computer Assisted Surgery (MRCAS 95), Baltimore, Maryland, November [Internet]. 1995 [cited 2013 Jan 1]. p. 5–7. Available from:
  9. 9.
    Poston T, Serra L. The virtual workbench: Dextrous VR. In: Virtual Reality Software and Technology [Internet]. World Scientific; 1994 [cited 2017 May 7]. p. 111–21. Available from:
  10. 10.
    Serra L, Nowinski WL, Poston T, Hern N, Meng LC, Guan CG, et al. The Brain Bench: virtual tools for stereotactic frame neurosurgery. Med Image Anal. 1997 Sep;1(4):317–29.CrossRefGoogle Scholar
  11. 11.
    Serra L, Poston T, Hern N, Choon CB, Waterworth JA. Interaction techniques for a virtual workspace. Proc ICATVRST’95. 1995;79–80.Google Scholar
  12. 12.
    Serra L, Poston T, Hern N, Pheng Ann H, Beng Choon C. Virtual space editing of tagged MRI heart data. In: Computer Vision, Virtual Reality and Robotics in Medicine [Internet]. 1995 [cited 2013 Jan 1]. p. 70–76. Available from:
  13. 13.
    Serra L, Hern N, Choon CB, Poston T. Interactive vessel tracing in volume data. In: Proceedings of the 1997 Symposium on Interactive 3D Graphics [Internet]. New York: ACM; 1997 [cited 2013 Jan 8]. p. 131-ff. (I3D ’97). Available from:
  14. 14.
    Serra L, Kockro R, Guan C, Hern N, Lee E, Lee Y, et al. Multimodal volume-based tumor neurosurgery planning in the virtual workbench. Med Image Comput Comput Assist Interv. 1998:1007–15.Google Scholar
  15. 15.
    Schmandt C. Spatial input/display correspondence in a stereoscopic computer graphic work station. In: Proceedings of the 10th Annual Conference on Computer Graphics and Interactive Techniques [Internet]. New York: ACM; 1983. p. 253–61. (SIGGRAPH ’83). Available from: Scholar
  16. 16.
    Sowizral HA. Interacting with virtual environments using augmented virtual tools. In: 1994. p. 409–416. Available from:
  17. 17.
    Fuhrmann A, Loffelmann H, Schmalstieg D, Gervautz M. Collaborative visualization in augmented reality. IEEE Comput Graph Appl. 1998 Jul;18(4):54–9.CrossRefGoogle Scholar
  18. 18.
    Billinghurst M, Baldis S, Matheson L, Philips M. 3D palette: a virtual reality content creation tool. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology [Internet]. New York: ACM; 1997. p. 155–156. (VRST ’97). Available from:
  19. 19.
    Deering MF. The HoloSketch VR sketching system. Commun ACM. 1996 May;39(5):54–61.CrossRefGoogle Scholar
  20. 20.
    Wells WM, Viola P, Atsumi H, Nakajima S, Kikinis R. Multi-modal volume registration by maximization of mutual information. Med Image Anal. 1996 Mar;1(1):35–51.CrossRefGoogle Scholar
  21. 21.
    Chia WK, Serra L. Contouring in 2D while viewing stereoscopic 3D volumes. Stud Health Technol Inform. 2006;119:93–5.PubMedGoogle Scholar
  22. 22.
    Stadie AT, Kockro RA, Serra L, Fischer G, Schwandt E, Grunert P, et al. Neurosurgical craniotomy localization using a virtual reality planning system versus intraoperative image–guided navigation. Int J Comput Assist Radiol Surg. 2011 Sep 1;6(5):565–72.CrossRefGoogle Scholar
  23. 23.
    Ferroli P, Tringali G, Acerbi F, Aquino D, Franzini A, Broggi G. Brain surgery in a stereoscopic virtual reality environment: a single institution’s experience with 100 cases. Neurosurgery. 2010 Sep;67(3 Suppl Operative):ons79–84; discussion ons84.CrossRefGoogle Scholar
  24. 24.
    Kockro RA, Serra L, Tseng-Tsai Y, Chan C, Yih-Yian S, Gim-Guan C, et al. Planning and simulation of neurosurgery in a virtual reality environment. Neurosurgery. 2000 Jan;46(1):118–35; discussion 135–7.CrossRefGoogle Scholar
  25. 25.
    Matis GK, Silva DO d A, Chrysou OI, Karanikas M, Pelidou S-H, Birbilis TA, et al. Virtual reality implementation in neurosurgical practice: the “can’t take my eyes off you” effect. Turk Neurosurg. 2013;23(5):690–1.PubMedGoogle Scholar
  26. 26.
    Kockro RA, Stadie A, Schwandt E, Reisch R, Charalampaki C, Ng I, et al. A collaborative virtual reality environment for neurosurgical planning and training. Neurosurgery. 2007 Nov;61(5 Suppl 2):379–91; discussion 391.CrossRefGoogle Scholar
  27. 27.
    Guo Y, Ke Y, Zhang S, Wang Q, Duan C, Jia H, et al. Combined application of virtual imaging techniques and three-dimensional computed tomographic angiography in diagnosing intracranial aneurysms. Chin Med J (Engl). 2008;121(24):2521.Google Scholar
  28. 28.
    Kockro RA, Killeen T, Ayyad A, Glaser M, Stadie A, Reisch R, et al. Aneurysm surgery with preoperative three-dimensional planning in a virtual reality environment: technique and outcome analysis. World Neurosurg. 2016 Dec;96:489–99.CrossRefGoogle Scholar
  29. 29.
    Stadie AT, Kockro RA, Reisch R, Tropine A, Boor S, Stoeter P, et al. Virtual reality system for planning minimally invasive neurosurgery. Technical note. J Neurosurg. 2008 Feb;108(2):382–94.CrossRefGoogle Scholar
  30. 30.
    Wong GKC, Zhu CXL, Ahuja AT, Poon WS. Craniotomy and clipping of intracranial aneurysm in a stereoscopic virtual reality environment. Neurosurgery. 2007 Sep;61(3):564–8; discussion 568–9.CrossRefGoogle Scholar
  31. 31.
    Ng I, Hwang PYK, Kumar D, Lee CK, Kockro RA, Sitoh YY. Surgical planning for microsurgical excision of cerebral arterio-venous malformations using virtual reality technology. Acta Neurochir (Wien). 2009 May;151(5):453–63; discussion 463.CrossRefGoogle Scholar
  32. 32.
    Wong GKC, Zhu CXL, Ahuja AT, Poon WS. Stereoscopic virtual reality simulation for microsurgical excision of cerebral arteriovenous malformation: case illustrations. Surg Neurol. 2009 Jul;72(1):69–72; discussion 72–3.CrossRefGoogle Scholar
  33. 33.
    Du Z-Y, Gao X, Zhang X-L, Wang Z-Q, Tang W-J. Preoperative evaluation of neurovascular relationships for microvascular decompression in the cerebellopontine angle in a virtual reality environment. J Neurosurg [Internet]. 2009 Oct 23 [cited 2010 Jun 3]; Available from:
  34. 34.
    González Sánchez JJ, Enseñat Nora J, Candela Canto S, Rumià Arboix J, Caral Pons LA, Oliver D, et al. New stereoscopic virtual reality system application to cranial nerve microvascular decompression. Acta Neurochir. 2010 Feb;152(2):355–60.CrossRefGoogle Scholar
  35. 35.
    Liu X-D, Xu Q-W, Che X-M, Yang D-L. Trigeminal neurinomas: clinical features and surgical experience in 84 patients. Neurosurg Rev. 2009 Oct;32(4):435–44.CrossRefGoogle Scholar
  36. 36.
    Cerebrovascular Diseases in Children: Anthony J. Raimondi: 9781461276760 [Internet]. [cited 2017 Jun 10]. Available from:
  37. 37.
    Stadie A, Reisch R, Kockro R, Fischer G, Schwandt E, Boor S, et al. Minimally invasive cerebral cavernoma surgery using keyhole approaches – solutions for technique-related limitations. Minim Invasive Neurosurg. 2009 Feb;52(01):9–16.CrossRefGoogle Scholar
  38. 38.
    Khu KJ, Ng I, Ng WH. The relationship between parasagittal and falcine meningiomas and the superficial cortical veins: a virtual reality study. Acta Neurochir. 2009;151(11):1459–64.CrossRefGoogle Scholar
  39. 39.
    Low D, Lee CK, Tay LL, Ng WH, Ang BT, Ng I. Augmented reality neurosurgical planning and navigation for surgical excision of parasagittal, falcine and convexity meningiomas. Br J Neurosurg. 2010 Feb;24(1):69–74.CrossRefGoogle Scholar
  40. 40.
    Tang H, Sun H, Xie L, Tang Q, Gong Y, Mao Y, et al. Intraoperative ultrasound assistance in resection of intracranial meningiomas. Chin J Cancer Res. 2013;25(3):339–45.PubMedPubMedCentralGoogle Scholar
  41. 41.
    Tang H-L, Sun H-P, Gong Y, Mao Y, Wu J-S, Zhang X-L, et al. Preoperative surgical planning for intracranial meningioma resection by virtual reality. Chin Med J (Engl). 2012 Jun;125(11):2057–61.Google Scholar
  42. 42.
    Anil SM, Kato Y, Hayakawa M, Yoshida K, Nagahisha S, Kanno T. Virtual 3-dimensional preoperative planning with the dextroscope for excision of a 4th ventricular ependymoma. Minim Invasive Neurosurg. 2007 Apr;50(2):65–70.CrossRefGoogle Scholar
  43. 43.
    Wang S-S, Li J-F, Zhang S-M, Jing J-J, Xue L. A virtual reality model of the clivus and surgical simulation via transoral or transnasal route. Int J Clin Exp Med. 2014;7(10):3270–9.PubMedPubMedCentralGoogle Scholar
  44. 44.
    Wang S-S, Xue L, Jing J-J, Wang R-M. Virtual reality surgical anatomy of the sphenoid sinus and adjacent structures by the transnasal approach. J Craniomaxillofac Surg. 2012 Sep;40(6):494–9.CrossRefGoogle Scholar
  45. 45.
    Di Somma A, de Notaris M, Stagno V, Serra L, Enseñat J, Alobid I, et al. Extended endoscopic endonasal approaches for cerebral aneurysms: anatomical, virtual reality and morphometric study. BioMed Res Int [Internet]. 2014 Jan 19 [cited 2014 Feb 5];2014. Available from:
  46. 46.
    Reisch R, Stadie A, Kockro R, Gawish I, Schwandt E, Hopf N. The minimally invasive supraorbital subfrontal key-hole approach for surgical treatment of temporomesial lesions of the dominant hemisphere. Minim Invasive Neurosurg. 2009 Aug;52(4):163–9.CrossRefGoogle Scholar
  47. 47.
    Fischer G, Stadie A, Schwandt E, Gawehn J, Boor S, Marx J, et al. Minimally invasive superficial temporal artery to middle cerebral artery bypass through a minicraniotomy: benefit of three-dimensional virtual reality planning using magnetic resonance angiography. Neurosurg Focus. 2009 May;26(5):E20.CrossRefGoogle Scholar
  48. 48.
    Reisch R, Stadie A, Kockro RA, Hopf N. The keyhole concept in neurosurgery. World Neurosurg. 2013 Feb;79(2 Suppl):S17.e9–13.Google Scholar
  49. 49.
    Serra C, Huppertz H-J, Kockro RA, Grunwald T, Bozinov O, Krayenbühl N, et al. Rapid and accurate anatomical localization of implanted subdural electrodes in a virtual reality environment. J Neurol Surg A Cent Eur Neurosurg. 2013 May;74(3):175–82.CrossRefGoogle Scholar
  50. 50.
    Wang S-S, Zhang S-M, Jing J-J. Stereoscopic virtual reality models for planning tumor resection in the sellar region. BMC Neurol. 2012;12:146.CrossRefGoogle Scholar
  51. 51.
    Yang DL, Xu QW, Che XM, Wu JS, Sun B. Clinical evaluation and follow-up outcome of presurgical plan by Dextroscope: a prospective controlled study in patients with skull base tumors. Surg Neurol. 2009 Dec;72(6):682–9.CrossRefGoogle Scholar
  52. 52.
    Chen L, Zhao Y, Zhou L, Zhu W, Pan Z, Mao Y. Surgical strategies in treating brainstem cavernous malformations. Neurosurgery. 2011 Mar;68(3):609–21.CrossRefGoogle Scholar
  53. 53.
    Qiu T, Zhang Y, Wu J-S, Tang W-J, Zhao Y, Pan Z-G, et al. Virtual reality presurgical planning for cerebral gliomas adjacent to motor pathways in an integrated 3-D stereoscopic visualization of structural MRI and DTI tractography. Acta Neurochir. 2010 Nov;152(11):1847–57.CrossRefGoogle Scholar
  54. 54.
    de Notaris M, Palma K, Serra L, Enseñat J, Alobid I, Poblete J, et al. A three-dimensional computer-based perspective of the skull base. World Neurosurg. 2014 Dec;82(6S):S41–8.CrossRefGoogle Scholar
  55. 55.
    Franzini A, Messina G, Marras C, Molteni F, Cordella R, Soliveri P, et al. Poststroke fixed dystonia of the foot relieved by chronic stimulation of the posterior limb of the internal capsule. J Neurosurg. 2009 Dec;111(6):1216–9.CrossRefGoogle Scholar
  56. 56.
    Gu S-X, Yang D-L, Cui D-M, Xu Q-W, Che X-M, Wu J-S, et al. Anatomical studies on the temporal bridging veins with Dextroscope and its application in tumor surgery across the middle and posterior fossa. Clin Neurol Neurosurg. 2011 Dec;113(10):889–94.CrossRefGoogle Scholar
  57. 57.
    Kockro RA, Hwang PYK. Virtual temporal bone: an interactive 3-dimensional learning aid for cranial base surgery. Neurosurgery. 2009 May;64(5 Suppl 2):216–29; discussion 229–30.CrossRefGoogle Scholar
  58. 58.
    Kockro RA. Neurosurgery simulators – beyond the experiment. World Neurosurg. 2013 Nov;80(5):e101–2.CrossRefGoogle Scholar
  59. 59.
    Lee CK, Tay LL, Ng WH, Ng I, Ang BT. Optimization of ventricular catheter placement via posterior approaches: a virtual reality simulation study. Surg Neurol. 2008 Sep;70(3):274–7; discussion 277–8.CrossRefGoogle Scholar
  60. 60.
    Robison RA, Liu CY, Apuzzo MLJ. Man, mind, and machine: the past and future of virtual reality simulation in neurologic surgery. World Neurosurg. 2011;76(5):419–30.CrossRefGoogle Scholar
  61. 61.
    Shen M, Zhang X-L, Yang D-L, Wu J-S. Stereoscopic virtual reality presurgical planning for cerebrospinal otorrhea. Neurosci Riyadh Saudi Arab. 2010 Jul;15(3):204–8.Google Scholar
  62. 62.
    Shi J, Xia J, Wei Y, Wang S, Wu J, Chen F, et al. Three-dimensional virtual reality simulation of periarticular tumors using Dextroscope reconstruction and simulated surgery: a preliminary 10-case study. Med Sci Monit Int Med J Exp Clin Res. 2014;20:1043–50.Google Scholar
  63. 63.
    Stadie AT, Kockro RA. Mono-Stereo-Autostereo. Neurosurgery. 2013 Jan;72:A63–77.CrossRefGoogle Scholar
  64. 64.
    Yang D-L, Che X, Lou M, Xu Q-W, Wu J-S, Li W, et al. Application of dextroscope virtual reality system. In: Anatomical research of inner structures in petrosal bone. [cited 2013 Apr 4]; Available from:
  65. 65.
    Goh KYC. Separation surgery for total vertical craniopagus twins. Childs Nerv Syst ChNS Off J Int Soc Pediatr Neurosurg. 2004 Aug;20(8–9):567–75.Google Scholar
  66. 66.
    R.A. Kockro RA, Goh K, Chan C, Hern N, Lee E, Chia GL, Serra L. Pre-Operative Planning Of The Separation Of Craniopagus Twins In A Collaborative Virtual Reality Environment. Proceedings of the XVIII Congress of the European Society for Pediatric Neurosurgery, Kiruna, Sweden, 14-18 June 2002. Child’s Nerv Syst. 2002;18:252–3.Google Scholar
  67. 67.
    Johns Hopkins Magazine [Internet]. [cited 2017 May 28]. Available from:
  68. 68.
    Kockro RA, Tsai YT, Ng I, Hwang P, Zhu C, Agusanto K, et al. Dex-ray: augmented reality neurosurgical navigation with a handheld video probe. Neurosurgery. 2009 Oct;65(4):795–807; discussion 807–8.CrossRefGoogle Scholar
  69. 69.
    Caversaccio M, Eichenberger A, Häusler R. Virtual simulator as a training tool for endonasal surgery. Am J Rhinol. 2003 Oct;17(5):283–90.CrossRefGoogle Scholar
  70. 70.
    Corey CL, Popelka GR, Barrera JE, Most SP. An analysis of malar fat volume in two age groups: implications for craniofacial surgery. Craniomaxillofac Trauma Reconstr. 2012 Dec;5(4):231–4.CrossRefGoogle Scholar
  71. 71.
    Kwon J, Barrera JE, Jung T-Y, Most SP. Measurements of orbital volume change using computed tomography in isolated orbital blowout fractures. Arch Facial Plast Surg. 2009 Dec;11(6):395–8.CrossRefGoogle Scholar
  72. 72.
    Kwon J, Barrera JE, Most SP. Comparative computation of orbital volume from axial and coronal CT using three-dimensional image analysis. Ophthal Plast Reconstr Surg. 2010 Jan;26(1):26–9.CrossRefGoogle Scholar
  73. 73.
    Li Y, Tang K, Xu X, Yi B. Application of Dextroscope virtual reality in anatomical research of the mandible part of maxillary artery. Beijing Da Xue Xue Bao. 2012 Feb 18;44(1):75–9.PubMedGoogle Scholar
  74. 74.
    Pau CY, Barrera JE, Kwon J, Most SP. Three-dimensional analysis of zygomatic-maxillary complex fracture patterns. Craniomaxillofac Trauma Reconstr. 2010 Sep;3(3):167–76.CrossRefGoogle Scholar
  75. 75.
    Correa CR, Litt HI, Hwang W-T, Ferrari VA, Solin LJ, Harris EE. Coronary Artery findings after left-sided compared with right-sided radiation treatment for early-stage breast cancer. J Clin Oncol. 2007 Jul 20;25(21):3031–7.CrossRefGoogle Scholar
  76. 76.
    Chen G, Yang S-Z, Wu G-Q, Wang Y, Fan G-H, Tan L-W, et al. Development and clinical application of 3D operative planning system of live in virtual reality environments. Zhonghua Wai Ke Za Zhi. 2009 Nov;47(21):1620–6.PubMedGoogle Scholar
  77. 77.
    Chen G, Li X, Wu G, Wang Y, Fang B, Xiong X, et al. The use of virtual reality for the functional simulation of hepatic tumors (case control study). Int J Surg. 2010;8(1):72–8.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of NeurosurgeryHirslanden HospitalZurichSwitzerland
  2. 2.Galgo Medical SLBarcelonaSpain

Personalised recommendations