Pedicle screw navigation using surface digitization on the Microsoft HoloLens

  • Florentin LiebmannEmail author
  • Simon Roner
  • Marco von Atzigen
  • Davide Scaramuzza
  • Reto Sutter
  • Jess Snedeker
  • Mazda Farshad
  • Philipp Fürnstahl
Original Article



In spinal fusion surgery, imprecise placement of pedicle screws can result in poor surgical outcome or may seriously harm a patient. Patient-specific instruments and optical systems have been proposed for improving precision through surgical navigation compared to freehand insertion. However, existing solutions are expensive and cannot provide in situ visualizations. Recent technological advancement enabled the production of more powerful and precise optical see-through head-mounted displays for the mass market. The purpose of this laboratory study was to evaluate whether such a device is sufficiently precise for the navigation of lumbar pedicle screw placement.


A novel navigation method, tailored to run on the Microsoft HoloLens, was developed. It comprises capturing of the intraoperatively reachable surface of vertebrae to achieve registration and tool tracking with real-time visualizations without the need of intraoperative imaging. For both surface sampling and navigation, 3D printable parts, equipped with fiducial markers, were employed. Accuracy was evaluated within a self-built setup based on two phantoms of the lumbar spine. Computed tomography (CT) scans of the phantoms were acquired to carry out preoperative planning of screw trajectories in 3D. A surgeon placed the guiding wire for the pedicle screw bilaterally on ten vertebrae guided by the navigation method. Postoperative CT scans were acquired to compare trajectory orientation (3D angle) and screw insertion points (3D distance) with respect to the planning.


The mean errors between planned and executed screw insertion were \(3.38^{\circ }\pm {1.73}^{\circ }\) for the screw trajectory orientation and 2.77±1.46 mm for the insertion points. The mean time required for surface digitization was 125±27 s.


First promising results under laboratory conditions indicate that precise lumbar pedicle screw insertion can be achieved by combining HoloLens with our proposed navigation method. As a next step, cadaver experiments need to be performed to confirm the precision on real patient anatomy.


Surgical navigation Augmented reality Surface digitization HoloLens Spine Pedicle screw 


Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Human and animals rights

This article does not contain any studies with human participants or animals performed by any of the authors.

Patient data

This articles does not contain patient data.


  1. 1.
    Raciborski F, Gasik R, Kłak A (2016) Disorders of the spine. A major health and social problem. Reumatologia 54(4):196CrossRefGoogle Scholar
  2. 2.
    Vos T, Flaxman AD, Naghavi M, Lozano R, Michaud C, Ezzati M, Shibuya K, Salomon JA, Abdalla S, Aboyans V et al (2012) Years lived with disability (ylds) for 1160 sequelae of 289 diseases and injuries 1990–2010: a systematic analysis for the global burden of disease study 2010. The Lancet 380(9859):2163–2196CrossRefGoogle Scholar
  3. 3.
    Van Tulder MW, Koes BW, Bouter LM (1997) Conservative treatment of acute and chronic nonspecific low back pain: a systematic review of randomized controlled trials of the most common interventions. Spine 22(18):2128–2156CrossRefGoogle Scholar
  4. 4.
    Mirza SK, Deyo RA (2007) Systematic review of randomized trials comparing lumbar fusion surgery to nonoperative care for treatment of chronic back pain. Spine 32(7):816–823CrossRefGoogle Scholar
  5. 5.
    Verlaan J, Diekerhof C, Buskens E, Van der Tweel I, Verbout A, Dhert W, Oner F (2004) Surgical treatment of traumatic fractures of the thoracic and lumbar spine: a systematic review of the literature on techniques, complications, and outcome. Spine 29(7):803–814CrossRefGoogle Scholar
  6. 6.
    Maruyama T, Takeshita K (2008) Surgical treatment of scoliosis: a review of techniques currently applied. Scoliosis 3(1):6CrossRefGoogle Scholar
  7. 7.
    Mason A, Paulsen R, Babuska JM, Rajpal S, Burneikiene S, Nelson EL, Villavicencio AT (2014) The accuracy of pedicle screw placement using intraoperative image guidance systems: a systematic review. J Neurosurg Spine 20(2):196–203CrossRefGoogle Scholar
  8. 8.
    Modi HN, Suh SW, Fernandez H, Yang JH, Song HR (2008) Accuracy and safety of pedicle screw placement in neuromuscular scoliosis with free-hand technique. Eur Spine J 17(12):1686–1696CrossRefGoogle Scholar
  9. 9.
    Farshad M, Betz M, Farshad-Amacker NA, Moser M (2017) Accuracy of patient-specific template-guided vs. free-hand fluoroscopically controlled pedicle screw placement in the thoracic and lumbar spine: a randomized cadaveric study. Eur Spine J 26(3):738–749CrossRefGoogle Scholar
  10. 10.
    Merc M, Drstvensek I, Vogrin M, Brajlih T, Recnik G (2013) A multi-level rapid prototyping drill guide template reduces the perforation risk of pedicle screw placement in the lumbar and sacral spine. Arch Orthop Trauma Surg 133(7):893–899CrossRefGoogle Scholar
  11. 11.
    Kantelhardt SR, Martinez R, Baerwinkel S, Burger R, Giese A, Rohde V (2011) Perioperative course and accuracy of screw positioning in conventional, open robotic-guided and percutaneous robotic-guided, pedicle screw placement. Eur Spine J 20(6):860–868CrossRefGoogle Scholar
  12. 12.
    Tian NF, Huang QS, Zhou P, Zhou Y, Wu RK, Lou Y, Xu HZ (2011) Pedicle screw insertion accuracy with different assisted methods: a systematic review and meta-analysis of comparative studies. Eur Spine J 20(6):846–859CrossRefGoogle Scholar
  13. 13.
    Narain AS, Hijji FY, Yom KH, Kudaravalli KT, Haws BE, Singh K (2017) Radiation exposure and reduction in the operating room: perspectives and future directions in spine surgery. World J Orthop 8(7):524CrossRefGoogle Scholar
  14. 14.
    Gebhard FT, Kraus MD, Schneider E, Liener UC, Kinzl L, Arand M (2006) Does computer-assisted spine surgery reduce intraoperative radiation doses? Spine 31(17):2024–2027CrossRefGoogle Scholar
  15. 15.
    Slomczykowski M, Roberto M, Schneeberger P, Ozdoba C, Vock P (1999) Radiation dose for pedicle screw insertion: fluoroscopic method versus computer-assisted surgery. Spine 24(10):975–983CrossRefGoogle Scholar
  16. 16.
    Nottmeier EW, Crosby TL (2007) Timing of paired points and surface matching registration in three-dimensional (3D) image-guided spinal surgery. Clin Spine Surg 20(4):268–270Google Scholar
  17. 17.
    Richter M, Cakir B, Schmidt R (2005) Cervical pedicle screws: conventional versus computer-assisted placement of cannulated screws. Spine 30(20):2280–2287CrossRefGoogle Scholar
  18. 18.
    Chiang CF, Tsai TT, Chen LH, Lai PL, Fu TS, Niu CC, Chen WJ (2012) Computed tomography-based navigation-assisted pedicle screw insertion for thoracic and lumbar spine fractures. Chang Gung Med J 35(4):332–338Google Scholar
  19. 19.
    Qian L, Unberath M, Yu K, Fuerst B, Johnson A, Navab N, Osgood G (2017) Towards virtual monitors for image guided interventions-real-time streaming to optical see-through head-mounted displays. arXiv preprint arXiv:171000808
  20. 20.
    Andress S, Johnson A, Unberath M, Winkler AF, Yu K, Fotouhi J, Weidert S, Osgood G, Navab N (2018) On-the-fly augmented reality for orthopedic surgery using a multimodal fiducial. J Med Imaging 5(2):021209CrossRefGoogle Scholar
  21. 21.
    Sielhorst T, Feuerstein M, Navab N (2008) Advanced medical displays: a literature review of augmented reality. J Disp Technol 4(4):451–467CrossRefGoogle Scholar
  22. 22.
    Navab N, Blum T, Wang L, Okur A, Wendler T (2012) First deployments of augmented reality in operating rooms. Computer 45(7):48–55CrossRefGoogle Scholar
  23. 23.
    Ma L, Zhao Z, Chen F, Zhang B, Fu L, Liao H (2017) Augmented reality surgical navigation with ultrasound-assisted registration for pedicle screw placement: a pilot study. Int J Comput Assis Radiol Surg 12(12):2205–2215CrossRefGoogle Scholar
  24. 24.
    Microsoft (2018) HoloLens Research mode. Accessed 1 Nov 2018
  25. 25.
    Olson E (2011) Apriltag: a robust and flexible visual fiducial system. In: 2011 IEEE international conference on robotics and automation (ICRA). IEEE, pp 3400–3407Google Scholar
  26. 26.
    Wang J, Olson E (2016) Apriltag 2: efficient and robust fiducial detection. In: IROS, pp 4193–4198Google Scholar
  27. 27.
    Microsoft (2018) Locatable camera. Accessed 5 Nov 2018
  28. 28.
    Garrido-Jurado S, Muñoz-Salinas R, Madrid-Cuevas FJ, Marín-Jiménez MJ (2014) Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognit 47(6):2280–2292CrossRefGoogle Scholar
  29. 29.
    Garrido-Jurado S, Munoz-Salinas R, Madrid-Cuevas FJ, Medina-Carnicer R (2016) Generation of fiducial marker dictionaries using mixed integer linear programming. Pattern Recognit 51:481–491CrossRefGoogle Scholar
  30. 30.
    Romero-Ramirez FJ, Muñoz-Salinas R, Medina-Carnicer R (2018) Speeded up detection of squared fiducial markers. Image Vis Comput 76:38–47CrossRefGoogle Scholar
  31. 31.
    Kalman RE (1960) A new approach to linear filtering and prediction problems. J Basic Eng 82(1):35–45CrossRefGoogle Scholar
  32. 32.
    Bradski G, Kaehler A (2000) Opencv. Dr Dobbs journal of software tools 3Google Scholar
  33. 33.
    Horn BK (1987) Closed-form solution of absolute orientation using unit quaternions. JOSA A 4(4):629–642CrossRefGoogle Scholar
  34. 34.
    Microsoft (2017) Use the HoloLens clicker. Accessed 3 Nov 2018
  35. 35.
    Microsoft (2018) HoloToolkit 2017.4.1.0. Accessed 3 Nov 2018
  36. 36.
    Besl PJ, McKay ND (1992) Method for registration of 3-D shapes. In: Sensor fusion IV: control paradigms and data structures, vol 1611. International society for optics and photonics, pp 586–607Google Scholar
  37. 37.
    Pearson K (1901) LIII. On lines and planes of closest fit to systems of points in space. Lond Edinb Dublin Philos Mag J Sci 2(11):559–572CrossRefGoogle Scholar
  38. 38.
    Schweizer A, Mauler F, Vlachopoulos L, Nagy L, Fürnstahl P (2016) Computer-assisted 3-dimensional reconstructions of scaphoid fractures and nonunions with and without the use of patient-specific guides: early clinical outcomes and postoperative assessments of reconstruction accuracy. J Hand Surg 41(1):59–69CrossRefGoogle Scholar
  39. 39.
    Roner S, Vlachopoulos L, Nagy L, Schweizer A, Fürnstahl P (2017) Accuracy and early clinical outcome of 3-dimensional planned and guided single-cut osteotomies of malunited forearm bones. J Hand Surg 42(12):1031–e1CrossRefGoogle Scholar
  40. 40.
    Walti J, Jost GF, Cattin PC (2014) A new cost-effective approach to pedicular screw placement. In: Workshop on augmented environments for computer-assisted interventions. Springer, pp 90–97Google Scholar
  41. 41.
    Gibby JT, Swenson SA, Cvetko S, Rao R, Javan R (2019) Head-mounted display augmented reality to guide pedicle screw placement utilizing computed tomography. Int J Comput Assis Radiol Surg 14(3):525–535CrossRefGoogle Scholar
  42. 42.
    Vassallo R, Rankin A, Chen EC, Peters TM (2017) Hologram stability evaluation for microsoft hololens. In: Medical imaging 2017: image perception, observer performance, and technology assessment, vol 10136. international society for optics and photonics, p 1013614Google Scholar

Copyright information

© CARS 2019

Authors and Affiliations

  1. 1.Computer Assisted Research and Development Group, Balgrist University HospitalUniversity of ZurichZurichSwitzerland
  2. 2.Laboratory for Orthopaedic BiomechanicsETH ZurichZurichSwitzerland
  3. 3.Orthopaedic Department, Balgrist University HospitalUniversity of ZurichZurichSwitzerland
  4. 4.Department of InformaticsUniversity of ZurichZurichSwitzerland
  5. 5.Department of NeuroinformaticsUniversity of Zurich and ETH ZurichZurichSwitzerland
  6. 6.Radiology Department, Balgrist University HospitalUniversity of ZurichZurichSwitzerland

Personalised recommendations