Robotic laser osteotomy through penscriptive structured light visual servoing

  • Jamil JivrajEmail author
  • Ryan Deorajh
  • Phillips Lai
  • Chaoliang Chen
  • Nhu Nguyen
  • Joel Ramjist
  • Victor X. D. Yang
Original Article



Planning osteotomies is a task that surgeons do as part of standard surgical workflow. This task, however, becomes more difficult and less intuitive when a robot is tasked with performing the osteotomy. In this study, we aim to provide a new method for surgeons to allow for highly intuitive trajectory planning, similar to the way an attending surgeon would instruct a junior.


Planning an osteotomy, especially during a craniotomy, is performed intraoperatively using a sterile surgical pen or pencil directly on the exposed bone surface. This paper presents a new method for generating osteotomy trajectories for a multi-DOF robotic manipulator using the same method and relaying the penscribed cut path to the manipulator as a three-dimensional trajectory. The penscribed cut path is acquired using structured light imaging, and detection, segmentation, optimization and orientation generation of the Cartesian trajectory are done autonomously after minimal user input.


A 7-DOF manipulator (KUKA IIWA) is able to follow fully penscribed trajectories with sub-millimeter accuracy in the target plane and perpendicular to it (0.46 mm and 0.36 mm absolute mean error, respectively).


The robot is able to precisely follow cut paths drawn by the surgeon directly onto the exposed boney surface of the skull. We demonstrate through this study that current surgical workflow does not have to be drastically modified to introduce robotic technology in the operating room. We show that it is possible to guide a robot to perform an osteotomy in much the same way a senior surgeon would show a trainee by using a simple surgical pen or pencil.


Laser osteotomy Trajectory generation Drilling Penscription Planning Structured light Image processing Feature detection 



Thank you to Jillian Cardinell for the graphics help.


Funding for this research was provided by The Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery Grant: “Optical Coherence Tomography, Optical Topographical Imaging and Fluorescence Guided Surgical Laser Ablation”—Grant Number RGPIN/6263-2014.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Humans and animal rights

This article does not contain and studies with human or animal participants performed by any of the authors. All ex-vivo animal parts were obtained in accordance with the research ethics guidelines of Ryerson University. All patient data were anonymized in accordance with Sunnybrook Health Sciences Centre policy. Informed consent was obtained from all individual participants included in the study.


  1. 1.
    Giraud JY, Villemin S, Darmana R, Cahuzac JP, Autefage A, Morucci JP (1991) Bone cutting. Clin Phys Physiol Meas 12(1):1CrossRefPubMedGoogle Scholar
  2. 2.
    AOT AG. Advanced osteotomy tools.
  3. 3.
    Vadera S, Chan A, Low T, Gill A, Morenkova A, Phiellip NM, Hermanowicz N, Hsu FP (2017) Frameless stereotactic robot-assisted subthalamic nucleus deep brain stimulation: case report. World Neurosurg 97:762-e11CrossRefGoogle Scholar
  4. 4.
    Giger A, Jud C, Cattin P (2017) Respiratory motion compensation for the robot-guided laser osteotome. Int J Comput Assist Radiol Surg 12(10):1751–1762CrossRefPubMedGoogle Scholar
  5. 5.
    Burgner J (2010) Robot assisted laser osteotomy. KIT Scientific Publishing, KarlsruheGoogle Scholar
  6. 6.
    Baek KW, Deibel W, Martinov D, Griessen M, Bruno A, Zeilhofer HF, Cattin P, Juergens P (2015) Clinical applicability of robot-guided contact-free laser osteotomy in cranio-maxillo-facial surgery: in-vitro simulation and in-vivo surgery in minipig mandibles. J Oral Maxillofac Surg 53(10):976–981CrossRefGoogle Scholar
  7. 7.
    Deibel W, Schneider A, Augello M, Bruno AE, Juergens P, Cattin P (2015) A compact, efficient and light weight laser head for CARLO®: integration, performance and benefits. In: Novel optical systems design and optimization XVIII, vol 9579, International Society for Optics and PhotonicsGoogle Scholar
  8. 8.
    Mönnich H, Stein D, Raczkowsky J, Wörn H (2010) Results of CO2 robotic laser oseotomy in surgery with motion compensation. In: Photonic therapeutics and diagnostics VI, vol 7548, International Society for Optics and PhotonicsGoogle Scholar
  9. 9.
    Xu D, andLinkun Wang ZJ, Tan M (2004) Features extraction for structured light image of welding seam with arc and splash disturbance. In: 8th International conference on control, automation, robotics and vision, Kunming, China, pp 1559–1563Google Scholar
  10. 10.
    Xu D, Wang L, Tan M (2004) Image processing and visual control method for arc welding robot. In: International conference on robotics and biomimetics, Shenyang, pp 727–732Google Scholar
  11. 11.
    Xu D, Tan M, Li Y (2006) Visual control system for robotic welding. In: Industrial robotics theory modelling control, Pro Literatur Verlag, AustriaGoogle Scholar
  12. 12.
    Xu D, Tan M, Zhao X, Tu Z (2004) Seam tracking and visual control for robotic arc welding based on structured light stereovision. Int J Autom Comput 1:63–75CrossRefGoogle Scholar
  13. 13.
    Yu Z, He Y, Xu Y, Chen H (2018) Vision-based deviation extraction for three-dimensional control in robotic welding with steel sheet. Int J Adv Manuf Technol 95(9–12):4449–4458CrossRefGoogle Scholar
  14. 14.
    Ye Z, Fang G, Chen S, Zou JJ (2012) Passive vision based seam tracking system for pulse-MAG welding. Int J Adv Manuf Technol 67:1987–1996CrossRefGoogle Scholar
  15. 15.
    Xu Y, Fang G, Chen S, Zou JJ, Ye Z (2014) Real-time image processing for vision-based weld seam tracking in robotic GMAW. Int J Adv Manuf Technol 73(9):1413–1425CrossRefGoogle Scholar
  16. 16.
    Guha D, Gupta S, Fehlings MG, Yang V (2017) Optical topographic imaging for spinal intraoperative three-dimensional navigation in minimally invasive approaches: initial preclinical experience. Spine J 17(10):S254CrossRefGoogle Scholar
  17. 17.
    Sciavicco L, Siciliano B (2000) Modelling and control of robot manipulators, 2nd edn. Springer, BerlinCrossRefGoogle Scholar
  18. 18.
    Boomgaard RVD, Balen RV (1992) Computer vision, methods for fast morphological image transforms using bitmapped images. Gr Image Process Gr Models Image Process 54(3):254–258Google Scholar
  19. 19.
    Sobel I, Feldman G (1968) A \(3\times 3\) isotropic gradient operator for image processing. In: A talk at the Stanford artificial project, pp 271–272Google Scholar
  20. 20.
    Zill D, Cullen M (2006) Advanced engineering mathematics, 3rd edn. Jones & Bartlett Learning, BurlingtonGoogle Scholar
  21. 21.
    Jolliffe I (2002) Principal component analysis, series: Springer series in statistics, 2nd edn. Springer, New YorkGoogle Scholar
  22. 22.
    Wiles A, Thompson D, Frantz D (2004) Accuracy assessment and interpretation for optical tracking systems. In: Proceedings of SPIE medical imaging 2004: visualization, image-guided procedures and display (SPIE), vol 5367, pp 421–432Google Scholar
  23. 23.
    Elfring R, de la Fuente M, Radermarcher K (2010) Assessment of optical localizer accuracy for computer aided surgery systems. Comput Aided Surg 15(1–3):1–12CrossRefPubMedGoogle Scholar
  24. 24.
    Dillon NP (2014) Preliminary testing of a compact bone-attached robot for otologic surgery. In: Medical imaging 2014: image-guided procedures, robotic interventions, and modeling, vol 9036, International Society for Optics and PhotonicsGoogle Scholar
  25. 25.
    Dillion NP, Balachandran R, Fitzpatrick JM, Siebold MA, Labadie RF, Wanna GB, Withrow TJ, Webster RJ (2015) A compact, bone-attached robot for mastoidectomy. J Med Devices 9(3):031003CrossRefGoogle Scholar
  26. 26.
    Gerber N, Bell B, Gavaghan K, Weisstanner C, Caversaccio M, Weber S (2014) Surgical planning tool for robotically assisted hearing aid implantation. Int J Comput Assist Radiol Surg 9(1):11–20CrossRefPubMedGoogle Scholar
  27. 27.
    Zagzoog N, Yang VX (2018) State of robotic mastoidectomy: literature review. World Neurosurg 116:347–351CrossRefPubMedGoogle Scholar

Copyright information

© CARS 2019

Authors and Affiliations

  • Jamil Jivraj
    • 1
    Email author
  • Ryan Deorajh
    • 1
  • Phillips Lai
    • 1
  • Chaoliang Chen
    • 1
  • Nhu Nguyen
    • 1
  • Joel Ramjist
    • 1
  • Victor X. D. Yang
    • 1
    • 2
  1. 1.Biophotonics & Bioengineering Laboratory, Department of Electrical & Computer EngineeringRyerson UniversityTorontoCanada
  2. 2.Division of Neurosurgery, Sunnybrook Health Sciences CentreTorontoCanada

Personalised recommendations