System Designs for Augmented Reality Based Ablation Probe Tracking

  • Hao Bo Yu
  • Harvey HoEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10749)


In this paper we present two Augmented Reality (AR) systems and associated algorithms to track and visualise a surgical ablation probe. The first system is based on the Kinect sensor while the second system makes use of a stereo-vision camera (OvrVision Pro) and a Head Mounted Display (HMD) device. Both systems utilise the fiducial markers on a custom-built rig attached to the ablation probe. We applied the first AR system to the navigation of a virtual liver, and the second AR system to the prediction of the 3D position of probe tip. The predication error for the tip was about 5–10 mm, with a computational speed of 10 FPS. In conclusion, two AR systems were designed and implemented with potential for further improvements to be applied in an actual clinical context.


Augmented reality Ablation probe Computer vision Virtual model 


  1. 1.
    Jolesz, F. (ed.): Intraoperative Imaging and Image-Guided Therapy. Springer Science & Business Media, New York (2014). Google Scholar
  2. 2.
    Ward, T., Goldman, R., Weintraub, J.: Electromagnetic navigation with multimodality image fusion for image-guided percutaneous interventions. Tech. Vasc. Intervent. Radiol. 16(3), 177–181 (2013)CrossRefGoogle Scholar
  3. 3.
    Clifford, M., Banovac, F., Levy, E., Cleary, K.: Assessment of hepatic motion secondary to respiration for computer assisted interventions. Comput. Aided Surg. 7(5), 291–299 (2002)CrossRefGoogle Scholar
  4. 4.
    Abolhassani, N., Patel, R., Moallem, M.: Needle insertion into soft tissue: a survey. Med. Eng. Phy. 29(4), 413–431 (2007)CrossRefGoogle Scholar
  5. 5.
    Loser, M.H., Navab, N.: A new robotic system for visually controlled percutaneous interventions under CT fluoroscopy. In: Delp, S.L., DiGoia, A.M., Jaramaz, B. (eds.) MICCAI 2000. LNCS, vol. 1935, pp. 887–896. Springer, Heidelberg (2000). CrossRefGoogle Scholar
  6. 6.
    Srikrishna, B.K., Musti, U., Heikkil, J.: Geometry based exhaustive line correspondence determination. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 4341–4348. IEEE (2016)Google Scholar
  7. 7.
    Patriciu, A., Awad, M., Solomon, S.B., Choti, M., Mazilu, D., Kavoussi, L., Stoianovici, D.: Robotic assisted radio-frequency ablation of liver tumors – randomized patient study. In: Duncan, J.S., Gerig, G. (eds.) MICCAI 2005. LNCS, vol. 3750, pp. 526–533. Springer, Heidelberg (2005). CrossRefGoogle Scholar
  8. 8.
    Okuma, T., Sakaue, K., Takemura, H., Yokoya, N.: Real-time camera parameter estimation from images for a mixed reality system. In: Proceedings of the 2000 15th International Conference on Pattern Recognition, vol. 4, pp. 482–486. IEEE (2000)Google Scholar
  9. 9.
    Nicolau, S., Pennec, X., Soler, L., et al.: An augmented reality system for liver thermal ablation: design and evaluation on clinical cases. Med. Image Anal. 13, 494–506 (2009)CrossRefGoogle Scholar
  10. 10.
    Rosenthal, M., State, A., Lee, J., Hirota, G., Ackerman, J., Keller, K., Fuchs, H.: Augmented reality guidance for needle biopsies: an initial randomized, controlled trial in phantoms. Med. Image Anal. 6(3), 313–320 (2002)CrossRefzbMATHGoogle Scholar
  11. 11.
    Rusu, R.B., Cousins, S.: 3D is here: point cloud library (PCL). In: 2011 IEEE International Conference on Robotics and automation (ICRA), pp. 1–4. IEEE (2011)Google Scholar
  12. 12.
    Kramer, J., Burrus, N., Echtler, F., Daniel, H.C., Parker, M.: Hacking the Kinect, vol. 268. Apress, New York (2012)CrossRefGoogle Scholar
  13. 13.
    Forsyth, D., Ponce, J.: Computer Vision: A Modern Approach. Pearson, New York (2015)Google Scholar
  14. 14.
    Ren, H., Liu, W., Lim, A.: Marker-based surgical instrument tracking using dual kinect sensors. IEEE Trans. Autom. Sci. Eng. 11(3), 921–924 (2014)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Auckland Bioengineering InstituteThe University of AucklandAucklandNew Zealand

Personalised recommendations