On-demand calibration and evaluation for electromagnetically tracked laparoscope in augmented reality visualization

  • Xinyang Liu
  • William Plishker
  • George Zaki
  • Sukryool Kang
  • Timothy D. Kane
  • Raj ShekharEmail author
Original Article



Common camera calibration methods employed in current laparoscopic augmented reality systems require the acquisition of multiple images of an entire checkerboard pattern from various poses. This lengthy procedure prevents performing laparoscope calibration in the operating room (OR). The purpose of this work was to develop a fast calibration method for electromagnetically (EM) tracked laparoscopes, such that the calibration can be performed in the OR on demand.


We designed a mechanical tracking mount to uniquely and snugly position an EM sensor to an appropriate location on a conventional laparoscope. A tool named fCalib was developed to calibrate intrinsic camera parameters, distortion coefficients, and extrinsic parameters (transformation between the scope lens coordinate system and the EM sensor coordinate system) using a single image that shows an arbitrary portion of a special target pattern. For quick evaluation of calibration results in the OR, we integrated a tube phantom with fCalib prototype and overlaid a virtual representation of the tube on the live video scene.


We compared spatial target registration error between the common OpenCV method and the fCalib method in a laboratory setting. In addition, we compared the calibration re-projection error between the EM tracking-based fCalib and the optical tracking-based fCalib in a clinical setting. Our results suggest that the proposed method is comparable to the OpenCV method. However, changing the environment, e.g., inserting or removing surgical tools, might affect re-projection accuracy for the EM tracking-based approach. Computational time of the fCalib method averaged 14.0 s (range 3.5 s–22.7 s).


We developed and validated a prototype for fast calibration and evaluation of EM tracked conventional (forward viewing) laparoscopes. The calibration method achieved acceptable accuracy and was relatively fast and easy to be performed in the OR on demand.


Augmented reality Electromagnetic tracking Camera calibration Laparoscopic procedure Laparoscopic visualization 



The authors would like to thank Dr. Joao P. Barreto and Mr. Rui Melo of Perceive3D, SA, for providing the rdCalib API and the associated calibration target pattern. The authors would also like to thank James McConnaughey for his assistance in 3D printing the mechanical EM tracking mount. This work was supported partially by the National Institutes of Health Grant 1R41CA192504.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Supplementary material

Supplementary material 1 (mov 26589 KB)


  1. 1.
    Feuerstein M, Mussack T, Heining SM, Navab N (2008) Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection. IEEE Trans Med Imaging 27(3):355–369CrossRefPubMedGoogle Scholar
  2. 2.
    Shekhar R, Dandekar O, Bhat V, Philip M, Lei P, Godinez C, Sutton E, George I, Kavic S, Mezrich R, Park A (2010) Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography. Surg Endosc 24(8):1976–1985CrossRefPubMedGoogle Scholar
  3. 3.
    Leven J, Burschka D, Kumar R, Zhang G, Blumenkranz S, Dai XD, Awad M, Hager GD, Marohn M, Choti M, Hasser C, Taylor RH (2005) DaVinci canvas: a telerobotic surgical system with integrated, robot-assisted, laparoscopic ultrasound capability. Proc Med Image Comput Comput Assist Interv 8(Pt 1):811–818Google Scholar
  4. 4.
    Cheung CL, Wedlake C, Moore J, Pautler SE, Peters TM (2010) Fused video and ultrasound images for minimally invasive partial nephrectomy: a phantom study. Proc Med Image Comput Comput Assist Interv 13(Pt 3):408–415Google Scholar
  5. 5.
    Kang X, Azizian M, Wilson E, Wu K, Martin AD, Kane TD, Peters CA, Cleary K, Shekhar R (2014) Stereoscopic augmented reality for laparoscopic surgery. Surg Endosc 28(7):2227–2235CrossRefPubMedGoogle Scholar
  6. 6.
    Shiu Y, Ahmad S (1989) Calibration of wrist-mounted robotic sensors by solving homogeneous transform equations of the form ax = xb. IEEE Trans Robot Autom 5(1):16–29CrossRefGoogle Scholar
  7. 7.
    Bouguet JY (2015) Camera calibration with OpenCV. Accessed 20 July 2015
  8. 8.
    Heikkila J, Silven O (1997) A four-step camera calibration procedure with implicit image correction. In: Proceedings of IEEE computer society conference computer vision pattern recognition. pp 1106–1112Google Scholar
  9. 9.
    Zhang Z (1999) Flexible camera calibration by viewing a plane from unknown orientations. In: Proceedings of international conference on computer vision. pp 666–673Google Scholar
  10. 10.
    Shekhar R, Liu X, Wilson E, Kang S, Petrosyan M, Kane TD (2015) Stereoscopic augmented reality visualization for laparoscopic surgery—initial clinical experience. In: Proceedings of annual meeting of society of American gastrointestinal and endoscopic surgeonsGoogle Scholar
  11. 11.
    Barreto JP, Roquette J, Sturm P, Fonseca F (2009) Automatic camera calibration applied to medical endoscopy. In: Proceedings of british machine vision conferenceGoogle Scholar
  12. 12.
    Melo R, Barreto JP, Falcão G (2012) A new solution for camera calibration and real-time image distortion correction in medical endoscopy-initial technical evaluation. IEEE Trans Biomed Eng 59(3):634–644CrossRefPubMedGoogle Scholar
  13. 13.
    Liu X, Su H, Kang S, Kane TD, Shekhar R (2015) Application of single-image camera calibration for ultrasound augmented laparoscopic visualization. In: Proceedings of SPIE medical, imaging. p 94151TGoogle Scholar
  14. 14.
    Feuerstein M, Reichl T, Vogel J, Traub J, Navab N (2009) Magneto-optical tracking of flexible laparoscopic ultrasound: model-based online detection and correction of magnetic tracking errors. IEEE Trans Med Imaging 28(6):951–967CrossRefPubMedGoogle Scholar
  15. 15.
    Yaniv Z, Wilson E, Lindisch D, Cleary K (2009) Electromagnetic tracking in the clinical environment. Med Phys 36(3):876–892CrossRefPubMedPubMedCentralGoogle Scholar
  16. 16.
    Franz AM, März K, Hummel J, Birkfellner W, Bendl R, Delorme S, Schlemmer HP, Meinzer HP, Maier-Hein L (2012) Electromagnetic tracking for US-guided interventions: standardized assessment of a new compact field generator. Int J Comput Assist Radiol Surg 7(6):813–818CrossRefPubMedGoogle Scholar
  17. 17.
    Maier-Hein L, Franz AM, Birkfellner W, Hummel J, Gergel I, Wegner I, Meinzer HP (2012) Standardized assessment of new electromagnetic field generators in an interventional radiology setting. Med Phys 39(6):3424–3434CrossRefPubMedGoogle Scholar
  18. 18.
    Moore JT, Wiles AD, Wedlake C, Bainbridge D, Kiaii B, Luisa Trejos A, Patel R, Peters TM (2010) Integration of trans-esophageal echocardiography with magnetic tracking technology for cardiac interventions. In: Proceedings of SPIE medical, imaging. p 76252YGoogle Scholar
  19. 19.
    Liu X, Kang S, Wilson E, Peters CA, Kane TD, Shekhar R (2014) Evaluation of electromagnetic tracking for stereoscopic augmented reality laparoscopic visualization. Proc MICCAI Workshop Clin Image Based Proced Transl Res Med Imaging 8361:84–91Google Scholar
  20. 20.
    Atcheson B, Heide F, Heidrich W (2010) CALTag: High precision fiducial markers for camera calibration. In: 15th International workshop on vision, modeling and visualization. SiegenGoogle Scholar
  21. 21.
    Johnson MP (2003) Exploiting quaternions to support expressive interactive character motion. Dissertation, MITGoogle Scholar
  22. 22.
    Hartley RI, Sturm P (1997) Triangulation. Comput Vis Image Underst 68(2):146–157CrossRefGoogle Scholar

Copyright information

© CARS 2016

Authors and Affiliations

  • Xinyang Liu
    • 1
  • William Plishker
    • 2
  • George Zaki
    • 2
  • Sukryool Kang
    • 1
  • Timothy D. Kane
    • 1
  • Raj Shekhar
    • 1
    Email author
  1. 1.Sheikh Zayed Institute for Pediatric Surgical InnovationChildren’s National Health SystemWashingtonUSA
  2. 2.IGI Technologies, Inc.College ParkUSA

Personalised recommendations