On-demand calibration and evaluation for electromagnetically tracked laparoscope in augmented reality visualization
- 363 Downloads
Common camera calibration methods employed in current laparoscopic augmented reality systems require the acquisition of multiple images of an entire checkerboard pattern from various poses. This lengthy procedure prevents performing laparoscope calibration in the operating room (OR). The purpose of this work was to develop a fast calibration method for electromagnetically (EM) tracked laparoscopes, such that the calibration can be performed in the OR on demand.
We designed a mechanical tracking mount to uniquely and snugly position an EM sensor to an appropriate location on a conventional laparoscope. A tool named fCalib was developed to calibrate intrinsic camera parameters, distortion coefficients, and extrinsic parameters (transformation between the scope lens coordinate system and the EM sensor coordinate system) using a single image that shows an arbitrary portion of a special target pattern. For quick evaluation of calibration results in the OR, we integrated a tube phantom with fCalib prototype and overlaid a virtual representation of the tube on the live video scene.
We compared spatial target registration error between the common OpenCV method and the fCalib method in a laboratory setting. In addition, we compared the calibration re-projection error between the EM tracking-based fCalib and the optical tracking-based fCalib in a clinical setting. Our results suggest that the proposed method is comparable to the OpenCV method. However, changing the environment, e.g., inserting or removing surgical tools, might affect re-projection accuracy for the EM tracking-based approach. Computational time of the fCalib method averaged 14.0 s (range 3.5 s–22.7 s).
We developed and validated a prototype for fast calibration and evaluation of EM tracked conventional (forward viewing) laparoscopes. The calibration method achieved acceptable accuracy and was relatively fast and easy to be performed in the OR on demand.
KeywordsAugmented reality Electromagnetic tracking Camera calibration Laparoscopic procedure Laparoscopic visualization
The authors would like to thank Dr. Joao P. Barreto and Mr. Rui Melo of Perceive3D, SA, for providing the rdCalib API and the associated calibration target pattern. The authors would also like to thank James McConnaughey for his assistance in 3D printing the mechanical EM tracking mount. This work was supported partially by the National Institutes of Health Grant 1R41CA192504.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
Informed consent was obtained from all individual participants included in the study.
Supplementary material 1 (mov 26589 KB)
- 3.Leven J, Burschka D, Kumar R, Zhang G, Blumenkranz S, Dai XD, Awad M, Hager GD, Marohn M, Choti M, Hasser C, Taylor RH (2005) DaVinci canvas: a telerobotic surgical system with integrated, robot-assisted, laparoscopic ultrasound capability. Proc Med Image Comput Comput Assist Interv 8(Pt 1):811–818Google Scholar
- 4.Cheung CL, Wedlake C, Moore J, Pautler SE, Peters TM (2010) Fused video and ultrasound images for minimally invasive partial nephrectomy: a phantom study. Proc Med Image Comput Comput Assist Interv 13(Pt 3):408–415Google Scholar
- 7.Bouguet JY (2015) Camera calibration with OpenCV. http://docs.opencv.org/doc/tutorials/calib3d/camera_calibration/camera_calibration.html. Accessed 20 July 2015
- 8.Heikkila J, Silven O (1997) A four-step camera calibration procedure with implicit image correction. In: Proceedings of IEEE computer society conference computer vision pattern recognition. pp 1106–1112Google Scholar
- 9.Zhang Z (1999) Flexible camera calibration by viewing a plane from unknown orientations. In: Proceedings of international conference on computer vision. pp 666–673Google Scholar
- 10.Shekhar R, Liu X, Wilson E, Kang S, Petrosyan M, Kane TD (2015) Stereoscopic augmented reality visualization for laparoscopic surgery—initial clinical experience. In: Proceedings of annual meeting of society of American gastrointestinal and endoscopic surgeonsGoogle Scholar
- 11.Barreto JP, Roquette J, Sturm P, Fonseca F (2009) Automatic camera calibration applied to medical endoscopy. In: Proceedings of british machine vision conferenceGoogle Scholar
- 13.Liu X, Su H, Kang S, Kane TD, Shekhar R (2015) Application of single-image camera calibration for ultrasound augmented laparoscopic visualization. In: Proceedings of SPIE medical, imaging. p 94151TGoogle Scholar
- 18.Moore JT, Wiles AD, Wedlake C, Bainbridge D, Kiaii B, Luisa Trejos A, Patel R, Peters TM (2010) Integration of trans-esophageal echocardiography with magnetic tracking technology for cardiac interventions. In: Proceedings of SPIE medical, imaging. p 76252YGoogle Scholar
- 19.Liu X, Kang S, Wilson E, Peters CA, Kane TD, Shekhar R (2014) Evaluation of electromagnetic tracking for stereoscopic augmented reality laparoscopic visualization. Proc MICCAI Workshop Clin Image Based Proced Transl Res Med Imaging 8361:84–91Google Scholar
- 20.Atcheson B, Heide F, Heidrich W (2010) CALTag: High precision fiducial markers for camera calibration. In: 15th International workshop on vision, modeling and visualization. SiegenGoogle Scholar
- 21.Johnson MP (2003) Exploiting quaternions to support expressive interactive character motion. Dissertation, MITGoogle Scholar