Abstract
Instrument localization and tracking is an important challenge for advanced computer assisted techniques in minimally invasive surgery and image-based solutions to instrument localization can provide a non-invasive, low cost solution. In this study, we present a novel algorithm capable of recovering the 3D pose of laparoscopic surgical instruments combining constraints from a classification algorithm, multiple point features, stereo views (when available) and a linear motion model to robustly track the tool in surgical videos. We demonstrate the improved robustness and performance of our algorithm with optically tracked ground truth and additionally qualitatively demonstrate its performance on in vivo images.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Azimian, H., Patel, R., Naish, M.: On constrained manipulation in robotics-assisted minimally invasive surgery. In: 3rd IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics, pp. 650–655 (2010)
van der Putten Westebring, E.P., Goossens, R.H.M., Jakimowicz, J.J., Dankelman, J.: Haptics in minimally invasive surgery a review. Minimally Invasive Therapy & Allied Technologies 17(1), 3–16 (2008)
Speidel, S., Sudra, G., Senemaud, J., Drentschew, M., Müller-Stich, B.P., Gutt, C., Dillmann, R.: Recognition of risk situations based on endoscopic instrument tracking and knowledge based situation modeling. In: Medical Imaging 2008: Visualization, Image-Guided Procedures, and Modeling, vol. 6918 (2008)
Chmarra, M.K., Grimbergen, C.A., Dankelman, J.: Systems for tracking minimally invasive surgical instruments. Minimally Invasive Therapy & Allied Technologies 16(6), 328–340 (2007)
Mirota, D.J., Ishii, M., Hager, G.D.: Vision-based navigation in image-guided interventions. Annual Review of Biomedical Engineering 13, 297–319 (2011)
Stoyanov, D.: Surgical vision. Annals of Biomedical Engineering 40(2), 332–345 (2012)
Uecker, D.R., Lee, C., Wang, Y.F., Wang, Y.: Automated instrument tracking in robotically assisted laparoscopic surgery. Journal of Image Guided Surgery 1(6), 308–325 (1995) PMID: 9080352
Sznitman, R., Ali, K., Richa, R., Taylor, R., Hager, G., Fua, P.: Data-driven visual tracking in retinal microsurgery. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012, Part II. LNCS, vol. 7511, pp. 568–575. Springer, Heidelberg (2012)
Richa, R., Balicki, M., Meisner, E., Sznitman, R., Taylor, R., Hager, G.: Visual tracking of surgical tools for proximity detection in retinal surgery. In: Taylor, R.H., Yang, G.-Z. (eds.) IPCAI 2011. LNCS, vol. 6689, pp. 55–66. Springer, Heidelberg (2011)
Zhao, T., Zhao, W., Halabe, D.J., Hoffman, B.D., Nowlin, W.C.: Fiducial marker design and detection for locating surgical instrument in images. Patent US 068 395, 07 08 (2010)
Voros, S., Long, J., Cinquin, P.: Automatic detection of instruments in laparoscopic images: A first step towards high-level command of robotic endoscopic holders. The International Journal of Robotics Research 26(11-12), 1173–1190 (2007)
Reiter, A., Allen, P.K., Zhao, T.: Appearance learning for 3d tracking of robotic surgical tools. The International Journal of Robotics Research (2013)
Austin, R., Allen, P.K., Tao, Z.: Articulated surgical tool detection using virtually-rendered templates. Computer Assisted Radiology and Surgery (2012)
Allan, M., Ourselin, S., Thompson, S., Hawkes, D.J., Kelly, J., Stoyanov, D.: Toward detection and localization of instruments in minimally invasive surgery. IEEE Transactions on Biomedical Engineering 60(4), 1050–1058 (2013)
Prisacariu, V.A., Reid, I.D.: PWP3D: Real-Time segmentation and tracking of 3D objects. Int. J. Computer Vision 98(3), 335–354 (2012)
Maier-Hein, L., Mountney, P., Bartoli, A., Elhawary, H., Elson, D., Groch, A., Kolb, A., Rodrigues, M., Sorger, J., Speidel, S., Stoyanov, D.: Optical techniques for 3d surface reconstruction in computer-assisted laparoscopic surgery. Medical Image Analysis 17(8), 974–996 (2013)
Cremers, D., Rousson, M., Deriche, R.: A review of statistical approaches to level set segmentation: Integrating color, texture, motion and shape. Int. J. Comput. Vision 72(2), 195–215 (2007)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)
Prince, S.: Computer Vision: Models Learning and Inference. Cambridge University Press (2012)
Tsai, R., Lenz, R.: A new technique for fully autonomous and efficient 3d robotics hand/eye calibration. IEEE Transactions on Robotics and Automation 5(3), 345–358 (1989)
Pezzementi, Z., Voros, S., Hager, G.D.: Articulated object tracking by rendering consistent appearance parts. In: IEEE International Conference on Robotics and Automation, ICRA 2009, pp. 3940–3947 (May 2009)
Prisacariu, V.A., Reid, I.: Nonlinear shape manifolds as shape priors in level set segmentation and tracking. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2011, pp. 2185–2192. IEEE Computer Society, Washington, DC (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Allan, M. et al. (2014). 2D-3D Pose Tracking of Rigid Instruments in Minimally Invasive Surgery. In: Stoyanov, D., Collins, D.L., Sakuma, I., Abolmaesumi, P., Jannin, P. (eds) Information Processing in Computer-Assisted Interventions. IPCAI 2014. Lecture Notes in Computer Science, vol 8498. Springer, Cham. https://doi.org/10.1007/978-3-319-07521-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-07521-1_1
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07520-4
Online ISBN: 978-3-319-07521-1
eBook Packages: Computer ScienceComputer Science (R0)