Abstract
Augmented reality (AR) is widely used in minimally invasive surgery (MIS), since it enhances the surgeon’s perception of spatial relationship by overlaying the invisible structures on the endoscopic images. Depth perception is the key problem in AR visualization. In this paper, we present a video-based AR system for aiding MIS of removing a tumor inside a kidney. We explore several different AR visualization techniques. They are transparent overlay, virtual window, random-dot mask and the ghosting method. We also introduce the depth-aware ghosting method to further enhance the depth perception of virtual structure which has complex spatial geometry. Both simulated and in vivo experiments were carried out to evaluate these AR visualization techniques. The experimental results demonstrate the feasibility of our AR system and AR visualization techniques. Finally, we conclude the characteristics of these AR visualization techniques.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Collins, T., Pizarro, D., Bartoli, A., Canis, M., Bourdel, N.: Computer-assisted laparoscopic myomectomy by augmenting the uterus with pre-operative MRI data. In: ISMAR, pp. 243–248 (2014)
Kim, J.-H., Bartoli, A., Collins, T., Hartley, R.: Tracking by detection for interactive image augmentation in laparoscopy. In: Dawant, B.M., Christensen, G.E., Fitzpatrick, J.M., Rueckert, D. (eds.) WBIR 2012. LNCS, vol. 7359, pp. 246–255. Springer, Heidelberg (2012)
Schneider, A., Pezold, S., Sauer, A., Ebbing, J., Wyler, S., Rosenthal, R., Cattin, P.C.: Augmented reality assisted laparoscopic partial nephrectomy. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014, Part II. LNCS, vol. 8674, pp. 357–364. Springer, Heidelberg (2014)
Puerto-Souza, G.A., Mariottini, G.L.: Toward Long-term and accurate augmented-reality display for minimally-invasive surgery. In: ICRA, pp. 5384–5389 (2013)
Teber, D., Guven, S., Simpfendörfer, T., Baumhauer, M., Güven, E.O., Yencilek, F., Gözen, A.S., Rassweiler, J.: Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? Preliminary in vitro and in vivo results. Eur. Urol. 56(2), 332–338 (2009)
Su, L.M., Vagvolgyi, B.P., Agarwal, R., Reiley, C.E., Taylor, R.H., Hager, G.D.: Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology 73(4), 896–900 (2009)
Kersten-Oertel, M., Jannin, P., Collins, D.L.: The state of the art of visualization in mixed reality image guided surgery. Comput. Med. Imaging Graph. 37(2), 98–112 (2013)
Nicolau, S., Soler, L., Mutter, D., Marescaux, J.: Augmented reality in laparoscopic surgical oncology. Surg. Oncol. 20(3), 189–201 (2011)
Kersten-Oertel, M., Gerard, I., Drouin, S., Mok, K., Sirhan, D., Sinclair, D.S., Collins, D.L.: Augmented reality in neurovascular surgery: feasibility and first uses in the operating room. Int. J. Comput. Assist. Radiol. Surg. 10(11), 1823–1836 (2015)
Sielhorst, T., Bichlmeier, C., Heining, S.M., Navab, N.: Depth perception – a major issue in medical AR: evaluation study by twenty surgeons. In: Larsen, R., Nielsen, M., Sporring, J. (eds.) MICCAI 2006. LNCS, vol. 4190, pp. 364–372. Springer, Heidelberg (2006)
Buchmann, V., Nilsen, T., Billinghurst, M.: Interaction with partially transparent hands and objects. In: AUIC, vol. 40, pp. 17–20 (2005)
Fuchs, H., Livingston, M.A., Raskar, R., Keller, K., State, A., Crawford, J.R., Rademacher, P., Drake, S.H., Meyer, A.A.: Augmented reality visualization for laparoscopic surgery. In: Wells, W.M., Colchester, A.C.F., Delp, S.L. (eds.) MICCAI 1998. LNCS, vol. 1496, pp. 934–943. Springer, Heidelberg (1998)
Otsuki, M., Kuzuoka, H., Milgram, P.: Analysis of depth perception with virtual mask in stereoscopic AR. In: International Conference on Artificial Reality and Telexistence, Eurographics Symposium on Virtual Environments (2015)
Zollmann, S., Kalkofen, D., Mendez, E., Reitmay, G.: Image-based ghostings for single layer occlusions in augmented reality. In: ISMAR, pp. 19–26 (2010)
3D Slicer. http://www.slicer.org
Newcombe, R.A., Lovegrove, S.J., Davison, A.J.: DTAM: dense tracking and mapping in real-time. In: ICCV, pp. 2320–2327 (2011)
Best, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)
Marques, B., Haouchine, N., Plantefeve, R., Cotin, S.: Improving depth perception during surgical augmented reality. In: SIGGRAPH 2015 Posters, p. 24 (2015)
Acknowledgments
This work has been supported by the National High-tech R&D Program (863 Program) of Institute of Automation, Chinese Academy of Sciences (CASIA), grant 2012AA011903 and 2015AA015905.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Wang, R., Geng, Z., Zhang, Z., Pei, R. (2016). Visualization Techniques for Augmented Reality in Endoscopic Surgery. In: Zheng, G., Liao, H., Jannin, P., Cattin, P., Lee, SL. (eds) Medical Imaging and Augmented Reality. MIAR 2016. Lecture Notes in Computer Science(), vol 9805. Springer, Cham. https://doi.org/10.1007/978-3-319-43775-0_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-43775-0_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-43774-3
Online ISBN: 978-3-319-43775-0
eBook Packages: Computer ScienceComputer Science (R0)