Complexation of Optical, Ultrasound, and X-ray Images in Intraoperative Navigation Systems

  • I. L. EgoshinaEmail author


A comparative analysis is performed of modern means of complexing images in laparoscopic surgery. A way of increasing the accuracy of complexing multispectral images using three-dimensional means, and of solving the problem of anatomical deformation via the application of augmented reality based on biophotonics, is proposed. The coincidence between additional information and a laparoscopic image is one advantage of approaches to augmented reality that are based on biophotonics, since the data originate from the same endoscope. Calibration of the camera is in this case not required, since the lens is naturally distorted in both augmented reality and the image.



This work was supported by the RF Ministry of Education and Science, project no. RFMEFI577170254, “An Intraoperational Navigation System for Minimally Invasive Surgery with the Support of Augmented Reality Technology Based on Virtual 3D Models of Organs, Obtained Using the Results from CT Diagnostics.”


  1. 1.
    Hayashi, Y., Misawa, K., Hawkes, D.J., et al., Int. J. Comput. Assisted Radiol. Surg., 2016, vol. 11, p. 837.CrossRefGoogle Scholar
  2. 2.
    Puerto-Souza, G., Cadeddu, J., Mariottini, G.L., et al., IEEE Trans. Biomed. Eng., 2014, vol. 61, p. 2609.CrossRefGoogle Scholar
  3. 3.
    Pratt, P., Mayer, E., Vale, J., Cohen, D., et al., J. Rob. Surg., 2012, vol. 6, p. 23.CrossRefGoogle Scholar
  4. 4.
    Conrad, C., Fusaglia, M., Peterhans, M., et al., J. Am. Coll. Surg., 2016, vol. 223, p. e31.CrossRefGoogle Scholar
  5. 5.
    Ieiri, S., Uemura, M., Konishi, K., et al., Pediatr. Surg. Int., 2012, vol. 28, p. 341.CrossRefGoogle Scholar
  6. 6.
    Souzaki, R., Ieiri, S., Uemura, M., et al., J. Pediatr. Surg., 2013, vol. 48, p. 2479.CrossRefGoogle Scholar
  7. 7.
    Tsutsumi, N., Tomikawa, M., Uemura, M., et al., Surg. Endoscopy, 2013, vol. 27, p. 2178.CrossRefGoogle Scholar
  8. 8.
    Cheng, A., Kang, J.U., Taylor, R.H., et al., J. Biomed. Opt., 2013, vol. 18, no. 6, p. 066013.ADSCrossRefGoogle Scholar
  9. 9.
    Cheng, A., Guo, X., Kang, H.J., et al., Proc. SPIE, 2014, vol. 8943, p. 89435J.CrossRefGoogle Scholar
  10. 10.
    Marques, B., Roy, F., Haouchine, N., Jeanvoine, E., et al., Proc. 17th Int. Conf. on E-health Networking, Application & Services, Boston, 2015, p. 22.Google Scholar
  11. 11.
    Lin, B., Sun, Y., Qian, X., et al., Int. J. Med. Rob. Comput. Assisted Surg., 2016, vol. 12, no. 2, p, 158.Google Scholar
  12. 12.
    Bergen, T. and Wittenberg, T., IEEE J. Biomed. Health Inf., 2016, vol. 20, p. 304.CrossRefGoogle Scholar
  13. 13.
    Geng, J. and Xie, J., IEEE Sens. J., 2014, vol. 14, p. 945.ADSCrossRefGoogle Scholar
  14. 14.
    Collins, T., Pizarro, D., Bartoli, A., et al., in Augmented Reality Environments for Medical Imaging and Computer-Assisted Interventions, Liao, H., Linte, C.A., Masamune, K., Peters, T., and Guayan, Z., Eds., Springer, 2013, p. 162.Google Scholar
  15. 15.
    Chang, P.L., Handa, A., Davison, A.J., et al., Proc. 5th Int. Conf. on Information Processing in Computer-Assisted Interventions (Fukuoka, 2014), Springer, 2014, p. 11.Google Scholar
  16. 16.
    Malti, A. and Bartoli, A., IEEE Trans. Biomed. Eng., 2014, vol. 61, p. 1684.CrossRefGoogle Scholar
  17. 17.
    Martinez Herrera, S.E., Malti, A., Morel, O., et al., Proc. 10th IEEE Int. Symp. on Biomedical Imaging, San Francisco, 2013, p. 1412.Google Scholar
  18. 18.
    Edgcumbe, P., Pratt, P., Yang, G.Z., et al., Med. Image Anal., 2015, vol. 25, p. 95.CrossRefGoogle Scholar
  19. 19.
    Furukawa, R., Masutani, R., Miyazaki, D., et al., Proc. 37th Annual Int. Conf. of the IEEE Engineering in Medicine and Biology Society, Milano, 2015, p. 7937.Google Scholar
  20. 20.
    Lin, J., Clancy, N.T., and Elson, D.S., Int. J. Comput. Assisted Radiol. Surg., 2015, vol. 10, p. 1941.CrossRefGoogle Scholar
  21. 21.
    Fusaglia, M., Hess, H., Schwalbe, M., et al., Int. J. Comput. Assisted Radiol. Surg., 2015, vol. 11, p. 1499.CrossRefGoogle Scholar
  22. 22.
    Maier-Hein, L., Groch, A., Bartoli, A., et al., IEEE Trans. Med. Imaging, 2014, vol. 33, p. 1913.CrossRefGoogle Scholar
  23. 23.
    Thompson, S., Totz, J., Song, Y., et al., Proc. SPIE, 2015, vol. 9415, p. 941 509.CrossRefGoogle Scholar
  24. 24.
    Collins, T., Pizarro, D., Bartoli, A., et al., Proc. IEEE Int. Symp. on Mixed and Augmented Reality, Munich, 2014, p. 243.Google Scholar
  25. 25.
    Dos Santos, T.R., Seitel, A., Kilgus, T., et al., Med. Image Anal., 2014, vol. 18, p. 1101.CrossRefGoogle Scholar
  26. 26.
    Chen, E.C., McLeod, A.J., Baxter, J.S., et al., Int. J. Comput. Assisted Radiol. Surg., 2015, vol. 10, no. 6, p. 867.CrossRefGoogle Scholar
  27. 27.
    Suwelack, S., Rohl, S., Bodenstedt, S., et al., Med. Phys., 2014, vol. 41, p. 111 901.Google Scholar
  28. 28. Scholar
  29. 29.
    Wang, D. and Tewfik, A.H., IEEE Trans. Med. Imaging, 2012, vol. 31, p. 924.CrossRefGoogle Scholar
  30. 30.
    Rohl, S., Bodenstedt, S., Suwelack, S., et al., Med. Phys., 2012, vol. 39, p. 1632.Google Scholar
  31. 31.
    Rueckert, D. and Schnabel, J.A., in Biomedical Image Processing, Deserno, T.M., Ed., Springer, 2011, p. 131.Google Scholar
  32. 32.
    Sotiras, A., Davatzikos, C., and Paragios, N., IEEE Trans. Med. Imaging, 2013, vol. 32, p. 1153.CrossRefGoogle Scholar
  33. 33.
    Mountney, P., Fallert, J., Nicolau, S.A., et al., Proc. 17th Int. Conf. on Medical Image Computing and Computer-Assisted Intervention (Boston, 2014), Springer, 2014, p. 423.Google Scholar
  34. 34.
    Ukimura, O., Nakamoto, M., Sato, Y., et al., in New Technologies in Urology, Dasgupta, P., Fitzpatrick, J.M., Kirby, R., and Gill, I.S., Eds., Springer, 2010, p. 215.Google Scholar
  35. 35.
    Song, Y., Totz, J., Thompson, S., et al., Int. J. Comput. Assisted Radiol. Surg., 2015, vol. 10, p. 1951.CrossRefGoogle Scholar
  36. 36.
    Schwabenland, I., Sunderbrink, D., Nollert, G., et al., in Imaging and Visualization in the Modern Operating Room: A Comprehensive Guide for Physicians, Fong, Y., Giulianotti, P.C., Lewis, J.S., Koerkamp, B.G., and Reiner, Th., Eds., Springer, 2015, p. 89.Google Scholar
  37. 37.
    Bernhardt, S., Nicolau, S.A., Agnus, V., et al., Proc. 11th IEEE Int. Symp. on Biomedical Imaging, Beijing, 2014, p. 563.Google Scholar
  38. 38.
    Bernhardt, S., Nicolau, S.A., Agnus, V., et al., Med. Image Anal., 2016, vol. 30, p. 130.CrossRefGoogle Scholar
  39. 39.
    Bernhardt, S., Nicolau, S.A., Bartoli, A., et al., in Computer-Assisted and Robotic Endoscopy, Luo, X., Reichl, T., Reiter, A., and Mariottini, G.-L., Eds., Springer, 2015, p. 59.Google Scholar
  40. 40.
    Bernhardt, S., Nicolau, S.A., Soler, L., et al., Med. Image Anal., 2017, vol. 37, p. 66.CrossRefGoogle Scholar
  41. 41.
    Schols, R.M., Bouvy, N.D., van Dam, R.M., et al., Surg. Endoscopy, 2013, vol. 27, p. 1851.CrossRefGoogle Scholar
  42. 42.
    Diana, M., Halvax, P., Dallemagne, B., et al., Surg. Endoscopy, 2014, vol. 28, p. 3108.CrossRefGoogle Scholar
  43. 43.
    Diana, M., Noll, E., Diemunsch, P., et al., Ann. Surg., 2014, vol. 259, p. 700.CrossRefGoogle Scholar
  44. 44.
    Van den Berg, N.S., Buckle, T., KleinJan, G.H., et al., Eur. Urol., 2017, vol. 72, p. 110.CrossRefGoogle Scholar
  45. 45.
    Clancy, N.T., Arya, S., Stoyanov, D., et al., Biomed. Opt. Express, 2015, vol. 6, p. 4179.CrossRefGoogle Scholar
  46. 46.
    Hill, E.R., Xia, W., Nikitichev, D.I., et al., Proc SPIE, 2016, vol. 9708, p. 97080B.Google Scholar

Copyright information

© Allerton Press, Inc. 2018

Authors and Affiliations

  1. 1.Volga State University of TechnologyYoshkar-OlaRussia

Personalised recommendations