Advertisement

Capturing a Surgical Area Using Multiple Depth Cameras Mounted on a Robotic Mechanical System

  • Masahiro Nonaka
  • Kaoru Watanabe
  • Hiroshi NoborioEmail author
  • Masatoshi Kayaki
  • Kiminori Mizushino
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10289)

Abstract

In our surgical navigation study, we construct a mechanical system for steadily capturing several surgical scenes by using two parallel robotic sliders and multiple vision cameras. In this paper, we first determine how to select an adequate time interval during which each camera projects a pattern to calculate depth against an organ. If multiple cameras project and receive patterns simultaneously, pattern interferences occur around the organ and, consequently, the cameras cannot capture depth images. Second, we investigate whether few or no occlusions occur in several surgical scenarios for an organ operation. Finally, we check experimentally whether distance precision in depth images is exactly maintained when a surgeon raises the camera to insert a microscope during a microsurgery. If the above functions are performed correctly, our proposed transcription algorithms for position, orientation, and shape from a real organ to its virtual polyhedron’s organ with STL-format play an active part during an actual surgery.

Keywords

Robotic system Depth cameras Capturing a surgical area 

Notes

Acknowledgments

This study was supported in part by 2014 Grants-in-Aid for Scientific Research (No. JP26289069) from the Ministry of Education, Culture, Sports, Science, and Technology, Japan. Further support was provided by the 2014 Cooperation Research Fund from the Graduate School at Osaka Electro- Communication University.

References

  1. 1.
    Logan, W.C., Prashanth, D., William, C.C., Benoit, M.D., Robert, L.G., Michael, I.M.: Organ surface deformation measurement and analysis in open hepatic surgery: Method and preliminary results from 12 clinical cases. IEEE Trans. Biomed. Eng. 58(8), 2280–2289 (2011). doi: 10.1109/TBME.2011.2146782 CrossRefGoogle Scholar
  2. 2.
    Xu, A., Zhu, J.F., Zhang, D.: Development of a measurement system for laparoendoscopic single-site surgery: Reliability and repeatability of digital image correlation for measurement of surface deformations in SILS port. JSLS 18(3) (2014). doi: 10.4293/JSLS.2014.00267, PMCID: PMC4154418
  3. 3.
    Kang, N., Lee, M.W., Rhee, T.: Simulating liver deformation during respiration using sparse local features. IEEE Comput. Graph. Appl. 32(5), 29–38 (2012). doi: 10.1109/MCG.2012.65 CrossRefGoogle Scholar
  4. 4.
    Watanabe, K., Kayaki, M., Mizushino, K., Nonaka, M., Noborio, H.: A mechanical system directly attaching beside a surgical bed for measuring surgical area precisely by depth camera. In: Proceedings of the 10th MedViz Conference and the 6th Eurographics Workshop on Visual Computing for Biology and Medicine (EG VCBM), Bergen, Norway, pp. 105–108, 7–9 September (2016). ISBN:978-82-998920-7-0 (Printed), ISBN:978-82-998920-8-7 (Electronic)Google Scholar
  5. 5.
    Watanabe, K., Kayaki, M., Mizushino, K., Nonaka, M., Noborio, H.: Brain shift simulation controlled by directly captured surface points. In: Proceedings of the 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2016), Category: Late Breaking Research Posters, Theme: BioMedical Imaging and Image Processing, Sessions: Ignite_Theme 2_Fr2, Poster Session III, Orlando Florida USA, 16–20 August (2016)Google Scholar
  6. 6.
    Watanabe, K., Kayaki, M., Mizushino, K., Nonaka, M., Noborio, H.: Capturing a brain shift directly by the depth camera kinect v2. In: Proceedings of the 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2016), Category: Late Breaking Research Posters, Theme: Computational Systems & Synthetic Biology; Multiscale Modeling, Sessions: Ignite_Theme 4_Fr1, Poster Session II, Orlando Florida USA, August 16–20 (2016)Google Scholar
  7. 7.
    Noborio, H., et al.: Tracking a real liver using a virtual liver and an experimental evaluation with kinect v2. In: Ortuño, F., Rojas, I. (eds.) IWBBIO 2016. LNCS, vol. 9656, pp. 149–162. Springer, Cham (2016). doi: 10.1007/978-3-319-31744-1_14 CrossRefGoogle Scholar
  8. 8.
    Noborio, H., et al.: Depth image matching algorithm for deforming and cutting a virtual liver via its real liver image captured using kinect v2. In: Ortuño, F., Rojas, I. (eds.) IWBBIO 2016. LNCS, vol. 9656, pp. 196–205. Springer, Cham (2016). doi: 10.1007/978-3-319-31744-1_18 CrossRefGoogle Scholar
  9. 9.
    Noborio, H., Watanabe, K., Yagi, M., Ida, Y., Nankaku, S., Onishi, K., Koeda, M., Kon, M., Matsui, K., Kaibori, M.: Experimental results of 2D depth-depth matching algorithm based on depth camera kinect v1. J. Bioinform. Neurosci. 1(1), 38–44 (2015)Google Scholar
  10. 10.
    Noborio, H., Watanabe, K., Yagi, M., Ida, Y., Onishi, K., Koeda, M., Nankaku, S., Matsui, K., Kon, M., Kaibori, M.: Image-based initial position/orientation adjustment system between real and virtual livers. Jurnal Teknologi, Med. Eng. 77(6), 41–45 (2015). doi: 10.11113/jt.v77.6225, Penerbit UTM Press, E-ISSN: 2180-3722
  11. 11.
    Intel® RealSense™ SDK 2016 R2 Reference Manual: Object Tracking, API Version 10.0 (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Masahiro Nonaka
    • 1
  • Kaoru Watanabe
    • 2
  • Hiroshi Noborio
    • 2
    Email author
  • Masatoshi Kayaki
    • 3
  • Kiminori Mizushino
    • 3
  1. 1.Department of Computer ScienceOsaka Electro-Communication UniversityNeyagawaJapan
  2. 2.Department of NeurosurgeryKansai Medical UniversityHirakataJapan
  3. 3.Embedded Wings CooperationOsakaJapan

Personalised recommendations