Capturing a Surgical Area Using Multiple Depth Cameras Mounted on a Robotic Mechanical System
In our surgical navigation study, we construct a mechanical system for steadily capturing several surgical scenes by using two parallel robotic sliders and multiple vision cameras. In this paper, we first determine how to select an adequate time interval during which each camera projects a pattern to calculate depth against an organ. If multiple cameras project and receive patterns simultaneously, pattern interferences occur around the organ and, consequently, the cameras cannot capture depth images. Second, we investigate whether few or no occlusions occur in several surgical scenarios for an organ operation. Finally, we check experimentally whether distance precision in depth images is exactly maintained when a surgeon raises the camera to insert a microscope during a microsurgery. If the above functions are performed correctly, our proposed transcription algorithms for position, orientation, and shape from a real organ to its virtual polyhedron’s organ with STL-format play an active part during an actual surgery.
KeywordsRobotic system Depth cameras Capturing a surgical area
This study was supported in part by 2014 Grants-in-Aid for Scientific Research (No. JP26289069) from the Ministry of Education, Culture, Sports, Science, and Technology, Japan. Further support was provided by the 2014 Cooperation Research Fund from the Graduate School at Osaka Electro- Communication University.
- 1.Logan, W.C., Prashanth, D., William, C.C., Benoit, M.D., Robert, L.G., Michael, I.M.: Organ surface deformation measurement and analysis in open hepatic surgery: Method and preliminary results from 12 clinical cases. IEEE Trans. Biomed. Eng. 58(8), 2280–2289 (2011). doi: 10.1109/TBME.2011.2146782 CrossRefGoogle Scholar
- 2.Xu, A., Zhu, J.F., Zhang, D.: Development of a measurement system for laparoendoscopic single-site surgery: Reliability and repeatability of digital image correlation for measurement of surface deformations in SILS port. JSLS 18(3) (2014). doi: 10.4293/JSLS.2014.00267, PMCID: PMC4154418
- 4.Watanabe, K., Kayaki, M., Mizushino, K., Nonaka, M., Noborio, H.: A mechanical system directly attaching beside a surgical bed for measuring surgical area precisely by depth camera. In: Proceedings of the 10th MedViz Conference and the 6th Eurographics Workshop on Visual Computing for Biology and Medicine (EG VCBM), Bergen, Norway, pp. 105–108, 7–9 September (2016). ISBN:978-82-998920-7-0 (Printed), ISBN:978-82-998920-8-7 (Electronic)Google Scholar
- 5.Watanabe, K., Kayaki, M., Mizushino, K., Nonaka, M., Noborio, H.: Brain shift simulation controlled by directly captured surface points. In: Proceedings of the 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2016), Category: Late Breaking Research Posters, Theme: BioMedical Imaging and Image Processing, Sessions: Ignite_Theme 2_Fr2, Poster Session III, Orlando Florida USA, 16–20 August (2016)Google Scholar
- 6.Watanabe, K., Kayaki, M., Mizushino, K., Nonaka, M., Noborio, H.: Capturing a brain shift directly by the depth camera kinect v2. In: Proceedings of the 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2016), Category: Late Breaking Research Posters, Theme: Computational Systems & Synthetic Biology; Multiscale Modeling, Sessions: Ignite_Theme 4_Fr1, Poster Session II, Orlando Florida USA, August 16–20 (2016)Google Scholar
- 8.Noborio, H., et al.: Depth image matching algorithm for deforming and cutting a virtual liver via its real liver image captured using kinect v2. In: Ortuño, F., Rojas, I. (eds.) IWBBIO 2016. LNCS, vol. 9656, pp. 196–205. Springer, Cham (2016). doi: 10.1007/978-3-319-31744-1_18 CrossRefGoogle Scholar
- 9.Noborio, H., Watanabe, K., Yagi, M., Ida, Y., Nankaku, S., Onishi, K., Koeda, M., Kon, M., Matsui, K., Kaibori, M.: Experimental results of 2D depth-depth matching algorithm based on depth camera kinect v1. J. Bioinform. Neurosci. 1(1), 38–44 (2015)Google Scholar
- 10.Noborio, H., Watanabe, K., Yagi, M., Ida, Y., Onishi, K., Koeda, M., Nankaku, S., Matsui, K., Kon, M., Kaibori, M.: Image-based initial position/orientation adjustment system between real and virtual livers. Jurnal Teknologi, Med. Eng. 77(6), 41–45 (2015). doi: 10.11113/jt.v77.6225, Penerbit UTM Press, E-ISSN: 2180-3722
- 11.Intel® RealSense™ SDK 2016 R2 Reference Manual: Object Tracking, API Version 10.0 (2016)Google Scholar