Abstract
The phase recognition task has been performed on different types of surgeries which ranges from cataract to neurological to laparoscopic intervention. The visual features of a surgical video can be used to identify the surgical phases in laparoscopic interventions. Owing to the significant improvement in performance exhibited by convolutional neural networks (CNN) on various challenging tasks like image classification, action recognition etc., they are widely used as feature extractors. In the proposed framework, features extracted by a CNN are used for phase recognition. The task of phase recognition in surgical videos is rendered challenging because of the presence of motion blur produced to the mobile nature of the recording device and the high variance in scenes observed during the course of the surgery. Also, blood stains on the camera lens and complete or partial occlusion of the scene captured by the laparoscopic camera poses additional challenges. These challenges can be overcome by using temporal features in addition to the spatial visual features. A long short-term memory (LSTM) network is used to learn the temporal information of the video. The m2cai16-workflow dataset consisting of videos of cholecystectomy is used for experimental validation of the performance. Surgical workflow, which refers to the statistical modelling of activities taking place in an operating room during a surgery be done in terms of the surgical phases.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
N. Padoy, T. Blum, H. Feussner, M.O. Berger, N. Navab, On-line recognition of surgical activity for monitoring in the operating room, in Association for the Advancement of Artificial Intelligence, pp. 1718–1724 (2008)
T. Blum, H. Feuner, N. Navab, Modeling and segmentation of surgical workflow from laparoscopic video, in International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 400–407 (2010)
F. Lalys, L. Riffaud, D. Bouget, P. Jannin, A framework for the recognition of high-level surgical tasks from video images for cataract surgeries. IEEE Trans. Biomed. Eng. 59(4), 966–976 (2012)
C. Lea, G.D. Hager, R. Vidal, An improved model for segmentation and recognition of fine-grained activities with application to surgical training tasks, in IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1123–1129 (2015)
U. Klank, N. Padoy, H. Feussner, N. Navab, Automatic feature generation in endoscopic images. Int. J. Comput. Assist. Radiol. Surg. 3(3–4), 331–339 (2008)
A.P. Twinanda, S. Shehata, D. Mutter, J. Marescaux, M. de Mathelin, N. Padoy, Endonet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans. Med. Imag. 36(1), 86–97 (2017)
Y. LeCun, Y. Bengio, Convolutional networks for images, speech, and time series, The Handbook of Brain Theory and Neural Networks, vol. 3361, no. 10 (1995)
S. Hochreiter, J. Schmidhuber, Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
S.J. Pan, Q. Yang, A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)
N. Tajbakhsh, J.Y. Shin, S.R. Gurudu, R.T. Hurst, C.B. Kendall, M.B. Gotway, J. Liang, Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans. Med. Imag. 35(5), 1299–312 (2016)
A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 1097–1105 (2012)
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan V. Vanhoucke, A. Rabinovich, Going deeper with convolutions, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)
K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
D.E. Rumelhart, G.E. Hinton, R.J. Williams, Learning representations by back-propagating errors. Cognit. Model. 5(3), 1 (1988)
J. Deng, W. Dong, R. Socher, L.J. Li, K. Li, L. Fei-Fei, Imagenet: a large-scale hierarchical image database, in Proceedings of the IEEE Conference on Conference Computer Vision and Pattern Recognition, pp. 248–255 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Mishra, K., Sathish, R., Sheet, D. (2019). Phase Identification and Workflow Modeling in Laparoscopy Surgeries Using Temporal Connectionism of Deep Visual Residual Abstractions. In: Balas, V., Roy, S., Sharma, D., Samui, P. (eds) Handbook of Deep Learning Applications. Smart Innovation, Systems and Technologies, vol 136. Springer, Cham. https://doi.org/10.1007/978-3-030-11479-4_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-11479-4_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-11478-7
Online ISBN: 978-3-030-11479-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)