Abstract
Twin-to-twin transfusion syndrome treatment requires fetoscopic laser photocoagulation of placental vascular anastomoses to regulate blood flow to both fetuses. Limited field-of-view (FoV) and low visual quality during fetoscopy make it challenging to identify all vascular connections. Mosaicking can align multiple overlapping images to generate an image with increased FoV, however, existing techniques apply poorly to fetoscopy due to the low visual quality, texture paucity, and hence fail in longer sequences due to the drift accumulated over time. Deep learning techniques can facilitate in overcoming these challenges. Therefore, we present a new generalized Deep Sequential Mosaicking (DSM) framework for fetoscopic videos captured from different settings such as simulation, phantom, and real environments. DSM extends an existing deep image-based homography model to sequential data by proposing controlled data augmentation and outlier rejection methods. Unlike existing methods, DSM can handle visual variations due to specular highlights and reflection across adjacent frames, hence reducing the accumulated drift. We perform experimental validation and comparison using 5 diverse fetoscopic videos to demonstrate the robustness of our framework.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
TTTS phantom from Surgical Touch Simulator: https://www.surgicaltouch.com/.
References
Baschat, A., et al.: Twin-to-twin transfusion syndrome (TTTS). J. Perinat. Med. 39(2), 107–112 (2011)
Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74(1), 59–73 (2007)
Daga, P., et al.: Real-time mosaicing of fetoscopic videos using SIFT. In: Medical Imaging: Image-Guided Procedures (2016)
DeTone, D., Malisiewicz, T., Rabinovich, A.: Deep image homography estimation. In: RSS Workshop on Limits and Potentials of Deep Learning in Robotics (2016)
Dwyer, G., et al.: A continuum robot and control interface for surgical assist in fetoscopic interventions. IEEE Robot. Autom. Lett. 2(3), 1656–1663 (2017)
Gaisser, F., et al.: Stable image registration for in-vivo fetoscopic panorama reconstruction. J. Imaging 4(1), 24 (2018)
Malis, E., Vargas, M.: Deeper understanding of the homography decomposition for vision-based control. Ph.D. thesis, INRIA (2007)
Nguyen, T., et al.: Unsupervised deep homography: a fast and robust homography estimation model. IEEE Robot. Autom. Lett. 3(3), 2346–2353 (2018)
Peter, L., et al.: Retrieval and registration of long-range overlapping frames for scalable mosaicking of in vivo fetoscopy. IJCARS 13(5), 713–720 (2018)
Reeff, M., Gerhard, F., Cattin, P., Gábor, S.: Mosaicing of endoscopic placenta images. In: INFORMATIK 2006-Informatik für Menschen, Band 1 (2006)
Tella-Amo, M., et al.: Probabilistic visual and electromagnetic data fusion for robust drift-free sequential mosaicking: application to fetoscopy. J. Med. Imaging 5(2), 021217 (2018)
Tella-Amo, M., et al.: Pruning strategies for efficient online globally-consistent mosaicking in fetoscopy. J. Med. Imaging 6, 035001 (2019)
Totz, J., Mountney, P., Stoyanov, D., Yang, G.-Z.: Dense surface reconstruction for enhanced navigation in MIS. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011. LNCS, vol. 6891, pp. 89–96. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23623-5_12
Vasconcelos, F., et al.: Towards computer-assisted TTTS: laser ablation detection for workflow segmentation from fetoscopic video. IJCARS 13(10), 1661–1670 (2018)
Acknowledgments
This work was supported through an Innovative Engineering for Health award by Wellcome [WT101957]; Engineering and Physical Sciences Research Council (EPSRC) [NS/A000027/1]. It was additionally supported by the Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS) at UCL [203145Z/16/Z] and EPSRC [EP/N027078/1, EP/P012841/1, EP/P027938/1, EP/R004080/1].
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Bano, S. et al. (2019). Deep Sequential Mosaicking of Fetoscopic Videos. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11764. Springer, Cham. https://doi.org/10.1007/978-3-030-32239-7_35
Download citation
DOI: https://doi.org/10.1007/978-3-030-32239-7_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32238-0
Online ISBN: 978-3-030-32239-7
eBook Packages: Computer ScienceComputer Science (R0)