Abstract
Regenerative therapies have recently shown potential in restoring sight lost due to degenerative diseases. Their efficacy requires precise intra-retinal delivery, which can be achieved by robotic systems accompanied by high quality visualization of retinal layers. Intra-operative Optical Coherence Tomography (iOCT) captures cross-sectional retinal images in real-time but with image quality that is inadequate for intra-retinal therapy delivery. This paper proposes a two-stage super-resolution methodology that enhances the image quality of the low resolution (LR) iOCT images leveraging information from pre-operatively acquired high-resolution (HR) OCT (preOCT) images. First, we learn the degradation process from HR to LR domain through CycleGAN and use it to generate pseudo iOCT (LR) images from the HR preOCT ones. Then, we train a Pix2Pix model on the pairs of pseudo iOCT and preOCT to learn the super-resolution mapping. Quantitative analysis using both full-reference and no-reference image quality metrics demonstrates that our approach clearly outperforms the learning-based state-of-the art techniques with statistical significance. Achieving iOCT image quality comparable to preOCT quality can help this medical imaging modality be established in vitreoretinal surgery, without requiring expensive hardware-related system updates.
Supported by King’s Centre for Doctoral Studies - Centre for Doctoral Training in Surgical & Interventional Engineering and funded in whole, or in part, by the Wellcome Trust [WT203148/Z/16/Z]. For the purpose of open access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
We interchange “super resolution” and “quality enhancement” as usual in the literature.
- 2.
References
Apostolopoulos, S., et al.: Automatically enhanced oct scans of the retina: a proof of concept study. Sci. Rep. 10(1), 1–8 (2020)
Bernardes, R., Maduro, C., Serranho, P., Araújo, A., Barbeiro, S., Cunha-Vaz, J.: Improved adaptive complex diffusion despeckling filter. Opt. Express 18(23), 24048–24059 (2010)
Bińkowski, M., Sutherland, D.J., Arbel, M., Gretton, A.: Demystifying MMD GANs. arXiv preprint arXiv:1801.01401 (2018)
Bulat, A., Yang, J., Tzimiropoulos, G.: To learn image super-resolution, use a GAN to learn how to do image degradation first. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 185–200 (2018)
Cornelissen, P., Ourak, M., Borghesan, G., Reynaerts, D., Vander Poorten, E.: Towards real-time estimation of a spherical eye model based on a single fiber OCT. In: 2019 19th International Conference on Advanced Robotics (ICAR), pp. 666–672. IEEE (2019)
da Cruz, L., et al.: Phase 1 clinical study of an embryonic stem cell-derived retinal pigment epithelium patch in age-related macular degeneration. Nat. Biotechnol. 36(4), 328 (2018)
Devalla, S.K., et al.: A deep learning approach to denoise optical coherence tomography images of the optic nerve head. Sci. Rep. 9(1), 1–13 (2019)
Fang, L., Li, S., Cunefare, D., Farsiu, S.: Segmentation based sparse reconstruction of optical coherence tomography images. IEEE Trans. Med. Imaging 36(2), 407–421 (2016)
Goodfellow, I., et al.: Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27 (2014)
Halupka, K.J., et al.: Retinal optical coherence tomography image enhancement via deep learning. Biomed. Opt. Express 9(12), 6205–6221 (2018)
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a local Nash equilibrium. Adv. Neural Inf. Process. Syst. 30 (2017)
Immerkaer, J.: Fast noise variance estimation. Comput. Vis. Image Underst. 64(2), 300–302 (1996)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
de Jong, E.K., Geerlings, M.J., den Hollander, A.I.: Age-related macular degeneration. In: Genetics and Genomics of Eye Disease, pp. 155–180 (2020)
Komninos, C., et al.: Intra-operative OCT (iOCT) image quality enhancement: a super-resolution approach using high quality iOCT 3D scans. In: Fu, H., Garvin, M.K., MacGillivray, T., Xu, Y., Zheng, Y. (eds.) OMIA 2021. LNCS, vol. 12970, pp. 21–31. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87000-3_3
Komninos, C., et al.: Surgical biomicroscopy-guided intra-operative optical coherence tomography (iOCT) image super-resolution. Int. J. Comput. Assist. Radiol. Surg. 17(5), 877–883 (2022). https://doi.org/10.1007/s11548-022-02603-5
Li, M., Idoughi, R., Choudhury, B., Heidrich, W.: Statistical model for OCT image denoising. Biomed. Opt. Express 8(9), 3903–3917 (2017)
Matkovic, K., Neumann, L., Neumann, A., Psik, T., Purgathofer, W.: Global contrast factor-a new approach to image contrast. Comput. Aesthetics 2005(159–168), 1 (2005)
Nazari, H., et al.: Stem cell based therapies for age-related macular degeneration: the promises and the challenges. Prog. Retin. Eye Res. 48, 1–39 (2015)
Ozcan, A., Bilenca, A., Desjardins, A.E., Bouma, B.E., Tearney, G.J.: Speckle reduction in optical coherence tomography images using digital filtering. JOSA A 24(7), 1901–1910 (2007)
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vision 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y
Sander, B., Larsen, M., Thrane, L., Hougaard, J.L., Jørgensen, T.M.: Enhanced optical coherence tomography imaging by multiple scan averaging. Br. J. Ophthalmol. 89(2), 207–212 (2005)
Viehland, C., et al.: Enhanced volumetric visualization for real time 4D intraoperative ophthalmic swept-source OCT. Biomed. Opt. Express 7(5), 1815 (2016)
Xu, J., Gong, E., Pauly, J., Zaharchuk, G.: 200x low-dose PET reconstruction using deep learning. arXiv preprint arXiv:1712.04119 (2017)
Yang, Q., et al.: Low-dose CT image denoising using a generative adversarial network with Wasserstein distance and perceptual loss. IEEE Trans. Med. Imaging 37(6), 1348–1357 (2018)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary material 1 (mp4 16790 KB)
Supplementary material 2 (mp4 16741 KB)
Supplementary material 3 (mp4 4555 KB)
Supplementary material 4 (mp4 4763 KB)
Supplementary material 5 (mp4 14856 KB)
Supplementary material 6 (mp4 15028 KB)
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Komninos, C. et al. (2022). Intra-operative OCT (iOCT) Super Resolution: A Two-Stage Methodology Leveraging High Quality Pre-operative OCT Scans. In: Antony, B., Fu, H., Lee, C.S., MacGillivray, T., Xu, Y., Zheng, Y. (eds) Ophthalmic Medical Image Analysis. OMIA 2022. Lecture Notes in Computer Science, vol 13576. Springer, Cham. https://doi.org/10.1007/978-3-031-16525-2_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-16525-2_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16524-5
Online ISBN: 978-3-031-16525-2
eBook Packages: Computer ScienceComputer Science (R0)
