Abstract
The Segment Anything Model (SAM) has gained significant attention in the field of image segmentation due to its impressive capabilities and prompt-based interface. While SAM has already been extensively evaluated in various domains, its adaptation to retinal OCT scans remains unexplored. To bridge this research gap, we conduct a comprehensive evaluation of SAM and its adaptations on a large-scale public dataset of OCTs from RETOUCH challenge. Our evaluation covers diverse retinal diseases, fluid compartments, and device vendors, comparing SAM against state-of-the-art retinal fluid segmentation methods. Through our analysis, we showcase adapted SAM’s efficacy as a powerful segmentation model in retinal OCT scans, although still lagging behind established methods in some circumstances. The findings highlight SAM’s adaptability and robustness, showcasing its utility as a valuable tool in retinal OCT image analysis and paving the way for further advancements in this domain.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Apostolopoulos, S., Ciller, C., Sznitman, R., De Zanet, S.: Simultaneous classification and segmentation of cysts in retinal oct. In: Proceedings of MICCAI Retinal OCT Fluid Challenge (RETOUCH), pp. 22–29 (2017)
Bogunovic, H., et al.: RETOUCH: the retinal OCT fluid detection and segmentation benchmark and challenge. IEEE Trans. Med. Imaging 38(8), 1858–1874 (2019). https://doi.org/10.1109/TMI.2019.2901398
Campochiaro, P.A., Aiello, L.P., Rosenfeld, P.J.: Anti-vascular endothelial growth factor agents in the treatment of retinal disease: from bench to bedside. Ophthalmology 123(10), S78–S88 (2016). https://doi.org/10.1016/j.ophtha.2016.04.056
Chen, Q., et al.: Automatic segmentation of fluid-associated abnormalities and pigment epithelial detachment in retinal sd-oct images. In: Proceedings of MICCAI Retinal OCT Fluid Challenge (RETOUCH), pp. 15–21 (2017)
Deng, R., et al.: Segment anything model (SAM) for digital pathology: assess zero-shot segmentation on whole slide imaging. In: MIDL (2023)
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
Gao, Y., Xia, W., Hu, D., Gao, X.: DeSAM: decoupling segment anything model for generalizable medical image segmentation, arxiv.org/abs/2306.00499 (2023)
Houlsby, N., et al.: Parameter-efficient transfer learning for nlp. In: International Conference on Machine Learning, pp. 2790–2799. PMLR (2019)
Hu, E.J., et al.: LoRA: low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)
Hu, X., Xu, X., Shi, Y.: How to efficiently adapt large segmentation model (SAM) to medical image (2023). https://doi.org/10.48550/arxiv.2306.13731, arxiv.org/abs/2306.13731
Huang, Y., et al.: Segment Anything Model for Medical Images? arXiv preprint arXiv:2304.14660 (2023)
Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.H.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18(2), 203–211 (2021). https://doi.org/10.1038/s41592-020-01008-z,www.nature.com/articles/s41592-020-01008-z
Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.H.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18(2), 203–211 (2021). https://doi.org/10.1038/s41592-020-01008-z, www.nature.com/articles/s41592-020-01008-z
Ji, G.P., Fan, D.P., Xu, P., Cheng, M.M., Zhou, B., Gool, L.V.: SAM Struggles in Concealed Scenes - Empirical Study on "Segment Anything". arXiv preprint arXiv:2304.06022 (2023)
Ji, W., Li, J., Bi, Q., Liu, T., Li, W., Cheng, L.: Segment anything is not always perfect: an investigation of SAM on different real-world applications. arXiv preprint arXiv:2304.05750 (2023)
Kang, S.H., Park, H.S., Jang, J., Jeon, K.: Deep neural networks for the detection and segmentation of the retinal fluid in oct images. In: MICCAI Retinal OCT Fluid Challenge (RETOUCH) (2017)
Kirillov, A., et al.: Segment Anything. arXiv (2023). https://doi.org/10.48550/arxiv.2304.02643, arxiv.org/abs/2304.02643
Kurtzer, G.M., Sochat, V., Bauer, M.W.: Singularity: scientific containers for mobility of compute. PLOS ONE 12(5), e0177459 (2017) https://doi.org/10.1371/journal.pone.0177459, www.journals.plos.org/plosone/article?id=10.1371/journal.pone.0177459
Lei, W., Wei, X., Zhang, X., Li, K., Zhang, S.: MedLSAM: localize and segment anything model for 3D medical images (2023). https://doi.org/10.48550/arxiv.2306.14752, arxiv.org/abs/2306.14752
Lu, D., et al.: Deep-learning based multiclass retinal fluid segmentation and detection in optical coherence tomography images using a fully convolutional neural network. Med. Image Anal. 54, 100–110 (2019)
Ma, J., Wang, B.: Segment anything in medical images. arXiv preprint arXiv:2304.12306 (2023)
Moor, M., et al.: Foundation models for generalist medical artificial intelligence. Nature 616(7956), 259–265 (2023). https://doi.org/10.1038/s41586-023-05881-4,www.nature.com/articles/s41586-023-05881-4
Morley, D., Foroosh, H., Shaikh, S., Bagci, U.: Simultaneous detection and quantification of retinal fluid with deep learning. arXiv preprint arXiv:1708.05464 (2017)
Ndipenoch, N., Miron, A., Wang, Z., Li, Y.: nnUNet RASPP for Retinal OCT Fluid Detection, Segmentation and Generalisation over Variations of Data Sources. arXiv preprint arXiv:2302.13195 (2023)
Radford, A., et al.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning, pp. 8748–8763. PMLR (2021)
Rashno, A., Koozekanani, D.D., Parhi, K.K.: Detection and segmentation of various types of fluids with graph shortest path and deep learning approaches. In: Proceedings of MICCAI Retinal OCT Fluid Challenge (RETOUCH), pp. 54–62 (2017)
Roy, S., et al.: SAM.MD: zero-shot medical image segmentation capabilities of the segment anything model. In: MIDL (2023)
Tennakoon, R., Gostar, A.K., Hoseinnezhad, R., Bab-Hadiashar, A.: Retinal fluid segmentation in OCT images using adversarial loss based convolutional neural networks. In: International Symposium on Biomedical Imaging (ISBI), pp. 1436–1440. IEEE Computer Society (May 2018). https://doi.org/10.1109/ISBI.2018.8363842
Wu, J., et al.: Medical SAM adapter: adapting segment anything model for medical image segmentation. arXiv preprint arXiv:2304.12620 (2023)
Yadav, S., Gopinath, K., Sivaswamy, J.: A generalized motion pattern and fcn based approach for retinal fluid detection and segmentation. arXiv preprint arXiv:1712.01073 (2017)
Zhang, K., Liu, D.: Customized Segment Anything Model for medical image segmentation (2023). https://doi.org/10.48550/arxiv.2304.13785, arxiv.org/abs/2304.13785
Zhang, Y., Jiao, R.: How Segment Anything Model (SAM) boost medical image segmentation: a survey (2023). https://doi.org/10.48550/arxiv.2305.03678, arxiv.org/abs/2305.03678
Acknowledgements
The financial support by the Christian Doppler Research Association, Austrian Federal Ministry for Digital and Economic Affairs, the National Foundation for Research, Technology and Development is gratefully acknowledged.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Fazekas, B., Morano, J., Lachinov, D., Aresta, G., Bogunović, H. (2023). Adapting Segment Anything Model (SAM) for Retinal OCT. In: Antony, B., Chen, H., Fang, H., Fu, H., Lee, C.S., Zheng, Y. (eds) Ophthalmic Medical Image Analysis. OMIA 2023. Lecture Notes in Computer Science, vol 14096. Springer, Cham. https://doi.org/10.1007/978-3-031-44013-7_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-44013-7_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44012-0
Online ISBN: 978-3-031-44013-7
eBook Packages: Computer ScienceComputer Science (R0)
