To automatically detect and classify the lesions of diabetic retinopathy (DR) in fundus fluorescein angiography (FFA) images using deep learning algorithm through comparing 3 convolutional neural networks (CNNs).
A total of 4067 FFA images from Eye Center at the Second Affiliated Hospital of Zhejiang University School of Medicine were annotated with 4 kinds of lesions of DR, including non-perfusion regions (NP), microaneurysms, leakages, and laser scars. Three CNNs including DenseNet, ResNet50, and VGG16 were trained to achieve multi-label classification, which means the algorithms could identify 4 retinal lesions above at the same time.
The area under the curve (AUC) of DenseNet reached 0.8703, 0.9435, 0.9647, and 0.9653 for detecting NP, microaneurysms, leakages, and laser scars, respectively. For ResNet50, AUC was 0.8140 for NP, 0.9097 for microaneurysms, 0.9585 for leakages, and 0.9115 for laser scars. And for VGG16, AUC was 0.7125 for NP, 0.5569 for microaneurysms, 0.9177 for leakages, and 0.8537 for laser scars.
Experimental results demonstrate that DenseNet is a suitable model to automatically detect and distinguish retinal lesions in the FFA images with multi-label classification, which lies the foundation of automatic analysis for FFA images and comprehensive diagnosis and treatment decision-making for DR.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Huang Y (2017) International Diabetes Federation (IDF) (2017) IDF diabetes atlas. 8th edition. http://www.diabetesatlas.org/resources/2017-atlas.html. Accessed 9 Jun 2019
Li Z, Keel S, Liu C, He Y, Meng W, Scheetz J, Lee PY, Shaw J, Ting D, Wong TY, Taylor H, Chang R, He M (2018) An automated grading system for detection of vision-threatening referable diabetic retinopathy on the basis of color fundus photographs. Diabetes Care 41:2509–2516. https://doi.org/10.2337/dc18-0147
Lamoke F, Shaw S, Yuan J, Ananth S, Duncan M, Martin P, Bartoli M (2015) Increased oxidative and nitrative stress accelerates aging of the retinal vasculature in the diabetic retina. PLoS One 10:e0139664. https://doi.org/10.1371/journal.pone.0139664
Dodo Y, Murakami T, Uji A, Yoshitake S, Yoshimura N (2015) Disorganized retinal lamellar structures in nonperfused areas of diabetic retinopathy. Invest Ophthalmol Vis Sci 56:2012–2020. https://doi.org/10.1167/iovs.14-15924
Unoki N, Nishijima K, Sakamoto A, Kita M, Watanabe D, Hangai M, Kimura T, Kawagoe N, Ohta M, Yoshimura N (2007) Retinal sensitivity loss and structural disturbance in areas of capillary nonperfusion of eyes with diabetic retinopathy. Am J Ophthalmol 144:755–760. https://doi.org/10.1016/j.ajo.2007.07.011
Kozak I, El-Emam SY, Cheng L, Bartsch DU, Chhablani J, Freeman WR, Arevalo JF (2014) Fluorescein angiography versus optical coherence tomography-guided planning for macular laser photocoagulation in diabetic macular edema. Retina 34:1600–1605. https://doi.org/10.1097/iae.0000000000000120
Zhang XH, Chutatape O, Ieee (2004) Detection and classification of bright lesions in color fundus images. Icip: 2004 international conference on image processing 1–5: 139–142
van Ginneken B (2017) Fifty years of computer analysis in chest imaging: rule-based, machine learning, deep learning. Radiol Phys Technol 10:23–32. https://doi.org/10.1007/s12194-017-0394-5
Weng SF, Reps J, Kai J, Garibaldi JM, Qureshi N (2017) Can machine-learning improve cardiovascular risk prediction using routine clinical data? PLoS One 12:e0174944. https://doi.org/10.1371/journal.pone.0174944
Bejnordi BE, Zuidhof G, Balkenhol M, Hermsen M, Bult P, van Ginneken B, Karssemeijer N, Litjens G, van der Laak J (2017) Context-aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images. J Med Imaging (Bellingham) 4:044504. https://doi.org/10.1117/1.jmi.4.4.044504
Mercan C, Aksoy S, Mercan E, Shapiro LG, Weaver DL, Elmore JG (2018) Multi-instance multi-label learning for multi-class classification of whole slide breast histopathology images. IEEE Trans Med Imaging 37:316–325. https://doi.org/10.1109/tmi.2017.2758580
Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2017) Corrigendum: dermatologist-level classification of skin cancer with deep neural networks. Nature 546:686. https://doi.org/10.1038/nature22985
Abramoff MD, Lou Y, Erginay A, Clarida W, Amelon R, Folk JC, Niemeijer M (2016) Improved automated detection of diabetic retinopathy on a publicly available dataset through integration of deep learning. Invest Ophthalmol Vis Sci 57:5200–5206. https://doi.org/10.1167/iovs.16-19964
Vidal-Alaball J, Royo Fibla D (2019) Artificial intelligence for the detection of diabetic retinopathy in primary care: protocol for algorithm development. JMIR Res Protoc 8:e12539. https://doi.org/10.2196/12539
Han SS, Kim MS, Lim W, Park GH, Park I, Chang SE (2018) Classification of the clinical images for benign and malignant cutaneous tumors using a deep learning algorithm. J Invest Dermatol 138:1529–1538. https://doi.org/10.1016/j.jid.2018.01.028
Arslan Tuncer S, Akilotu B, Toraman S (2019) A deep learning-based decision support system for diagnosis of OSAS using PTT signals. Med Hypotheses 127:15–22. https://doi.org/10.1016/j.mehy.2019.03.026
Kamel M, Belkassim S, Mendonca AM, Campilho A, Ieee; Ieee; Ieee I (2001) A neural network approach for the automatic detection of microaneurysms in retinal angiograms. Ijcnn'01: International Joint Conference on Neural Networks 1–4, Proceedings: 2695–2699
Hafez M, Azeem SA, Ieee; Ieee; Ieee I (2002) Using adaptive edge technique for detecting microaneurysms in fluorescein angiograms of the ocular fundus. 11th Ieee Mediterranean Electrotechnical Conference, Proceedings: 479–483
Tavakoli M, Shahri RP, Pourreza H, Mehdizadeh A, Banaee T, Toosi MHB (2013) A complementary method for automated detection of microaneurysms in fluorescein angiography fundus images to assess diabetic retinopathy. Pattern Recogn 46:2740–2753. https://doi.org/10.1016/j.patcog.2013.03.011
Frame AJ, Undrill PE, Cree MJ, Olson JA, McHardy KC, Sharp PF, Forrester JV (1998) A comparison of computer based classification methods applied to the detection of microaneurysms in ophthalmic fluorescein angiograms. Comput Biol Med 28:225–238. https://doi.org/10.1016/s0010-4825(98)00011-0
Tan JH, Fujita H, Sivaprasad S, Bhandary SV, Rao AK, Kuang CC, Acharya UR (2017) Automated segmentation of exudates, haemorrhages, microaneurysms using single convolutional neural network. Inf Sci 420:S0020025517308927
Lam C, Yu C, Huang L, Rubin D (2018) Retinal lesion detection with deep learning using image patches. Invest Ophthalmol Vis Sci 59:590–596. https://doi.org/10.1167/iovs.17-22721
Lu W, Tong Y, Yu Y, Xing Y, Chen C, Shen Y (2018) Deep learning-based automated classification of multi-categorical abnormalities from optical coherence tomography images. Transl Vis Sci Technol 7:41. https://doi.org/10.1167/tvst.7.6.41
Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J, Kim R, Raman R, Nelson PC, Mega JL, Webster DR (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Jama 316:2402–2410. https://doi.org/10.1001/jama.2016.17216
Cao W, Czarnek N, Shan J, Li L (2018) Microaneurysm detection using principal component analysis and machine learning methods. IEEE Trans Nanobiosci 17:191–198. https://doi.org/10.1109/tnb.2018.2840084
Khojasteh P, Aliahmad B, Kumar DK (2018) Fundus images analysis using deep features for detection of exudates, hemorrhages and microaneurysms. BMC Ophthalmol 18:288. https://doi.org/10.1186/s12886-018-0954-4
This work was financially supported by Zhejiang Provincial Key Research and Development Plan (grant number 2019C03020), the Natural Science Foundation of China (grant number 81670888), and the Natural Science Foundation of China (grant number 81870635).
This study was conducted in compliance with the principles of the Declaration of Helsinki and was approved by the Ethics Committees of the Second Affiliated Hospital of Zhejiang University School of Medicine. Written informed consents were obtained from all subjects for the publication of this study and accompanying images.
Conflict of interest
The authors declare that they have no conflict of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Pan, X., Jin, K., Cao, J. et al. Multi-label classification of retinal lesions in diabetic retinopathy for automatic analysis of fundus fluorescein angiography based on deep learning. Graefes Arch Clin Exp Ophthalmol 258, 779–785 (2020). https://doi.org/10.1007/s00417-019-04575-w
- Diabetic retinopathy
- Fundus fluorescein angiography
- Deep learning
- Multi-label classification