Multi-label classification of retinal lesions in diabetic retinopathy for automatic analysis of fundus fluorescein angiography based on deep learning



To automatically detect and classify the lesions of diabetic retinopathy (DR) in fundus fluorescein angiography (FFA) images using deep learning algorithm through comparing 3 convolutional neural networks (CNNs).


A total of 4067 FFA images from Eye Center at the Second Affiliated Hospital of Zhejiang University School of Medicine were annotated with 4 kinds of lesions of DR, including non-perfusion regions (NP), microaneurysms, leakages, and laser scars. Three CNNs including DenseNet, ResNet50, and VGG16 were trained to achieve multi-label classification, which means the algorithms could identify 4 retinal lesions above at the same time.


The area under the curve (AUC) of DenseNet reached 0.8703, 0.9435, 0.9647, and 0.9653 for detecting NP, microaneurysms, leakages, and laser scars, respectively. For ResNet50, AUC was 0.8140 for NP, 0.9097 for microaneurysms, 0.9585 for leakages, and 0.9115 for laser scars. And for VGG16, AUC was 0.7125 for NP, 0.5569 for microaneurysms, 0.9177 for leakages, and 0.8537 for laser scars.


Experimental results demonstrate that DenseNet is a suitable model to automatically detect and distinguish retinal lesions in the FFA images with multi-label classification, which lies the foundation of automatic analysis for FFA images and comprehensive diagnosis and treatment decision-making for DR.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4


  1. 1.

    Huang Y (2017) International Diabetes Federation (IDF) (2017) IDF diabetes atlas. 8th edition. Accessed 9 Jun 2019

  2. 2.

    Li Z, Keel S, Liu C, He Y, Meng W, Scheetz J, Lee PY, Shaw J, Ting D, Wong TY, Taylor H, Chang R, He M (2018) An automated grading system for detection of vision-threatening referable diabetic retinopathy on the basis of color fundus photographs. Diabetes Care 41:2509–2516.

    Article  PubMed  Google Scholar 

  3. 3.

    Lamoke F, Shaw S, Yuan J, Ananth S, Duncan M, Martin P, Bartoli M (2015) Increased oxidative and nitrative stress accelerates aging of the retinal vasculature in the diabetic retina. PLoS One 10:e0139664.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  4. 4.

    Dodo Y, Murakami T, Uji A, Yoshitake S, Yoshimura N (2015) Disorganized retinal lamellar structures in nonperfused areas of diabetic retinopathy. Invest Ophthalmol Vis Sci 56:2012–2020.

    Article  PubMed  Google Scholar 

  5. 5.

    Unoki N, Nishijima K, Sakamoto A, Kita M, Watanabe D, Hangai M, Kimura T, Kawagoe N, Ohta M, Yoshimura N (2007) Retinal sensitivity loss and structural disturbance in areas of capillary nonperfusion of eyes with diabetic retinopathy. Am J Ophthalmol 144:755–760.

    Article  PubMed  Google Scholar 

  6. 6.

    Kozak I, El-Emam SY, Cheng L, Bartsch DU, Chhablani J, Freeman WR, Arevalo JF (2014) Fluorescein angiography versus optical coherence tomography-guided planning for macular laser photocoagulation in diabetic macular edema. Retina 34:1600–1605.

    Article  PubMed  PubMed Central  Google Scholar 

  7. 7.

    Zhang XH, Chutatape O, Ieee (2004) Detection and classification of bright lesions in color fundus images. Icip: 2004 international conference on image processing 1–5: 139–142

  8. 8.

    van Ginneken B (2017) Fifty years of computer analysis in chest imaging: rule-based, machine learning, deep learning. Radiol Phys Technol 10:23–32.

    Article  PubMed  PubMed Central  Google Scholar 

  9. 9.

    Weng SF, Reps J, Kai J, Garibaldi JM, Qureshi N (2017) Can machine-learning improve cardiovascular risk prediction using routine clinical data? PLoS One 12:e0174944.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  10. 10.

    Bejnordi BE, Zuidhof G, Balkenhol M, Hermsen M, Bult P, van Ginneken B, Karssemeijer N, Litjens G, van der Laak J (2017) Context-aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images. J Med Imaging (Bellingham) 4:044504.

    Article  Google Scholar 

  11. 11.

    Mercan C, Aksoy S, Mercan E, Shapiro LG, Weaver DL, Elmore JG (2018) Multi-instance multi-label learning for multi-class classification of whole slide breast histopathology images. IEEE Trans Med Imaging 37:316–325.

    Article  PubMed  Google Scholar 

  12. 12.

    Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2017) Corrigendum: dermatologist-level classification of skin cancer with deep neural networks. Nature 546:686.

    CAS  Article  PubMed  Google Scholar 

  13. 13.

    Abramoff MD, Lou Y, Erginay A, Clarida W, Amelon R, Folk JC, Niemeijer M (2016) Improved automated detection of diabetic retinopathy on a publicly available dataset through integration of deep learning. Invest Ophthalmol Vis Sci 57:5200–5206.

    Article  PubMed  Google Scholar 

  14. 14.

    Vidal-Alaball J, Royo Fibla D (2019) Artificial intelligence for the detection of diabetic retinopathy in primary care: protocol for algorithm development. JMIR Res Protoc 8:e12539.

    Article  PubMed  PubMed Central  Google Scholar 

  15. 15.

    Han SS, Kim MS, Lim W, Park GH, Park I, Chang SE (2018) Classification of the clinical images for benign and malignant cutaneous tumors using a deep learning algorithm. J Invest Dermatol 138:1529–1538.

    CAS  Article  PubMed  Google Scholar 

  16. 16.

    Arslan Tuncer S, Akilotu B, Toraman S (2019) A deep learning-based decision support system for diagnosis of OSAS using PTT signals. Med Hypotheses 127:15–22.

    Article  PubMed  Google Scholar 

  17. 17.

    Kamel M, Belkassim S, Mendonca AM, Campilho A, Ieee; Ieee; Ieee I (2001) A neural network approach for the automatic detection of microaneurysms in retinal angiograms. Ijcnn'01: International Joint Conference on Neural Networks 1–4, Proceedings: 2695–2699

  18. 18.

    Hafez M, Azeem SA, Ieee; Ieee; Ieee I (2002) Using adaptive edge technique for detecting microaneurysms in fluorescein angiograms of the ocular fundus. 11th Ieee Mediterranean Electrotechnical Conference, Proceedings: 479–483

  19. 19.

    Tavakoli M, Shahri RP, Pourreza H, Mehdizadeh A, Banaee T, Toosi MHB (2013) A complementary method for automated detection of microaneurysms in fluorescein angiography fundus images to assess diabetic retinopathy. Pattern Recogn 46:2740–2753.

    Article  Google Scholar 

  20. 20.

    Frame AJ, Undrill PE, Cree MJ, Olson JA, McHardy KC, Sharp PF, Forrester JV (1998) A comparison of computer based classification methods applied to the detection of microaneurysms in ophthalmic fluorescein angiograms. Comput Biol Med 28:225–238.

    CAS  Article  PubMed  Google Scholar 

  21. 21.

    Tan JH, Fujita H, Sivaprasad S, Bhandary SV, Rao AK, Kuang CC, Acharya UR (2017) Automated segmentation of exudates, haemorrhages, microaneurysms using single convolutional neural network. Inf Sci 420:S0020025517308927

    Article  Google Scholar 

  22. 22.

    Lam C, Yu C, Huang L, Rubin D (2018) Retinal lesion detection with deep learning using image patches. Invest Ophthalmol Vis Sci 59:590–596.

    Article  PubMed  PubMed Central  Google Scholar 

  23. 23.

    Lu W, Tong Y, Yu Y, Xing Y, Chen C, Shen Y (2018) Deep learning-based automated classification of multi-categorical abnormalities from optical coherence tomography images. Transl Vis Sci Technol 7:41.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  24. 24.

    Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J, Kim R, Raman R, Nelson PC, Mega JL, Webster DR (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Jama 316:2402–2410.

    Article  PubMed  Google Scholar 

  25. 25.

    Cao W, Czarnek N, Shan J, Li L (2018) Microaneurysm detection using principal component analysis and machine learning methods. IEEE Trans Nanobiosci 17:191–198.

    Article  Google Scholar 

  26. 26.

    Khojasteh P, Aliahmad B, Kumar DK (2018) Fundus images analysis using deep features for detection of exudates, hemorrhages and microaneurysms. BMC Ophthalmol 18:288.

    Article  PubMed  PubMed Central  Google Scholar 

Download references


This work was financially supported by Zhejiang Provincial Key Research and Development Plan (grant number 2019C03020), the Natural Science Foundation of China (grant number 81670888), and the Natural Science Foundation of China (grant number 81870635).

Author information



Corresponding author

Correspondence to Juan Ye.

Ethics declarations

This study was conducted in compliance with the principles of the Declaration of Helsinki and was approved by the Ethics Committees of the Second Affiliated Hospital of Zhejiang University School of Medicine. Written informed consents were obtained from all subjects for the publication of this study and accompanying images.

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Pan, X., Jin, K., Cao, J. et al. Multi-label classification of retinal lesions in diabetic retinopathy for automatic analysis of fundus fluorescein angiography based on deep learning. Graefes Arch Clin Exp Ophthalmol 258, 779–785 (2020).

Download citation


  • Diabetic retinopathy
  • Fundus fluorescein angiography
  • Deep learning
  • Multi-label classification