Classification of Melanoma Images with Fisher Vectors and Deep Learning

  • Gastón Liberman
  • Daniel AcevedoEmail author
  • Marta Mejail
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11401)


The present work corresponds to the application of techniques of data mining and deep training of neural networks (deep learning) with the objective of classifying images of moles in ‘Melanomas’ or ‘No Melanomas’. For this purpose an ensemble of three classifiers will be created. The first corresponds to a convolutional network VGG-16, the other two correspond to two hybrid models. Each hybrid model is composed of a VGG-16 input network and a Support Vector Machine (SVM) as a classifier. These models will be trained with Fisher Vectors (FVs) calculated with the descriptors that are the output of the convolutional network aforementioned. The difference between these two last classifiers lies in the fact that one has segmented images as input of the VGG-16 network, while the other uses non-segmented images. Segmentation is done by means of an U-NET network. Finally, we will analyze the performance of the hybrid models: the VGG-16 network and the ensemble that incorporates the three classifiers.


Melanoma classification Deep learning Fisher vectors 


  1. 1.
    Stolz, W., et al.: ABCD rule of dermatoscopy: a new practical method for early recognition of malignant melanoma. Eur. J. Dermatol. 4, 521–527 (1994)Google Scholar
  2. 2.
    Menzies, S.W., Ingvar, C., Crotty, K.A., McCarthy, W.H.: Frequency and morphologic characteristics of invasive melanoma lacking specific surface microscopic features. Arch. Dermatol. 132, 1178–1182 (1996)CrossRefGoogle Scholar
  3. 3.
    Argenziano, G., Fabbrocini, G., Carli, P., De Giorgi, V., Sammarco, E., Delfino, M.: Epiluminescence microscopy for the diagnosis of doubtful melanocytic skin lesions comparison of the ABCD rule of dermatoscopy and a new 7-point checklist based on pattern analysis. Arch Dermatol. 134, 1563–1570 (1998)CrossRefGoogle Scholar
  4. 4.
    Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)CrossRefGoogle Scholar
  5. 5.
    Iyatomi, H., Oka, H., Hashimoto, M., Tanaka, M., Ogawa., K.: An internet based melanoma diagnostic system Toward the practical application. In: Proceedings of the 2005 IEEE Symposium on Computational Intelligence in Bioinformatics and Comp. Biology, pp. 1–4 (2005)Google Scholar
  6. 6.
    Capdehourat, G., Corez, A., Bazzano, A., Musé, P.: Pigmented skin lesions classification using dermatoscopic images. In: Bayro-Corrochano, E., Eklundh, J.-O. (eds.) CIARP 2009. LNCS, vol. 5856, pp. 537–544. Springer, Heidelberg (2009). Scholar
  7. 7.
    Alcon, J.F., et al.: Automatic imaging system with decision support for inspection of pigmented skin lesions and melanoma diagnosis. IEEE J. Sel. Top. Signal Process. 3, 14–25 (2009)CrossRefGoogle Scholar
  8. 8.
    Leo, G.D., Paolillo, A., Sommella, P., Fabbrocini, G.: Automatic diagnosis of melanoma: a software system based on the 7-point check-list. In: HICSS (2010)Google Scholar
  9. 9.
    Ruiz, D., et al.: A decision support system for the diagnosis of melanoma: a comparative approach. Expert. Syst. Appl. 38, 15217–15223 (2011)CrossRefGoogle Scholar
  10. 10.
    Capdehourat, G., et al.: Toward a combined tool to assist dermatologists in melanoma detection from dermoscopic images of pigmented skin lesions. Pattern Recognit. Lett. 32, 1–10 (2011)CrossRefGoogle Scholar
  11. 11.
    Xie, F., Fan, H., Yang, L., Jiang, Z., Meng, R., Bovik, A.: Melanoma classification on dermoscopy images using a neural network ensemble model. IEEE Trans. Med. Imaging 36, 849–858 (2016)CrossRefGoogle Scholar
  12. 12.
    Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation arXiv:1505.04597 (2015)
  13. 13.
    Codella, N., Cai, J., Abedini, M., Garnavi, R., Halpern, A., Smith, J.R.: Deep learning, sparse coding, and SVM for melanoma recognition in dermoscopy images. In: Zhou, L., Wang, L., Wang, Q., Shi, Y. (eds.) MLMI 2015. LNCS, vol. 9352, pp. 118–126. Springer, Cham (2015). Scholar
  14. 14.
    Demyanov, S., et al.: Classification of dermoscopy patterns using deep convolutional neural networks. In: IEEE 13th ISBI (2016)Google Scholar
  15. 15.
    Kawahara, J., et al.: Deep features to classify skin lesions. In: ISBI, pp. 1397–1400 (2016)Google Scholar
  16. 16.
    Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639), 115 (2017)CrossRefGoogle Scholar
  17. 17.
    Yu, Z., et al.: Hybrid dermoscopy image classification framework based on deep CNN and Fisher vector. In: 14th IEEE ISBI, Melbourne, Australia (2017)Google Scholar
  18. 18.
    Lecun, Y., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  19. 19.
    Sanchez, J., et al.: Image classification with the fisher vector: theory and practice. IJCV 105, 222–245 (2013)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Gijsenij, A., et al.: Computational color constancy: survey and experiments. IEEE Trans. Image Process. 20, 2475–2489 (2011)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv 1409.1556 (2014)Google Scholar
  22. 22.
    Selvaraju, R., et al.: Grad-CAM: why did you say that? Visual explanations from deep networks via gradient-based localization (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Gastón Liberman
    • 1
    • 2
  • Daniel Acevedo
    • 1
    • 2
    Email author
  • Marta Mejail
    • 1
    • 2
  1. 1.Facultad de Ciencias Exactas y Naturales, Departamento de ComputaciónUniversidad de Buenos AiresBuenos AiresArgentina
  2. 2.Instituto de Investigación en Ciencias de la Computación (ICC)CONICET-Universidad de Buenos AiresBuenos AiresArgentina

Personalised recommendations