Deep Transfer Learning Ensemble for Classification

  • Chetak KandaswamyEmail author
  • Luís M. Silva
  • Luís A. Alexandre
  • Jorge M. Santos
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9094)


Transfer learning algorithms typically assume that the training data and the test data come from different distribution. It is better at adapting to learn new tasks and concepts more quickly and accurately by exploiting previously gained knowledge. Deep Transfer Learning (DTL) emerged as a new paradigm in transfer learning in which a deep model offer greater flexibility in extracting high-level features. DTL offers selective layer based transference, and it is problem specific. In this paper, we propose the Ensemble of Deep Transfer Learning (EDTL) methodology to reduce the impact of selective layer based transference and provide optimized framework to work for three major transfer learning cases. Empirical results on character, object and biomedical image recognition tasks achieves that the proposed method indicate statistically significant classification accuracy over the other established transfer learning method.


Deep learning Transfer learning Ensemble 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Thrun, S.: Learning to learn: Introduction. In Learning To Learn (1996)Google Scholar
  2. 2.
    Caruana, R.: Multitask learning. Machine Learning 28(1), 41–75 (1997)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Daumé III, H., Marcu, D.: Domain Adaptation for Statistical Classifiers. J. Artif. Intell. Res. (JAIR) 26, 101–126 (2006)zbMATHGoogle Scholar
  4. 4.
    Raina, R., Battle, A., Lee, H., Packer, B., Ng, A.Y.: Self-taught learning: transfer learning from unlabeled data. In: Proc. of the ACM Conference on (ICML), pp. 759–766 (2007)Google Scholar
  5. 5.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. proceedings of the IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  6. 6.
    Hinton, G.E., Osindero, S., Teh, Y.: A fast learning algorithm for deep belief nets. The Journal of Neural computation 7, 1527–1554 (2006)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.-A.: Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010)zbMATHMathSciNetGoogle Scholar
  8. 8.
    Bengio, Y., et al.: Towards Biologically Plausible Deep Learning. arXiv preprint arXiv:1502.04156 (2015)
  9. 9.
    Kandaswamy, C., Silva, L., Alexandre, L., Sousa, R., Santos, J.M., Marques de Sá, J.: Improving transfer learning accuracy by reusing Stacked Denoising Autoencoders. In: IEEE Conference on Systems Man and Cybernetics. IEEE (2014)Google Scholar
  10. 10.
    Kandaswamy, C., Silva, L.M., Alexandre, L.A., Santos, J.M., de Sá, J.M.: Improving deep neural network performance by reusing features trained with transductive transference. In: Wermter, S., Weber, C., Duch, W., Honkela, T., Koprinkova-Hristova, P., Magg, S., Palm, G., Villa, A.E.P. (eds.) ICANN 2014. LNCS, vol. 8681, pp. 265–272. Springer, Heidelberg (2014) CrossRefGoogle Scholar
  11. 11.
    Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, pp. 3320–3328 (2014)Google Scholar
  12. 12.
    Kandaswamy, C., Silva, L., Cardoso, J.S.: Source-target-source classification using Stacked Denoising Autoencoders. In: Proc. of the 7th Iberian Conference on Pattern Recognition and Image Analysis, Santiago de Compostela, Spain, June 2015Google Scholar
  13. 13.
    Razavian, A.S., Azizpour, H., Sullivan, J., Carlsson, S.: CNN features off-the-shelf: an astounding baseline for recognition. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 512–519. IEEE (2014)Google Scholar
  14. 14.
    Deng, L., Platt, J.C.: Ensemble deep learning for speech recognition. In: Proceedings of the Annual Conference of International Speech Communication Association (INTERSPEECH) (2014)Google Scholar
  15. 15.
    Abdullah, A., Veltkamp, R.C., Wiering, M.A.: An ensemble of deep support vector machines for image categorization. In: International Conference of Soft Computing and Pattern Recognition, SOCPAR 2009, pp. 301–306. IEEE (2009)Google Scholar
  16. 16.
    Ciresan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2012)Google Scholar
  17. 17.
    Lin, J.: Divergence measures based on the Shannon entropy. IEEE Transactions on Information Theory 37, 145–151 (1991)zbMATHCrossRefGoogle Scholar
  18. 18.
    Ljosa, V.: Katherine L. Sokolnicki, and Anne E. Carpenter.: Annotated high-throughput microscopy image sets for validation. Nat Methods 9(7), 637 (2012)CrossRefGoogle Scholar
  19. 19.
    Ljosa, V., et al.: Comparison of methods for image-based profiling of cellular morphological responses to small-molecule treatment. Journal of biomolecular screening (2013)Google Scholar
  20. 20.
    Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley Press (2004)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Chetak Kandaswamy
    • 1
    • 2
    • 3
    Email author
  • Luís M. Silva
    • 2
    • 4
  • Luís A. Alexandre
    • 5
  • Jorge M. Santos
    • 2
    • 6
  1. 1.Instituto de Investigação e Inovação em SaúdeUniversidade do PortoPortoPortugal
  2. 2.INEB - Instituto de Engenharia BiomédicaPortoPortugal
  3. 3.Department of Electrical and Computer Engineering, Faculty of EngineeringUniversity of PortoPortoPortugal
  4. 4.Departamento de MatemáticaUniversidade de AveiroAveiroPortugal
  5. 5.Universidade da Beira Interior & Instituto de TelecomunicaçõesCovilhãPortugal
  6. 6.Instituto Superior de EngenhariaPolitécnico do PortoPortoPortugal

Personalised recommendations