Advertisement

Transfer Learning Using Rotated Image Data to Improve Deep Neural Network Performance

  • Telmo AmaralEmail author
  • Luís M. Silva
  • Luís A. Alexandre
  • Chetak Kandaswamy
  • Joaquim Marques de Sá
  • Jorge M. Santos
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8814)

Abstract

In this work we explore the idea that, in the presence of a small training set of images, it could be beneficial to use that set itself to obtain a transformed training set (by performing a random rotation on each sample), train a source network using the transformed data, then retrain the source network using the original data. Applying this transfer learning technique to three different types of character data, we achieve average relative improvements between 6 % and 16 % in the classification test error. Furthermore, we show that it is possible to achieve relative improvements between 8 % and 42 % in cases where the amount of original training samples is very limited (30 samples per class), by introducing not just one rotation but several random rotations per sample.

Keywords

Transfer learning Deep learning Stacked auto-encoders 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bengio, Y.: Learning deep architectures for AI. Foundations and Trends in Machine Learning 2(1), 1–127 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  2. Bengio, Y., Courville, A., Vincent, P.: Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(8), 1798–1828 (2013)CrossRefGoogle Scholar
  3. Ciresan, D., Meier, U., Gambardella, L., Schmidhuber, J.: Deep, big, simple neural nets for handwritten digit recognition. Neural Computation 22(12), 3207–3220 (2010)CrossRefGoogle Scholar
  4. Ciresan, D., Meier, U., Schmidhuber, J.: Transfer learning for Latin and Chinese characters with deep neural networks. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–6 (2012)Google Scholar
  5. Deng, L., Yu, D.: Deep learning for signal and information processing. Microsoft Research Monograph (2013)Google Scholar
  6. Glorot, X., Bordes, A., Bengio, Y.: Domain adaptation for large-scale sentiment classification: a deep learning approach. In: International Conference on Machine Learning (ICML), pp. 513–520 (2011)Google Scholar
  7. Hinton, G., Osindero, S., Teh, Y.-W.: A fast learning algorithm for deep belief nets. Neural Computation 18(7), 1527–1554 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  8. Larochelle, H., Erhan, D., Courville, A., Bergstra, J., Bengio, Y.: An empirical evaluation of deep architectures on problems with many factors of variation. In: International Conference on Machine Learning (ICML), pp. 473–480 (2007)Google Scholar
  9. Pan, S., Yang, Q.: A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 22(10), 1345–1359 (2010)CrossRefGoogle Scholar
  10. Simard, P., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks applied to visual document analysis. In: International Conference on Document Analysis and Recognition (ICDAR), vol. 3, pp. 958–962 (2003)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Telmo Amaral
    • 1
    Email author
  • Luís M. Silva
    • 1
    • 2
  • Luís A. Alexandre
    • 3
  • Chetak Kandaswamy
    • 1
  • Joaquim Marques de Sá
    • 1
    • 4
  • Jorge M. Santos
    • 1
    • 5
  1. 1.Instituto de Engenharia Biomédica (INEB)Universidade do PortoPortoPortugal
  2. 2.Departamento de MatemáticaUniversidade de AveiroAveiroPortugal
  3. 3.Instituto de TelecomunicaçõesUniversidade da Beira InteriorCovilhãPortugal
  4. 4.Dep. de Eng. Electrotécnica e de ComputadoresFac. de Eng. da Univ. do PortoPortoPortugal
  5. 5.Dep. de MatemáticaInstituto Superior de Engenharia do Instituto Politécnico do PortoPortoPortugal

Personalised recommendations