Advertisement

Semi Supervised Autoencoder

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9948)

Abstract

Autoencoders are self-supervised learning tools, but are unsupervised in the sense that class information is not required for training; but almost invariably they are used for supervised classification tasks. We propose to learn the autoencoder for a semi-supervised paradigm, i.e. with both labeled and unlabeled samples available. Given labeled and unlabeled data, our proposed autoencoder automatically adjusts – for unlabeled data it acts as a standard autoencoder (unsupervised) and for labeled data it additionally learns a linear classifier. We use our proposed semi-supervised autoencoder to (greedily) construct a stacked architecture. We demonstrate the efficacy our design in terms of both accuracy and run time requirements for the case of image classification. Our model is able to provide high classification accuracy with even simple classification schemes as compared to existing models for deep architectures.

Keywords

Autoencoder Feature extraction Classification Semi-supervised learning 

References

  1. 1.
    Hinton, G.E., Zemel, R.S.: Autoencoders, minimum description length, and Helmholtz free energy. Adv. Neural Inf. Process. Syst. 6, 3–10 (1994) Google Scholar
  2. 2.
    Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P-A.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1096–1103. ACM (2008)Google Scholar
  3. 3.
    Chen, Y., Lin, Z., Zhao, X., Wang, G., Gu, Y.: Deep learning-based classification of hyperspectral data. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 7(6), 2094–2107 (2014)CrossRefGoogle Scholar
  4. 4.
    Maldonado, S., Weber, R., Basak, J.: Simultaneous feature selection and classification using kernel-penalized support vector machines. Inf. Sci. 181(1), 115–128 (2011)CrossRefGoogle Scholar
  5. 5.
    Kong, H., Li, X., Wang, L., Teoh, E.K., Wang, J.-G., Venkateswarlu, R.: Generalized 2D principal component analysis. In: Proceedings of 2005 IEEE International Joint Conference on Neural Networks, IJCNN 2005, vol. 1, pp. 108–113. IEEE (2005)Google Scholar
  6. 6.
    Bengio, Y.: Learning deep architectures for AI. Found. Trends® Mach. Learn. 2(1), 1–127 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Hinton, G.E., Osindero, S., Teh, Y.-W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Ciresan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3642–3649. IEEE (2012)Google Scholar
  9. 9.
    Gao, S., Zhang, Y., Jia, K., Lu, J., Zhang, Y.: Single sample face recognition via learning deep supervised autoencoders. IEEE Trans. Inf. Forensics Secur. 10(10), 2108–2118 (2015)CrossRefGoogle Scholar
  10. 10.
    Huang, G., Song, S., Gupta, J.N.D., Wu, C.: Semi-supervised and unsupervised extreme learning machines. IEEE Trans. Cybern. 44(12), 2405–2417 (2014)CrossRefGoogle Scholar
  11. 11.
    Ranzato, M., Szummer, M.: Semi-supervised learning of compact document representations with deep networks. In: Proceedings of the 25th International Conference on Machine Learning, pp. 792–799. ACM (2008)Google Scholar
  12. 12.
    Larochelle, H., Mandel, M., Pascanu, R., Bengio, Y.: Learning algorithm for the classification restricted boltzmann machine. J. Mach. Learn. Res. 13(1), 643–669 (2012)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Almousli, H., Vincent, P.: Semi supervised autoencoders: better focusing model capacity during feature extraction. In: Lee, M., Hirose, A., Hou, Z.-G., Kil, R.M. (eds.) ICONIP 2013. LNCS, vol. 8226, pp. 328–335. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  14. 14.
    Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.-A.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1096–1103. ACM (2008)Google Scholar
  15. 15.
    Lemme, A., Reinhart, R.F., Steil, J.J.: Online learning and generalization of parts-based image representations by non-negative sparse autoencoders. Neural Netw. 33, 194–203 (2012)CrossRefGoogle Scholar
  16. 16.
    Jiang, Z., Lin, Z., Davis, L.S.: Learning a discriminative dictionary for sparse coding via label consistent K-SVD. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1697–1704. IEEE (2011)Google Scholar
  17. 17.
    Shrivastava, A., Pillai, J.K., Patel, V.M., Chellappa, R.: Learning discriminative dictionaries with partially labeled data. In: 2012 19th IEEE International Conference on Image Processing (ICIP), pp. 3113–3116. IEEE (2012)Google Scholar
  18. 18.
  19. 19.
  20. 20.
  21. 21.
    Lawson, C.L., Hanson, R.J.: Solving least squares problems, vol. 161. Prentice-hall, Englewood Cliffs (1974)zbMATHGoogle Scholar
  22. 22.
    Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
  24. 24.
    Ng, A.: Sparse autoencoder. CS294A Lecture notes 72 (2011)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Indraprasatha Institute of Information Technology-DelhiNew DelhiIndia

Personalised recommendations