Abstract
Autoencoders are self-supervised learning tools, but are unsupervised in the sense that class information is not required for training; but almost invariably they are used for supervised classification tasks. We propose to learn the autoencoder for a semi-supervised paradigm, i.e. with both labeled and unlabeled samples available. Given labeled and unlabeled data, our proposed autoencoder automatically adjusts – for unlabeled data it acts as a standard autoencoder (unsupervised) and for labeled data it additionally learns a linear classifier. We use our proposed semi-supervised autoencoder to (greedily) construct a stacked architecture. We demonstrate the efficacy our design in terms of both accuracy and run time requirements for the case of image classification. Our model is able to provide high classification accuracy with even simple classification schemes as compared to existing models for deep architectures.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hinton, G.E., Zemel, R.S.: Autoencoders, minimum description length, and Helmholtz free energy. Adv. Neural Inf. Process. Syst. 6, 3–10 (1994)
Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P-A.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1096–1103. ACM (2008)
Chen, Y., Lin, Z., Zhao, X., Wang, G., Gu, Y.: Deep learning-based classification of hyperspectral data. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 7(6), 2094–2107 (2014)
Maldonado, S., Weber, R., Basak, J.: Simultaneous feature selection and classification using kernel-penalized support vector machines. Inf. Sci. 181(1), 115–128 (2011)
Kong, H., Li, X., Wang, L., Teoh, E.K., Wang, J.-G., Venkateswarlu, R.: Generalized 2D principal component analysis. In: Proceedings of 2005 IEEE International Joint Conference on Neural Networks, IJCNN 2005, vol. 1, pp. 108–113. IEEE (2005)
Bengio, Y.: Learning deep architectures for AI. Found. Trends® Mach. Learn. 2(1), 1–127 (2009)
Hinton, G.E., Osindero, S., Teh, Y.-W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)
Ciresan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3642–3649. IEEE (2012)
Gao, S., Zhang, Y., Jia, K., Lu, J., Zhang, Y.: Single sample face recognition via learning deep supervised autoencoders. IEEE Trans. Inf. Forensics Secur. 10(10), 2108–2118 (2015)
Huang, G., Song, S., Gupta, J.N.D., Wu, C.: Semi-supervised and unsupervised extreme learning machines. IEEE Trans. Cybern. 44(12), 2405–2417 (2014)
Ranzato, M., Szummer, M.: Semi-supervised learning of compact document representations with deep networks. In: Proceedings of the 25th International Conference on Machine Learning, pp. 792–799. ACM (2008)
Larochelle, H., Mandel, M., Pascanu, R., Bengio, Y.: Learning algorithm for the classification restricted boltzmann machine. J. Mach. Learn. Res. 13(1), 643–669 (2012)
Almousli, H., Vincent, P.: Semi supervised autoencoders: better focusing model capacity during feature extraction. In: Lee, M., Hirose, A., Hou, Z.-G., Kil, R.M. (eds.) ICONIP 2013. LNCS, vol. 8226, pp. 328–335. Springer, Heidelberg (2013)
Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.-A.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1096–1103. ACM (2008)
Lemme, A., Reinhart, R.F., Steil, J.J.: Online learning and generalization of parts-based image representations by non-negative sparse autoencoders. Neural Netw. 33, 194–203 (2012)
Jiang, Z., Lin, Z., Davis, L.S.: Learning a discriminative dictionary for sparse coding via label consistent K-SVD. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1697–1704. IEEE (2011)
Shrivastava, A., Pillai, J.K., Patel, V.M., Chellappa, R.: Learning discriminative dictionaries with partially labeled data. In: 2012 19th IEEE International Conference on Image Processing (ICIP), pp. 3113–3116. IEEE (2012)
http://www.iro.umontreal.ca/~lisa/twiki/bin/view.cgi/Public/DeepVsShallowComparisonICML2007
Lawson, C.L., Hanson, R.J.: Solving least squares problems, vol. 161. Prentice-hall, Englewood Cliffs (1974)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Ng, A.: Sparse autoencoder. CS294A Lecture notes 72 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Gogna, A., Majumdar, A. (2016). Semi Supervised Autoencoder. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds) Neural Information Processing. ICONIP 2016. Lecture Notes in Computer Science(), vol 9948. Springer, Cham. https://doi.org/10.1007/978-3-319-46672-9_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-46672-9_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46671-2
Online ISBN: 978-3-319-46672-9
eBook Packages: Computer ScienceComputer Science (R0)