Abstract
Higher level features learning algorithms have been applied on handwritten digit recognition and got more promising results than just using raw intensity values with classification algorithms. However, the approaches of these algorithms still not take the advantage of specific characteristics of data. We propose a new method to learn higher level features from specific characteristics of data using sparse autoencoder. The main key of our appoarch is to divide the handwritten digits into subsets corresponding to specific characteristics. The experimental results show that the proposed method achieves lower error rates and time complexity than the original approach of sparse autoencoder. The results also show that the more correlated characteristics we define, the better higher level features we learn.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bagnell, J.A., Bradley, D.M.: Differentiable sparse coding. In: NIPS. pp. 113–120 (2008)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR. vol. 1, pp. 886–893 (2005)
DeCoste, D., Schölkopf, B.: Training invariant support vector machines. Mach. Learn. 46(1–3), 161–190 (2002)
Grosse, R., Raina, R., Kwong, H., Andrew, Y.Ng.: Shift-invariance sparse coding for audio classification. CoRR, abs/1206.5241, 2012.
Hinton, G., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)
Lennig M., Hunt, M., Mermelstein, P.: Experiments in syllable-based recognition of continuous speech. In: Proceedings of International Conference on Acoustics, Speech and, Signal Processing, pp. 880–883 (1996)
Huth, A.G., Gallant, J.L., Vu, A.T., Nishimoto, S.: A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron 76(6), 1210–1224 (2012)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, H.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE. vol. 86, pp. 2278–2324 (1998)
Lee, H., Battle, A., Raina, R., Andrew, Y.Ng.: Efficient sparse coding algorithms. In: Advances in Neural Information Processing Systems. vol. 19, pp. 801–808 (2007)
Lee, H., Grosse, R., Ranganath, R., Andrew, Y.Ng.: Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: ICML. p. 77 (2009)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)
Martens, J.: Deep learning via hessian-free optimization. In: Fürnkranz, J., Joachims, T.,(eds.) Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel, pp. 735–742 June 2010
Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)
Raina, R., Battle, A., Lee, H., Packer, B., Andrew Y.Ng.: Self-taught learning: Transfer learning from unlabeled data. In: ICML ’07: Proceedings of the 24th International Conference on, Machine learning (2007)
Simard, P., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks applied to visual document analysis. In: ICDAR, pp. 958–962 (2003)
Socher, R., Eric, H., Pennin, J., Andrew, Y.Ng, Manning, C.D.: Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In: Shawe-Taylor, J., Zemel, R.S., Bartlett, P., Pereira, F.C.N., Weinberger, K.Q., (eds.) Proceeding of 24th Advances in Neural Information Processing Systems, pp. 801–809 (2011)
Kai. Y., Zhang, T., Gong, Y.: Nonlinear learning using local coordinate coding. In: Bengio, Y., Schuurmans, D., Lafferty, J.D., Williams, C.K.I., Culotta, A. (eds.) Proceeding of 24th Advances in Neural Information Processing Systems, NIPS, pp. 2223–2231 (2009) (Curran Associates Inc.)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Duong, A.T., Phan, H.T., Le, N.DH., Tran, S.T. (2014). A Hierarchical Approach for Handwritten Digit Recognition Using Sparse Autoencoder. In: Kóczy, L., Pozna, C., Kacprzyk, J. (eds) Issues and Challenges of Intelligent Systems and Computational Intelligence. Studies in Computational Intelligence, vol 530. Springer, Cham. https://doi.org/10.1007/978-3-319-03206-1_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-03206-1_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-03205-4
Online ISBN: 978-3-319-03206-1
eBook Packages: EngineeringEngineering (R0)