Abstract
Learning robust and invariant feature representations is always a crucial task in visual recognition and analysis. Mean square error (MSE) has been used in many feature encoding methods as a feature reconstruction criterion. However, due to the non-Gaussian noises and non-linearity structures in natural images, second order statistics like MSE are usually not sufficient to capture these information from image data. In this paper, motivated by the information-theoretic learning framework and kernel machine learning, we adopt a similarity measure called correntropy in the auto-encoder model to tackle this problem. The proposed maximum correntropy auto-encoder (MCAE) learns more robust and discriminative representations than MSE based model by performing computation in an infinite dimensional kernel space. Moreover, we further exploit the power of kernel by learning a kernel embedding neural network which explicitly maps data from Euclidean space to an approximated kernel space. Experimental results on standard object categorization datasets show the effectiveness of kernel learning in feature representation for visual recognition task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Jianchao, Y., Kai, Y., Yihong, G., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 1794–1801 (2009)
Le, Q.V., Karpenko, A., Ngiam, J., Ng, A.Y.: ICA with Reconstruction Cost for Efficient Overcomplete Feature Learning. In: Advances in Neural Information Processing Systems (2011)
Bengio, Y.: Learning deep architectures for ai. Foundations and Trends® in Machine Learning 2(1), 1–127 (2009)
Hinton, G., Osindero, S., Teh, Y.: A fast learning algorithm for deep belief nets. Neural Computation 18(7), 1527–1554 (2006)
Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H.: Greedy layer-wise training of deep networks. Advances in Neural Information Processing Systems 19, 153 (2007)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)
Liu, W., Pokharel, P.P., Principe, J.C.: Correntropy: Properties and Applications in Non-Gaussian Signal Processing. IEEE Transactions on Signal Processing 55, 5286–5298 (2007)
Yuan, X.T., Hu, B.G.: Robust feature extraction via information theoretic learning. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML 2009, pp. 1193–1200. ACM, New York (2009)
He, R., Zheng, W.S., Hu, B.G.: Maximum correntropy criterion for robust face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(8), 1561–1576 (2011)
Rahimi, A., Recht, B.: Random features for large-scale kernel machines. Advances in Neural Information Processing Systems 20, 1177–1184 (2007)
Maji, S., Berg, A.: Max-margin additive classifiers for detection. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 40–47. IEEE (2009)
Vedaldi, A., Zisserman, A.: Efficient additive kernels via explicit feature maps. Pattern Analysis and Machine Intellingence 34(3) (2011)
Yu, K., Xu, W., Gong, Y.: Deep learning with kernel regularization for visual recognition. In: NIPS 2008, pp. 1889–1896 (2008)
Rumelhart, D., Hintont, G., Williams, R.: Learning representations by back-propagating errors. Nature 323(6088), 533–536 (1986)
Baldi, P., Hornik, K.: Neural networks and principal component analysis: Learning from examples without local minima. Neural Networks 2(1), 53–58 (1989)
Hyvärinen, A., Hurri, J., Hoyer, P.: Natural Image Statistics: A Probabilistic Approach to Early Computational Vision, vol. 39. Springer (2009)
Rifai, S., Vincent, P., Muller, X., Glorot, X., Bengio, Y.: Contracting auto-encoders: Explicit invariance during feature extraction. In: Proceedings of the Twenty-Eight International Conference on Machine Learning (ICML 2011) (June 2011)
Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1096–1103. ACM (2008)
Principe, J., Xu, D.: Information-theoretic learning using reny’s quadratic entropy. In: First International Workshop on Independent Component Analysis (ICA 1999), pp. 407–412. Citeseer (1999)
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images. Master’s thesis, Department of Computer Science, University of Toronto (2009)
Coates, A., Lee, H., Ng, A.: An analysis of single-layer networks in unsupervised feature learning. In: AISTATS 14, vol. 1001, p. 48109 (2011)
Fan, R., Chang, K., Hsieh, C., Wang, X., Lin, C.: Liblinear: A library for large linear classification. The Journal of Machine Learning Research 9, 1871–1874 (2008)
Yu, K., Zhang, T.: Improved local coordinate coding using local tangents (June 2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ren, W., Yu, Y., Zhang, J., Huang, K. (2013). Exploring the Power of Kernel in Feature Representation for Object Categorization. In: Lee, M., Hirose, A., Hou, ZG., Kil, R.M. (eds) Neural Information Processing. ICONIP 2013. Lecture Notes in Computer Science, vol 8228. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-42051-1_67
Download citation
DOI: https://doi.org/10.1007/978-3-642-42051-1_67
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-42050-4
Online ISBN: 978-3-642-42051-1
eBook Packages: Computer ScienceComputer Science (R0)