Abstract
In this paper, we suggest multiple kernel learning with hierarchical feature representations. Recently, deep learning represents excellent performance to extract hierarchical feature representations in unsupervised manner. However, since fine-tuning step of deep learning only considers global level of features for classification problems, it makes each layers hierarchical features intractable. Therefore, we propose a method to employ the combined multiple levels of pre-trained features via Multiple Kernel Learning (MKL). MKL is lately proposed optimization problem in classification and is applied to various machine learning problems. MKL automatically finds the best combination of kernels. By applying multiple kernel learning to hierarchical features pre-trained by deep learning, we obtain the optimal combinations of multiple levels of features for the classification task. Also, MKL is applied to analyze the contribution of each layer of features for classification by obtained weight of each kernel.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bengio, Y.: Learning deep architectures for AI. Foundations and Trends in Machine Learning (2009)
Le Roux, N., Bengio, Y.: Representational power of restricted boltzmann machines and deep belief networks. Neural Computation 20, 1631–1649 (2008)
Hinton, G.E., Salakhutdinov, R.: Reducing the Dimensionality of Data with Neaural Netwoks. Science 313, 504–507 (2006)
Lee, H., Grosse, R., Ranganath, R., Ng, A.: Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: Proceedings of the 26th International Conference on Machine Learning, ICML (2009)
Bach, F., Lanckriet, G., Jordan, M.: Multiple kernel learning, conic duality, and the SMO algorithm. In: International Conference on Machine Learning (2004)
Yang, J., Li, Y., Tian, Y.: Group-sensitive multiple kernel learning for object categorization. In: Computer Vision (2009)
Vedaldi, A., Gulshan, V.: Multiple kernels for object detection. In: IEEE 12th International Conference on Computer Vision (2009)
Kloft, M., Rückert, U., Bartlett, P.L.: A Unifying View of Multiple Kernel Learning. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds.) ECML PKDD 2010, Part II. LNCS, vol. 6322, pp. 66–81. Springer, Heidelberg (2010)
Binder, A., Nakajima, S., Kloft, M., Müller, C.: Insights from Classifying Visual Concepts with Multiple Kernel Learning. PloS One 7 (2012)
Nakajima, S., Binder, A., Müller, C.: Multiple kernel learning for object classification. In: Proceedings of the 12th Workshop on Information-Based Induction Sciences (2009)
Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 297, 273–297 (1995)
Hinton, G.E.: Training Products of Experts by Minimizing Contrastive Divergence. Neural Computation 14, 1771–1800 (2002)
Orabona, F., Jie, L.: Ultra-fast optimization algorithm for sparse multi kernel learning. In: Conference on Machine Learning (2011)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradien-Based Learning Applied to Document Recognition. Proceedings of the IEEE (1998)
Coates, A., Lee, H., Ng, A.: An analysis of single-layer networks in unsupervised feature learning. Ann Arbor (2010)
Chang, C., Lin, C.: LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 1–39 (2011)
Salakhutdinov, R., Mnih, A., Hinton, G.: Restricted Boltzmann machines for collaborative filtering. In: Proceedings of the 24th International Conference on Machine Learning, ICML, pp. 791–798 (2007)
Kloft, M., Laskov, P., Zien, A.: Efficient and Accurate Lp -Norm Multiple Kernel Learning. In: Advances in Neural Information Processing Systems, NIPS (2009)
Gönen, M., Alpaydın, E.: Multiple kernel learning algorithms. Journal of Machine Learning Research 12, 2211–2268 (2011)
Hinton, G., Osindero, S., Teh, Y.: A fast learning algorithm for deep belief nets. Neural Computation 1554, 1527–1554 (2006)
Swersky, K., Chen, B., Marlin, B., de Freitas, N.: A tutorial on stochastic approximation algorithms for training Restricted Boltzmann Machines and Deep Belief Nets. In: 2010 Information Theory and Applications Workshop (ITA), pp. 1–10 (2010)
Salakhutdinov, R.: Learning deep generative models (2009)
MNIST database of handwritten digits, http://yann.lecun.com/exdb/mnist/
STL-10 dataset, http://cs.stanford.edu/~acoates/stl10
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lee, J., Lim, J.H., Choi, H., Kim, DS. (2013). Multiple Kernel Learning with Hierarchical Feature Representations. In: Lee, M., Hirose, A., Hou, ZG., Kil, R.M. (eds) Neural Information Processing. ICONIP 2013. Lecture Notes in Computer Science, vol 8228. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-42051-1_64
Download citation
DOI: https://doi.org/10.1007/978-3-642-42051-1_64
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-42050-4
Online ISBN: 978-3-642-42051-1
eBook Packages: Computer ScienceComputer Science (R0)