Abstract
Novelty detection from multiple information sources is an important problem and selecting appropriate features is a crucial step for solving this problem. In this paper, we propose a novel data domain description algorithm which is inspired by multiple kernel learning and elastic-net-type constrain on the kernel weight. Most Multiple kernel learning algorithms employ the 1-norm constraints on the kernel combination weights, which enforce a sparsity solution but maybe lose useful information. In contrast, imposing the p-norm(p>1) constraint on the kernel weights will keep all the information in the base kernels, which lead to non-sparse solutions and brings the risk of being sensitive to noise and incorporating redundant information. To address this problem, we introduce an elastic-net-type constrain on the kernel weights. It finds the best trade-off between sparsity and accuracy. Furthermore, our algorithm facilitates the grouping effect. The proposed algorithm can be equivalently formalized as a convex-concave problem that can be effectively resolved with level method. Experimental results show that the proposed algorithm converges rapidly and demonstrate its efficiency comparing to other data description algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Rakotomamonjy, A., et al.: SimpleMKL. Journal of Machine Learning Research 9, 2491–2521 (2008)
Ritter, G., Gallegos, M.T.: Outliers in statistical pattern recognition and an application to automatic chromosome classification. Pattern Recognition Letters 18(6), 525–539 (1997)
Tarassenko, L., et al.: Novelty detection for the identification of masses in mammograms. In: Proceedings of the 4th IEE International Conference on Artificial Neural Networks 1995, pp. 442–447 (1995)
Tax, D.M.J., Duin, R.P.W.: Support vector domain description. Pattern Recognition Letters 20 (1999)
Scholkopf, B., et al.: Support Vector Method for Novelty Detection. In: Advances in Neural Information Processing Systems (December 2000)
Lanckriet, G.R.G., et al.: Learning the kernel matrix with semidefinite programming. Journal of Machine Learning Research 5, 27–72 (2004)
Sonnenburg, S., et al.: Large scale multiple kernel learning. Journal of Machine Learning Research 7, 1531–1565 (2006)
Aflalo, J., et al.: Variable Sparsity Kernel Learning. Journal of Machine Learning Research 12, 565–592 (2011)
Kloft, M., et al.: l(p)-Norm Multiple Kernel Learning. Journal of Machine Learning Research 12, 953–997 (2011)
Mehmet, G., Ethem, A.: Multiple Kernel Learning Algorithms. Journal of Machine Learning Research 12, 2211–2268 (2011)
Lampert, C.H., Blaschko, M.B.: A multiple kernel learning approach to joint multi-class object detection. In: Annual Symposium of the Deutsche-Arbeitsgemeinschaft-fur-Mustererkennung (DAGM), Munich, Germany (2008)
Longworth, C., Gales, M.J.F., IEEE: Multiple Kernel Learning for speaker verification. In: 33rd IEEE International Conference on Acoustics, Speech and Signal Processing, Las Vegas, NV (2008)
Qiu, S., Lane, T.: Multiple Kernel Support Vector Regression for siRNA Efficacy Prediction. In: Măndoiu, I., Wang, S.-L., Zelikovsky, A. (eds.) ISBRA 2008. LNCS (LNBI), vol. 4983, pp. 367–378. Springer, Heidelberg (2008)
Lin, Y.Y., Liu, T.L., Fuh, C.S.: Multiple Kernel Learning for Dimensionality Reduction. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(6), 1147–1160 (2011)
Yeh, C.Y., Huang, C.W., Lee, S.J.: A multiple-kernel support vector regression approach for stock market price forecasting. Expert Systems with Applications 38(3), 2177–2186 (2011)
Suard, F., et al.: Model selection in pedestrian detection using multiple kernel learning. In: IEEE Intelligent Vehicles Symposium, Istanbul, Turkey (2007)
Bach, F.R.: Consistency of the group Lasso and multiple kernel learning. Journal of Machine Learning Research 9, 1179–1225 (2008)
Cristianini, N., et al.: On kernel-target alignment. In: Advances in Neural Information Processing Systems 14, vol. 1,2, pp. 367–373 (2002)
Ye, J.P., Ji, S.W., Chen, J.H.: Multi-class discriminant kernel learning via convex programming. Journal of Machine Learning Research 9, 719–758 (2008)
Chapelle, O., et al.: Choosing multiple parameters for support vector machines. Machine Learning 46(1-3), 131–159 (2002)
Srebro, N., Ben-David, S.: Learning Bounds for Support Vector Machines with Learned Kernels. In: Lugosi, G., Simon, H.U. (eds.) COLT 2006. LNCS (LNAI), vol. 4005, pp. 169–183. Springer, Heidelberg (2006)
Bach, F.R., Lanckriet, G.R.G., Jordan, M.I.: Multiple kernel learning, conic duality, and the SMO algorithm. In: Proceedings of the Twenty-first International Conference on Machine Learning, pp. 6–13. ACM, Banff (2004)
Xu, Z., et al.: An extended level method for efficient multiple kernel learning. In: Advances in Neural Information Processing Systems, vol. 21, pp. 1825–1832 (2009)
Orabona, F., Jie, L.: Ultra-Fast Optimization Algorithm for Sparse Multi Kernel Learning. In: Proceedings of the 28th International Conference on Machine Learning, Bellevue, Washington (2011)
Kloft, M., Brefeld, U., Laskov, P.: Non-sparse multiple kernel learning. In: NIPS Workshop on Kernel Learning: Automatic Selection of Optimal Kernels (2008)
Kloft, M., Nakajima, S., Brefeld, U.: Feature Selection for Density Level-Sets. In: Buntine, W., Grobelnik, M., Mladenić, D., Shawe-Taylor, J. (eds.) ECML PKDD 2009. LNCS, vol. 5781, pp. 692–704. Springer, Heidelberg (2009)
Shawe-Taylor, J., Hussain, Z.: Kernel learning for novelty detection. In: Proceedings of the NIPS Workshop on Kernel Learning 2008 (2008)
Haiqin, Y., et al.: Efficient Sparse Generalized Multiple Kernel Learning. IEEE Transactions on Neural Networks 22(3), 433–446 (2011)
Grandvalet, Y.: Least absolute shrinkage is equivalent to quadratic penalization. In: Niklasson, L., Bod´en, M., Ziemske, T. (eds.) Proceedings of the 8th International Conference on Artificial Neural Networks, pp. 201–206 (1998)
Mosek. The MOSEK Optimization Software (2011), http://www.mosek.com/index.php?id=2 (cited 2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chen, X., Ma, Y., Chang, L., Chen, G. (2011). Variable Sparse Multiple Kernels Learning for Novelty Detection. In: Lee, G. (eds) Advances in Automation and Robotics, Vol.1. Lecture Notes in Electrical Engineering, vol 122. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-25553-3_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-25553-3_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-25552-6
Online ISBN: 978-3-642-25553-3
eBook Packages: EngineeringEngineering (R0)