Abstract
Kernel methods comprise a class of machine learning algorithms that utilize Mercer kernels for producing nonlinear versions of conventional linear learning algorithms. This kernelizing approach has been applied, for example, to the famed least mean squares (LMS) [1] algorithm to give rise to the kernel least mean squares (KLMS) algorithm [2]. However, a major drawback of the LMS algorithm (and also of its kernelized version) is the performance degradation in scenarios with outliers. Bearing this in mind, we introduce instead a kernel classifier based on the least mean M-estimate (LMM) algorithm [3] which is a robust variant of the LMS algorithm based on M-estimation techniques. The proposed Kernel LMM (KLMM) algorithm is evaluated in pattern classification tasks with outliers using both synthetic and real-world datasets. The obtained results indicate the superiority of the proposed approach over the standard KLMS algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Widrow, B., Winter, R.: Neural nets for adaptive filtering and adaptive pattern recognition. IEEE Computer 21(3), 25–39 (1988)
Liu, W., Pokharel, P., Principe, J.: The kernel least-mean-square algorithm. IEEE Transactions on Signal Processing 56(2), 543–554 (2008)
Chan, S.C., Zhou, Y.: On the performance analysis of the least mean M-estimate and normalized least mean M-estimate algorithms with gaussian inputs and additive gaussian and contaminated gaussian noises. Journal of Signal Processing Systems 80(1), 81–103 (2010)
Vapnik, V.: The Nature of Statistical Learning Theory. Springer (1995)
Suykens, J.A.K., Gestel, V.T., DeBrabanter, J., DeMoor, B., Vandewalle, J.: Least Squares Support Vector Machines. World Scientific (2002)
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)
Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural networks architectures. Neural Computation 7, 219–269 (1995)
Bolzern, P., Colaneri, P., De Nicolao, G.: H ∞ -robustness of adaptive filters against measurement noise and parameter drift. Automatica 35(9), 1509–1520 (1999)
Hassibi, B., Sayed, A.H., Kailath, T.: H ∞ optimality of the LMS algorithm algorithm. IEEE Transactions on Signal Processing 44(2), 267–280 (1996)
Frénay, B., Verleysen, M.: Classification in the presence of label noise: a survey. IEEE Transactions on Neural Networks and Learning Systems 25(5), 845–869 (2014)
Friess, T.T., Cristianini, N., Campbell, C.: The kernel Adatron algorithm: A fast and simple learning procedure for support vector machines. In: Proceedings of the 15th International Conference of Machine Learning (ICML 1998), pp. 188–196 (1998)
Huber, P.J.: Robust estimation of a location parameter. Annals of Mathematical Statistics 35(1), 73–101 (1964)
Zou, Y., Chan, S.C., Ng, T.S.: Least mean M-estimate algorithms for robust adaptive filtering in impulsive noise. IEEE Transactions on Circuits and Systems II 47(12), 1564–1569 (2000)
de Paula Barros, A.L.B., Barreto, G.A.: Improving the classification performance of optimal linear associative memory in the presence of outliers. In: Rojas, I., Joya, G., Gabestany, J. (eds.) IWANN 2013, Part I. LNCS, vol. 7902, pp. 622–632. Springer, Heidelberg (2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Santos, J.D.A., Mattos, C.L.C., Barreto, G.A. (2014). A Novel Recursive Kernel-Based Algorithm for Robust Pattern Classification. In: Corchado, E., Lozano, J.A., Quintián, H., Yin, H. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2014. IDEAL 2014. Lecture Notes in Computer Science, vol 8669. Springer, Cham. https://doi.org/10.1007/978-3-319-10840-7_19
Download citation
DOI: https://doi.org/10.1007/978-3-319-10840-7_19
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10839-1
Online ISBN: 978-3-319-10840-7
eBook Packages: Computer ScienceComputer Science (R0)