Abstract
In this paper, we propose and study a new on-line algorithm for learning a SVM based on Radial Basis Function Kernel: Local Incremental Learning of SVM or LISVM. Our method exploits the “locality” of RBF kernels to update current machine by only considering a subset of support candidates in the neighbourhood of the input. The determination of this subset is conditioned by the computation of the variation of the error estimate. Implementation is based on the SMO one, introduced and developed by Platt [13]. We study the behaviour of the algorithm during learning when using different generalization error estimates. Experiments on three data sets (batch problems transformed into on-line ones) have been conducted and analyzed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
L. Bottou and V. Vapnik. Local learning algorithms. Neural Computation, 4(6):888–900, 1992.
L. Breiman. Bias, variance and arcing classifiers. Technical Report 460, University of California, Berkeley, CA, USA, 1996.
C. Burges. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2(2):955–974, 1998.
G. Cauwenberghs and T. Poggio. Incremental and decremental support vector machine learning. In Adv. Neural Information Processing, volume 13. MIT Press, 2001.
O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee. Choosing kernel parameters for support vector machines. Technical report, AT&T Labs, March 2000.
C. Cortes and V. Vapnik. Support vector networks. Machine Learning, 20:1–25, 1995.
Nello Cristianini and John Shawe-Taylor. An Introduction to Support Vector Machines and other kernel-based learning methods, chapter 4 Generalisation Theory, page 68. Cambridge University Press, 2000.
T. Friess, F. Cristianini, and N. Campbell. The kernel-adatron algorithm: a fast and simple learning procedure for support vector machines. In J. Shavlik, editor, Machine Learning: Proc. of the 15 th Int. Conf. Morgan Kaufmann Publishers, 1998.
T. Joachims. Making large-scale support vector machine learning practical. In B. Schölkopf, C. Burges, and A. Smola, editors, Advances in Kernel Methods-Support Vector Learning, pages 169–184. MIT Press, Cambridge, MA, 1998.
T. Joachims. Estimating the generalization performance of a svm efficiently. In Proc. of the 17 th Int. Conf. on Machine Learning. Morgan Kaufmann, 2000.
R. Klinkenberg and J. Thorsten. Detecting concept drift with support vector machines. In Proc. of the 17 th Int. Conf. on Machine Learning. Morgan Kaufmann, 2000.
E. Osuna, R. Freund, and F. Girosi. Improved training algorithm for support vector machines. In Proc. IEEE Workshop on Neural Networks for Signal Processing, pages 276–285, 1997.
J.C. Platt. Sequential minimal optimization: A fast algorithm for training support vector machines. Technical Report 98-14, Microsof Research, April 1998.
G. Rätsch, T. Onoda, and K.-R. Müller. Soft margins for AdaBoost. Technical Report NC-TR-1998-021, Department of Computer Science, Royal Holloway, University of London, Egham, UK, 1998.
N. Syed, H. Liu, and K. Sung. Incremental learning with support vector machines. In Proc. of the Int. Joint Conf. on Artificial Intelligence (IJCAI), 1999.
V. Vapnik. The nature of statistical learning theory. Springer, New York, 1995.
V. Vapnik. Statistical learning theory. John Wiley and Sons, inc., 1998.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ralaivola, L., d’Alché-Buc, F. (2001). Incremental Support Vector Machine Learning: A Local Approach. In: Dorffner, G., Bischof, H., Hornik, K. (eds) Artificial Neural Networks — ICANN 2001. ICANN 2001. Lecture Notes in Computer Science, vol 2130. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44668-0_46
Download citation
DOI: https://doi.org/10.1007/3-540-44668-0_46
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42486-4
Online ISBN: 978-3-540-44668-2
eBook Packages: Springer Book Archive