Abstract
Incremental and decremental processes of training a support vector machine (SVM) resumes to the migration of vectors in and out of the support set along with modifying the associated thresholds. This paper gives an overview of all the boundary conditions implied by vector migration through the incremental / decremental process. The analysis will show that the same procedures, with very slight variations, can be used for both the incremental and decremental learning. The case of vectors with duplicate contribution is also considered. Migration of vectors among sets on decreasing the regularization parameter is given particularly attention. Experimental data show the possibility of modifying this parameter on a large scale, varying it from complete training (overfitting) to a calibrated value.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bottou, L., Le Cun, Y.: Large Scale Online Learning. In: Thrun, S., Saul, L., Schölkopf, B. (eds.) Advances in Neural Information Processing Systems, vol. 16. MIT Press, Cambridge (2004)
Chang, C.C., Lin, C.J.: LIBSVM: a Library for Support Vector Machines (2001), http://www.csie.ntu.edu.tw/~cjlin/libsvm
Burges, C.J., Crisp, D.J.: Uniqueness of the SVM Solution. In: Solla, S.A., Leen, T.K., Muller, K.R. (eds.) Advances in Neural Information Processing Systems, vol. 12. Morgan Kaufmann, San Francisco (2000)
Blake, C.L., Merz, C.J.: UCI repository of machine learning databases, University of California, Irvine, Dept. of Information and Computer Sciences (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
Diehl, C.P., Cauwenberghs, G.: SVM Incremental Learning, Adaptation and Optimization. In: Proceedings of the IJCNN, vol. 4 (2003)
Alpaydin, E.: Introduction to Machine Learning. MIT Press, Cambridge (2004)
d’Alche-Buc, F., Ralaivola, L.: Incremental Learning Algorithms for Classification and Regression: local strategies. In: American Institute of Physics Conference Proc., vol. 627, pp. 320–329 (2002)
Cauwenberghs, G., Poggio, T.: Incremental and Decremental Support Vector Machine Learning. In: Neural Information Processing Systems, Denver (2000)
Fung, G., Mangasarian, O.: Incremental Support Vector Machine Classification. In: Advances in Neural Information Processing Systems. MIT Press, Cambridge (2003)
Hull, J.J.: A Database for Handwritten Text Recognition Research. IEEE Transactions on Pattern Analysis and Machine Intelligence 16(5), 550–554 (1994)
Taylor, J.S., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)
Ma, J., Thelier, J., Perkins, S.: Accurate On-line Support Vector Regression. Neural Computation (2003)
Martin, M.: On-line Support Vector Machine Regression. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) ECML 2002. LNCS (LNAI), vol. 2430. Springer, Heidelberg (2002)
Syed, N.A., Liu, H., Sung, K.K.: Incremental Learning with Support Vector Machines. In: Proc. of the Workshop on Support Vector Machines at the International Joint Conference on Artificial Intelligence, IJCAI 1999, Stockholm, Sweden (1999)
Laskov, P., Gehl, C., Krüger, S., Müller, K.R.: Incremental Support Vector Learning: Analysis, Implementation and Applications. Journal of Machine Learning Research 7, 1909–1936 (2006)
Reuters 21578 Text Categorization Collection, Dataset, http://kdd.ics.uci.edu/databases/reuters21578/reuters21578.html
Rüping, S.: Incremental Learning with Support Vector Machines. In: Proceedings of the 2001 IEEE International Conference on Data Mining, ICDM 2001 (2001)
Theodoridis, S., Koutroumbas, K.: Pattern Recognition, 2nd edn. Elsevier Academic Press, Amsterdam (2003)
Joachims, T.: Learning to Classify Text using Support Vector Machines: Methods, Theory and Algorithms. Kluwer Academic Publishers, Dordrecht (2002)
Wang, W., Men, C., Lu, W.: Online Prediction Model Based on Support Vector Machine. Neurocomputing 71, 550–558 (2008)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gâlmeanu, H., Andonie, R. (2008). Implementation Issues of an Incremental and Decremental SVM. In: Kůrková, V., Neruda, R., Koutník, J. (eds) Artificial Neural Networks - ICANN 2008. ICANN 2008. Lecture Notes in Computer Science, vol 5163. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87536-9_34
Download citation
DOI: https://doi.org/10.1007/978-3-540-87536-9_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87535-2
Online ISBN: 978-3-540-87536-9
eBook Packages: Computer ScienceComputer Science (R0)