Abstract
Incremental learning has been widely addressed in machine learning literature to deal with tasks where the learning environment is steadily changing or training samples become available one after another over time. Support Vector Machine has been successfully used in pattern recognition and function estimation. In order to tackle with incremental learning problems with new features, an incremental feature learning algorithm based on Least Square Support Vector Machine is proposed in this paper. In this algorithm, features of newly joined samples contain two parts: already existing features and new features. Using historic structural parameters which are trained from the already existing features, the algorithm only trains the new features with Least Square Support Vector Machine. Experiments show that this algorithm has two outstanding properties. First, different kernel functions can be used for the already existing features and the new features according to the distribution of samples. Consequently, this algorithm is more suitable to deal with classification tasks which can not be well solved by using a single kernel function. Second, the training time and the memory space can be reduced because the algorithm fully uses the structural parameters of classifiers trained formerly and only trains the new features with Least Square Support Vector Machine. Some UCI datasets are used to demonstrate the less training time and comparable or better performance of this algorithm than the Least Square Support Vector Machine.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Vapnik, V.: The nature of statistical learning theory. Springer, New York (1995)
Vapnik, V.: Statistical learning theory. John Wiley, New York (1998)
Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge (2000)
Schökopf, B., Burges, C., Smola, A. (eds.): Advances in Kernel Methods-Support Vector Learning. MIT press, Cambridge (1999)
Schökopf, B., Mika, S., Burge, C., Knirsch, P., Müller, K.-R., Rätsch, G., Smola, A.: Input Space vs. Feature Space in Kernel-Based Methods. IEEE Transactions on Neural Networks 10(5), 1000–1017 (1999)
Suykens, J.A.K., Vandewalle, J.: Least squares support vector machine classifiers. Neural Processing Letters 9(3), 293–300 (1999)
Domeniconi, C., Gunopulos, D.: Incremental support vector machine construction. In: Proceedings IEEE International Conference on Data Mining, 2001, ICDM 2001, pp. 589–592 (2001)
Alistair Shilton, M.: Incremental Training of Support Vector Machines. IEEE transactions on neural networks 16(1), 114–131 (2005)
Ong, C.S., Smola, A.J., Williamson, R.C.: Learning the Kernel with Hyperkernels. Journal of Machine Learning Research 6, 1043–1071 (2005)
Zhao, Y., He, Q.: An Incremental Learning Algorithm Based on Support Vector Domain Classifier. In: Proc. 5th IEEE Int. Conf. on Cognitive Information, pp. 805–809 (2006)
Shilton, A., Palaniswami, M., Ralph, D., Tsoi, A.: Incremental training of support vector machines. IEEE Trans. Neural Netw. 16, 114–131 (2005)
Pelckmans, K., Karsmakers, P., Suykens, J.A.K., De Moor, B.: Ordinal Least Squares Support Vector Machines – a Discriminant Analysis Approach. In: Proc. of the Machine Learning for Signal Processing (MLSP 2006), Maynooth, Ireland, September 2006, pp. 1–8 (2006)
Kazushi, I., Takemasa, Y.: Incremental support vector machines and their geometrical analyses. In: Neurocomputing, pp. 2528–2533 (2007)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, X., Zhang, G., Zhan, Y., Zhu, E. (2008). An Incremental Feature Learning Algorithm Based on Least Square Support Vector Machine. In: Preparata, F.P., Wu, X., Yin, J. (eds) Frontiers in Algorithmics. FAW 2008. Lecture Notes in Computer Science, vol 5059. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69311-6_34
Download citation
DOI: https://doi.org/10.1007/978-3-540-69311-6_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69310-9
Online ISBN: 978-3-540-69311-6
eBook Packages: Computer ScienceComputer Science (R0)