Abstract
It has been recently shown that the quadratic programming formulation underlying a number of kernel methods can be treated as a minimal enclosing ball (MEB) problem in a feature space where data has been previously embedded. Core Vector Machines (CVMs) in particular, make use of this equivalence in order to compute Support Vector Machines (SVMs) from very large datasets in the batch scenario. In this paper we study two algorithms for online classification which extend this family of algorithms to deal with large data streams. Both algorithms use analytical rules to adjust the model extracted from the stream instead of recomputing the entire solution on the augmented dataset. We show that these algorithms are more accurate than the current extension of CVMs to handle data streams using an analytical rule instead of solving large quadratic programs. Experiments also show that the online approaches are considerably more efficient than periodic computation of CVMs even though warm start is being used.
This work was supported by Research Grants 1110854 Fondecyt and Basal FB0821, “Centro Científico-Tecnológico de Valparaíso”, UTFSM.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Aggarwal, C. (ed.): Data Streams, Models and Algorithms. Springer, Heidelberg (2007)
Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines (2010)
Clarkson, K.: Coresets, sparse greedy approximation, and the frank-wolfe algorithm. In: Proceedings of SODA 2008, pp. 922–931. SIAM, Philadelphia (2008)
Crammer, K., Dekel, O., Keshet, J., Shalev-Shwartz, S., Singer, Y.: Online passive-aggressive algorithms. J. of Machine Learning Research 7, 551–585 (2006)
Dekel, O., Shalev-Shwartz, S., Singer, Y.: The forgetron: A kernel-based perceptron on a budget. SIAM Journal of Computing 37(5), 1342–1372 (2008)
Frandi, E., Gasparo, M.-G., Lodi, S., Ñanculef, R., Sartori, C.: A new algorithm for training sVMs using approximate minimal enclosing balls. In: Bloch, I., Cesar Jr., R.M. (eds.) CIARP 2010. LNCS, vol. 6419, pp. 87–95. Springer, Heidelberg (2010)
Hettich, S., Bay, S.: The UCI KDD Archive (2010), http://kdd.ics.uci.edu
Kivinen, J.: Online learning of linear classifiers, pp. 235–257 (2003)
Kivinen, J., Smola, A.J., Williamson, R.C.: Online learning with kernels. IEEE Transactions on Signal Processing 52(8), 2165–2176 (2004)
Lodi, S., Ñanculef, R., Sartori, C.: Single-pass distributed learning of multi-class svms using core-sets. In: Proceedings of the SDM 2010, pp. 257–268. SIAM, Philadelphia (2010)
Léon Bottou, D.D., Chapelle, O., Weston, J. (eds.): Large Scale Kernel Machines. MIT Press, Cambridge (2007)
Rai, P., Daumé, H., Venkatasubramanian, S.: Streamed learning: one-pass svms. In: IJCAI 2009: Proceedings of the 21st International Jont Conference on Artifical Intelligence, pp. 1211–1216. Morgan Kaufmann Publishers, San Francisco (2009)
Schölkopf, B., Smola, A.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2001)
Shalev-Shwartz, S., Singer, Y.: A primal-dual perspective of online learning algorithms. Machine Learning 69(2-3), 115–142 (2007)
Tsang, I., Kocsor, A., Kwok, J.: Simpler core vector machines with enclosing balls. In: ICML 2007, pp. 911–918. ACM, New York (2007)
Tsang, I., Kocsor, A., Kwok, J.: LibCVM Toolkit (2009)
Tsang, I., Kwok, J., Cheung, P.-M.: Core vector machines: Fast svm training on very large data sets. Journal of Machine Learning Research 6, 363–392 (2005)
Tsang, I., Kwok, J., Zurada, J.: Generalized core vector machines. IEEE Transactions on Neural Networks 17(5), 1126–1140 (2006)
Wang, D., Zhang, B., Zhang, P., Qiao, H.: An online core vector machine with adaptive meb adjustment. Pattern Recognition 43(10), 3468–3482 (2010)
Yildirim, E.A.: Two algorithms for the minimum enclosing ball problem. SIAM Journal on Optimization 19(3), 1368–1391 (2008)
Zarrabi-Zadeh, H., Chan, T.M.: A simple streaming algorithm for minimum enclosing balls. In: Proceedings of the CCCG 2006 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ñanculef, R., Allende, H., Lodi, S., Sartori, C. (2011). Two One-Pass Algorithms for Data Stream Classification Using Approximate MEBs. In: Dobnikar, A., Lotrič, U., Šter, B. (eds) Adaptive and Natural Computing Algorithms. ICANNGA 2011. Lecture Notes in Computer Science, vol 6594. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-20267-4_38
Download citation
DOI: https://doi.org/10.1007/978-3-642-20267-4_38
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-20266-7
Online ISBN: 978-3-642-20267-4
eBook Packages: Computer ScienceComputer Science (R0)