Skip to main content
SpringerLink
Log in

Incremental SVM Learning: Review

  • Chapter
  • First Online:
Learning from Data Streams in Evolving Environments

Part of the book series: Studies in Big Data ((SBD,volume 41))

Abstract

The aim of this paper is to present a review of methods for incremental Support Vector Machines (SVM) learning and their adaptation for data stream classification in evolving environments. We formalize a taxonomy of these methods based on their characteristics and the type of solution they provide. We discuss the strength and weakness of the various learning methods and also highlight some applications involving data stream, where incremental SVM learning has been used.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/diehl/Incremental-SVM-Learning-in-MATLAB.

  2. 2.

    For brevity and clarity, the derivation of the solution of Eq. (13) is not shown in this paper, the interested reader is referred to [2] for a detail explanation.

References

  1. Anguita, D., Ghio, A., Lawal, I.A., Oneto, L.: A heuristic approach to model selection for online support vector machines. In: Proceedings of the International Workshop on Advances in Regularization, Optimization, Kernel Methods and Support Vector Machines: Theory and Application, pp. 77–78 (2013)

    Google Scholar 

  2. Boukharouba, K., Bako, L., Lecoeuche, S.: Incremental and decremental multi-category classification by support vector machines. In: Proceedings of the International Conference on Machine Learning and Applications, pp. 294–300 (2009)

    Google Scholar 

  3. Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Disc. 2(2), 121–167 (1998)

    Article  Google Scholar 

  4. Cauwenberghs, G., Poggio, T.: Incremental and decremental support vector machine learning. In: Proceedings of the International Conference on Advances in Neural Information Processing Systems, pp. 409–415 (2000)

    Google Scholar 

  5. Chapelle, O.: Training a support vector machine in the primal. Neural Comput. 19(5), 1155–1178 (2007)

    Article  MathSciNet  Google Scholar 

  6. Chen, M.S., Ho, T.Y., Huang, D.Y.: Online transductive support vector machines for classification. In: Proceedings of the International Conference on Information Security and Intelligent Control, pp. 258–261 (2012)

    Google Scholar 

  7. Cortes, C., Vapnik, V.: Supportvector networks. Mach. Learn. 20, 273–297 (1995)

    MATH  Google Scholar 

  8. Dehestani, D., Eftekhari, F., Guo, Y., Ling, S., Su, S., Nguyen, H.: Online support vector machine application for model based fault detection and isolation of HVAC system. Int. J. Mach. Learn. Comput. 1(1), 66–72 (2011)

    Article  Google Scholar 

  9. Diehl, C., Cauwenberghs, G.: SVM incremental learning, adaptation and optimization. In: Proceedings of the International Joint Conference on Neural Networks, pp. 2685–2690 (2003)

    Google Scholar 

  10. Diethe, T., Girolami, M.: Online learning with multiple kernels: a review. Neural Comput. 25(3), 567–625 (2013)

    Article  MathSciNet  Google Scholar 

  11. Domeniconi, C., Gunopulos, D.: Incremental support vector machine construction. In: Proceedings of the IEEE International Conference on Data Mining, pp. 589–592 (2001)

    Google Scholar 

  12. Duan, H., Shao, X., Hou, W., He, G., Zeng, Q.: An incremental learning algorithm for lagrangian support vector machines. Pattern Recogn. Lett. 30(15), 1384–1391 (2009)

    Article  Google Scholar 

  13. Fan, H., Song, Q., Yang, X., Xu, Z.: Kernel online learning algorithm with state feedbacks. Knowl.-Based Syst. 89, 173–180 (2015)

    Article  Google Scholar 

  14. Galmeanu, H., Andonie, R.: A multi-class incremental and decremental svm approach using adaptive directed acyclic graphs. In: Proceedings of the International Conference on Adaptive and Intelligent Systems, pp. 114–119 (2009)

    Google Scholar 

  15. Gâlmeanu, H., Sasu, L.M., Andonie, R.: Incremental and decremental svm for regression. Int. J. Comput. Commun. Control 11(6), 755–775 (2016)

    Article  Google Scholar 

  16. Gama, J.: A survey on learning from data streams: current and future trends. Prog. Artif. Intell. 1(1), 45–55 (2012)

    Article  Google Scholar 

  17. Gepperth, A., Hammer, B.: Incremental learning algorithms and applications. In: Proceedings of the European Sympoisum on Artificial Neural Networks, pp. 357–368 (2016)

    Google Scholar 

  18. Guo, J.: An improved incremental training approach for large scaled dataset based on support vector machine. In: Proceedings of the International Conference on Big Data Computing Applications and Technologies, pp. 149–157 (2016)

    Google Scholar 

  19. He, H., Chen, S., Li, K., Xu, X.: Incremental learning from stream data. IEEE Trans. Neural Netw. 22(12), 1901–1914 (2011)

    Article  Google Scholar 

  20. Ho, A.K.N., Ragot, N., Ramel, J.Y., Eglin, V., Sidere, N.: Document classification in a non-stationary environment: a one-class svm approach. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 616–620 (2013)

    Google Scholar 

  21. Hsieh, C.J., Si, S., Dhillon, I.: A divide-and-conquer solver for kernel support vector machines. In: Proceedings of the International Conference on Machine Learning, pp. 566–574 (2014)

    Google Scholar 

  22. Ikeda, K., Yamasaki, T.: Incremental support vector machines and their geometrical analyses. Neurocomputing 70(13–15), 2528–2533 (2007)

    Article  Google Scholar 

  23. JinHyuk, H., Sung-Bue, C.: Incremental support vector machine for unlabeled data classification. In: Proceedings of the International Conference on Neural Information Processing, pp. 1403–1407 (2002)

    Google Scholar 

  24. Joachims, T.: Making large-scale support vector machine learning practical. In: Schölkopf, B., Burges, C.J.C., Smola, A.J. (eds.) Advances in Kernel Methods, pp. 169–184. MIT Press, Cambridge, MA (1999)

    Google Scholar 

  25. Joachims, T.: Transductive inference for text classification using support vector machines. In: International Conference on Machine Learning, pp. 200–209 (1999)

    Google Scholar 

  26. Kapp, M.N., Sabourin, R., Maupin, P.: Adaptive incremental learning with an ensemble of support vector machines. In: Proceedings of the International Conference on Pattern Recognition, pp. 4048–4051 (2010)

    Google Scholar 

  27. Karasuyama, M., Takeuchi, I.: Multiple incremental decremental learning of support vector machines. IEEE Trans. Neural Netw. 21(7), 1048–1059 (2010)

    Article  Google Scholar 

  28. Katagiri, S., Abe, S.: Incremental training of support vector machines using hyperspheres. Pattern Recogn. Lett. 27(13), 1495–1507 (2006)

    Article  Google Scholar 

  29. Kembhavi, A., Siddiquie, B., Miezianko, R., McCloskey, S., Davis, L.S.: Incremental multiple kernel learning for object recognition. In: Proceedings of the International Conference on Computer Vision, pp. 638–645 (2009)

    Google Scholar 

  30. Kohonen, T.: Self-organized formation of topologically correct feature maps. In: Anderson, J.A., Rosenfeld, E. (eds.) Neurocomputing: Foundations of Research, pp. 509–521. MIT Press, Cambridge, MA (1988)

    Google Scholar 

  31. Laskov, P., Gehl, C., Kruger, S., Muller, K.R.: Incremental support vector learning: analysis, implementation and applications. J. Mach. Learn. Res. 7, 1909–1936 (2006)

    MathSciNet  MATH  Google Scholar 

  32. Lawal, I.A., Poiesi, F., Anguita, D., Cavallaro, A.: Support vector motion clustering. IEEE Trans. Circuits Syst. Video Technol. 27(11), 2395–2408 (2017)

    Article  Google Scholar 

  33. Liang, Z., Li, Y.: Incremental support vector machine learning in the primal and applications. Neurocomputing 72, 2249–2258 (2009)

    Article  Google Scholar 

  34. Lin, H., Deng, J.D., Woodford, B.J.: Anomaly detection in crowd scenes via online adaptive one-class support vector machines. In: Proceedings of the International Conference on Image Processing, pp. 2434–2438 (2015)

    Google Scholar 

  35. Lu, Y., Boukharouba, K., Boonært, J., Fleury, A., Lecuche, S.: Application of an incremental svm algorithm for on-line human recognition from video surveillance using texture and color features. Neurocomputing 126, 132–140 (2014)

    Article  Google Scholar 

  36. Luo, J., Pronobis, A., Caputo, B., Jensfelt, P.: Incremental learning for place recognition in dynamic environments. In: Proceedings of the IEEE International Conference on Intelligent Robots and Systems, pp. 721–728 (2007)

    Google Scholar 

  37. Naqa, I., Yang, Y., Galatsanos, N., Wernick, M.: Relevance feedback based on incremental learning for mammogram retrieval. In: Proceedings of the International Conference on Image Processing, pp. 729–732 (2003)

    Google Scholar 

  38. Ngo Ho, A.K., Eglin, V., Ragot, N., Ramel, J.Y.: Multi one-class incremental svm for document stream digitization. In: Proceedings of the International Workshop on Document Analysis Systems, pp. 5–6 (2016)

    Google Scholar 

  39. Nguyen, H.M., Cooper, E.W., Kamei, K.: Online learning from imbalanced data streams. In: Proceedings of the International Conference of Soft Computing and Pattern Recognition, pp. 347–352 (2011)

    Google Scholar 

  40. Nguyen, H.L., Woon, Y.K., Ng, W.K.: A survey on data stream clustering and classification. Knowl. Inf. Syst. 45(3), 535–569 (2015)

    Article  Google Scholar 

  41. Orabona, F., Castellini, C., Caputo, B., Jie, L., Sandini, G.: On-line independent support vector machines. Pattern Recogn. 43(4), 1402–1412 (2010)

    Article  Google Scholar 

  42. Pronobis, A., Jie, L., Caputo, B.: The more you learn, the less you store: memory-controlled incremental SVM for visual place recognition. Image Vis. Comput. 28(7), 1080–1097 (2010)

    Article  Google Scholar 

  43. Rai, P., Daumé, H., Venkatasubramanian, S.: Streamed learning: one-pass svms. In: Proceedings of the International Jont Conference on Artifical Intelligence, pp. 1211–1216 (2009)

    Google Scholar 

  44. Ralaivola, L., d’Alché Buc, F.: Incremental support vector machine learning: a local approach. In: Proceedings of the International Conference on Artificial Neural Networks, pp. 322–329. Springer, Berlin (2001)

    Google Scholar 

  45. Rifkin, R., Klautau, A.: In defense of one-vs-all classification. J. Mach. Learn. Res. 5, 101–141 (2004)

    MathSciNet  MATH  Google Scholar 

  46. Ruping, S.: Incremental learning with support vector machines. In: Proceedings of the International Conference on Data Mining, pp. 641–642 (2001)

    Google Scholar 

  47. Sayed-Mouchaweh, M.: Learning from Data Streams in Dynamic Environments. Springer International Publishing, Berlin (2016)

    Book  Google Scholar 

  48. Syed, N.A., Huan, S., Kah, L., Sung, K.: Incremental learning with support vector machines. In: Proceedings of the 16th International Joint Conference on Artificial Intelligence, pp. 161–168 (1999)

    Google Scholar 

  49. Tsai, C.H., Lin, C.Y., Lin, C.J.: Incremental and decremental training for linear classification. In: Proceedings of the International Conference on Knowledge Discovery and Data Mining, pp. 343–352 (2014)

    Google Scholar 

  50. Vapnik, V.N.: The Nature Of Statistical Learning Theory. Springer, New York (1995)

    Book  Google Scholar 

  51. Wang, J., Yang, D., Jiang, W., Zhou, J.: Semisupervised incremental support vector machine learning based on neighborhood kernel estimation. IEEE Trans. Syst. Man Cybern. Syst. PP(99), 1–11 (2017)

    Google Scholar 

  52. Wu, C., Wang, X., Bai, D., Zhang, H.: Fast SVM incremental learning based on the convex hulls algorithm. In: Proceedings of the International Conference on Computational Intelligence and Security, vol. 1, pp. 249–252 (2008)

    Google Scholar 

  53. Xiao, R., Wang, J., Zhang, F.: An approach to incremental SVM learning algorithm. In: Proceedings of the International Conference on Tools with Artificial Intelligence, pp. 268–273 (2000)

    Google Scholar 

  54. Xie, W., Uhlmann, S., Kiranyaz, S., Gabbouj, M.: Incremental learning with support vector data description. In: Proceedings of the International Conference on Pattern Recognition, pp. 3904–3909 (2014)

    Google Scholar 

  55. Xihuang, Z., Wenbo, X.: The implementation of online transductive support vector machine. In: Li, D., Wang, B. (eds.) Artificial Intelligence Applications and Innovations. AIAI 2005. IFIP—The International Federation for Information Processing, vol. 187, pp. 231–238. Springer, Boston (2005)

    Google Scholar 

  56. Xing, Y., Shen, F., Luo, C., Zhao, J.: L3-svm: a lifelong learning method for svm. In: Proceedings of the International Joint Conference on Neural Networks, pp. 1–8 (2015)

    Google Scholar 

  57. Xu, S., Wang, J.: A fast incremental extreme learning machine algorithm for data streams classification. Expert Syst. Appl. 65, 332–344 (2016)

    Article  Google Scholar 

  58. Yin, G., Zhang, Y.T., Li, Z.N., Ren, G.Q., Fan, H.B.: Online fault diagnosis method based on incremental support vector data description and extreme learning machine with incremental output structure. Neurocomputing 128, 224–231 (2014)

    Article  Google Scholar 

  59. Zang, W., Zhang, P., Zhou, C., Guo, L.: Comparative study between incremental and ensemble learning on data streams: case study. J. Big Data 1(1), 1–16 (2014)

    Article  Google Scholar 

  60. Zheng, J., Shen, F., Fan, H., Zhao, J.: An online incremental learning support vector machine for large-scale data. Neural Comput. Appl. 22(5), 1023–1035 (2013)

    Article  Google Scholar 

  61. Zhou, X., Zhang, X., Wang, B.: Online support vector machine: a survey. In: Kim, J., Geem, Z. (eds.) Harmony Search Algorithm. Advances in Intelligent Systems and Computing, vol. 382, pp. 269–278. Springer, Berlin (2016)

    Chapter  Google Scholar 

  62. Zhu, Z., Zhu, X., Guo, Y.F., Xue, X.: Transfer incremental learning for pattern classification. In: Proceedings of the 19th ACM International Conference on Information and Knowledge Management, pp. 1709–1712 (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering and Networks, Jouf University, Sakaka, Saudi Arabia

    Isah Abdullahi Lawal

Authors
  1. Isah Abdullahi Lawal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Isah Abdullahi Lawal .

Editor information

Editors and Affiliations

  1. Institute Mines-Telecom Lille Douai, Douai, France

    Moamar Sayed-Mouchaweh

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer International Publishing AG, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Lawal, I.A. (2019). Incremental SVM Learning: Review. In: Sayed-Mouchaweh, M. (eds) Learning from Data Streams in Evolving Environments. Studies in Big Data, vol 41. Springer, Cham. https://doi.org/10.1007/978-3-319-89803-2_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-89803-2_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-89802-5

  • Online ISBN: 978-3-319-89803-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation