Evolutionary Intelligence

, Volume 12, Issue 2, pp 97–112 | Cite as

Incremental supervised learning: algorithms and applications in pattern recognition

  • Aida ChefrourEmail author
Review Article


The most effective well-known methods in the context of static machine learning offer no alternative to evolution and dynamic adaptation to integrate new data or to restructure problems already partially learned. In this area, the incremental learning represents an interesting alternative and constitutes an open research field, becoming one of the major concerns of the machine learning and classification community. In this paper, we study incremental supervised learning techniques and their applications, especially in the field of pattern recognition. This article presents an overview of the main concepts and supervised algorithms of incremental learning, including a synthesis of research studies done in this field and focusing on neural networks, decision trees and support vector machines.


Machine learning Incremental clustering Pattern recognition Neural network Decision tree SVM 



  1. 1.
    Almaksour A (2011) Incremental learning of evolving fuzzy inference systems: application to handwritten gesture recognition. Ph.D. thesis, INSA de RennesGoogle Scholar
  2. 2.
    Bai X, Ren P, Zhang H, Zhou J (2015) An incremental structured part model for object recognition. Neurocomputing 154:189–199CrossRefGoogle Scholar
  3. 3.
    Cauwenberghs G, Poggio T (2001) Incremental and decremental support vector machine learning. In: Advances in neural information processing systems, pp 409–415Google Scholar
  4. 4.
    Chefrour A, Souici-Meslati L (2013) Un panorama de méthodes d’apprentissage incrémental. In: International conference on extraction and knowledge management, Toulouse, France, pp 57–70Google Scholar
  5. 5.
    Déniz O, Castrillón M, Lorenzo J, Hernández M (2002) An incremental learning algorithm for face recognition. International workshop on biometric authentication. Springer, New York, pp 1–9zbMATHGoogle Scholar
  6. 6.
    Diehl CP, Cauwenberghs G (2003) SVM incremental learning adaptation and optimization. In: Proceedings of neural networks conference, vol 4, pp 2685–2690Google Scholar
  7. 7.
    Erdem Z, Polikar R, Gurgen F, Yumusak N (2005) Ensemble of SVMs for incremental learning. In: Proceedings of international workshop on multiple classifier systems. Springer, New York, pp 246–256Google Scholar
  8. 8.
    Fung G, Mangasarian OL (2002) Incremental support vector machine classification. In: Proceedings of the international conference on data mining (SIAM). Society for Industrial and Applied Mathematics, pp 247–260Google Scholar
  9. 9.
    Ghassabeh YA, Moghaddam HA (2007) A face recognition system using neural networks with incremental learning ability.In: Proceedings of the international symposium on computational intelligence in robotics and automation, pp 291–296Google Scholar
  10. 10.
    Gurney KR, Baker D, Rayner P, Denning S (2008) Interannual variations in continental-scale net carbon exchange and sensitivity to observing networks estimated from atmospheric CO2 inversions for the period 1980 to 2005. Global Biogeochem Cycles 22(3):1–17CrossRefGoogle Scholar
  11. 11.
    Hacene G, Gripon V, Farrugia N, Arzel M, Jezequel M (2017) Incremental learning with pretrained convolutional neural networks and binary associative memoriesGoogle Scholar
  12. 12.
    Han S, Meng Z, Khan AS, Tong Y (2016) Incremental boosting convolutional neural network for facial action unit recognition. In: Advances in neural information processing systems, pp 109–117Google Scholar
  13. 13.
    Huang C, Ai H, Yamashita T, Lao S, Kawade M (2007) Incremental learning of boosted face detector. In: Proceedings of the 11th on computer vision IEEE, pp 1–8 Google Scholar
  14. 14.
    Hulley G, Marwala T (2007) Evolving classifiers: Methods for incremental learning. arXiv preprint arXiv:0709.3965
  15. 15.
    Joshi P, Kulkarni P (2012) Incremental learning: areas and methods-a survey. Int J Data Min Knowl Manag Process 2(5):43CrossRefGoogle Scholar
  16. 16.
    Kawewong A, Pimup R, Hasegawa O (2013) Incremental learning framework for indoor scene recognition. In: AAAI, pp 1–7Google Scholar
  17. 17.
    Lawal IA, Abdulkarim SA (2017) Adaptive svm for data stream classification, S Afr Comput J 29(1):27–42Google Scholar
  18. 18.
    Liu Y (2015) Incremental learning in deep neural networks. Master of Science Thesis, Tampere University of TechnologyGoogle Scholar
  19. 19.
    Loosli G (2010) Méthodes à noyaux pour la détection de contexte. Academic editions EGoogle Scholar
  20. 20.
    Lu Y, Boukharouba K, Boonært J, Fleury A, Lecoeuche S (2014) Application of an incremental svm algorithm for on-line human recognition from video surveillance using texture and color features. Neurocomputing 126:132–140CrossRefGoogle Scholar
  21. 21.
    Luo J, Pronobis A, Caputo B, Jensfelt P (2007) Incremental learning for place recognition in dynamic environments. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems, pp 721–728Google Scholar
  22. 22.
    Mańdziuk J, Shastri L (2002) Incremental class learning approach and its application to handwritten digit recognition. Inf Sci 141(3–4):193–217.CrossRefzbMATHGoogle Scholar
  23. 23.
    Mohemmed A, Lu G, Kasabov N (2012) Evaluating span incremental learning for handwritten digit recognition. International conference on neural information processing. Springer, New York, pp 670–677Google Scholar
  24. 24.
    Molina JFG, Zheng L, Sertdemir M, Dinter DJ, Schönberg S, Rädle M (2014) Incremental learning with svm for multimodal classification of prostatic adenocarcinoma, PLoS One 9(4):e93600CrossRefGoogle Scholar
  25. 25.
    Ozawa S, Pang S, Kasabov N (2008) Incremental learning of chunk data for online pattern classification systems. IEEE Trans Neural Netw 19(6):1061–1074CrossRefGoogle Scholar
  26. 26.
    Ozawa S, Toh SL, Abe S, Pang S, Kasabov N (2005) Incremental learning of feature space and classifier for face recognition. Neural Netw 18(5–6):575–584CrossRefGoogle Scholar
  27. 27.
    Polikar R, Upda L, Upda SS, Honavar V (2001) Learn++: an incremental learning algorithm for supervised neural networks. IEEE Trans Syst Man Cybern Part C 31(4):497–508CrossRefGoogle Scholar
  28. 28.
    Prudent Y (2006) Système d’apprentissage incrémental et hybride. PhD thesis, INSA University, RouenGoogle Scholar
  29. 29.
    Ralaivola L, dAlché Buc F (2001) Incremental support vector machine learning: A local approach. In: Proceedings of the international conference on artificial neural networks. Springer, New York, pp 322–330Google Scholar
  30. 30.
    Reddy KK, Liu J, Shah M (2009) Incremental action recognition using feature-tree. In: Proceedings of the 12th international conference on computer vision IEEE, pp 1010–1017 Google Scholar
  31. 31.
    Ruping S (2001) Incremental learning with support vector machines. In: Proceedings of the international conference on Data Mining IEEE, pp 641–642 Google Scholar
  32. 32.
    Salperwyck C, Lemaire V, de Bois DUDP (2010) Classification incrémentale supervisée: un panel introductif. In: AAFD, pp 121–148 Google Scholar
  33. 33.
    Sarwar SS, Ankit A, Roy K (2017) Incremental learning in deep convolutional neural networks using partial network sharing. arXiv:1712.02719Google Scholar
  34. 34.
    Schlimmer JC, Fisher D (1986) A case study of incremental concept induction. AAAI 86, pp 496–501Google Scholar
  35. 35.
    Syed NA, Liu H, Sung KK (1999) Handling concept drifts in incremental learning with support vector machines. In: Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining, pp 317–321 Google Scholar
  36. 36.
    Toh SL, Ozawa S (2003) A face recognition system using neural networks with incremental learning ability. In: Proceedings of the 8th Australian and New Zealand Conference on intelligent information systems, Citeseer, pp 389–394Google Scholar
  37. 37.
    Utgoff PE (1989) Incremental induction of decision trees. Mach Learn 4(2):161–186CrossRefGoogle Scholar
  38. 38.
    Zhao H, Yuen PC, Kwok JT (2006) A novel incremental principal component analysis and its application for face recognition. IEEE Trans Syst Man Cybern Part B 36(4):873–886CrossRefGoogle Scholar
  39. 39.
    Zou L, Zhang T, Cao Z (2009) An incremental learning algorithm based on Support Vector Machine for pattern recognition. In: Proceedings of the International Society for Optics and Photonics, vol 7496Google Scholar
  40. 40.
    Zribi M, Boujelbene Y (2016) The neural networks with an incremental learning algorithm approach for mass classification in breast cancer. Biomed Data Min 5(118):2Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.LISCO Laboratory, Computer Science DepartmentBadji Mokhtar-Annaba UniversityAnnabaAlgeria
  2. 2.Computer Science DepartmentMohamed Cherif Messaadia UniversitySouk AhrasAlgeria

Personalised recommendations