Skip to main content
SpringerLink
Log in

Adaptation in Classification Systems

  • Chapter
Book cover Foundations of Computational Intelligence Volume 2

Part of the book series: Studies in Computational Intelligence ((SCI,volume 202))

Summary

The persistence and evolution of systems essentially depend of their ability to self-adapt to new situations. As an expression of intelligence, adaptation is a distinguishing quality of any system that is able to learn and to adjust itself in a flexible manner to new environmental conditions. Such ability ensures selfcorrection over time as new events happen, new input becomes available, or new operational conditions occur. This requires self-monitoring of the performance in an ever changing environment. The relevance of adaptation is established in numerous domains and by versatile real world applications.

The primary goal of this contribution is to investigate adaptation issues in learning classification systems formdifferent perspectives. Being a scheme of adaptation, life long incremental learning will be examined. However, special attention will be given to adaptive neural networks and the most visible incremental learning algorithms (fuzzy ARTMAP, nearest generalized exemplar, growing neural gas, generalized fuzzy minmax neural network, IL based on function decomposition) and their adaptation mechanisms will be discussed. Adaptation can also be incorporated in the combination of such incremental classifiers in different ways so that adaptive ensemble learners can be obtained too. These issues and other pertaining to drift will be investigated and illustrated by means of a numerical simulation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Battiti, R., Colla, A.: Democracy in neural nets: Voting schemes for classification. Neural Networks 7(4), 691–707 (1994)

    Article  Google Scholar 

  2. Blum, A.: Empirical support for winnow and weighted-majority based algorithms: results on a calendar scheduling domain. Machine Learning 26, 5–23 (1997)

    Article  MathSciNet  Google Scholar 

  3. Bouchachia, A.: Incremental learning via function decomposition. In: Proc. of the Int. Conf. on machine learning and applications, pp. 63–68 (2006)

    Google Scholar 

  4. Bouchachia, A.: Incremental Learning. In: Encyclopedia of Data Warehousing and Mining, 2nd edn., Idea-Group (in press) (2008)

    Google Scholar 

  5. Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)

    MATH  MathSciNet  Google Scholar 

  6. Breiman, L.: Pasting small votes for classification in large databases and on-line. Machine Learning 36, 85–103 (1999)

    Article  Google Scholar 

  7. Carpenter, G., Grossberg, S., Rosen, D.: Fuzzy ART: Fast Stable Learning and Categorization of Analog Patterns by an Adaptive Resonance System. Neural Networks 4(6), 759–771 (1991)

    Article  Google Scholar 

  8. Dietrich, C., Palm, G., Schwenker, F.: Decision templates for the classification of bioacoustic time series. Information Fusion 4(2), 101–109 (2003)

    Article  Google Scholar 

  9. Fern, A., Givan, R.: Machine learning. Machine Learning 53, 71–109 (2003)

    Article  MATH  Google Scholar 

  10. French, R.: Catastrophic forgetting in connectionist networks: Causes, consequences and solutions. Trends in Cognitive Sciences, Trends in Cognitive Sciences 3(4), 128–135 (1999)

    MathSciNet  Google Scholar 

  11. Freund, Y., Schapire, R.: A decision-theoretic generalization of on-line learning and an application to boosting. In: Proceedings of the Second European Conference on Computational Learning Theory, pp. 23–37 (1995)

    Google Scholar 

  12. Fritzke, B.: A growing neural gas network learns topologies. In: Advances in neural information processing systems, pp. 625–632 (1995)

    Google Scholar 

  13. Gabrys, B., Bargiela, A.: General fuzzy min-max neural network for clustering and classification. IEEE Trans. on Neural Networks 11(3), 769–783 (2000)

    Article  Google Scholar 

  14. Grossberg, S.: Nonlinear neural networks: principles, mechanism, and architectures. Neural Networks 1, 17–61 (1988)

    Article  Google Scholar 

  15. Hettich, S., Blake, C., Merz, C.: UCI repository of machine learning databases (1998), www.ics.uci.edu/~mlearn/MLRepository.html

  16. Huang, Y., Suen, C.: A method of combining multiple experts for the recognition of unconstrained handwritten numerals. IEEE Transactions on Pattern Analysis and Machine Intelligence 17(1), 90–94 (1995)

    Article  Google Scholar 

  17. Jacobs, R.: Methods of combining experts’ probability assessments. Neural Computing 7, 865–888 (1995)

    Article  Google Scholar 

  18. Jacobs, R., Jordan, M., Nowlan, S., Hinton, G.: Adaptive mixtures of local experts. Neural Computing 3, 79–87 (1991)

    Article  Google Scholar 

  19. Stanley, K.: Learning concept drift with a committee of decision trees. Technical Report TR-03-302, Dept of Computer Science, University of Texas at Austin, USA (2003)

    Google Scholar 

  20. Kittler, J., Hatef, M., Duin, R., Matas, J.: On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(3), 226–239 (1998)

    Article  Google Scholar 

  21. Klinkenberg, R.: Learning drifting concepts: example selection vs. example weighting. Intelligent Data Analysis 8(3), 281–300 (2004)

    Google Scholar 

  22. Kolter, J., Maloof, M.: Dynamic weighted majority: a new ensemble method for tracking concept drift. In: Proceedings of the 3rd International Conference on Data Mining ICDM 2003, pp. 123–130. IEEE CS Press, Los Alamitos (2003)

    Google Scholar 

  23. Kuncheva, L.: A theoretical study on six classifier fusion strategies. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(2), 281–286 (2002)

    Article  Google Scholar 

  24. Kuncheva, L.: Classifier ensembles for changing environments. In: Proc. of the 5th international workshop on multiple classifier systems, pp. 1–15 (2004)

    Google Scholar 

  25. Kuncheva, L., Bezdek, J., Duin, R.: Decision templates for multiple classifier fusion: An experimental comparison. Pattern Recognition 34(2), 299–314 (2001)

    Article  MATH  Google Scholar 

  26. Lam, L., Suen, C.: Optimal combinations of pattern classifiers. Pattern Recognition Letters 16, 945–954 (1995)

    Article  Google Scholar 

  27. Littlestone, N.: Learning quickly when irrelevant attributes abound: A new linear threshold algorithm. Machine Learning 2, 285–318 (1988)

    Google Scholar 

  28. Martinetz, T., Berkovich, S., Schulten, K.: Neural gas network for vector quantization and its application to time-series prediction. IEEE Trans. Neural Networks 4(4), 558–569 (1993)

    Article  Google Scholar 

  29. McCloskey, M., Cohen, N.: Catastrophic interference in connectionist networks: the sequential learning problem. The psychology of learning and motivation 24, 109–164 (1999)

    Article  Google Scholar 

  30. Oza, N.: Online Ensemble Learning. Phd thesis, University of California, Berkeley (2001)

    Google Scholar 

  31. Promise. Software engineering repository (May 2008), http://promise.site.uottawa.ca/SERepository

  32. Quartz, S., Sejnowski, T.: The neural basis of cognitive development: a constructivist manifesto. Behavioral and Brain Sciences 20(4), 537–556 (1997)

    Google Scholar 

  33. Ratcliff, R.: Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological Review 97, 285–308 (1990)

    Article  Google Scholar 

  34. Sahel, Z., Bouchachia, A., Gabrys, B.: Adaptive mechanisms for classification problems with drifting data. In: Apolloni, B., Howlett, R.J., Jain, L. (eds.) KES 2007, Part II. LNCS, vol. 4693, pp. 419–426. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  35. Salzberg, S.: A nearest hyperrectangle learning method. Machine learning 6, 277–309 (1991)

    Google Scholar 

  36. Sharkey, N., Sharkey, A.: Catastrophic forgetting in connectionist networks: Causes, consequences and solutions. An analysis of catastrophic interference 7(3-4), 301–329 (1995)

    Google Scholar 

  37. Sirosh, J., Miikkulainen, R., Choe, Y. (eds.): Lateral Interactions in the Cortex: Structure and Function, The UTCS Neural Networks Research Group, Austin, TX, Electronic book (1996)

    Google Scholar 

  38. Slaganicoff, M.: Density-adaptive learning and forgetting. In: Proc. of the 10th Int. Conf. on Machine Learning, pp. 276–283 (1993)

    Google Scholar 

  39. Street, W., Kim, Y.: A streaming ensemble algorithm (sea) for large-scale classification. In: Proceedings of the 7th International Conference on Knowledge Discovery and Data Mining KDDM 2001, pp. 377–382 (2001)

    Google Scholar 

  40. Tax, D., van Breukelen, M., Duin, R., Kittler, J.: Combining multiple classifiers by averaging or by multiplying? Pattern Recognition 33(9), 1475–1485 (2000)

    Article  Google Scholar 

  41. Tsymbal, A., Pechenizkiy, M., Cunningham, P., Puuronen, S.: Handling local concept drift with dynamic integration of classifiers: Domain of antibiotic resistance in nosocomial infections. In: Proc. of the 19th IEEE Symposium on Computer-Based Medical Systems, pp. 679–684 (2006)

    Google Scholar 

  42. Wettschereck, D., Dietterich, T.: An experimental comparison of the nearest-neighbor and nearest-hyperrectangle algorithms. Machine Learning 19(1), 5–27 (1995)

    Google Scholar 

  43. Widmer, G., Kubat, M.: Learning in the presence of concept drift and hidden contexts. Machine Learning 23, 69–101 (1996)

    Google Scholar 

  44. Wolpert, D.: Stacked generalization. Neural Networks 5(2), 241–259 (1992)

    Article  Google Scholar 

  45. Woods, K., Kegelmeyer, W., Bowyer, K.: Combination of multiple classifiers using local accuracy estimates. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 405–410 (1997)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Group of Software Engineering & Soft Computing, Dept. of Informatics, University of Klagenfurt, Austria

    Abdelhamid Bouchachia

Authors
  1. Abdelhamid Bouchachia
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculty of Computers and Information Information Technology Department, Cairo University, 5 Ahmed Zewal St., Orman, Giza

    Aboul-Ella Hassanien

  2. Machine Intelligence Research Labs (MIR Labs), Scientific Network for Innovation and Research Excellence, P.O. Box 2259, 98071-2259, Auburn,Washington, USA

    Ajith Abraham

  3. Soft Computing and Intelligent Information Systems Department of Computer Science and Artificial Intelligence ETS de Ingenierias Informática y de Telecomunicación, University of Granada, E-18071, Granada, Spain

    Francisco Herrera

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Bouchachia, A. (2009). Adaptation in Classification Systems. In: Hassanien, AE., Abraham, A., Herrera, F. (eds) Foundations of Computational Intelligence Volume 2. Studies in Computational Intelligence, vol 202. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-01533-5_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-01533-5_9

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-01532-8

  • Online ISBN: 978-3-642-01533-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation