Art-Based Autonomous Learning Systems: Part I — Architectures and Algorithms

  • C. P. Lim
  • R. F. Harrison
Part of the Studies in Fuzziness and Soft Computing book series (STUDFUZZ, volume 43)


This chapter describes the design of novel ART-based intelligent systems that are able to learn and, at the same time, to refine their knowledge in perpetuity. Fuzzy ARTMAP and the Probabilistic Neural Network are integrated to form a hybrid system that possesses the desirable properties for incremental, causal learning as well as for Bayesian probability estimation. Subsequently, a multiple neural network architecture is devised to aggregate outputs from several individual networks into a unified decision. A number of algorithms is proposed to increase the generalization and adaptability of the resulting systems.


Input Pattern Probabilistic Neural Network Target Class Belief Function Adaptive Resonance Theory 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Rumelhart, D.E, Hinton, G.E., and Williams, RJ. (1986), “Learning internal representation by error propagation,” in Rumelhart, D.E. and McLelland, J.L. (Eds.), Parallel Distributed Processing, I, pp. 318–362, Cambridge, MA: MIT Press.Google Scholar
  2. [2]
    Broomhead, D.S. and Lowe, D. (1988), “Multivariate functional interpolation and adaptive networks,” Complex Systems, Vol. 2, pp. 321–355.MathSciNetMATHGoogle Scholar
  3. [3]
    Cybenko, G. (1989), “Approximation by superposition of a sigmoidal function,” Mathematics of Control, Signals and Systems, Vol. 2, pp. 303–314.MathSciNetMATHCrossRefGoogle Scholar
  4. [4]
    Poggio, T. and Girosi, F. (1990), “Network approximation and learning,” Proceedings of IEEE, Vol. 78, pp. 1481–1497.CrossRefGoogle Scholar
  5. [5]
    White, H. (1989), “Learning in artificial neural networks: a statistical perspective,” Neural Computation, Vol. 1, pp. 425–464.CrossRefGoogle Scholar
  6. [6]
    Richard, M.D. and Lippmann, R.P. (1991), “Neural network classifiers estimate Bayesian a posteriori probabilities,” Neural Computation, Vol. 3, pp. 461–483.CrossRefGoogle Scholar
  7. [7]
    Grossberg, S. (1980), “How does a brain build a cognitive code,” Psychological Review, Vol. 87, pp. 1–51.CrossRefGoogle Scholar
  8. [8]
    Carpenter, G.A. and Grossberg, S. (1987), “A massively parallel architecture for a self-organizing neural pattern recognition machine,” Computer Vision, Graphics and Image Processing, Vol. 37, pp. 54–115.MATHCrossRefGoogle Scholar
  9. [9]
    Carpenter, G.A. and Grossberg, S. (1988), “The ART of adaptive pattern recognition by a self-organizing neural network,” IEEE Computer, Vol. 21, pp. 77–88.CrossRefGoogle Scholar
  10. [10]
    Carpenter, G.A. and Grossberg, S. (1987), “ART 2: stable self-organization of pattern recognition codes for analogue input patterns,” Applied Optics, Vol. 26, pp. 4919–4930.CrossRefGoogle Scholar
  11. [11]
    Carpenter, G.A. and Grossberg, S. (1990), “ART 3 hierarchical search: chemical transmitters in self-organizing pattern recognition architectures,” Neural Networks, Vol. 3, pp. 129–152.CrossRefGoogle Scholar
  12. [12]
    Carpenter, G.A., Grossberg, S., and Rosen, D.B. (1991), “Fuzzy ART: fast stable learning and categorization of analogue patterns by an adaptive resonance system,” Neural Networks, Vol. 4, pp. 759–771.CrossRefGoogle Scholar
  13. [13]
    Carpenter, G.A., Grossberg, S., and Reynolds, J.H. (1991), “ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network,” Neural Networks, Vol. 4, pp. 565–588.CrossRefGoogle Scholar
  14. [14]
    Carpenter, G.A., Grossberg, S., Markuzon, N., Reynolds, J.H., and Rosen, D.B. (1992), “Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analogue multidimensional maps,” IEEE Transactions on Neural Networks, Vol. 3, pp. 698–712.CrossRefGoogle Scholar
  15. [15]
    Carpenter, G.A. and Ross, W.D. (1995), “ART-EMAP: a neural network architecture for object recognition by evidence accumulation,” IEEE Transactions on Neural Networks, Vol. 6, pp. 805–818.CrossRefGoogle Scholar
  16. [16]
    Specht, D.F. (1990), “Probabilistic neural networks,” Neural Networks, Vol. 3, pp. 109–118.Google Scholar
  17. [17]
    Zadeh, L. (1965), “Fuzzy sets,” Information and Control, Vol. 8, pp. 338–353.MathSciNetMATHCrossRefGoogle Scholar
  18. [18]
    Lim, C.P. and Harrison, R.F. (1997), “Modified Fuzzy ARTMAP approaches Bayes optimal classification rates: an empirical demonstration,” Neural Networks, Vol. 10, pp. 755–774.CrossRefGoogle Scholar
  19. [19]
    Parzen, E. (1962), “On estimation of a probability density function and mode,” Annals of Mathematical Statistics, Vol. 33, pp. 1065–1076.MathSciNetMATHCrossRefGoogle Scholar
  20. [20]
    Cacoullos, T. (1966), “Estimation of a multivariate density,” Annals of the Institute of Statistical Mathematics (Tokyo), Vol. 18, pp. 179–189.MathSciNetMATHCrossRefGoogle Scholar
  21. [21]
    Fukunaga, K. (1972), Introduction to Statistical Pattern Recognition, New York: Academic Press.Google Scholar
  22. [22]
    Duda, R.O. and Hart, P.E. (1973), Pattern Classification and Scene Analysis, New York: John Wiley and Sons.MATHGoogle Scholar
  23. [23]
    Lim, C.P. and Harrison, R.F. (1997), “An incremental adaptive network for on-line supervised learning and probability estimation,” Neural Networks, Vol. 10, pp. 925–939.CrossRefGoogle Scholar
  24. [24]
    Masters, T. (1993), Practical Neural Network Recipes in C++, San Diego, CA: Academic Press.Google Scholar
  25. [25]
    Masters, T. (1995), Advanced Algorithms for Neural Networks: A C++ Sourcebook, New York: John Wiley and Sons.Google Scholar
  26. [26]
    Musavi, M.T., Ahmed, W., Chan, K.H., Faris, K.B., and Hummels, D.M. (1992), “On the training of radial basis function classifiers” Neural Networks, Vol. 5, pp. 595–603.CrossRefGoogle Scholar
  27. [27]
    Xu, L., Krzyzak, A., and Suen, C.Y. (1992), “Methods of combining multiple classifiers and their applications to handwriting recognition,” IEEE Transactions on Systems, Man, and Cybernetics, Vol. 22, pp. 418–435.CrossRefGoogle Scholar
  28. [28]
    Ho, T.K., Hull, J.J., and Srihari, S.N. (1994), “Decision combination in multiple classifier systems,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 16, pp. 66–75.CrossRefGoogle Scholar
  29. [29]
    Huang, Y.S., and Suen, C.Y. (1995), “A method of combining multiple experts for the recognition of unconstrained handwritten numerals,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 17, pp. 90–94.CrossRefGoogle Scholar
  30. [30]
    Pearl, J. (1988), Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, San Mateo, CA.: Morgan Kaufmann.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • C. P. Lim
    • 1
  • R. F. Harrison
    • 2
  1. 1.School of Industrial TechnologyUniversiti Sains MalaysiaPenangMalaysia
  2. 2.Department of Automatic Control and Systems EngineeringThe University of SheffieldSheffieldUK

Personalised recommendations