Innovations in ART Neural Networks pp 133-166 | Cite as

# Art-Based Autonomous Learning Systems: Part I — Architectures and Algorithms

## Abstract

This chapter describes the design of novel ART-based intelligent systems that are able to learn and, at the same time, to refine their knowledge in perpetuity. Fuzzy ARTMAP and the Probabilistic Neural Network are integrated to form a hybrid system that possesses the desirable properties for incremental, causal learning as well as for Bayesian probability estimation. Subsequently, a multiple neural network architecture is devised to aggregate outputs from several individual networks into a unified decision. A number of algorithms is proposed to increase the generalization and adaptability of the resulting systems.

## Keywords

Input Pattern Probabilistic Neural Network Target Class Belief Function Adaptive Resonance Theory## Preview

Unable to display preview. Download preview PDF.

## References

- [1]Rumelhart, D.E, Hinton, G.E., and Williams, RJ. (1986), “Learning internal representation by error propagation,” in Rumelhart, D.E. and McLelland, J.L. (Eds.),
*Parallel Distributed Processing, I*, pp. 318–362, Cambridge, MA: MIT Press.Google Scholar - [2]Broomhead, D.S. and Lowe, D. (1988), “Multivariate functional interpolation and adaptive networks,”
*Complex Systems*, Vol. 2, pp. 321–355.MathSciNetMATHGoogle Scholar - [3]Cybenko, G. (1989), “Approximation by superposition of a sigmoidal function,”
*Mathematics of Control, Signals and Systems*, Vol. 2, pp. 303–314.MathSciNetMATHCrossRefGoogle Scholar - [4]Poggio, T. and Girosi, F. (1990), “Network approximation and learning,”
*Proceedings of IEEE*, Vol. 78, pp. 1481–1497.CrossRefGoogle Scholar - [5]White, H. (1989), “Learning in artificial neural networks: a statistical perspective,”
*Neural Computation*, Vol. 1, pp. 425–464.CrossRefGoogle Scholar - [6]Richard, M.D. and Lippmann, R.P. (1991), “Neural network classifiers estimate Bayesian a posteriori probabilities,” Neural Computation, Vol. 3, pp. 461–483.CrossRefGoogle Scholar
- [7]Grossberg, S. (1980), “How does a brain build a cognitive code,”
*Psychological Review*, Vol. 87, pp. 1–51.CrossRefGoogle Scholar - [8]Carpenter, G.A. and Grossberg, S. (1987), “A massively parallel architecture for a self-organizing neural pattern recognition machine,”
*Computer Vision, Graphics and Image Processing*, Vol. 37, pp. 54–115.MATHCrossRefGoogle Scholar - [9]Carpenter, G.A. and Grossberg, S. (1988), “The ART of adaptive pattern recognition by a self-organizing neural network,”
*IEEE Computer*, Vol. 21, pp. 77–88.CrossRefGoogle Scholar - [10]Carpenter, G.A. and Grossberg, S. (1987), “ART 2: stable self-organization of pattern recognition codes for analogue input patterns,”
*Applied Optics*, Vol. 26, pp. 4919–4930.CrossRefGoogle Scholar - [11]Carpenter, G.A. and Grossberg, S. (1990), “ART 3 hierarchical search: chemical transmitters in self-organizing pattern recognition architectures,”
*Neural Networks*, Vol. 3, pp. 129–152.CrossRefGoogle Scholar - [12]Carpenter, G.A., Grossberg, S., and Rosen, D.B. (1991), “Fuzzy ART: fast stable learning and categorization of analogue patterns by an adaptive resonance system,”
*Neural Networks*, Vol. 4, pp. 759–771.CrossRefGoogle Scholar - [13]Carpenter, G.A., Grossberg, S., and Reynolds, J.H. (1991), “ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network,”
*Neural Networks*, Vol. 4, pp. 565–588.CrossRefGoogle Scholar - [14]Carpenter, G.A., Grossberg, S., Markuzon, N., Reynolds, J.H., and Rosen, D.B. (1992), “Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analogue multidimensional maps,”
*IEEE Transactions on Neural Networks*, Vol. 3, pp. 698–712.CrossRefGoogle Scholar - [15]Carpenter, G.A. and Ross, W.D. (1995), “ART-EMAP: a neural network architecture for object recognition by evidence accumulation,”
*IEEE Transactions on Neural Networks*, Vol. 6, pp. 805–818.CrossRefGoogle Scholar - [16]Specht, D.F. (1990), “Probabilistic neural networks,”
*Neural Networks*, Vol. 3, pp. 109–118.Google Scholar - [17]Zadeh, L. (1965), “Fuzzy sets,” Information and Control, Vol. 8, pp. 338–353.MathSciNetMATHCrossRefGoogle Scholar
- [18]Lim, C.P. and Harrison, R.F. (1997), “Modified Fuzzy ARTMAP approaches Bayes optimal classification rates: an empirical demonstration,”
*Neural Networks*, Vol. 10, pp. 755–774.CrossRefGoogle Scholar - [19]Parzen, E. (1962), “On estimation of a probability density function and mode,”
*Annals of Mathematical Statistics*, Vol. 33, pp. 1065–1076.MathSciNetMATHCrossRefGoogle Scholar - [20]Cacoullos, T. (1966), “Estimation of a multivariate density,”
*Annals of the Institute of Statistical Mathematics*(Tokyo), Vol. 18, pp. 179–189.MathSciNetMATHCrossRefGoogle Scholar - [21]Fukunaga, K. (1972),
*Introduction to Statistical Pattern Recognition*, New York: Academic Press.Google Scholar - [22]Duda, R.O. and Hart, P.E. (1973),
*Pattern Classification and Scene Analysis*, New York: John Wiley and Sons.MATHGoogle Scholar - [23]Lim, C.P. and Harrison, R.F. (1997), “An incremental adaptive network for on-line supervised learning and probability estimation,”
*Neural Networks*, Vol. 10, pp. 925–939.CrossRefGoogle Scholar - [24]Masters, T. (1993),
*Practical Neural Network Recipes in C++*, San Diego, CA: Academic Press.Google Scholar - [25]Masters, T. (1995),
*Advanced Algorithms for Neural Networks: A C++ Sourcebook*, New York: John Wiley and Sons.Google Scholar - [26]Musavi, M.T., Ahmed, W., Chan, K.H., Faris, K.B., and Hummels, D.M. (1992), “On the training of radial basis function classifiers”
*Neural Networks*, Vol. 5, pp. 595–603.CrossRefGoogle Scholar - [27]Xu, L., Krzyzak, A., and Suen, C.Y. (1992), “Methods of combining multiple classifiers and their applications to handwriting recognition,”
*IEEE Transactions on Systems, Man, and Cybernetics*, Vol. 22, pp. 418–435.CrossRefGoogle Scholar - [28]Ho, T.K., Hull, J.J., and Srihari, S.N. (1994), “Decision combination in multiple classifier systems,”
*IEEE Transactions on Pattern Analysis and Machine Intelligence*, Vol. 16, pp. 66–75.CrossRefGoogle Scholar - [29]Huang, Y.S., and Suen, C.Y. (1995), “A method of combining multiple experts for the recognition of unconstrained handwritten numerals,”
*IEEE Transactions on Pattern Analysis and Machine Intelligence*, Vol. 17, pp. 90–94.CrossRefGoogle Scholar - [30]Pearl, J. (1988),
*Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference*, San Mateo, CA.: Morgan Kaufmann.Google Scholar