Self-Organizing Neural Networks for Supervised and Unsupervised Learning and Prediction

  • Gail A. Carpenter
  • Stephen Grossberg
Part of the NATO ASI Series book series (volume 136)


A neural network architecture for fast, stable, incremental learning of recognition categories and multidimensional maps is described. The architecture, called fuzzy ARTMAP, achieves a synthesis of fuzzy logic and Adaptive Resonance Theory (ART) neural networks. Fuzzy ARTMAP realizes a Minimax Learning Rule that conjointly minimizes predictive error and maximizes code compression. The system automatically learns a minimal number of recognition categories, or “hidden units”, to meet accuracy criteria, and the final code can include both fine and coarse categories. At each learning stage, system weights may be translated into a set of if-then rules that characterize the decision making process. Prediction is improved by training the system several times using different orderings of the input set, then voting on the outcomes. ART and ARTMAP networks are being applied to problems such as medical prediction, airplane design, electrocardiogram analysis, seismic recognition, adaptive software, and radar scene analysis.


Predictive Error Adaptive Weight Adaptive Resonance Theory Category Choice Fuzzy ARTMAP 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Parker, D.B.: Learning-logic. Invention Report, 581–64, File 1, Office of Technology Licensing. Stanford University, October, 1982.Google Scholar
  2. 2.
    Rumelhart, D.E., Hinton, G., and Williams, R.: Learning internal representations by error propagation. In D.E. Rumelhart and J.L. McClelland (Eds.), Parallel distributed processing. Cambridge, MA: MIT Press, 1986.Google Scholar
  3. 3.
    Werbos, P.: Beyond regression: New tools for prediction and analysis in the behavioral sciences. PhD Thesis, Harvard University, Cambridge, MA, 1974.Google Scholar
  4. 4.
    Ratcliff, R.: Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions. Psych. Rev. 97, 285–308 (1990).CrossRefGoogle Scholar
  5. 5.
    Carpenter, G.A., Grossberg, S., and Reynolds, J.H.: ARTMAP: Supervised real-time learning and classification of nonstationary data by a self-organizing neural network. Neural Networks 4, 565–588 (1991).CrossRefGoogle Scholar
  6. 6.
    Carpenter, G.A., Grossberg, S., Markuzon, N., Reynolds, J.H., and Rosen, D.B.: Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans. Neural Networks 3 698–713 (1992).CrossRefGoogle Scholar
  7. 7.
    Grossberg, S.: Adaptive pattern classification and universal recoding, II: Feedback, expectation, olfaction, and illusions. Biol. Cybern. 23, 187–202 (1976).CrossRefMATHMathSciNetGoogle Scholar
  8. 8.
    Grossberg, S.: How does a brain build a cognitive code? Psych. Rev. 1, 1–51 (1980).CrossRefGoogle Scholar
  9. 9.
    Carpenter, G.A. and Grossberg, S.: A massively parallel architecture for a self-organizing neural pattern recognition machine. Comp. Vis. Graph. Image Proc. 37, 54–115 (1987).CrossRefGoogle Scholar
  10. 10.
    Carpenter, G.A. and Grossberg, S.: ART 2: Stable self-organization of pattern recognition codes for analog input patterns. Appl. Opt. 26, 4919–4930 (1987).CrossRefGoogle Scholar
  11. 11.
    Carpenter, G.A. and Grossberg, S.: ART 3: Hierarchical search using chemical transmitters in self-organizing pattern recognition architectures. Neural Networks 3, 129–152 (1990).CrossRefGoogle Scholar
  12. 12.
    Grossberg, S.: Neural expectation: Cerebellar and retinal analogs of cells fired by learnable or unlearned pattern classes. Kybernetik 10, 49–57 (1972).CrossRefMATHGoogle Scholar
  13. 13.
    Grossberg, S.: On the development of feature detectors in the visual cortex with applications to learning and reaction-diffusion systems. Biol. Cybern. 21, 145–159 (1976).CrossRefMATHMathSciNetGoogle Scholar
  14. 14.
    Grossberg, S.: Adaptive pattern classification and universal recoding, I: Parallel development and coding of neural feature detectors. Biol. Cybern. 23, 121–134 (1976).CrossRefMATHMathSciNetGoogle Scholar
  15. 15.
    Grossberg, S.: A theory of human memory: Self-organization and performance of sensory-motor codes, maps, and plan. In R. Rosen and F. Snell (Eds.), Progress in theoretical biology, Vol. 5. New York: Academic Press, 1978. [Reprinted in S. Grossberg, Studies of mind and brain: Neural principles of learning, perception, development, cognition, and motor control. Boston: Reidel Press, 1982].Google Scholar
  16. 16.
    Grossberg, S.: Studies of mind and brain: Neural principles of learning, perception, development, cognition, and motor control. Boston: Reidel Press, 1982.MATHGoogle Scholar
  17. 17.
    von der Malsburg, C.: Self-organization of orientation sensitive cells in the striate cortex. Kybernetik 14, 85–100 (1973).CrossRefGoogle Scholar
  18. 18.
    Willshaw, D.J. and von der Malsburg, C.: How patterned neural connections can be set up by self-organization. Proc. Roy. Soc. London (B) 194, 431–445 (1976).CrossRefGoogle Scholar
  19. 19.
    Cohen M.A. and Grossberg, S.: Neural dynamics of speech and language coding: Developmental programs, perceptual grouping, and competition for short term memory. Human Neurobiol. 5, 1–22 (1986).Google Scholar
  20. 20.
    Cohen, M.A. and Grossberg, S.: Masking fields: A massively parallel architecture for learning, recognizing, and predicting multiple groupings of patterned data. Appl. Opt. 26, 1866–1891 (1987).Google Scholar
  21. 21.
    Grossberg, S. and Kuperstein, M.: Neural dynamics of adaptive sensory-motor control: Ballistic eye movements. Amsterdam: Elsevier/North Holland, 1986.Google Scholar
  22. 21a.
    Grossberg, S. and Kuperstein, M.: Neural dynamics of adaptive sensory-motor control: Expanded edition. Elmsford, NY: Pergamon Press, 1989.Google Scholar
  23. 22.
    Kohonen, T.: Self-organization and associative memory. New York: Springer-Verlag, 1988.MATHGoogle Scholar
  24. 23.
    Carpenter, G.A. and Grossberg, S. (Eds.): Pattern recognition by self-organizing neural networks. Cambridge, MA: MIT Press, 1991.Google Scholar
  25. 24.
    Grossberg, S. (Ed.): The adaptive brain, I: Cognition, learning, reinforcement, and rhythm. Amsterdam: Elsevier/North-Holland, 1987.Google Scholar
  26. 25.
    Grossberg, S. (Ed.): The adaptive brain, II: Vision, speech, language, and motor control. Amsterdam: Elsevier/North-Holland, 1987.Google Scholar
  27. 26.
    Grossberg, S. (Ed.): Neural networks and natural intelligence. Cambridge, MA: MIT Press, 1988.Google Scholar
  28. 27.
    Carpenter, G.A. and Grossberg, S.: Neural dynamics of category learning and recognition: Attention, memory consolidation, and amnesia. In S. Grossberg (Ed.), The adaptive brain, I: Cognition, learning, reinforcement, and rhythm. Amsterdam: Elsevier/North Holland, pp. 238–286, 1987.Google Scholar
  29. 28.
    Grossberg, S. and Merrill, J.W.L.: A neural network model of adaptively timed reinforcement learning and hippocampal dynamics. Cog. Brain Res. 1, 3–38 (1992).CrossRefGoogle Scholar
  30. 29.
    Desimone, R.: Neural circuits for visual attention in the primate brain. In G.A. Carpenter and S. Grossberg (Eds.), Neural networks for vision and image processing. Cambridge, MA: MIT Press, pp. 343–364, 1992.Google Scholar
  31. 30.
    Laird, J.E., Newell, A., and Rosenbloom, P.S.: SOAR: An architecture for general intelligence. Artificial Intell. 33, 1–64 (1987).CrossRefGoogle Scholar
  32. 31.
    Carpenter, G.A., Grossberg, S., and Rosen, D.B.: Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system. Neural Networks 4, 759–771 (1991).CrossRefGoogle Scholar
  33. 32.
    Kosko, B.: Fuzzy entropy and conditioning. Info. Sci. 40, 165–174 (1986).CrossRefMATHMathSciNetGoogle Scholar
  34. 33.
    Zadeh, L.: Fuzzy sets. Info. Control 8, 338–353 (1965).CrossRefMATHMathSciNetGoogle Scholar
  35. 34.
    Frey, P.W. and Slate, D.J.: Letter recognition using Holland-style adaptive classifiers. Mach. Learn. 6, 161–182 (1991).Google Scholar
  36. 35.
    Holland, J.H.: Adaptive algorithms for discovering and using general patterns in growing knowledge bases. Int. J. Pol. Anal. Info. Sys. 4, 217–240 (1980).Google Scholar
  37. 36.
    Baloch, A.J. and Waxman, A.M.: Visual learning, adaptive expectations, and learning behavioral conditioning of the mobil robot MAVIN. Neural Networks 4, 271–302 (1991).CrossRefGoogle Scholar
  38. 37.
    Caudell, T., Smith, S., Johnson, C., Wunsch, D., and Escobedo, R.: An industrial application of neural networks to reusable design. Adaptive neural systems, Technical Report BCS-CS-ACS-91–001, Seattle, WA: The Boeing Company, pp. 185–190, 1991.Google Scholar
  39. 38.
    Gjerdingen, R.O.: Categorization of musical patterns by self-organizing neuronlike networks. Music Percep. 7, 339–370 (1990).Google Scholar
  40. 39.
    Goodman, P., Kaburlasos, V., Egbert, D., Carpenter, G.A., Grossberg, S., Reynolds, J.H., Hammermeister, K., Marshall, G., and Grover, F.: Fuzzy ARTMAP neural network prediction of heart surgery mortality. Proceedings of the Wang Institute research conference: Neural networks for learning, recognition, and control. Boston, MA: Boston University, p. 48, 1992.Google Scholar
  41. 40.
    Ham, F.M. and Han, S.W.: Quantitative study of the QRS complex using fuzzy ARTMAP and MIT/BIH arrythmia database. Proceedings of the World Congress on Neural Networks (WCNN’93) I, 207–211 (1993).Google Scholar
  42. 41.
    Keyvan, S., Durg, A., and Rabelo, L.C.: Application of artificial neural networks for development of diagnostic monitoring system in nuclear plants. Trans. Amer. Nuclear Soc. 1, 515–522 (1993).Google Scholar
  43. 42.
    Mehta, B.V., Vij, L., and Rabelo, L.C.: Prediction of secondary structures of proteins using fuzzy ARTMAP. Proceedings of the World Congress on Neural Networks (WCNN’93) I, 228–232 (1993).Google Scholar
  44. 43.
    Seibert, M. and Waxman, A.M.: Learning and recognizing 3D objects from multiple views in a neural system. In H. Wechsler (Ed.) Neural networks for perception, Vol. 1. New York: Academic Press, 1991.Google Scholar
  45. 44.
    Salzberg, S.L.: Learning with nested generalized exemplars. Boston: Kluwer Academic Publishers, 1990.CrossRefMATHGoogle Scholar
  46. 45.
    Moore, B.: ART 1 and pattern clustering. In D. Touretzky, G. Hinton, and T. Sejnowski (Eds.), Proceedings of the 1988 connectionist models summer school. San Mateo, CA: Morgan Kaufmann, pp. 174–185, 1989.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1994

Authors and Affiliations

  • Gail A. Carpenter
    • 1
  • Stephen Grossberg
    • 1
  1. 1.Center for Adaptive Systems and Department of Cognitive and Neural SystemsBoston UniversityBostonUSA

Personalised recommendations