Skip to main content
SpringerLink
Log in

Self-Organizing Neural Networks for Supervised and Unsupervised Learning and Prediction

  • Conference paper
From Statistics to Neural Networks

Part of the book series: NATO ASI Series ((NATO ASI F,volume 136))

Abstract

A neural network architecture for fast, stable, incremental learning of recognition categories and multidimensional maps is described. The architecture, called fuzzy ARTMAP, achieves a synthesis of fuzzy logic and Adaptive Resonance Theory (ART) neural networks. Fuzzy ARTMAP realizes a Minimax Learning Rule that conjointly minimizes predictive error and maximizes code compression. The system automatically learns a minimal number of recognition categories, or “hidden units”, to meet accuracy criteria, and the final code can include both fine and coarse categories. At each learning stage, system weights may be translated into a set of if-then rules that characterize the decision making process. Prediction is improved by training the system several times using different orderings of the input set, then voting on the outcomes. ART and ARTMAP networks are being applied to problems such as medical prediction, airplane design, electrocardiogram analysis, seismic recognition, adaptive software, and radar scene analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Parker, D.B.: Learning-logic. Invention Report, 581–64, File 1, Office of Technology Licensing. Stanford University, October, 1982.

    Google Scholar 

  2. Rumelhart, D.E., Hinton, G., and Williams, R.: Learning internal representations by error propagation. In D.E. Rumelhart and J.L. McClelland (Eds.), Parallel distributed processing. Cambridge, MA: MIT Press, 1986.

    Google Scholar 

  3. Werbos, P.: Beyond regression: New tools for prediction and analysis in the behavioral sciences. PhD Thesis, Harvard University, Cambridge, MA, 1974.

    Google Scholar 

  4. Ratcliff, R.: Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions. Psych. Rev. 97, 285–308 (1990).

    Article  Google Scholar 

  5. Carpenter, G.A., Grossberg, S., and Reynolds, J.H.: ARTMAP: Supervised real-time learning and classification of nonstationary data by a self-organizing neural network. Neural Networks 4, 565–588 (1991).

    Article  Google Scholar 

  6. Carpenter, G.A., Grossberg, S., Markuzon, N., Reynolds, J.H., and Rosen, D.B.: Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans. Neural Networks 3 698–713 (1992).

    Article  Google Scholar 

  7. Grossberg, S.: Adaptive pattern classification and universal recoding, II: Feedback, expectation, olfaction, and illusions. Biol. Cybern. 23, 187–202 (1976).

    Article  MATH  MathSciNet  Google Scholar 

  8. Grossberg, S.: How does a brain build a cognitive code? Psych. Rev. 1, 1–51 (1980).

    Article  Google Scholar 

  9. Carpenter, G.A. and Grossberg, S.: A massively parallel architecture for a self-organizing neural pattern recognition machine. Comp. Vis. Graph. Image Proc. 37, 54–115 (1987).

    Article  Google Scholar 

  10. Carpenter, G.A. and Grossberg, S.: ART 2: Stable self-organization of pattern recognition codes for analog input patterns. Appl. Opt. 26, 4919–4930 (1987).

    Article  Google Scholar 

  11. Carpenter, G.A. and Grossberg, S.: ART 3: Hierarchical search using chemical transmitters in self-organizing pattern recognition architectures. Neural Networks 3, 129–152 (1990).

    Article  Google Scholar 

  12. Grossberg, S.: Neural expectation: Cerebellar and retinal analogs of cells fired by learnable or unlearned pattern classes. Kybernetik 10, 49–57 (1972).

    Article  MATH  Google Scholar 

  13. Grossberg, S.: On the development of feature detectors in the visual cortex with applications to learning and reaction-diffusion systems. Biol. Cybern. 21, 145–159 (1976).

    Article  MATH  MathSciNet  Google Scholar 

  14. Grossberg, S.: Adaptive pattern classification and universal recoding, I: Parallel development and coding of neural feature detectors. Biol. Cybern. 23, 121–134 (1976).

    Article  MATH  MathSciNet  Google Scholar 

  15. Grossberg, S.: A theory of human memory: Self-organization and performance of sensory-motor codes, maps, and plan. In R. Rosen and F. Snell (Eds.), Progress in theoretical biology, Vol. 5. New York: Academic Press, 1978. [Reprinted in S. Grossberg, Studies of mind and brain: Neural principles of learning, perception, development, cognition, and motor control. Boston: Reidel Press, 1982].

    Google Scholar 

  16. Grossberg, S.: Studies of mind and brain: Neural principles of learning, perception, development, cognition, and motor control. Boston: Reidel Press, 1982.

    MATH  Google Scholar 

  17. von der Malsburg, C.: Self-organization of orientation sensitive cells in the striate cortex. Kybernetik 14, 85–100 (1973).

    Article  Google Scholar 

  18. Willshaw, D.J. and von der Malsburg, C.: How patterned neural connections can be set up by self-organization. Proc. Roy. Soc. London (B) 194, 431–445 (1976).

    Article  Google Scholar 

  19. Cohen M.A. and Grossberg, S.: Neural dynamics of speech and language coding: Developmental programs, perceptual grouping, and competition for short term memory. Human Neurobiol. 5, 1–22 (1986).

    Google Scholar 

  20. Cohen, M.A. and Grossberg, S.: Masking fields: A massively parallel architecture for learning, recognizing, and predicting multiple groupings of patterned data. Appl. Opt. 26, 1866–1891 (1987).

    Google Scholar 

  21. Grossberg, S. and Kuperstein, M.: Neural dynamics of adaptive sensory-motor control: Ballistic eye movements. Amsterdam: Elsevier/North Holland, 1986.

    Google Scholar 

  22. Grossberg, S. and Kuperstein, M.: Neural dynamics of adaptive sensory-motor control: Expanded edition. Elmsford, NY: Pergamon Press, 1989.

    Google Scholar 

  23. Kohonen, T.: Self-organization and associative memory. New York: Springer-Verlag, 1988.

    MATH  Google Scholar 

  24. Carpenter, G.A. and Grossberg, S. (Eds.): Pattern recognition by self-organizing neural networks. Cambridge, MA: MIT Press, 1991.

    Google Scholar 

  25. Grossberg, S. (Ed.): The adaptive brain, I: Cognition, learning, reinforcement, and rhythm. Amsterdam: Elsevier/North-Holland, 1987.

    Google Scholar 

  26. Grossberg, S. (Ed.): The adaptive brain, II: Vision, speech, language, and motor control. Amsterdam: Elsevier/North-Holland, 1987.

    Google Scholar 

  27. Grossberg, S. (Ed.): Neural networks and natural intelligence. Cambridge, MA: MIT Press, 1988.

    Google Scholar 

  28. Carpenter, G.A. and Grossberg, S.: Neural dynamics of category learning and recognition: Attention, memory consolidation, and amnesia. In S. Grossberg (Ed.), The adaptive brain, I: Cognition, learning, reinforcement, and rhythm. Amsterdam: Elsevier/North Holland, pp. 238–286, 1987.

    Google Scholar 

  29. Grossberg, S. and Merrill, J.W.L.: A neural network model of adaptively timed reinforcement learning and hippocampal dynamics. Cog. Brain Res. 1, 3–38 (1992).

    Article  Google Scholar 

  30. Desimone, R.: Neural circuits for visual attention in the primate brain. In G.A. Carpenter and S. Grossberg (Eds.), Neural networks for vision and image processing. Cambridge, MA: MIT Press, pp. 343–364, 1992.

    Google Scholar 

  31. Laird, J.E., Newell, A., and Rosenbloom, P.S.: SOAR: An architecture for general intelligence. Artificial Intell. 33, 1–64 (1987).

    Article  Google Scholar 

  32. Carpenter, G.A., Grossberg, S., and Rosen, D.B.: Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system. Neural Networks 4, 759–771 (1991).

    Article  Google Scholar 

  33. Kosko, B.: Fuzzy entropy and conditioning. Info. Sci. 40, 165–174 (1986).

    Article  MATH  MathSciNet  Google Scholar 

  34. Zadeh, L.: Fuzzy sets. Info. Control 8, 338–353 (1965).

    Article  MATH  MathSciNet  Google Scholar 

  35. Frey, P.W. and Slate, D.J.: Letter recognition using Holland-style adaptive classifiers. Mach. Learn. 6, 161–182 (1991).

    Google Scholar 

  36. Holland, J.H.: Adaptive algorithms for discovering and using general patterns in growing knowledge bases. Int. J. Pol. Anal. Info. Sys. 4, 217–240 (1980).

    Google Scholar 

  37. Baloch, A.J. and Waxman, A.M.: Visual learning, adaptive expectations, and learning behavioral conditioning of the mobil robot MAVIN. Neural Networks 4, 271–302 (1991).

    Article  Google Scholar 

  38. Caudell, T., Smith, S., Johnson, C., Wunsch, D., and Escobedo, R.: An industrial application of neural networks to reusable design. Adaptive neural systems, Technical Report BCS-CS-ACS-91–001, Seattle, WA: The Boeing Company, pp. 185–190, 1991.

    Google Scholar 

  39. Gjerdingen, R.O.: Categorization of musical patterns by self-organizing neuronlike networks. Music Percep. 7, 339–370 (1990).

    Google Scholar 

  40. Goodman, P., Kaburlasos, V., Egbert, D., Carpenter, G.A., Grossberg, S., Reynolds, J.H., Hammermeister, K., Marshall, G., and Grover, F.: Fuzzy ARTMAP neural network prediction of heart surgery mortality. Proceedings of the Wang Institute research conference: Neural networks for learning, recognition, and control. Boston, MA: Boston University, p. 48, 1992.

    Google Scholar 

  41. Ham, F.M. and Han, S.W.: Quantitative study of the QRS complex using fuzzy ARTMAP and MIT/BIH arrythmia database. Proceedings of the World Congress on Neural Networks (WCNN’93) I, 207–211 (1993).

    Google Scholar 

  42. Keyvan, S., Durg, A., and Rabelo, L.C.: Application of artificial neural networks for development of diagnostic monitoring system in nuclear plants. Trans. Amer. Nuclear Soc. 1, 515–522 (1993).

    Google Scholar 

  43. Mehta, B.V., Vij, L., and Rabelo, L.C.: Prediction of secondary structures of proteins using fuzzy ARTMAP. Proceedings of the World Congress on Neural Networks (WCNN’93) I, 228–232 (1993).

    Google Scholar 

  44. Seibert, M. and Waxman, A.M.: Learning and recognizing 3D objects from multiple views in a neural system. In H. Wechsler (Ed.) Neural networks for perception, Vol. 1. New York: Academic Press, 1991.

    Google Scholar 

  45. Salzberg, S.L.: Learning with nested generalized exemplars. Boston: Kluwer Academic Publishers, 1990.

    Book  MATH  Google Scholar 

  46. Moore, B.: ART 1 and pattern clustering. In D. Touretzky, G. Hinton, and T. Sejnowski (Eds.), Proceedings of the 1988 connectionist models summer school. San Mateo, CA: Morgan Kaufmann, pp. 174–185, 1989.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Adaptive Systems and Department of Cognitive and Neural Systems, Boston University, 111 Cummington Street, Boston, Massachusetts, 02215, USA

    Gail A. Carpenter & Stephen Grossberg

Authors
  1. Gail A. Carpenter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephen Grossberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Electrical Engineering, University of Minnesota, Minneapolis, MN, 55455, USA

    Vladimir Cherkassky

  2. Department of Statistics, Stanford University, Stanford, CA, 94309, USA

    Jerome H. Friedman

  3. Computer Science Department, George Mason University, Fairfax, VA, 22030, USA

    Harry Wechsler

Rights and permissions

Reprints and permissions

Copyright information

© 1994 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Carpenter, G.A., Grossberg, S. (1994). Self-Organizing Neural Networks for Supervised and Unsupervised Learning and Prediction. In: Cherkassky, V., Friedman, J.H., Wechsler, H. (eds) From Statistics to Neural Networks. NATO ASI Series, vol 136. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-79119-2_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-79119-2_16

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-79121-5

  • Online ISBN: 978-3-642-79119-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation