Skip to main content
SpringerLink
Log in

Self-organizing Maps and Adaptive Filters

  • Chapter
Models of Neural Networks

Part of the book series: Physics of Neural Networks ((NEURAL NETWORKS))

Synopsis

Topographically organized maps and adaptive filters fulfill important roles for information processing in the brain and are also promising to facilitate tasks in digital information processing. In this contribution, we report results on two important network models. A first network model comprises the “self-organizing feature maps” of Kohonen. We discuss their relation to optimal representation of data, present results of a mathematical analysis of their behavior near a stationary state, demonstrate the formation of “striped projections”, if higher-dimensional feature spaces are to be mapped onto a two-dimensional cortical surface, and present recent simulation results for the somatosensory map of the skin surface and the retinal map in the visual cortex. The second network model is a hierarchical network for principal component analysis. Such a network, when trained with correlated random patterns, develops cells the receptive fields of which correspond to Gabor filters and resemble the receptive fields of “simple cells” in the visual cortex.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 74.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. D.J. Amit: Modeling Brain Function (Cambridge University Press 1989)

    MATH  Google Scholar 

  2. D J. Amit, H. Gutfreund, H. Sompolinsky: Storing infinite number of patterns in a spin-glass model of neural networks, Phys. Rev. Lett. 55, 1530–1533 (1985)

    Article  ADS  Google Scholar 

  3. D J. Amit, H. Gutfreund, H. Sompolinsky: Information storage in neural networks with low level of activity, Phys. Rev. A 35, 2293–2303 (1987)

    Article  MathSciNet  ADS  Google Scholar 

  4. S. Grossberg: Adaptive pattern classification and universal recoding: I. Parallel development and coding of neural feature detectors, Biol. Cybem. 23, 121–134 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  5. D.E. Rumelhart, D. Zipser: Feature discovery by competitive learning, Cognitive Science 9, 75–112 (1985)

    Article  Google Scholar 

  6. J. Buhmann, R. Divko, K. Schulten: Associative memory with high information content, Phys. Rev. A 39, 2689–2692 (1989)

    Article  MathSciNet  ADS  Google Scholar 

  7. E. Gardner, B. Derrida: Optimal storage properties of neural network models, J. Phys. A 21, 271–284 (1988)

    Article  MathSciNet  ADS  Google Scholar 

  8. T. Kohonen: Self-organized formation of topologically correct feature maps, Biol. Cybem. 43, 59–69 (1982)

    Article  MATH  Google Scholar 

  9. T. Kohonen: Analysis of a simple self-organizing process, Biol. Cybem. 44, 135–140 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  10. T. Kohonen: Self-Organization and Associative Memory, Springer Series in Information Sciences 8 (Springer, Berlin, Heidelberg 1984)

    MATH  Google Scholar 

  11. K. Obermayer, H. Ritter, K. Schulten: Large-scale simulation of self-organizing neural network on parallel computers: Application to biological modelling, Parallel Computing 14, 381–404 (1990), in press

    Article  Google Scholar 

  12. H. Ritter, T. Martinetz, K. Schulten: Topology conserving maps for learning visuomotor- coordination, Neural Networks 2, 159–168 (1989)

    Article  Google Scholar 

  13. T. Martinetz, H. Ritter, K. Schulten: Three-dimensional neural net for learning visuomotor- coordination of a robot arm, IEEE Transactions on Neural Networks 1, 131–136 (1990)

    Article  Google Scholar 

  14. H. Ritter, T. Martinetz, K. Schulten: Neuronale Netze—Eine Einführung in die Neuroinformatik selbstorganisierender Netzwerke (Addison-Wesley, Bonn 1990) (in German)

    Google Scholar 

  15. D.H. Hubel, T.N. Wiesel: Receptive fields of single neurones in cat’s striate cortex, J. Physiol. 148, 574–591 (1959)

    Google Scholar 

  16. J.P. Jones, L.A. Palmer: An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex, J. Neurophysiol. 58, 1133–1258 (1987)

    Google Scholar 

  17. WM. Cowan: The development of the brain, Sci. Am. 241, 107–117 (1979)

    Article  Google Scholar 

  18. R. Linsker: Self-organization in a perceptual network, IEEE Computer 21, 105–117 (1988)

    Article  Google Scholar 

  19. Y. Linde, A. Buzo, R.M. Gray: An algorithm for vector quantizer design, IEEE Trans. Comm. 28, 84–95 (1980)

    Article  Google Scholar 

  20. J. Makhoul, S. Roucos, H. Gish: Vector quantization in speech coding, Proc. IEEE (1985), 73–1551–1558

    Google Scholar 

  21. H. Ritter, K. Schulten: Kohonen’s self-organizing maps: Exploring their computational capabilities, Proc. IEEE ICNN 88, San Diego (IEEE Computer Society press 1988), Vol. I, pp.109–116

    Google Scholar 

  22. S.P. Luttrell: “Self-organisation: A derivation from first principles of a class of learning algorithms”, in: Proc. IJCNN 89, Washington DC (IEEE Computer Society Press, 1989), Vol. II pp. 495–498

    Google Scholar 

  23. H. Ritter, K. Schulten: Convergence properties of Kohonen’s topology conserving maps: Fluctuations, stability and dimension selection, Biol. Cybern. 60, 59–71 (1989)

    MathSciNet  Google Scholar 

  24. S. LeVay, T.N. Wiesel, D.H. Hubei: The development of ocular dominance columns in normal and visually deprived monkeys, J. Comp. Neurol. 191, 1–51 (1974)

    Article  Google Scholar 

  25. K.D. Miller, J.B. Keller, M.P. Stryker: Ocular dominance column development: analysis and simulation, Science 245, 605–615 (1989)

    Article  ADS  Google Scholar 

  26. G.G. Blasdel, G. Salama: Voltage-sensitive dyes reveal a modular organization in monkey striate cortex, Nature 321, 579–585 (1986)

    Article  ADS  Google Scholar 

  27. D.H. Hubel, T.N. Wiesel: Sequence regularity and geometry of orientation columns in the monkey striate cortex, J. Comp. Neurol. 158, 267–294 (1974)

    Article  Google Scholar 

  28. C. von der Malsburg: Development of ocularity domains and growth behavior of axon terminals, Biol. Cybem. 32, 49–62 (1979)

    Article  Google Scholar 

  29. C. von der Malsburg: Outline of a theory for the ontogenesis of iso-orientation domains in visual cortex (1982)

    Google Scholar 

  30. A. Takeuchi, S. Amari: Formation of topographic maps and columnar microstructures, Biol. Cybem. 35, 63–72 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  31. D.J. Willshaw, C. von der Malsburg: How patterned neural connections can be set up by self-organization, Proc. R. Soc. London B 194, 431–445 (1976)

    Article  ADS  Google Scholar 

  32. H. Ritter, T. Kohonen: Self-organizing semantic maps, Biol. Cybem. 61, 241–254 (1989)

    Article  Google Scholar 

  33. J.H. Kaas, RJ. Nelson, M. Sur, C.S. Lin, M.M. Merzenich: Multiple representations of the body within the primary somatosensory cortex of primates, Science 204, 521–523 (1979)

    Article  ADS  Google Scholar 

  34. J.H. Kaas, M.M. Merzenich, H.P. Killackey: The reorganization of somatosensory cortex following peripheral neve damage in adult and developing mammals, Annual Rev. Neurosci. 6, 325–256 (1983)

    Article  Google Scholar 

  35. M.M. Merzenich et al.: Somatosensory cortical map changes following digit amputation in adult monkeys, J. Comp. Neu. 224, 591 (1984)

    Article  Google Scholar 

  36. C. von der Malsburg, DJ. Willshaw: How to label nerve cells so that they can interconnect in an ordered fashion, Proc. Natl. Acad. Sci. USA 74, 5176–5178 (1977)

    Article  ADS  Google Scholar 

  37. J.C. Pearson, L.H. Finkel, G.M. Edelman: Plasticity in the organization of adult cerebral maps: A computer simulation based on neuronal group selection, J: Neurosci. 12,4209–4223 (1987)

    Google Scholar 

  38. K. Obermayer, H. Ritter, K. Schulten: Large-scale simulation of a self-organizing neural network: Formation of a somatotopic Map, Parallel Processing in Neural Systems and Computers, edited by R. Eckmiller, G. Hartmann, and G. Hauske (North-Holland, Amsterdam 1990) 71–74

    Google Scholar 

  39. K. Obermayer, H. Ritter, K. Schulten: A neural network model for the formation of topographic maps in the CNS: Development of receptive fields, IJCNN-90, Conf. Proceedings II, 423–429, San Diego (1990)

    Google Scholar 

  40. J. Rubner, K. Schulten: A self-organizing network for complete feature extraction, Biol. Cybem. 62, 193–199 (1990)

    Article  Google Scholar 

  41. D.N. Lawlwy, A.E. Maxwell: Factor Analysis as a Statistical Method (Butterworths, London 1963)

    Google Scholar 

  42. E. Oja: A simplified neuron model as a principal component analyzer, J. Math. Biology 15, 267–272 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  43. JJ. Hopfield, D.I. Feinstein, R.G. Palmer: “Unlearning” has a stabilizing effect in collective memories, Nature 304, 158–159 (1983)

    Article  ADS  Google Scholar 

  44. J. Rubner, P. Tavan: A self-organizing network for principal component analysis, Europhys. Lett. (1989)

    Google Scholar 

  45. H. Ritter: Asymptotic level density for a class of vector quantization processes, Internal Report A9, Helsinki Univ. of Technology (1989)

    Google Scholar 

  46. C. von der Malsburg: Self-organization of orientation sensitive cells in the striate cortex, Kybernetik 14, 85–100 (1973)

    Article  Google Scholar 

  47. A.L. Yuille, D.M. Kammen, D.S. Cohen: Quadrature and the development of orientation selective cortical cells by Hebb rules, Biol. Cybem. 61, 183–194 (1989)

    Article  MATH  Google Scholar 

  48. P. Baldi, K. Homik: Neural networks and principal component analysis: leraning from examples without local minima, Neural Networks 2, 53–58 (1989)

    Article  Google Scholar 

  49. J.P. Jones, L.A. Palmer: The two-dimensional spatial structure of simple receptive fields in cat striate cortex, J. Neurophysiol. 58, 1187–1211 (1987)

    Google Scholar 

  50. J.P. Jones, A. Stepnoski, L.A. Palmer: The two-dimensional spectral structure of simple receptive fields in cat striate cortex, J. Neurophysiol. 58, 1112–1232 (1987)

    Google Scholar 

  51. M. Cottrell, J.C. Fort: A stochastic model of retinotopy: A self-organizing process, Biol. Cybem. 53, 405–411 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  52. N.G. van Kampen: Stochastic Processes in Physics and Chemistry (North-Holland, Amsterdam 1981)

    MATH  Google Scholar 

  53. E.I. Knudsen, S. du Lac, S.D. Esterly: Computational maps in the brain, Ann. Rev. Neurosci. 10, 41–65 (1987)

    Article  Google Scholar 

  54. T. Kohonen: Clustering, Taxonomy and Topological Maps of Patterns, Proc 6th Int. Conf. on Pattem Recognition, Munich pp. 114–128 (1982)

    Google Scholar 

  55. H. Ritter, K. Schulten: On the stationary state of Kohonen’s self-organizing sensory mapping, Biol. Cybem. 54, 99–106 (1984)

    Article  Google Scholar 

Download references

Authors
  1. Helge Ritter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Klaus Obermayer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Klaus Schulten
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jeanne Rubner
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Electronics, Weizmann Institute of Science, 76100, Rehovot, Israel

    Eytan Domany

  2. Institut für Theoretische Physik, Technische Universität München, D-8046, Garching bei München, Fed. Rep. of Germany

    J. Leo van Hemmen

  3. Department of Physics and Beckman Institute, University of Illinois, Urbana, IL, 61801, USA

    Klaus Schulten

Rights and permissions

Reprints and permissions

Copyright information

© 1991 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Ritter, H., Obermayer, K., Schulten, K., Rubner, J. (1991). Self-organizing Maps and Adaptive Filters. In: Domany, E., van Hemmen, J.L., Schulten, K. (eds) Models of Neural Networks. Physics of Neural Networks. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-97171-6_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-97171-6_8

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-97173-0

  • Online ISBN: 978-3-642-97171-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation