Advertisement

Self-organizing Maps and Adaptive Filters

  • Helge Ritter
  • Klaus Obermayer
  • Klaus Schulten
  • Jeanne Rubner
Part of the Physics of Neural Networks book series (NEURAL NETWORKS)

Synopsis

Topographically organized maps and adaptive filters fulfill important roles for information processing in the brain and are also promising to facilitate tasks in digital information processing. In this contribution, we report results on two important network models. A first network model comprises the “self-organizing feature maps” of Kohonen. We discuss their relation to optimal representation of data, present results of a mathematical analysis of their behavior near a stationary state, demonstrate the formation of “striped projections”, if higher-dimensional feature spaces are to be mapped onto a two-dimensional cortical surface, and present recent simulation results for the somatosensory map of the skin surface and the retinal map in the visual cortex. The second network model is a hierarchical network for principal component analysis. Such a network, when trained with correlated random patterns, develops cells the receptive fields of which correspond to Gabor filters and resemble the receptive fields of “simple cells” in the visual cortex.

Keywords

Receptive Field Input Pattern Synaptic Weight Gabor Filter Output Unit 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 8.1
    D.J. Amit: Modeling Brain Function (Cambridge University Press 1989)MATHGoogle Scholar
  2. 8.2
    D J. Amit, H. Gutfreund, H. Sompolinsky: Storing infinite number of patterns in a spin-glass model of neural networks, Phys. Rev. Lett. 55, 1530–1533 (1985)ADSCrossRefGoogle Scholar
  3. 8.3
    D J. Amit, H. Gutfreund, H. Sompolinsky: Information storage in neural networks with low level of activity, Phys. Rev. A 35, 2293–2303 (1987)MathSciNetADSCrossRefGoogle Scholar
  4. 8.4
    S. Grossberg: Adaptive pattern classification and universal recoding: I. Parallel development and coding of neural feature detectors, Biol. Cybem. 23, 121–134 (1976)MathSciNetMATHCrossRefGoogle Scholar
  5. 8.5
    D.E. Rumelhart, D. Zipser: Feature discovery by competitive learning, Cognitive Science 9, 75–112 (1985)CrossRefGoogle Scholar
  6. 8.6
    J. Buhmann, R. Divko, K. Schulten: Associative memory with high information content, Phys. Rev. A 39, 2689–2692 (1989)MathSciNetADSCrossRefGoogle Scholar
  7. 8.7
    E. Gardner, B. Derrida: Optimal storage properties of neural network models, J. Phys. A 21, 271–284 (1988)MathSciNetADSCrossRefGoogle Scholar
  8. 8.8
    T. Kohonen: Self-organized formation of topologically correct feature maps, Biol. Cybem. 43, 59–69 (1982)MATHCrossRefGoogle Scholar
  9. 8.9
    T. Kohonen: Analysis of a simple self-organizing process, Biol. Cybem. 44, 135–140 (1982)MathSciNetMATHCrossRefGoogle Scholar
  10. 8.10
    T. Kohonen: Self-Organization and Associative Memory, Springer Series in Information Sciences 8 (Springer, Berlin, Heidelberg 1984)MATHGoogle Scholar
  11. 8.11
    K. Obermayer, H. Ritter, K. Schulten: Large-scale simulation of self-organizing neural network on parallel computers: Application to biological modelling, Parallel Computing 14, 381–404 (1990), in pressCrossRefGoogle Scholar
  12. 8.12
    H. Ritter, T. Martinetz, K. Schulten: Topology conserving maps for learning visuomotor- coordination, Neural Networks 2, 159–168 (1989)CrossRefGoogle Scholar
  13. 8.13
    T. Martinetz, H. Ritter, K. Schulten: Three-dimensional neural net for learning visuomotor- coordination of a robot arm, IEEE Transactions on Neural Networks 1, 131–136 (1990)CrossRefGoogle Scholar
  14. 8.14
    H. Ritter, T. Martinetz, K. Schulten: Neuronale NetzeEine Einführung in die Neuroinformatik selbstorganisierender Netzwerke (Addison-Wesley, Bonn 1990) (in German)Google Scholar
  15. 8.15
    D.H. Hubel, T.N. Wiesel: Receptive fields of single neurones in cat’s striate cortex, J. Physiol. 148, 574–591 (1959)Google Scholar
  16. 8.16
    J.P. Jones, L.A. Palmer: An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex, J. Neurophysiol. 58, 1133–1258 (1987)Google Scholar
  17. 8.17
    WM. Cowan: The development of the brain, Sci. Am. 241, 107–117 (1979)CrossRefGoogle Scholar
  18. 8.18
    R. Linsker: Self-organization in a perceptual network, IEEE Computer 21, 105–117 (1988)CrossRefGoogle Scholar
  19. 8.19
    Y. Linde, A. Buzo, R.M. Gray: An algorithm for vector quantizer design, IEEE Trans. Comm. 28, 84–95 (1980)CrossRefGoogle Scholar
  20. 8.20
    J. Makhoul, S. Roucos, H. Gish: Vector quantization in speech coding, Proc. IEEE (1985), 73–1551–1558Google Scholar
  21. 8.21
    H. Ritter, K. Schulten: Kohonen’s self-organizing maps: Exploring their computational capabilities, Proc. IEEE ICNN 88, San Diego (IEEE Computer Society press 1988), Vol. I, pp.109–116Google Scholar
  22. 8.22
    S.P. Luttrell: “Self-organisation: A derivation from first principles of a class of learning algorithms”, in: Proc. IJCNN 89, Washington DC (IEEE Computer Society Press, 1989), Vol. II pp. 495–498Google Scholar
  23. 8.23
    H. Ritter, K. Schulten: Convergence properties of Kohonen’s topology conserving maps: Fluctuations, stability and dimension selection, Biol. Cybern. 60, 59–71 (1989)MathSciNetGoogle Scholar
  24. 8.24
    S. LeVay, T.N. Wiesel, D.H. Hubei: The development of ocular dominance columns in normal and visually deprived monkeys, J. Comp. Neurol. 191, 1–51 (1974)CrossRefGoogle Scholar
  25. 8.25
    K.D. Miller, J.B. Keller, M.P. Stryker: Ocular dominance column development: analysis and simulation, Science 245, 605–615 (1989)ADSCrossRefGoogle Scholar
  26. 8.26
    G.G. Blasdel, G. Salama: Voltage-sensitive dyes reveal a modular organization in monkey striate cortex, Nature 321, 579–585 (1986)ADSCrossRefGoogle Scholar
  27. 8.27
    D.H. Hubel, T.N. Wiesel: Sequence regularity and geometry of orientation columns in the monkey striate cortex, J. Comp. Neurol. 158, 267–294 (1974)CrossRefGoogle Scholar
  28. 8.28
    C. von der Malsburg: Development of ocularity domains and growth behavior of axon terminals, Biol. Cybem. 32, 49–62 (1979)CrossRefGoogle Scholar
  29. 8.29
    C. von der Malsburg: Outline of a theory for the ontogenesis of iso-orientation domains in visual cortex (1982)Google Scholar
  30. 8.30
    A. Takeuchi, S. Amari: Formation of topographic maps and columnar microstructures, Biol. Cybem. 35, 63–72 (1979)MathSciNetMATHCrossRefGoogle Scholar
  31. 8.31
    D.J. Willshaw, C. von der Malsburg: How patterned neural connections can be set up by self-organization, Proc. R. Soc. London B 194, 431–445 (1976)ADSCrossRefGoogle Scholar
  32. 8.32
    H. Ritter, T. Kohonen: Self-organizing semantic maps, Biol. Cybem. 61, 241–254 (1989)CrossRefGoogle Scholar
  33. 8.33
    J.H. Kaas, RJ. Nelson, M. Sur, C.S. Lin, M.M. Merzenich: Multiple representations of the body within the primary somatosensory cortex of primates, Science 204, 521–523 (1979)ADSCrossRefGoogle Scholar
  34. 8.34
    J.H. Kaas, M.M. Merzenich, H.P. Killackey: The reorganization of somatosensory cortex following peripheral neve damage in adult and developing mammals, Annual Rev. Neurosci. 6, 325–256 (1983)CrossRefGoogle Scholar
  35. 8.35
    M.M. Merzenich et al.: Somatosensory cortical map changes following digit amputation in adult monkeys, J. Comp. Neu. 224, 591 (1984)CrossRefGoogle Scholar
  36. 8.36
    C. von der Malsburg, DJ. Willshaw: How to label nerve cells so that they can interconnect in an ordered fashion, Proc. Natl. Acad. Sci. USA 74, 5176–5178 (1977)ADSCrossRefGoogle Scholar
  37. 8.37
    J.C. Pearson, L.H. Finkel, G.M. Edelman: Plasticity in the organization of adult cerebral maps: A computer simulation based on neuronal group selection, J: Neurosci. 12,4209–4223 (1987)Google Scholar
  38. 8.38
    K. Obermayer, H. Ritter, K. Schulten: Large-scale simulation of a self-organizing neural network: Formation of a somatotopic Map, Parallel Processing in Neural Systems and Computers, edited by R. Eckmiller, G. Hartmann, and G. Hauske (North-Holland, Amsterdam 1990) 71–74Google Scholar
  39. 8.39
    K. Obermayer, H. Ritter, K. Schulten: A neural network model for the formation of topographic maps in the CNS: Development of receptive fields, IJCNN-90, Conf. Proceedings II, 423–429, San Diego (1990)Google Scholar
  40. 8.40
    J. Rubner, K. Schulten: A self-organizing network for complete feature extraction, Biol. Cybem. 62, 193–199 (1990)CrossRefGoogle Scholar
  41. 8.41
    D.N. Lawlwy, A.E. Maxwell: Factor Analysis as a Statistical Method (Butterworths, London 1963)Google Scholar
  42. 8.42
    E. Oja: A simplified neuron model as a principal component analyzer, J. Math. Biology 15, 267–272 (1982)MathSciNetMATHCrossRefGoogle Scholar
  43. 8.43
    JJ. Hopfield, D.I. Feinstein, R.G. Palmer: “Unlearning” has a stabilizing effect in collective memories, Nature 304, 158–159 (1983)ADSCrossRefGoogle Scholar
  44. 8.44
    J. Rubner, P. Tavan: A self-organizing network for principal component analysis, Europhys. Lett. (1989)Google Scholar
  45. 8.45
    H. Ritter: Asymptotic level density for a class of vector quantization processes, Internal Report A9, Helsinki Univ. of Technology (1989)Google Scholar
  46. 8.46
    C. von der Malsburg: Self-organization of orientation sensitive cells in the striate cortex, Kybernetik 14, 85–100 (1973)CrossRefGoogle Scholar
  47. 8.47
    A.L. Yuille, D.M. Kammen, D.S. Cohen: Quadrature and the development of orientation selective cortical cells by Hebb rules, Biol. Cybem. 61, 183–194 (1989)MATHCrossRefGoogle Scholar
  48. 8.48
    P. Baldi, K. Homik: Neural networks and principal component analysis: leraning from examples without local minima, Neural Networks 2, 53–58 (1989)CrossRefGoogle Scholar
  49. 8.49
    J.P. Jones, L.A. Palmer: The two-dimensional spatial structure of simple receptive fields in cat striate cortex, J. Neurophysiol. 58, 1187–1211 (1987)Google Scholar
  50. 8.50
    J.P. Jones, A. Stepnoski, L.A. Palmer: The two-dimensional spectral structure of simple receptive fields in cat striate cortex, J. Neurophysiol. 58, 1112–1232 (1987)Google Scholar
  51. 8.51
    M. Cottrell, J.C. Fort: A stochastic model of retinotopy: A self-organizing process, Biol. Cybem. 53, 405–411 (1986)MathSciNetMATHCrossRefGoogle Scholar
  52. 8.52
    N.G. van Kampen: Stochastic Processes in Physics and Chemistry (North-Holland, Amsterdam 1981)MATHGoogle Scholar
  53. 8.53
    E.I. Knudsen, S. du Lac, S.D. Esterly: Computational maps in the brain, Ann. Rev. Neurosci. 10, 41–65 (1987)CrossRefGoogle Scholar
  54. 8.54
    T. Kohonen: Clustering, Taxonomy and Topological Maps of Patterns, Proc 6th Int. Conf. on Pattem Recognition, Munich pp. 114–128 (1982)Google Scholar
  55. 8.55
    H. Ritter, K. Schulten: On the stationary state of Kohonen’s self-organizing sensory mapping, Biol. Cybem. 54, 99–106 (1984)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1991

Authors and Affiliations

  • Helge Ritter
  • Klaus Obermayer
  • Klaus Schulten
  • Jeanne Rubner

There are no affiliations available

Personalised recommendations