Advertisement

Unsupervised Learning and Self-Organization in Networks of Spiking Neurons

  • Thomas Natschläger
  • Berthold Ruf
  • Michael Schmitt
Part of the Studies in Fuzziness and Soft Computing book series (STUDFUZZ, volume 78)

Abstract

One of the most prominent features of biological neural systems is that individual neurons communicate via short electrical pulses, the so-called action potentials or spikes. In this chapter we investigate possible mechanisms of unsupervised learning and self-organization in networks of spiking neurons. After giving a brief introduction to spiking neuron networks we describe a biologically plausible algorithm for these networks to find clusters in a high dimensional input space or a subspace of it. The algorithm is shown to work even in a dynamically changing environment. Furthermore, we study self-organizing maps of spiking neurons showing that networks of spiking neurons using temporal coding can achieve a topology preserving behavior quite similar to that of Kohonen’s self-organizing map. For these networks a mechanism of competitive computation is proposed that is based on action potential timing. Thus, the winner in a population of competing neurons can be determined locally and in generally faster than in approaches which use rate coding. The models and algorithms presented in this chapter establish further steps toward more realistic descriptions of unsupervised learning in biological neural systems.

Keywords

Radial Basis Function Spike Train Rate Code Unsupervised Learn Input Neuron 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    L. F. Abbott and S. B. Nelson. Synaptic plasticity: taming the beast. Nature Neuroscience, 3 (Supp): 1178–1183, 2000.CrossRefGoogle Scholar
  2. 2.
    L. E. Abbott, J. A. Varela, K. Sen, and S. B. Nelson. Synaptic depression and gain control. Science, 275: 220–224, 1997.CrossRefGoogle Scholar
  3. 3.
    M. Abeles, H. Bergman, E. Margalit, and E. Vaadia. Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. Journal of Neurophysiology, 70 (4): 1629–1638, 1993.Google Scholar
  4. 4.
    M. A. Arbib, editor. The Handbook of Brain Theory and Neural Networks. MIT Press, Cambridge, Mass., 1995.Google Scholar
  5. 5.
    G. Blasel and K. Obermayer. Putative strategies of scene segmentation in monkey visual cortex. Neural Networks, 7: 865–881, 1994.CrossRefGoogle Scholar
  6. 6.
    Y. Choe and R. Miikkulainen. Self-organization and segmentation in a laterally connected orientation map of spiking neurons. Neurocomputing, 21: 139–157, 1998.CrossRefMATHGoogle Scholar
  7. 7.
    T. J. Gawne, T. Kjaer, and B. Richmond. Latency: Another potential code for feature binding in striate cortex. Journal of Neurophysiology, 76 (2): 1356–1360, 1996.Google Scholar
  8. 8.
    W. Gerstner. Spiking neurons. In W. Maass and C. M. Bishop, editors, Pulsed Neural Networks, pages 3–53. MIT Press, Cambridge, Mass., 1999.Google Scholar
  9. 9.
    W. Gerstner, R. Kempter, L. van Hemmen, and H. Wagner. A neuronal learning rule for sub-millisecond temporal coding. Nature, 383: 76–78, 1996.CrossRefGoogle Scholar
  10. 10.
    G. J. Goodhill and T. J. Sejnowski. A unifying objective function for topographic mappings. Neural Computation, 9: 1291–1303, 1997.CrossRefGoogle Scholar
  11. 11.
    S. Grossberg. Adaptive pattern classification and universal recording: II. Feedback, expectation, olfaction, illusions. Biological Cybernetics, 23: 187–202, 1976.MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    L. Haberly. Neuronal circuitry in olfactory cortex: Anatomy and functional implications. Chemical Senses, 10 (2): 219–238, 1985.CrossRefGoogle Scholar
  13. 13.
    P. Häfliger, M. Mahowald, and L. Watts. A spike based learning neuron in analog VLSI. In Advances in Neural Information Processing Systems 9, pages 692–698, MIT Press, Cambridge, Mass., 1997.Google Scholar
  14. 14.
    J. J. Hopfield. Pattern recognition computation using action potential timing for stimulus representation. Nature, 367: 33–36, 1995.CrossRefGoogle Scholar
  15. 15.
    D. Johnston and S. M. S. Wu. Foundations of Cellular Neurophysiology. MIT Press, Cambridge, Mass., 1995.Google Scholar
  16. 16.
    C. Koch. Biophysics of Computation: Information Processing in Single Neurons. Oxford University Press, 1999.Google Scholar
  17. 17.
    C. Koch and I. Segev. Methods in Neural Modeling: From Ions to Networks. MIT Press, Cambridge, Mass., 1998.Google Scholar
  18. 18.
    T. Kohonen. Physiological interpretation of the self-organizing map algorithm. Neural Networks, 6: 895–905, 1993.Google Scholar
  19. 19.
    T. Kohonen. Self-Organizing Maps. Springer, Berlin, 1995.CrossRefGoogle Scholar
  20. 20.
    B. Krekelberg and J. G. Taylor. Nitric oxide: What can it compute? Network: Computation in Neural Systems, 8: 1–16, 1997.CrossRefMATHGoogle Scholar
  21. 21.
    T. Lehmann and R. Woodburn. Biologically-inspired on-chip learning in pulsed neural networks. Analog Integrated Circuits and Signal Processing, 18: 117–131, 1999.CrossRefGoogle Scholar
  22. 22.
    W. Maass. Fast sigmoidal networks via spiking neurons. Neural Computation, 9: 279–304, 1997.CrossRefMATHGoogle Scholar
  23. 23.
    W. Maass. Networks of spiking neurons: The third generation of neural network models. Neural Networks, 10: 1659–1671, 1997.CrossRefGoogle Scholar
  24. 24.
    W. Maass. Computing with spiking neurons. In W. Maass and C. M. Bishop, editors, Pulsed Neural Networks, chapter 2, pages 55–85. MIT Press, Cambridge, Mass., 1999.Google Scholar
  25. 25.
    W. Maass and C. M. Bishop, editors. Pulsed Neural Networks. MIT Press, Cambridge, Mass., 1999.Google Scholar
  26. 26.
    H. Markram, Y. Wang, and M. Tsodyks. Differential signaling via the same axon of neocortical pyramidal neurons. Proc. Nat. Acad. Sci. USA, 95: 5323–8, 1998.CrossRefGoogle Scholar
  27. 27.
    J. O’Keefe and M. L. Reece. Phase relationship between hippocampal place units and the EEG theta rhythm. Hippocampus, 3 (3): 3317–30, 1993.Google Scholar
  28. 28.
    E Rieke, D. Warland, W. Bialek, and R. de Ruyter van Steveninck. SPIKES: Exploring the Neural Code. MIT Press, Cambridge, Mass., 1999.Google Scholar
  29. 29.
    H. Ritter. Self-organizing feature maps: Kohonen maps. In M. A. Arbib, editor, The Handbook of Brain Theory and Neural Networks, pages 846–851. MIT Press, Cambridge, Mass., 1995.Google Scholar
  30. 30.
    H. Ritter, T. Martinetz, and K. Schulten. Neural Computation and Self-Organizing Maps. Addison-Wesley, Reading, Mass., 1992.MATHGoogle Scholar
  31. 31.
    R. Ritz and T. J. Sejnowski. Synchronous oscillatory activity in sensory systems: new vistas on mechanisms. Current Opinion in Neurobiology, 7: 536–546, 1997.CrossRefGoogle Scholar
  32. 32.
    B. Ruf. Computing and Learning with Spiking Neurons-Theory and Simulations. PhD thesis, Institute for Theoretical Computer Science, Technische Universität Graz, Austria, 1998.Google Scholar
  33. 33.
    I. Segev. Dendritic processing. In M. A. Arbib, editor, The Handbook of Brain Theory and Neural Networks, pages 282–289. MIT Press, Cambridge, Mass., 1995.Google Scholar
  34. 34.
    J. Sirosh and R. Miikkulainen. Topographic receptive fields and patterned lateral interaction in a self-organizing model of the primary visual cortex. Neural Computation, 9: 577–594, 1997.CrossRefGoogle Scholar
  35. 35.
    K.-Y. Siu, V. Roychowdhury, and T. Kailath. Discrete Neural Computation: A Theoretical Foundation. Information and System Sciences Series. Prentice-Hall, Englewood Cliffs, NJ, 1995.MATHGoogle Scholar
  36. 36.
    D. W. Tank and J. J. Hopfield. Neural computation by concentrating information in time. Proc. Nat. Acad. Sci. USA, 84: 1896–1900, Apr. 1987.MathSciNetCrossRefGoogle Scholar
  37. 37.
    S. Thorpe, D. Fize, and C. Marlot. Speed of processing in the human visual system. Nature, 381: 520–522, 1996.CrossRefGoogle Scholar
  38. 38.
    J. A. Varela, K. Sen, J. Gibson, J. Fost, L. F. Abbott, and S. B. Nelson. A quantitative description of short-term plasticity at excitatory synapses in layer 2/3 of rat primary visual cortex. J. Neurosci, 17: 220–4, 1997.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Thomas Natschläger
  • Berthold Ruf
  • Michael Schmitt

There are no affiliations available

Personalised recommendations