The Principles of Neural Networks

  • Cornelia Denz


This chapter wants to give an overview over established ideas, algorithms and methods of neural computing. It contains the basic definitions necessary for further reading. Beginning with the model that is most equivalent to the brain’s structure — the single neuron — some simple examples of binary and analog network models are presented in their structure and in their principles. These examples have been chosen because they are fairly well-suited for optical implementations. Therefore, all following examples of optical neural net architectures will refer to the basic explanations in the present chapter. For the reader who is already familiar with the principle functions of several net structures, some of the explanations and algorithms might already be well-known. Then this chapter might be briefly read to familiarize with the foundations of the subject or even skipped for a first reading.


Energy Function Weight Vector Output Node Spin Glass Input Pattern 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Further Reading

  1. 1.
    D. Hebb, The Organization of Behaviour, J. Wiley & Sons (1949) . The original ideas of Hebb concerning learning by reinforcement with special emphasize on learning in simple neurons and the one-layer perceptron.Google Scholar
  2. 2.
    F. Rosenblatt, The perceptron: a probabilistic model for information storage and organization in the brain, Psych. Rev. 65 (1958), p. 386. One of the original publications on the beginning of the perceptron model.MathSciNetCrossRefGoogle Scholar
  3. 3.
    M.A. Minsky, S.A. Papert, Perceptrons. An introduction to Computational Geometry, MIT Press (1969) and Perceptrons. Expanded edition, MIT Press (1988). Concepts and criticism on single-layer perceptrons, with a lot of mathematical derivations and proofs.Google Scholar
  4. 4.
    P. Kanerva, Sparse Distributed Memory, MIT Press, Bradford Books, Cambridge (Mass.) (1986) An overview over models that are derived from the backpropagation paradigm.Google Scholar
  5. 5.
    J.S. Albus, Brains, Behavior and Robotics, McGraw-Hill, New York (1981) A summary of models that originate from the multilayer perceptron idea.Google Scholar
  6. 6.
    T. Kohonen, Self-Organization and Associative Memory, Springer Verlag, Berlin (1984). An introduction to concepts of associative memory and discussion of selforganization principles.Google Scholar
  7. 7.
    T. Kohonen, Adaptive, associative, and self-organizing functions in neural computing, Appl. Opt. 26 (1987), p. 4910. This paper describes certain adaptive and cooperative functions encountered in neural networks, including biological and mathematical models of description.CrossRefGoogle Scholar
  8. 8.
    H. Ritter, T. Martinetz and K. Schulten, Neural computing and self-organizing maps, Addison-Wesley, Reading, Massachusetts (1992). Self-organization is prescribed for neural computing, including the formation of feature nets and applications for computational tasks.Google Scholar
  9. 9.
    J.J. Hopfield, D.W. Tank, Computing with neural circuits: a model, Science 233 (1986), p. 633. A well-written description of the Hopfield model.CrossRefGoogle Scholar
  10. 10.
    G.E. Hinton and T.J. Sejnowski, Learning and relearning in Boltzmann machines, in J.L. McClelland, D.E. Rumelhardt, Parallel Distributed Processing, Vol. 1, MIT Press (1986), chapter 7. This chapter of the famous Parallel distributed Processing book gives an excellent overview over the principles of Boltzmann machines.Google Scholar
  11. 11.
    G.A. Carpenter, S. Grossberg, The ART of Adaptive Pattern Recognition, IEEE Computer 21 (1988), no. 3. A well-structured introduction to ART and a big source for further references.CrossRefGoogle Scholar

Copyright information

© Springer Fachmedien Wiesbaden 1998

Authors and Affiliations

  • Cornelia Denz
    • 1
  1. 1.Institut für Angewandte Optik, AG Photorefraktive OptikTechnische Universität DarmstadtDarmstadtGermany

Personalised recommendations