Statistical Physics of Learning and Generalization

  • Wolfgang Kinzel
Chapter

Abstract

Since 1982, starting with the work of Hopfield, theoretical physics is contributing to the theory of neural networks. In his pioneering work, Hopfield pointed out a relation between models of disordered magnets (spin glasses) and models of neurons interacting by competing synaptic couplings. This work started an extensive research effort: using models, methods and principles of statistical physics one has described the cooperative behavior of a large system of interacting neurons. Now, almost two decades later, much has been achieved in this field: associative memory, learning from examples, generalization from examples to an unknown rule, time series prediction, optimizing architectures and learning rules, all this has been expressed in a mathematical language which allows to calculate the cooperative properties of infinitely large systems being trained on infinitely many patterns [1, 2].

Keywords

Entropy Dition 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Hertz, J. and Krogh, A., and Palmer, R.G.: Introduction to the Theory of Neural Computation, ( Addison Wesley, Redwood City, 1991 )Google Scholar
  2. Engel, A. and Van den Broeck, C.: Statistical Mechanics of Learning, (Cambridge University Press, 2001 )Google Scholar
  3. W. Kinzel,: Z. Phys. B 60, 205 (1985)MathSciNetCrossRefADSGoogle Scholar
  4. D. Amit, H. Gutfreund and H. Sompolinsly, Ann. Phys. (NY) 173, 30 (1987)CrossRefADSGoogle Scholar
  5. M. Opper and W.Kinzel: Statistical Mechanics of Generalization, Models of Neural Networks III, ed. by E. Domany and J.L. van Hemmen and K. Schulten, 151-209 ( Springer Verlag, Heidelberg 1995 )Google Scholar
  6. M. Opper and D. Haussier: Phys. Rev. Lett. 66, 2677 (1991)MathSciNetCrossRefMATHADSGoogle Scholar
  7. H. Schwarze, M. Opper and W. Kinzel: Phys. Rev. A 46, R6185 (1992)CrossRefADSGoogle Scholar
  8. M. Opper: Phys. Rev. Lett. 72, 2113 (1994)CrossRefADSGoogle Scholar
  9. A. Weigand and N. S. Gershenfeld: Time Series Prediction, Santa Fe, ( Addison Wesley, 1994 )Google Scholar
  10. W. Kinzel, G. Reents: Physics by Computer, (Springer Verlag, 1998 )Google Scholar
  11. E. Eisenstein and I. Kanter and D.A. Kessler and W. Kinzel: Generation and Prediction of Time Series by a Neural Network, Phys. Rev. Letters 74 1, 6 - 9 (1995)CrossRefADSGoogle Scholar
  12. A. Priel and I. Kanter: Robust chaos generation by a perceptron, Europhys. Lett. 51, 244 - 250 (2000)Google Scholar
  13. A. Freking and W. Kinzel and I. Kanter: Phys. Rev. E (2002)Google Scholar
  14. W. Kinzel, R. Metzler, I. Kanter: J. Phys. A 33 L141 - L147 (2000);MathSciNetCrossRefMATHADSGoogle Scholar
  15. R. Metzler and W. Kinzel and I. Kanter: Phys. Rev. E 62 2, 2555 (2000)MathSciNetGoogle Scholar
  16. I. Kanter, W. Kinzel and E. Kanter, Europhys. Lett. 57, 141-147 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Wolfgang Kinzel

There are no affiliations available

Personalised recommendations