Advertisement

Statistical Physics of Learning and Generalization

  • Wolfgang Kinzel
Chapter

Abstract

Since 1982, starting with the work of Hopfield, theoretical physics is contributing to the theory of neural networks. In his pioneering work, Hopfield pointed out a relation between models of disordered magnets (spin glasses) and models of neurons interacting by competing synaptic couplings. This work started an extensive research effort: using models, methods and principles of statistical physics one has described the cooperative behavior of a large system of interacting neurons. Now, almost two decades later, much has been achieved in this field: associative memory, learning from examples, generalization from examples to an unknown rule, time series prediction, optimizing architectures and learning rules, all this has been expressed in a mathematical language which allows to calculate the cooperative properties of infinitely large systems being trained on infinitely many patterns [1, 2].

Keywords

Hide Unit Generalization Error Chaotic Sequence Time Series Prediction Multilayer Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Hertz, J. and Krogh, A., and Palmer, R.G.: Introduction to the Theory of Neural Computation, ( Addison Wesley, Redwood City, 1991 )Google Scholar
  2. Engel, A. and Van den Broeck, C.: Statistical Mechanics of Learning, (Cambridge University Press, 2001 )Google Scholar
  3. W. Kinzel,: Z. Phys. B 60, 205 (1985)MathSciNetCrossRefADSGoogle Scholar
  4. D. Amit, H. Gutfreund and H. Sompolinsly, Ann. Phys. (NY) 173, 30 (1987)CrossRefADSGoogle Scholar
  5. M. Opper and W.Kinzel: Statistical Mechanics of Generalization, Models of Neural Networks III, ed. by E. Domany and J.L. van Hemmen and K. Schulten, 151-209 ( Springer Verlag, Heidelberg 1995 )Google Scholar
  6. M. Opper and D. Haussier: Phys. Rev. Lett. 66, 2677 (1991)MathSciNetCrossRefMATHADSGoogle Scholar
  7. H. Schwarze, M. Opper and W. Kinzel: Phys. Rev. A 46, R6185 (1992)CrossRefADSGoogle Scholar
  8. M. Opper: Phys. Rev. Lett. 72, 2113 (1994)CrossRefADSGoogle Scholar
  9. A. Weigand and N. S. Gershenfeld: Time Series Prediction, Santa Fe, ( Addison Wesley, 1994 )Google Scholar
  10. W. Kinzel, G. Reents: Physics by Computer, (Springer Verlag, 1998 )Google Scholar
  11. E. Eisenstein and I. Kanter and D.A. Kessler and W. Kinzel: Generation and Prediction of Time Series by a Neural Network, Phys. Rev. Letters 74 1, 6 - 9 (1995)CrossRefADSGoogle Scholar
  12. A. Priel and I. Kanter: Robust chaos generation by a perceptron, Europhys. Lett. 51, 244 - 250 (2000)Google Scholar
  13. A. Freking and W. Kinzel and I. Kanter: Phys. Rev. E (2002)Google Scholar
  14. W. Kinzel, R. Metzler, I. Kanter: J. Phys. A 33 L141 - L147 (2000);MathSciNetCrossRefMATHADSGoogle Scholar
  15. R. Metzler and W. Kinzel and I. Kanter: Phys. Rev. E 62 2, 2555 (2000)MathSciNetGoogle Scholar
  16. I. Kanter, W. Kinzel and E. Kanter, Europhys. Lett. 57, 141-147 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Wolfgang Kinzel

There are no affiliations available

Personalised recommendations