Abstract
Twenty years ago, Pastur and Figotin [FP1,FP2] first introduced and studied what has become known as the Hopfield model and which turned out, over the years, to be one of the more successful and important models of a disordered system. This is also reflected in the fact that several contributions in this book are devoted to it. The Hopfield model is quite versatile and models various situations. Pastur and Figotin introduced it as a simple model for a spin glass, and, in 1982, Hopfield independently considered it as a model for associative memory.
L’intuition ne peut nous donner la rigueur, ni même la certitude, on s’en est aperçu de plus en plus.
Henri Poincaré,
“La Valeur de La Science”
Work partially supported by the Commission of the European Union under contract CHRX-CT93-0411
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
D.J. Amit, Modelling brain function, Cambridge University Press, Cambridge, 1989.
D.J. Amit, H. Gutfreund and H. Sompolinsky, Statistical mechanics of neural networks near saturation, Ann. Phys. 173, 30–67 (1987).
D.J. Amit, H. Gutfreund, and H. Sompolinsky, Spin-glass model of neural network, Phys. Rev. A 32, 1007–1018 (1985).
A. Bovier, Self-averaging in a class of generalized Hopfield models, J. Phys. A 27, 7069–7077 (1994).
A. Bovier and V. Gayrard, Rigorous bounds on the storage capacity for the dilute Hopfield model, J. Stat. Phys. 69, 597–627 (1992)
A. Bovier and V. Gayrard, Rigorous results on the thermodynamics of the dilute Hopfield model, J. Stat. Phys. 72, 643–664 (1993).
A. Bovier and V. Gayrard, Rigorous results on the Hopfield model of neural networks, Resenhas do IME-USP 2, 161–172 (1994).
A. Bovier and V. Gayrard, An almost sure large deviation principle for the Hopfield model, Ann. Probab. 24, 1444–1475 (1996).
A. Bovier and V. Gayrard, The retrieval phase of the Hopfield model, A rigorous analysis of the overlap distribution, Probab. Theor. Rel. Fields 107, 61–98 (1995).
A. Bovier and V. Gayrard, Metatstates in the Hopfield model in the replica symmetric regime, preprint (1997).
A. Bovier, V. Gayrard, and P. Picco, Gibbs states of the Hopfield model in the regime of perfect memory, Probab. Theor. Rel. Fields 100, 329–363 (1994).
A. Bovier, V. Gayrard, and P. Picco, Large deviation principles for the Hopfield model and the Kac-Hopfield model, Probab. Theor. Rel. Fields 101, 511–546 (1995).
A. Bovier, V. Gayrard, and P. Picco, Gibbs states of the Hopfield model with extensively many patterns, J. Stat. Phys. 79, 395–414 (1995).
A. Bovier, V. Gayrard, and P. Picco, Distribution of overlap profiles in the one-dimensional Kac-Hopfield model, WIAS-preprint 221, to appear in Commun. Math. Phys. (1997).
A. Bovier and J. Fröhlich, A heuristic theory of the spin glass phase, J. Stat. Phys. 44, 347–391 (1986).
H.J. Brascamp and E.H. Lieb, On extensions of the Brunn-Minkowski and Pékopa-Leindler theorems, including inequalities for log concave functions, and with an application to the diffusion equation, J. Functional Anal. 22, 366–389 (1976).
F. Comets, Large deviation estimates for a conditional probability distribution. Applications to random Gibbs measures, Probab. Theor. Rel. Fields 80, 407–432 (1989).
Y.S. Chow and H. Teicher, Probability Theory, 2nd edition, Springer, Berlin-Heidelberg-New York, 1988.
E. Domany, J.L. van Hemmen, and K. Schulten (eds.), Models of Neural Networks, Springer-Verlag, Berlin, 1991.
J.-D. Deuschel and D. Stroock, Large Deviations, Academic Press, Boston, 1989.
A. Dembo and O. Zeitouni, Large Deviation Techniques and Applications, Jones and Bartlett, Boston, 1992.
R.S. Ellis, Entropy, Large Deviations, and Statistical Mechanics, Springer-Verlag, Berlin, 1985.
S.F. Edwards and P.W. Anderson, Theory of spin glasses, J. Phys. F 5, 965–974 (1975).
T. Eisele and R.S. Ellis, Multiple phase transitions in the generalized Curie-Weiss model, J. Stat Phys. 52, 161–202 (1988).
A.C.D. van Enter, Stiffness exponent, number of pure states, and Almeida-Thouless line in spin glasses, J. Stat. Phys. 60, 275–279 (1990).
A.C.D. van Enter, J.L. van Hemmen and C. Pospiech, Mean-field theory of random-site q-state Potts models, J. Phys. A 21, 791–801 (1988).
D.S. Fisher and D.A. Huse, Pure phases in spin glasses, J. Phys. A 20, L997-L1003 (1987)
D.S. Fisher and D.A. Huse, Absence of many states in magnetic spin glasses, J. Phys. A 20, L1005-L1010 (1987).
L.A. Pastur and A.L. Figotin, Exactly soluble model of a spin glass, Sov. J. Low Temp. Phys. 3(6), 378–383 (1977).
L.A. Pastur and A.L. Figotin, On the theory of disordered spin systems, Theor. Math. Phys. 35, 403–414 (1978).
L.A. Pastur and A.L. Figotin, Infinite range limit for a class of disordered spin systems, Theor. Math. Phys. 51, 564–569 (1982).
C. Fassnacht and A. Zippelius, Recognition and Categorization in a structured neural network with attractor dynamics, Network 2, 63–84 (1991).
V. Gayrard, The thermodynamic limit of the g-state Potts-Hopfield model with infinitely many patterns, J. Stat. Phys. 68, 977–1011 (1992).
S. Geman, A limit theorem for the norms of random matrices, Ann. Probab. 8, 252–261 (1980).
H.O. Georgii, Gibbs Measures and Phase Transitions, Walter de Gruyter (de Gruyter Studies in Mathematics, Vol. 19), Berlin-New York, 1988.
V.L. Girko, Limit theorems for maximal and minimal eigenvalues of random matrices, Theor. Probab. Appl. 35, 680–695 (1989).
E. Golez and S. Martínez, Neural and Automata Networks, Kluwer Academic Publ., Dodrecht, 1990.
J. Hertz, A. Krogh, and R. Palmer, Introduction to the Theory of Neural Computation, Addison-Wesley, Redwood City, 1991.
J.J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982).
R.L. Stratonovich, On a method of calculating quantum distribution functions, Doklady Akad. Nauk S.S.S.R. 115, 1097 (1957) [translation: Soviet Phys. Doklady 2, 416–419 (1958)
J. Hubbard, Calculation of partition functions, Phys. Rev. Lett. 3, 77–78 (1959).
J. Jedrzejewski and A. Kombda, On equivalent-neighbour, random-site models of disordered systems, Z. Phys. B 63, 247–257 (1986).
J.L. van Hemmen, Equilibrium theory of spin-glasses: mean-field theory and beyond, in Heidelberg Colloquium on Spin Glasses, eds. J.L. van Hemmen and I. Morgenstern, 203–233 (1983), LNP 192 Springer, Berlin-Heidelberg-New York, 1983.
J.L. van Hemmen, Spin glass models of a neural network, Phys. Rev. A 34, 3435–3445 (1986).
J.L. van Hemmen, D. Grensing, A. Huber and R. Kühn, Elementary solution of classical spin-glass models, Z. Phys. B 65, 53–63 (1986).
J.L. van Hemmen and A.C.D. van Enter, Chopper model for pattern recognition, Phys. Rev. A 34, 2509–2512 (1986).
J.L. van Hemmen, A.C.D. van Enter, and J. Canisius. On a classical spin-glass model, Z. Phys. B 50, 311–336 (1983).
H. Koch, A free energy bound for the Hopfield model, J. Phys. A 26, L353-L355 (1993).
H. Koch and J. Piasko, Some rigorous results on the Hopfield neural network model, J. Stat. Phys. 55, 903–928 (1989).
Ch. Külske, private communication.
M. Ledoux and M. Talagrand, Probability in Banach Spaces, Springer, Berlin-Heidelberg-New York, 1991.
D. Loukianova, Two rigorous bounds in the Hopfield model of associative memory, to appear in Probab. Theor. Rel. Fields (1996).
J.M. Luttinger, Exactly soluble spin-glass model, Phys. Rev. Lett. 37, 778–782 (1976).
S. Martinez, Introduction to neural networks, preprint, Temuco, (1992).
D.C. Mattis, Solvable spin system with random interactions, Phys. Lett. 56A, 421–422 (1976).
R.J. McEliece, E.C. Posner, E.R. Rodemich, and S.S. Venkatesh, The capacity of the Hopfield associative memory, IEEE Trans. Inform. Theory 33, 461–482 (1987).
Y. Miyashita, Neuronal correlate of visual associative long term memory in the primate temporal cortex, Nature 335, 817–819 (1988).
E. Marinari, G. Parisi, and F. Ritort, On the 3D Ising spin glass, J. Phys. A 27, 2687–2708 (1994).
M. Mézard, G. Parisi, and M.A. Virasoro, Spin-Glass Theory and Beyond, World Scientific, Singapore, 1988.
B. Müller and J. Reinhardt, Neural Networks: An Introduction, Springer-Verlag, Berlin, 1990.
V.A. Malyshev and F.M. Spieksma, Dynamics of binary neural networks with a finite number of patterns. Part 1: General picture of the asynchronous zero temperature dynamics, MPEJ 3, 1–36 (1997).
Ch.M. Newman, Memory capacity in neural network models: Rigorous results, Neural Networks 1, 223–238 (1988).
Ch.M. Newman and D.L. Stein, Non-mean-field behavior in realistic spin glasses, Phys. Rev. Lett. 76, 515–518 (1996);
Ch.M. Newman and D.L. Stein, Spatial inhomogeneity and thermodynamic chaos, Phys. Rev. Lett. 76, 4821–4824 (1996);
Ch.M. Newman and D.L. Stein, Topies in Disordered Systems, to appear in Birkhäuser, Boston, 1997; Thermodynamic Chaos and the Structure of Short Range Spin Glasses, this volume.
D. Petritis, Thermodynamic formalism of neural computing, preprint, Université de Rennes, 1995.
L. Pastur and M. Shcherbina, Absence of self-averaging of the order parameter in the Sherrington-Kirkpatrick model, J. Stat. Phys. 62, 1–19 (1991).
L. Pastur, M. Shcherbina, and B. Tirozzi, The replica symmetric solution without the replica trick for the Hopfield model, J. Stat. Phys. 74, 1161–1183 (1994).
P. Peretto and J.J. Niez, Long term memory storage capacity of multiconnected neural networks, Biolog. Cybernetics 39, 53–63 (1986).
R. T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, 1970.
A.W. Roberts and D.E. Varberg, Convex Functions, Academic Press, New York and London, 1973.
D. Sherrington and S. Kirkpatrick, Solvable model of a spin glass, Phys. Rev. Lett. 35, 1792–1796 (1972).
J. Silverstein, Eigenvalues and eigenvectors of large sample covariance matrices, Contemporary Math. 50, 153–159 (1986).
A.-S. Snitzman, Equations de type Boltzmann spatialement homogènes, Z. Wahrscheinlichkeitstheorie verw. Gebiete 66, 559–592 (1986).
M. Shcherbina and B. Tirozzi, The free energy for a class of Hopfield models, J. Stat Phys. 72, 113–125 (1992).
M. Schlüter and E. Wagner, Phys. Rev. E49, 1690 (1994).
V.V. Yurinskii, Exponential inequalities for sums of random vectors, J. Multivariate Anal. 6, 473–499 (1976).
M. Talagrand, Concentration of measure and isoperimetric inequalities in product space, Publ. Math. I.H.E.S., 81, 73–205 (1995).
M. Talagrand, A new look at independence, Ann. Probab. 24, 1–34 (1996).
M. Talagrand, Résultats rigoureux pour le modèle de Hopfield, C. R. Acad. Sci. Paris, t. 321, Série I, 109–112 (1995).
M. Talagrand, Rigorous results for the Hopfield model with many patterns, preprint 1996, to appear in Probab. Theor. Rel. Fields.
M. Talagrand, The Sherrington-Kirkpatrick model: A challenge for mathematicians, preprint (1996), to appear in Probab. Theor. Rel. Fields.
D.J. Thouless, P.W. Anderson, and R.G. Palmer, Phil Mag. 35, 593 (1977).
Y.Q. Yin, Z.D. Bai, and P.R. Krishnaiah, On the limit of the largest eigenvalue of the large dimensional sample covariance matrix, Probab. Theor. Rel. Fields 78, 509–521 (1988).
V.V. Yurinskii, Exponential inequalities for sums of random vectors, J. Multivariate Anal. 6, 473–499 (1976)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1998 Birkhäuser Boston
About this chapter
Cite this chapter
Bovier, A., Gayrard, V. (1998). Hopfield Models as Generalized Random Mean Field Models. In: Bovier, A., Picco, P. (eds) Mathematical Aspects of Spin Glasses and Neural Networks. Progress in Probability, vol 41. Birkhäuser Boston. https://doi.org/10.1007/978-1-4612-4102-7_1
Download citation
DOI: https://doi.org/10.1007/978-1-4612-4102-7_1
Publisher Name: Birkhäuser Boston
Print ISBN: 978-1-4612-8653-0
Online ISBN: 978-1-4612-4102-7
eBook Packages: Springer Book Archive