Abstract
General stochastic binary Hopfield models are viewed from the angle of statistical mechanics. Both the general unconstrained binary stochastic Hopfield model and a certain constrained one are analyzed yielding explicit expressions of the free energy. Moreover, conditions are given for which some of these free energy expressions are Lyapunov functions of the corresponding differential equations. In mean field approximation, either stochastic model appears to coincide with a specific continuous model. Physically, the models are related to spin and to Potts glass models. By means of an alternative derivation, an expression of a ‘complementary’ free energy is presented. Some surveying computational results are reported and an alternative use of the discussed models in resolving constrained optimization problems is discussed.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
J. van den Berg and J. C. Bioch. About the Statistical Mechanics of (Un)Constrained Stochastic Hopfield and ‘Elastic’ Neural Networks. Technical Report EUR-CS-94-08, Comp. Sc. Dept. (AI Section), Faculty of Economics, Erasmus University Rotterdam, 1994.
J. van den Berg and J. C. Bioch. Constrained Optimization with the Hopfield-Lagrange Model. In Proceedings of the 14th IMACS World Congress, pages 470–473, Atlanta, GA 30332, USA, 1994.
J. van den Berg and J. C. Bioch. On the (Free) Energy of Hopfield Networks. Submitted to: Workshop on the Theory of Neural Networks: The Statistical Mechanics Perspective, Pohang, Korea, 1995.
D. E. Van den Bout and T. K. Miller. Improving the Performance of the Hopfield-Tank Neural Network Through Normalization and Annealing. Biological Cybernetics, 62:129–139, 1989.
J. Hertz, A. Krogh, and R. G. Palmer. Introduction to the Theory of Neural Computation. Addison-Wesley Publishing Company, The Advanced Book Program, 1991.
J. J. Hopfield. Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proceedings of the National Academy of Sciences, USA, 79:2554–2558, 1982.
J. J. Hopfield. Neurons with Graded Responses Have Collective Computational Properties Like Those of Two-State Neurons. Proceedings of the National Academy of Sciences, USA, 81:3088–3092, 1984.
J. J. Hopfield and D. W. Tank. “Neural” Computation of Decisions in Optimization Problems. Biological Cybernetics, 52:141–152, 1986.
C. Peterson and B. Söderberg. A New Method for Mapping Optimization Problems onto Neural Networks. International Journal of Neural Systems, 1:3–22, 1989.
D. Sherrington. Neural Networks: The Spin Glass Approach. In J.G. Taylor, editor, Mathematical Approaches to Neural Networks, pages 261–292. North-Holland, 1993.
P. D. Simic. Statistical Mechanics as the Underlying Theory of ‘Elastic’ and ‘Neural’ Optimisations. Network, 1:88–103, 1990.
G. V. Wilson and G. S. Pawley. On the Stabilitity of the Travelling Salesman Problem Algorithm of Hopfield and Tank. Biological Cybernetics, 58:63–70, 1988.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1995 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
van den Berg, J., Bioch, J.C. (1995). Some theorems concerning the free energy of (Un) constrained stochastic Hopfield neural networks. In: Vitányi, P. (eds) Computational Learning Theory. EuroCOLT 1995. Lecture Notes in Computer Science, vol 904. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-59119-2_186
Download citation
DOI: https://doi.org/10.1007/3-540-59119-2_186
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-59119-1
Online ISBN: 978-3-540-49195-8
eBook Packages: Springer Book Archive