Collective Phenomena in Neural Networks

  • J. Leo van Hemmen
  • Reimer Kühn
Part of the Physics of Neural Networks book series (NEURAL NETWORKS)

Synopsis and Note

In this paper we review some central notions of the theory of neural networks. In so doing we concentrate on collective aspects of the dynamics of large networks. The neurons are usually taken to be formal but this is not a necessary requirement for the central notions to be applicable. Formal neurons just make the theory simpler.

There are at least two ways of reading this review. It may be read as a self-contained introduction to the theory of neural networks. Alternatively, one may regard it as a vade mecum that goes with the other articles in the present book and may be consulted if one needs further explanation or meets an unknown idea. In order to allow the second approach as well we have tried to keep the level of redundancy much higher than is strictly necessary. So the attentive reader should not be annoyed if (s)he notices that some arguments are repeated.

Equations are labeled by (x.y.z). Referring to an equation within a subsection we only mention (z), within a section (y.z), and elsewhere in the paper (x.y.z). The chapter number is ignored.

The article also contains some new and previously unpublished results.

Keywords

Entropy Covariance Nism Suffix Verse 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.1
    N. Wiener, Cybernetics (Wiley, New York, and Hermann, Paris, 1948)Google Scholar
  2. 1.2
    J. J. Hopfield, Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982)MathSciNetADSCrossRefGoogle Scholar
  3. 1.3
    T. E. Posch, USCEE report 290 (1968)Google Scholar
  4. 1.4
    M. Minsky and S. Papert, Perceptrons: An Introduction to Computational Geometry (MIT Press, Cambridge, Mass., 1969) An expanded 2nd edition appeared in 1988. This book is a gold mine of insight.MATHGoogle Scholar
  5. 1.5
    J. J. Hopfield, Proc. Natl. Acad. Sci. USA 81, 3088–3092 (1984)ADSCrossRefGoogle Scholar
  6. 1.6
    W. S. McCulloch and W. Pitts, Bull. Math. Biophys. 5, 115–133 (1943)MathSciNetMATHCrossRefGoogle Scholar
  7. 1.7
    K. Huang, Statistical Mechanics (Wiley, New York, 1963); a 2nd edition appeared in 1987Google Scholar
  8. 1.8
    P. Peretto, Biol. Cybern. 50, 51–62 (1984)MATHCrossRefGoogle Scholar
  9. 1.9
    K. Binder, in Monte Carlo Methods in Statistical Physics, edited by K. Binder (Springer, Berlin, Heidelberg, 1979) pp. 1–45Google Scholar
  10. 1.10
    R. J. Glauber, J. Math. Phys. 4, 294–307 (1963)MathSciNetADSMATHCrossRefGoogle Scholar
  11. 1.11
    N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, J. Chem. Phys. 21, 1087–1092 (1953)ADSCrossRefGoogle Scholar
  12. 1.12
    H. E. Stanley, Introduction to Phase Transitions and Critical Phenomena (Oxford University Press, New York, 1971)Google Scholar
  13. 1.13
    D. J. Amit, H. Gutfreund, and H. Sompolinsky, (a) Phys. Rev. A 32, 1007–1018 (1985); (b) Phys. Rev. Lett. 55, 1530–1533 (1985); (c) Ann. Phys. (N.Y.) 173, 30–67 (1987)MathSciNetADSCrossRefGoogle Scholar
  14. 1.14
    J. J. Hopfield and D. W. Tank, Science 233, 625–633 (1986)ADSCrossRefGoogle Scholar
  15. 1.15
    D. O. Hebb, The Organization of Behavior (Wiley, New York, 1949) p. 62Google Scholar
  16. 1.16
    J. L. van Hemmen and R. Kühn, Phys. Rev. Lett. 57, 913–916 (1986)ADSCrossRefGoogle Scholar
  17. 1.17
    J. L. van Hemmen, D. Grensing, A. Huber, and R. Kühn, J. Stat. Phys. 50, 231–257 and 259–293 (1988)ADSMATHCrossRefGoogle Scholar
  18. 1.18
    S. R. S. Varadhan, Large Deviations and Applications (Society for Industrial and Applied Mathematics, Philadelphia, PA, 1984). This work has become a classic.CrossRefGoogle Scholar
  19. 1.19
    O. E. Lanford, “Entropy and equilibrium states in classical statistical mechanics”, in Statistical Mechanics and Mathematical Problems, edited by A. Lenard, Lecture Notes in Physics, Vol. 20 (Springer, New York, Berlin, Heidelberg, 1973) pp. 1–113. This elegant paper was seminal to, e.g., Refs. [1.20] and [1.21].CrossRefGoogle Scholar
  20. 1.20
    R. S. Ellis, Entropy, Large Deviations, and Statistical Mechanics (Springer, New York, Berlin, Heidelberg, 1985); Ann. Prob. 12, 1–12 (1984)MATHCrossRefGoogle Scholar
  21. 1.21
    J. L. van Hemmen, “Equilibrium theory of spin glasses: Mean field theory and beyond”, in Heidelberg Colloquium on Spin Glasses, edited by J. L. van Hemmen and I. Morgenstern, Lecture Notes in Physics, Vol. 192 (Springer, New York, Berlin, Heidelberg 1983), in particular, the Appendix; “The theory of large deviation and its applications in statistical mechanics”, in Mark Kac Seminar on Probability and Physi3cs, Syllabus 1985–1987, edited by F. den Hollander and R. Maassen, CWI Syllabus Series No. 17 (CWI, Amsterdam, 1988) pp. 41–47CrossRefGoogle Scholar
  22. 1.22
    J. Lamperti, Probability (Benjamin, New York, 1966)MATHGoogle Scholar
  23. 1.23
    A. C. D. van Enter and J. L. van Hemmen, Phys. Rev. A 29, 355–365 (1984)MathSciNetADSCrossRefGoogle Scholar
  24. 1.24
    See, for instance, Ref. [1.12, Sect. 1.6.5]Google Scholar
  25. 1.25
    N. G. de Bruyn, Asymptotic Methods in Analysis, 2nd Edition (North-Holland, Amsterdam, 1961) Sect. 1.4.2; a Dover edition has been published recently.Google Scholar
  26. 1.26
    J. L. van Hemmen, Phys. Rev. Lett. 49, 409–412 (1982);MathSciNetADSCrossRefGoogle Scholar
  27. 1.26
    J. L. van Hemmen, A. C. D. van Enter, and J. Canisius, Z. Phys. B 50, 311–336 (1983)MathSciNetADSCrossRefGoogle Scholar
  28. 1.27
    A. W. Roberts and D. E. Varberg, Convex Functions (Academic, New York, 1973)MATHGoogle Scholar
  29. 1.28
    J. L. van Hemmen, Phys. Rev. A 34, 3435–3445 (1986)MathSciNetADSCrossRefGoogle Scholar
  30. 1.29
    J. L. van Hemmen, D. Grensing, A. Huber, and R. Kühn, Z. Phys. B 65, 53–63 (1986)MathSciNetADSCrossRefGoogle Scholar
  31. 1.30
    J. L. van Hemmen and R. G. Palmer, J. Phys. A: Math. Gen. 12, 3881–3890 (1986)Google Scholar
  32. 1.31
    D. Grensing and R. Kühn, J. Phys. A: Math. Gen. 19, L1153-L1157 (1986)ADSCrossRefGoogle Scholar
  33. 1.32
    J. L. van Hemmen and R. G. Palmer, J. Phys. A: Math. Gen. 12, 563–580 (1979)ADSCrossRefGoogle Scholar
  34. 1.33
    In fact, as is also discussed at length in Ref. [1.32], in practical work the extension is not unique.Google Scholar
  35. 1.34
    C. M. Newman, Neural Networks 1, 223–238 (1988)CrossRefGoogle Scholar
  36. 1.35
    J. L. van Hemmen and V. A. Zagrebnov, J. Phys. A: Math. Gen. 20, 3989–3999 (1987)ADSCrossRefGoogle Scholar
  37. 1.36
    J. L. van Hemmen would like to thank M. Bouten (LUC, Diepenbeek) for his insistence on physical transparenceGoogle Scholar
  38. 1.37
    L. Breiman, Probability (Addison-Wesley, Reading, Mass., 1968) Sects. 11.3 and 11.4, including problem 11.6MATHGoogle Scholar
  39. 1.38
    D. Grensing, R. Kühn, and J. L. van Hemmen, J. Phys. A: Math. Gen. 20, 2935–2947 (1987)ADSCrossRefGoogle Scholar
  40. 1.39
    J. L. van Hemmen, Phys. Rev. A 36, 1959–1962 (1987)ADSCrossRefGoogle Scholar
  41. 1.40
    J. Marcinkiewicz, Sur une propriété de la loi de Gauss, Math. Z. 44, 612–618 (1939). The theorem has been rediscovered several times. For a textbook presentation, see: H. Richter, Wahrscheinlichkeitstheorie, 2nd Edition (Springer, Berlin, Heidelberg, 1966) pp. 213–214MathSciNetCrossRefGoogle Scholar
  42. 1.41
    Heidelberg Colloquium on Spin Glasses, edited by J. L. van Hemmen and I. Morgenstern, Lecture Notes in Physics, Vol. 192 (Springer, Berlin, Heidelberg, 1983)Google Scholar
  43. 1.42
    D. Sherrington and S. Kirkpatrick, Phys. Rev. Lett. 35, 1792–1796 (1975). The SK model is expected to describe a spin glass in sufficiently high dimensions (d > 8).ADSCrossRefGoogle Scholar
  44. 1.42
    See also: M. Mézard, G. Parisi, and M. A. Virasoro, Spin Glass Theory and Beyond (World Scientific, Singapore, 1987). It is fair to say that this book is devoted almost exclusively to the SK modelMATHGoogle Scholar
  45. 1.43
    W. A. Little, Math. Biosci. 19, 101–120 (1974);MATHCrossRefGoogle Scholar
  46. 1.43
    W. A. Little and G. L. Shaw, Math. Biosci. 39,281–290 (1978)MathSciNetMATHCrossRefGoogle Scholar
  47. 1.44
    A. Crisanti, D. J. Amit, and H. Gutfreund, Europhys. Lett. 2, 337–341 (1986)ADSCrossRefGoogle Scholar
  48. 1.45
    J. A. Hertz, G. Grinstein, and S. A. Solla, in Heidelberg Colloquium on Glassy Dynamics, edited by J. L. van Hemmen and I. Morgenstern, Lecture Notes in Physics, Vol. 275 (Springer, New York, Berlin, Heidelberg, 1987) pp. 538–546CrossRefGoogle Scholar
  49. 1.46
    D. J. Amit, in: Ref. [1.45: pp. 466–471]; A. Treves and D. J. Amit, J. Phys. A: Math. Gen. 21, 3155–3169 (1988)MathSciNetADSMATHCrossRefGoogle Scholar
  50. 1.47
    A. Crisanti and H. Sompolinsky, Phys. Rev. A 36, 4922–4939 (1987) and 37, 4865–4874 (1988)MathSciNetADSCrossRefGoogle Scholar
  51. 1.48
    M. V. Feigel’man and L. B. Ioffe, Intern. J. Mod. Phys. B 1, 51–68 (1987)ADSCrossRefGoogle Scholar
  52. 1.49
    A simple and elegant proof can be found in: J. Lamperti, Stochastic Processes (Springer, New York, 1977) pp. 107–112. See also R. Kindermann and J. L. Snell, Markov Random Fields and their Applications, Contemporary Mathematics Vol. 1 (American Mathematical Society, Providence, Rhode Island, 1980) pp. 52–61MATHGoogle Scholar
  53. 1.50
    J. F. Fontanari and R. Köberle, Phys. Rev. A 36, 2475–2477 (1987)MathSciNetADSCrossRefGoogle Scholar
  54. 1.51
    S. Grossberg, Neural Networks 1, 17–61 (1988), in particular, Sect. 1.9CrossRefGoogle Scholar
  55. 1.52
    R. Kühn and J. L. van Hemmen, Graded-Response Neurons (Heidelberg, 1987, unpublished)Google Scholar
  56. 1.52
    R. Kühn, S. Bös, and J.L. van Hemmen: Phys. Rev. A 43, RC (1991)Google Scholar
  57. 1.53
    Equations (2.2.8) and (2.2.9) were discovered independently by J. Jędrzejewski and A. Komoda, Z. Phys. B 63, 247–257 (1986)ADSCrossRefGoogle Scholar
  58. 1.54
    J.-P. Nadal, G. Toulouse, J.-P. Changeux, and S. Dehaene, Europhys. Lett. 1 (1986) 535–542 and 2, 343 (E) (1986)ADSCrossRefGoogle Scholar
  59. 1.55
    M. Mézard, J.-P. Nadal, and G. Toulouse, J. Phys. (Paris) 47, 1457–1462 (1986)CrossRefGoogle Scholar
  60. 1.56
    J. J. Hopfield, in Modelling in Analysis and Biomedicine, edited by C. Nicolini (World Scientific, Singapore, 1984) pp. 369–389, especially p. 381Google Scholar
  61. 1.57
    G. Parisi, J. Phys. A: Math. Gen. 19, L617-L620 (1986)ADSCrossRefGoogle Scholar
  62. 1.58
    J. L. van Hemmen, G. Keller, and R. Kühn, Europhys. Lett. 5, 663–668 (1988)ADSCrossRefGoogle Scholar
  63. 1.59
    G. Toulouse, S. Dehaene, and J.-P. Changeux, Proc. Natl. Acad. Sci. USA 83, 1695–1698 (1986)MathSciNetADSCrossRefGoogle Scholar
  64. 1.60
    See Table I on p. 271 of Ref. [1.17]Google Scholar
  65. 1.61
    H. Sompolinsky, Phys. Rev. 34, 2571–2574 (1986)ADSCrossRefGoogle Scholar
  66. 1.62
    H. Sompolinsky, in Heidelberg Colloquium on Glassy Dynamics, edited by J. L. van Hemmen and I. Morgenstern, Lecture Notes in Physics, Vol. 275 (Springer, New York, Berlin, Heidelberg, 1987) pp. 485–527CrossRefGoogle Scholar
  67. 1.63
    There is the physiological rule “low-efficacy synapses degenerate.” See: J.-P. Changeux, T. Heidmann, and P. Patte, in The Biology of Learning, edited by P. Marler and H. Terrace (Springer, New York, Berlin, Heidelberg 1984) pp. 115–133CrossRefGoogle Scholar
  68. 1.64
    J. L. van Hemmen and K. Rzązewski, J. Phys. A: Math. Gen. 20, 6553–6560 (1987)ADSMATHCrossRefGoogle Scholar
  69. 1.65
    G. Toulouse, in: Ref. [1.62: pp. 569–576]. Toulouse considers a slightly different model (learning within bounds) with αc ≈ 0.015. Estimating the connectivity Z of neurons involved in short-term memory to be of the order Z ≈ 500, he finds that at most αc Z ≈ 7 items can be stored. It is known from experimental psychology that the short-term memory capacity of humans is 7 ± 2 items (a rather famous number). If more items have to be stored, none of them can be retrieved, i.e., they are all forgotten. If the Hopfield model is overloaded, no retrieval is possible eitherGoogle Scholar
  70. 1.66
    R. Penrose, Proc. Cambridge Philos. Soc. 51, 406–413 (1955) and 52, 17–19 (1956); these papers are strongly recommended reading. The mathematics and numerics of the pseudoinverse is discussed at length in: T. N. E. Greville, SIAM Review 2, 15–43 (1960), and A. Albert, Regression and the Moore-Penrose Pseudoinverse (Academic, New York, 1972)MathSciNetADSMATHCrossRefGoogle Scholar
  71. 1.67
    T. Kohonen, IEEE Trans. Comput. C-23, 444–445 (1974); see also Kohonen’s book, Associative Memory (Springer, New York, Berlin, Heidelberg, 1977)CrossRefGoogle Scholar
  72. 1.68
    L. Personnaz, I. Guyon, and G. Dreyfus, J. Phys. (Paris) Lett. 46, L359-L365 (1985). These authors rediscovered the pseudoinverse in the nonlinear context (5.2.1), which they reduced to (5.2.2). A slightly more general, also linear, problem had been solved previously by Kohonen, see Ref. [1.67].Google Scholar
  73. 1.69
    F. R. Gantmacher, The Theory of Matrices, Vol. I (Chelsea, New York, 1977) Sects. IX. 3 and 4.Google Scholar
  74. 1.70
    S. Diederich and M Opper, Phys. Rev. Lett. 58, 949–952 (1987)MathSciNetADSCrossRefGoogle Scholar
  75. 1.71
    I. Kanter and H. Sompolinsky, Phys. Rev. A 35, 380–392 (1987)ADSCrossRefGoogle Scholar
  76. 1.72
    A. M. Odlyzko, J. Combin. Theory Ser. A 47, 124–133 (1988)MathSciNetMATHCrossRefGoogle Scholar
  77. 1.73
    L. Personnaz, I. Guyon, and G. Dreyfus, Phys. Rev. A 34, 4217–4228 (1986). The authors use a parallel dynamics and show, for instance, that cycles cannot occur.MathSciNetADSCrossRefGoogle Scholar
  78. 1.74
    F. Rosenblatt, Principles of Neurodynamics (Spartan Books, Washington, DC, 1961)Google Scholar
  79. 1.75
    Ref. [1.4: Chap. 11] gives a lucid discussionGoogle Scholar
  80. 1.76
    E. Gardner, J. Phys. A: Math. Gen. 21, 257–270 (1988)ADSCrossRefGoogle Scholar
  81. 1.77
    See also Chap. 3 by Forrest and Wallace. It contains a nice appendix, which supplements well the arguments presented here; in particular, the case κ > 0 in (5.3.9)Google Scholar
  82. 1.78
    T. M. Cover, IEEE Trans Electron. Comput. EC-14, 326–334 (1965);CrossRefGoogle Scholar
  83. 1.78
    P. Baldi and S. Venkatesh, Phys. Rev. Lett. 58, 913–916 (1987)MathSciNetADSCrossRefGoogle Scholar
  84. 1.79
    W. Krauth and M. Mézard, J. Phys. A: Math. Gen. 20, L745-L752 (1987)ADSCrossRefGoogle Scholar
  85. 1.80
    C. F. Stevens, Nature 338, 460–461 (1989) and references quoted therein; Nature 347, 16 (1990)ADSCrossRefGoogle Scholar
  86. 1.81
    The signal may also be “smeared out” by the capacitance of the dendritic tree. This gives rise to an exponential delay with an RC time τ′. Since the time window associated with τ′ is rather narrow (a few milliseconds), certainly when compared with the axonal delay τ, it will be neglected here. See, however, A. Herz, B. Sulzer, R. Kühn, and J. L. van Hemmen, in Neural Networks: From Models to Applications, edited by L. Personnaz and G. Dreyfus (I.D.S.E.T., Paris, 1989) pp. 307–315Google Scholar
  87. 1.82
    V. Braitenberg, in Brain Theory, edited by G. Palm and A. Aertsen (Springer, New York, Berlin, Heidelberg, 1986) pp. 81–96CrossRefGoogle Scholar
  88. 1.83
    R. Miller, Psychobiology 15, 241–247 (1987)ADSGoogle Scholar
  89. 1.84
    K. H. Lee, K. Chung, J. M. Chung, and R. E. Coggeshall, Comp. Neurol. 243, 335–346 (1986)CrossRefGoogle Scholar
  90. 1.85
    A.V.M. Herz, B. Sulzer, R. Kühn, and J. L. van Hemmen, Europhys. Lett. 7, 663–669 (1988)ADSCrossRefGoogle Scholar
  91. 1.86
    A.V.M. Herz, B. Sulzer, R. Kühn, and J. L. van Hemmen, Biol. Cybern. 60, 457–467 (1989)MATHCrossRefGoogle Scholar
  92. 1.87
    S. R. Kelso, A. H. Ganong, and T. H. Brown, Proc. Natl. Acad. Sci. USA 83, 5326–5330 (1986)ADSCrossRefGoogle Scholar
  93. 1.88
    R. Malinow and J. P. Miller, Nature 320, 529–530 (1986)ADSCrossRefGoogle Scholar
  94. 1.89
    The fact that Prob {ξiμiμ = ±1} = p = 0.5 also allows numerical simulations at a reasonable system size N; cf. Refs. [1.85] and [1.86]. For small p, numerical simulation is out of the question since either AT is so small that the statistics is no good or N is so large that even most supercomputers have memory problemsGoogle Scholar
  95. 1.90
    L. N. Cooper, in Nobel Symposia, Vol. 24, edited by B. and S. Lundqvist (Academic, New York, 1973) pp. 252–264Google Scholar
  96. 1.91
    D. Kleinfeld, Proc. Natl. Acad. Sci. USA 83, 9469–9473 (1986)MathSciNetADSCrossRefGoogle Scholar
  97. 1.92
    H. Sompolinsky and I. Kanter, Phys. Rev. Lett. 57, 2861–2864 (1986)ADSCrossRefGoogle Scholar
  98. 1.93
    R. Kühn and J. L. van Hemmen, this volume, Chap. 7Google Scholar
  99. 1.94
    U. Riedel, R. Kühn, and J. L. van Hemmen, Phys. Rev. A 38, 1105–1108 (1988);MathSciNetADSCrossRefGoogle Scholar
  100. 1.94
    U. Riedel, diploma thesis (Heidelberg, February 1988)Google Scholar
  101. 1.95
    J. Hale, Theory of Functional Differential Equations (Springer, New York, Berlin, Heidelberg, 1977)MATHCrossRefGoogle Scholar
  102. 1.96
    R. Bellman and K. L. Cooke, Differential Difference Equations (Academic, New York, 1963)MATHGoogle Scholar
  103. 1.97
    N. D. Hayes, J. London Math. Soc. 25, 226–232 (1950)MathSciNetMATHCrossRefGoogle Scholar
  104. 1.98
    L. S. Pontryagin, Amer. Math. Soc. Transl. series 2, 1, 95–110 (1955)Google Scholar
  105. 1.99
    P. Peretto, Neural Networks 1, 309–321 (1988)CrossRefGoogle Scholar
  106. 1.100
    L. F. Abbott and T. B. Kepler, J. Phys. A: Math. Gen. 22, L711-L717 (1989)MathSciNetADSCrossRefGoogle Scholar
  107. 1.101
    B. Derrida, E. Gardner, and A. Zippelius, Europhys. Lett. 4, 167–173 (1987)ADSCrossRefGoogle Scholar
  108. 1.102
    R. Kree and A. Zippelius, this volume, Chap. 6Google Scholar
  109. 1.103
    The equations for the Hopfield case with finitely many patterns, i.e., (13) with ε = 0, have been rediscovered by A. C. C. Coolen and Th. W. Ruijgrok, Phys. Rev. 38 (1988) 4253–4255 and M. Shiino, H. Nishimori, and M. Ono, J. Phys. Soc. Jpn. 58 (1989) 763–766. Here too the notion of sublattice is instrumental.Google Scholar
  110. 1.104
    S. Amari, Neural Networks 1, 63–73 (1988)CrossRefGoogle Scholar
  111. 1.105
    An illustration that should not be taken too seriously can be found on p. 561 in: Ref. [1.62]Google Scholar
  112. 1.106
    B. Forrest, J. Phys. A: Math. Gen. 21, 245–255 (1988)MathSciNetADSCrossRefGoogle Scholar
  113. 1.107
    H. Homer, D. Bormaim, M. Frick, H. Kinzelbach, and A. Schmidt, Z. Phys. B 76, 381–398 (1989)ADSCrossRefGoogle Scholar
  114. 1.108
    W. Gerstner, J. L. van Hemmen, and A.V.M. Herz, manuscript in preparation; J.L. van Hemmen, W. Gerstner, A.V.M. Herz, R. Kühn, and M. Vaas, in Konnektionismus in Artificial Intelligence und Kognitionsforschung, edited by G. Dorffner (Springer, Berlin, Heidelberg, 1990) pp. 153–162CrossRefGoogle Scholar
  115. 1.109
    S. Bös, R. Kühn, and J. L. van Hemmen, Z. Phys. B 71, 261–271 (1988); S. Bös, diploma thesis (Heidelberg, August 1988)MathSciNetADSCrossRefGoogle Scholar
  116. 1.110
    M. V. Feigerman and L. B. Ioffe, Int. J. Mod. Phys. B 1, 51–68 (1987)ADSCrossRefGoogle Scholar
  117. 1.111
    D. J. Amit, H. Gutfreund, and H. Sompolinsky, Phys. Rev. A 35, 2293–2303 (1987)MathSciNetADSCrossRefGoogle Scholar
  118. 1.112
    F. Crick and G. Mitchison, Nature 304, 111–114 (1983)ADSCrossRefGoogle Scholar
  119. 1.113
    E. R. Kandel and J. H. Schwartz, Principles of Neural Science, 2nd Edition (Elsevier, New York, 1985) Chap. 49Google Scholar
  120. 1.114
    J. J. Hopfield, D. I. Feinstein, and R. G. Palmer, Nature 304, 158–159 (1983)ADSCrossRefGoogle Scholar
  121. 1.115
    J. L. van Hemmen, L. B. Ioffe, R. Kühn, and M. Vaas, Physica A 163 (1990); M. Vaas, diploma thesis (Heidelberg, October 1989); J.L. van Hemmen, in Neural Networks and Spin Glasses, edited by W.K. Theumann and R. Köberle (World Scientific, Singapore 1990), pp. 91–114Google Scholar
  122. 1.116
    A.V.M. Herz, in Connectionism in Perspective, edited by R. Pfeifer, Z, Schreter, F. Fogelman- Soulié, and L. Steels (North-Holland, Amsterdam, 1989), Ph.D. thesis (Heidelberg, September 1990), and work in preparationGoogle Scholar
  123. 1.117
    W. Feller, An Introduction to Probability Theory and Its Applications, Vol. I, 3rd Edition (Wiley, New York, 1970) Sect. 1.XVI.3Google Scholar
  124. 1.118
    B. Derrida and J.-P. Nadal, J. Stat. Phys. 49, 993–1009 (1987)MathSciNetADSCrossRefGoogle Scholar
  125. 1.119
    N. Burgess, M. A. Moore, and J. L. Shapiro, in Neural Networks and Spin Glasses, edited by W. K. Theumann and R. Köberle (World Scientific, Singapore, 1990) pp. 291–307Google Scholar
  126. 1.120
    J. C. Eccles, The Understanding of the Brain, 2nd Edition (McGraw-Hill, New York, 1977)Google Scholar
  127. 1.121
    C. Meunier, D. Hansel, and A. Verga, J. Stat. Phys. 55, 859–901 (1989)MathSciNetADSMATHCrossRefGoogle Scholar
  128. 1.122
    V. S. Dotsenko, J. Phys. C 18, L1017-L1022; Physica A 140, 410–415 (1986)ADSCrossRefGoogle Scholar
  129. 1.123
    H. Gutfreund, Phys. Rev. A 37, 570–577 (1988)MathSciNetADSCrossRefGoogle Scholar
  130. 1.124
    N. Parga and M. A. Virasaro, J. Phys. (Paris) 47, 1857–1864 (1986)CrossRefGoogle Scholar
  131. 1.125
    J. Lamperti, Stochastic Processes (Springer, New York, Berlin, Heidelberg 1977); for the mathematically minded there is a neat summary of conditioning in Appendix 2.MATHGoogle Scholar
  132. 1.126
    M. V. Feigel’man and L. B. Ioffe, this volume, Chap. 5Google Scholar
  133. 1.127
    J. Doob, Am. Math. Month. 78, 451–463 (1971)MathSciNetMATHCrossRefGoogle Scholar
  134. 1.128
    K. L. Chung, A Course in Probability Theory, 2nd Edition (Academic, New York, 1974) Chap. 9MATHGoogle Scholar
  135. 1.129
    C. Cortes, A. Krogh, and J. A. Hertz, J. Phys. A: Math. Gen. 20, 4449–4455 (1987)MathSciNetADSCrossRefGoogle Scholar
  136. 1.130
    N. Parga, private communicationGoogle Scholar
  137. 1.131
    A. Krogh and J. A. Hertz, J. Phys. A: Math. Gen. 21, 2211–2224 (1988)MathSciNetADSMATHCrossRefGoogle Scholar
  138. 1.132
    R. J. McEliece, E. C. Posner, E. R. Rodemich, and S.S. Venkatesh, IEEE Trans. Inf. Theory IT-33, 461–492 (1987)MathSciNetCrossRefGoogle Scholar
  139. 1.133
    A. R. Damasio, H. Damasio, and G. W. van Hessen, Neurology (NY) 32, 331–341 (1982)Google Scholar
  140. 1.134
    M. A. Virasoro, Europhys. Lett. 7, 293–298 (1988)ADSCrossRefGoogle Scholar
  141. 1.135
    B. Derrida, E. Gardner, and P. Mottishaw, J. Phys. (Paris) 48, 741–755 (1987)CrossRefGoogle Scholar
  142. 1.136
    F. J. Dyson, Commun. Math. Phys. 12, 91–107 (1969) and 212–215. For a back-of-the- envelope discussion, see C. J. Thompson in Nonlinear Problems in the Physical Sciences and Biology, edited by I. Stakgold, D. D. Joseph, and D. H. Sattinger, Lecture Notes in Mathematics, Vol. 322 (Springer, Berlin, Heidelberg, 1973) pp. 308–342, in particular, pp. 329–330MathSciNetADSCrossRefGoogle Scholar
  143. 1.137
    W. Kinzel. Phys. Rev. B 33, 5086–5088 (1986)ADSCrossRefGoogle Scholar
  144. 1.138
    E. Domany and R. Meir, this volume, Chap. 9Google Scholar
  145. 1.139
    R. Kühn, J. Lindenberg, G. Sawitzki, and J. L. van Hemmen, manuscript in preparationGoogle Scholar
  146. 1.140
    D. J. Amit, K. Y. M. Wong, and C. Campbell, J. Phys. A: Math. Gen. 22, 2039–2043 (1989)MathSciNetADSCrossRefGoogle Scholar
  147. 1.141
    A. Treves and D. J. Amit, J. Phys. A: Math. Gen. 22, 2205–2226 (1989);MathSciNetADSCrossRefGoogle Scholar
  148. 1.141
    H. Sompolinsky, Physics Today 41 / 12, 70–80 (1988)MathSciNetADSCrossRefGoogle Scholar
  149. 1.142
    J. Buhmann, preprint (USC, 1989)Google Scholar
  150. 1.143
    A. Framkin and E. Moses, Phys. Rev. A 34, 714–716 (1986);ADSCrossRefGoogle Scholar
  151. 1.142
    E. Goles and G. Y. Vichniac in Neural Networks for Computing, edited by J. S. Denker, AIP Conf. Proc. 151 (American Institute of Physics, New York, 1986) pp. 165–181Google Scholar
  152. 1.144
    G.A. Kohring: J. Stat. Phys. 59, 1077–1086 (1990)ADSCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1991

Authors and Affiliations

  • J. Leo van Hemmen
  • Reimer Kühn

There are no affiliations available

Personalised recommendations