Advertisement

Elimination of a Catastrophic Destruction of a Memory in the Hopfield Model

  • Iakov Karandashev
  • Boris Kryzhanovsky
  • Leonid Litinskii
Part of the Communications in Computer and Information Science book series (CCIS, volume 311)

Abstract

For the standard Hopfield model a catastrophic destruction of the memory has place when the last is overfull (so called catastrophic forgetting). We eliminate the catastrophic forgetting assigning different weights to input patterns. As the weights one can use the frequencies of appearance of the patterns during the learning process. We show that only patterns whose weights are larger than some critical weight would be recognized. The case of the weights that are the terms of a geometric series is studied in details. The theoretical results are in good agreement with computer simulations.

Keywords

Hopfield model catastrophic forgetting quasi-Hebbian matrix 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Amit, D., Gutfreund, H., Sompolinsky, H.: Statistical Mechanics of Neural Networks Near Saturation. Annals of Physics 173, 30–67 (1987)CrossRefGoogle Scholar
  2. 2.
    Hertz, J., Krogh, A., Palmer, R.: Introduction to the Theory of Neural Computation. Addison-Wesley, Massachusetts (1991)Google Scholar
  3. 3.
    Parisi, G.: A memory which forgets. Journal of Physics A 19, L617–L620 (1986)Google Scholar
  4. 4.
    Nadal, J.P., Toulouse, G., Changeux, J.P., Dehaene, S.: Networks of Formal Neurons and Memory Palimpsest. Europhysics Letters 1(10), 535–542 (1986)CrossRefGoogle Scholar
  5. 5.
    van Hemmen, J.L., Keller, G., Kuhn, R.: Forgetful Memories. Europhysics Letters 5(7), 663–668 (1988)CrossRefGoogle Scholar
  6. 6.
    Sandberg, A., Ekeberg, O., Lansner, A.: An incremental Bayesian learning rule for palimpsest memory (preprint)Google Scholar
  7. 7.
    Karandashev, Y., Kryzhanovsky, B., Litinskii, L.: Local Minima of a Quadratic Binary Functional with a Quasi-Hebbian Connection Matrix. In: Diamantaras, K., Duch, W., Iliadis, L.S. (eds.) ICANN 2010, Part III. LNCS, vol. 6354, pp. 41–51. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  8. 8.
    Karandashev, I., Kryzhanovsky, B., Litinskii, L.: Weighted Patterns as a Tool for Improving the Hopfield Model. Phys. Rev. E 85, 041925 (2012)Google Scholar
  9. 9.
    van Hemmen, J.L., Zagrebnov, V.A.: Storing extensively many weighted patterns in a saturated neural network. J. of Phys. A 20, 3989–3999 (1987)CrossRefGoogle Scholar
  10. 10.
    van Hemmen, J.L., Kuhn, R.: Collective Phenomena in Neural Networks. In: Domany, E., van Hemmen, J.L., Shulten, K. (eds.) Models of Neural Networks, pp. 1–105. Springer, Berlin (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Iakov Karandashev
    • 1
  • Boris Kryzhanovsky
    • 1
  • Leonid Litinskii
    • 1
  1. 1.CONT SRISA RASMoscowRussia

Personalised recommendations