ICANN ’93 pp 707-710 | Cite as

Capacity and Error Correction Ability of Sparsely Encoded Associative Memory with Forgetting Process

  • Shotaro Akaho
Conference paper


Associative memory model of neural networks can not store items more than its memory capacity. When new items are given one after another, its connection weights should be decayed so that the number of stored items does not exceed the memory capacity (forgetting process). This paper analyzes the sparsely encoded associative memory, and presents the optimal decay rate that maximizes the number of stored items. The maximal number of stored items is given by O(n/a log n) when the decay rate is 1-O (a log n/n), where the network consists of n neurons with activity a.


Decay Rate Memory Capacity Associative Memory Connection Weight Generalization Ability 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    S. Akaho: Optimal decay rate of connection weights in covariance learning. Technical Report 92-37, Electrotechnical Laboratory, 1992.Google Scholar
  2. [2]
    S. Amari: Characteristics of sparsely encoded associative memory. Neural Networks, Vol. 2, No. 6, pp. 451–457, 1989.CrossRefGoogle Scholar
  3. [3]
    J.A. Anderson: A simple neural network generating interactive memory. Mathematical Bio-sciences, Vol. 14, pp. 197–220, 1972.CrossRefMATHGoogle Scholar
  4. [4]
    M. Mézard, J.P. Nadal, and G. Toulouse: Solvable models of working memories. J. Physique, Vol. 47, pp. 1457–1462, 1986.MathSciNetCrossRefGoogle Scholar
  5. [5]
    V.A. Vapnik: Estimation of Dependences Based on Empirical Data. Springer-Verlag, 1984.Google Scholar

Copyright information

© Springer-Verlag London Limited 1993

Authors and Affiliations

  • Shotaro Akaho
    • 1
  1. 1.Electrotechnical LaboratoryMathematical Informatics SectionTsukuba-shi, Ibaraki 305Japan

Personalised recommendations