Capacity and Error Correction Ability of Sparsely Encoded Associative Memory with Forgetting Process
Associative memory model of neural networks can not store items more than its memory capacity. When new items are given one after another, its connection weights should be decayed so that the number of stored items does not exceed the memory capacity (forgetting process). This paper analyzes the sparsely encoded associative memory, and presents the optimal decay rate that maximizes the number of stored items. The maximal number of stored items is given by O(n/a log n) when the decay rate is 1-O (a log n/n), where the network consists of n neurons with activity a.
KeywordsDecay Rate Memory Capacity Associative Memory Connection Weight Generalization Ability
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
S. Akaho: Optimal decay rate of connection weights in covariance learning. Technical Report 92-37, Electrotechnical Laboratory, 1992.Google Scholar
S. Amari: Characteristics of sparsely encoded associative memory. Neural Networks
, Vol. 2, No. 6, pp. 451–457, 1989.CrossRefGoogle Scholar
J.A. Anderson: A simple neural network generating interactive memory. Mathematical Bio-sciences
, Vol. 14, pp. 197–220, 1972.CrossRefMATHGoogle Scholar
M. Mézard, J.P. Nadal, and G. Toulouse: Solvable models of working memories. J. Physique
, Vol. 47, pp. 1457–1462, 1986.MathSciNetCrossRefGoogle Scholar
V.A. Vapnik: Estimation of Dependences Based on Empirical Data
. Springer-Verlag, 1984.Google Scholar
© Springer-Verlag London Limited 1993