Capacity and Error Correction Ability of Sparsely Encoded Associative Memory with Forgetting Process

Abstract

Associative memory model of neural networks can not store items more than its memory capacity. When new items are given one after another, its connection weights should be decayed so that the number of stored items does not exceed the memory capacity (forgetting process). This paper analyzes the sparsely encoded associative memory, and presents the optimal decay rate that maximizes the number of stored items. The maximal number of stored items is given by O(n/a log n) when the decay rate is 1-O (a log n/n), where the network consists of n neurons with activity a.