Associative Memory: Advanced Learning Strategies

Abstract

As we discussed in Sects. 3.1 and 4.3, the ability to recall memories correctly breaks down if the number p of stored patterns exceeds a certain limit. When the synaptic connections are determined according to Hebb’s rule (3.12), this happens at the storage density α = p/N ≈ 0.14. The reason for this behavior was the influence of the other stored patterns as expressed by the fluctuating noise term in (3.13). As we already pointed out at the end of Sect. 3.1, this influence vanishes exactly if the patterns are orthogonal to each other as defined in (3.22). On the other hand, the power of recollection deteriorates even earlier if the stored patterns are strongly correlated. Unfortunately, this happens in many practical examples. Just think of the graphical representation of roman letters, where “E” closely resembles “F” and “C” resembles “G”, or of a typical list of names from the telephone book, which are probably highly correlated.