Tailoring the performance of attractor neural networks

Abstract

First, we study the effects of introducing training noise on the retrieval behaviours of dilute attractor neural networks. We found that, in general, training noise enhances associativity, but also reduces the attractor overlap. At a narrow range of storage levels, however, the system exhibits re-entrant retrieval behaviour on increasing training noise.

Secondly, we consider optimization of network performance, and subsequently the storage capacity, in the presence of retrieval noise (temperature). This is achieved by adapting the network to an appropriate training overlap, which is determined self-consistently by the optimal attractor overlap. The maximum storage capacity deviates from the storage capacity of the maximally stable network on increasing temperature, and in the high temperature regime (T ≥ 0.38 for Gaussian noise, the Hebb-rule network yields the maximum storage capacity. Our analysis demonstrates the principles of specialization and adaptation in neural networks.