Coupled Neural Networks

Abstract

Multilayered feed-forward networks (perceptrons) are special cases of the general McCulloch-Pitts neural network with arbitrarily interconnected neurons. On the other hand, any general “recurrent” neural network can be considered to be represented by a feed-forward perceptron, albeit one with possibly very many layers. The reason for this strange equivalence is that the temporal evolution (3.5) of an arbitrary network constructed from binary neurons

$$ {s_i}\left( {t\,\, + \,\,1} \right)\,\, = \,\,{\mathop{\rm sgn}} \left[ {{h_i}\left( t \right)} \right]\,\, \equiv \,\,{\mathop{\rm sgn}} \left[ {\sum\limits_{j\, = \,1}^N {{w_{ij}}{s_j}\left( t \right)} } \right] $$

(13.1)

is necessarily periodic. This statement follows immediately from the observation that the N neurons can only assume 2^N configurations altogether, and hence some state of the network must reoccur after at most 2^N steps. Since only the present state of the network enters on the right-hand side of the evolution law (13.1), the sub se quent evolution proceeds strictly periodically from that moment on. If one considers the neural network at a certain moment t = n as the nth layer of a perceptron (with all layers identical!), the temporal-evolution law can be viewed as the law governing the flow of information from one layer to the next. It is then sufficient to take into account only a finite number of such layers, just as many as there are time steps leading up to the first repetition of a network configuration.