Introduction

Abstract

Generally, neural networks can be divided into two large classes. One class contains feedforward neural networks (FNNs), and the other contains recurrent neural networks (RNNs). This book focused on RNNs only. The essential difference between FNNs and RNNs is the presence of a feedback mechanism among the neurons in the latter. A FNN is a network without any feedback connections among its neurons, while a RNN has at least one feedback connection. Since RNNs allow feedback connections in neurons, the network topology can be very general: any neuron can be connected to any other, even to itself. Allowing the presence of feedback connections among neurons has an advantage, it leads naturally to an analysis of the networks as dynamic systems, in which the state of a network, at one moment in time, depends on the state at a previous moment in time. The topology of RNNs is shown in Figure 1.1.