Abstract
Recurrent networks have proved to be more powerful than feedforward neural networks in terms of classes of functions they can compute. But, because training of recurrent networks is a difficult task, it is not clear that these networks provide an advantage over feedforward networks for learning from examples. This communication proposes a general computation model that lays the foundations for characterizing the classes of functions computed by feedforward nets and convergent recurrent nets. Then a mathematical statement proves that convergent nets outperform feedforward nets on data fitting problems. It provides the basis to devise a new learning procedure that constraints the attractor set of a recurrent net and assures a convergent dynamic by using orthogonal inverse tools. The learning algorithm is based on an evolutionary selection mechanism. Using the previous procedure as evaluation function, it has been shown to be robust and well adapted to train convergent recurrent nets when feedforward nets cannot approximate a real parameter mapping.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
R.P. Agaxwal. Difference Equations and Inequalities: Theory, Methods and Applications. Pure and Applied Mathematics. Springer, E.J. Taft and Z. Nashed ed. edition, 1991.
American Institute of Physics. Neural Information Processing Systems Conference, volume 1, 1987.
Y. Bengio, P. Simard, and P. Frasconi. Learning Long-Term Dependencies with gradient is difficult. IEEE Trans, on Neural Networks, 1993. Special issue on Recurrent Networks.
O. Christensen. Frames and pseudo-inverses. J. Math. Anal. Appl., 2, 1995.
C. Gégout. Improvement of Multilayer Perceptron Trainings with an Evolutionary Initialization. In ICANN’95, volume 2, pages 153–158. EC2, 1995.
C. Gégout. Stable and Convergent Dynamics for Discrete-time Recurrent Networks. In World Congress of Nonlinear Analysts. IFNA, July 1996. to be published in 1997.
M.W. Hirsch. Convergent Activation Dynamics in Continuous Time Networks. Neural Networks, 2:331–349, 1989.
M. Schoenauer, E. Ronald, and S. Damour. Evolving Networks for Control. In Neuronîmes 93, Paris, 1993. EC2.
H.T. Siegelmann, B.G. Horne, and C.L. Giles. Computational capabilities of recurrent narx neural networks. Technical Report 95–78, University of Maryland, College Park, Md, 1995. Accepted also IEEE Trans, on Syst., Man and Cybern.
P.Y. Simard. Learning State Space Dynamics in Recurrent Networks. PhD thesis, University of Rochester, New York, mars 1991. 383.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 1998 Springer-Verlag Wien
About this paper
Cite this paper
Gégout, C. (1998). Evolutionary Learning of Recurrent Networks by Successive Orthogonal Inverse Approximations. In: Artificial Neural Nets and Genetic Algorithms. Springer, Vienna. https://doi.org/10.1007/978-3-7091-6492-1_83
Download citation
DOI: https://doi.org/10.1007/978-3-7091-6492-1_83
Publisher Name: Springer, Vienna
Print ISBN: 978-3-211-83087-1
Online ISBN: 978-3-7091-6492-1
eBook Packages: Springer Book Archive