Abstract
Artificial Neural Networks are bioinspired mathematical models that have been widely used to solve many complex problems. However, the training of a Neural Network is a difficult task since the traditional training algorithms may get trapped into local solutions easily. This problem is greater in Recurrent Neural Networks, where the traditional training algorithms sometimes provide unsuitable solutions. Some evolutionary techniques have also been used to improve the training stage, and to overcome such local solutions, but they have the disadvantage that the time taken to train the network is high. The objective of this work is to show that the use of some non-linear programming techniques is a good choice to train a Neural Network, since they may provide suitable solutions quickly. In the experimental section, we apply the models proposed to train an Elman Recurrent Neural Network in real-life Time Series Prediction problems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Blanco, Delgado, Pegalajar. 2001. A Real-Coded genetic algorithm for training recurrent neural networks. Neural Networks, vol. 14, pp. 93–105.
C. Zhu, R. H. Byrd and J. Nocedal. 1997. L-BFGS-B: Algorithm 778: L-BFGS-B, FORTRAN routines for large scale bound constrained optimization, ACM Transactions on Mathematical Software, Vol 23, Num. 4, pp 550–560
Cuéllar M.P., Delgado M., Pegalajar M.C. 2004. A Comparative study of Evolutionary Algorithms for Training Elman Recurrent Neural Networks to predict the Autonomous Indebtedness. in Proc. ICEIS, Porto, Portugal, pp. 457–461.
Danilo P. Mandic, Jonathon A. Chambers. 2001. Recurrent Neural Networks for Prediction. Wiley, John & Sons, Incorporated.
D. W. Marquardt. 1963. An algorithm for least-squares estimation of nonlinear parameters, Journal of the Society for Industrialand Applied Mathematics, pp. 11431–441.
Martin T. Hagan, Mohammed B. Menhaj. 1994. Training Feed Forward networks with the Marquardt algorithm, IEEE transactions on Neural networks, vol 5, no. 6, pp. 989–993.
Michael Hüsken, Peter Stagge. 2003. Recurrent Neural Networks for Time Series classification, Neurocomputing, vol. 50, pp. 223–235.
More, J. J. 1977. The Levenberg-Marquardt algorithm: Implementation and theory. Lecture notes in mathematics, Edited by G. A. Watson, SpringerVerlag.
R. H. Byrd, P. Lu and J. Nocedal. 1995. A Limited Memory Algorithm for Bound Constrained Optimization, SIAM Journal on Scientific and Statistical Computing, 16, 5, pp. 1190–1208.
R. Martí, A. El-Fallahi. 2002. Multilayer Neural Networks: An experimental evaluation of on-line training methods. Computers and Operations Research 31, pp. 1491–1513.
Ryad Zemomi, Daniel Racaceanu, Nouredalime Zerhonn. 2003. Recurrent Radial Basis fuction network for Time Seties prediction, Engineering appl. of Artificial Intelligence, vol. 16, no. 5–6, pp. 453–463.
Simon Haykin. 1999. Neural Networks (a Comprehensive foundation). Second Edition. Prentice Hall.
Williams R.J., Peng J. 1990. An efficient Gradient—Based Algorithm for On-Line Training of Recurrent Network trajectories,” Neural Computation, vol. 2, pp. 491–501.
Williams R.J., Zipser D. 1989. A learning algorithm for continually running fully recurrent neural networks, Neural Computation, vol. 1, pp. 270–280.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2007 Springer
About this paper
Cite this paper
Cuéllar, M., Delgado, M., Pegalajar, M. (2007). AN APPLICATION OF NON-LINEAR PROGRAMMING TO TRAIN RECURRENT NEURAL NETWORKS IN TIME SERIES PREDICTION PROBLEMS. In: Chen, CS., Filipe, J., Seruca, I., Cordeiro, J. (eds) Enterprise Information Systems VII. Springer, Dordrecht. https://doi.org/10.1007/978-1-4020-5347-4_11
Download citation
DOI: https://doi.org/10.1007/978-1-4020-5347-4_11
Publisher Name: Springer, Dordrecht
Print ISBN: 978-1-4020-5323-8
Online ISBN: 978-1-4020-5347-4
eBook Packages: Computer ScienceComputer Science (R0)