Abstract
New learning algorithms for an adaptive non-linear forward predictor which is based on a Pipelined Recurrent Neural Network (PRNN) are presented. A computationally efficient Gradient Descent (GD) algorithm, as well as a novel Extended Recursive Least Squares (ERLS) algorithm are tested on the predictor. Simulation studies, based on three speech signals, which have been made public and are available on the World Wide Web (WWW), show that the non-linear predictor does not perform satisfactorily when the previously proposed gradient descent algorithm was used. The steepest descent algorithm is shown to yield a poor performance in terms of the prediction error gain, whereas consistently improved results are obtained using the ERLS algorithm. The merit of the non-linear predictor structure is confirmed by yielding approximately 2 dB higher prediction gain than only a linear structure predictor, which uses the conventional Recursive Least Squares (RLS) algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
J. Shynk. Adaptive IIR filtering. IEEE ASSP Magazine, 6(2):4–21, 1989.
S. Haykin. Adaptive Filter Theory (Second Edition). Prentice Hall, 1989.
S. M. Kay. Fundamentals of Statistical Signal Processing: Estimation Theory. Prentice Hall, 1989.
J. Makhoul. Linear Prediction: A Tutorial Overview. Proceedings of the IEEE, 63(4):561–580, 1975.
S. Haykin and L. Li. Non-linear Adaptive Prediction of Non-stationary Signals. IEEE Transactions on Signal Processing, 43(2):526–535, 1995.
L. Li and S. Haykin. A Cascaded Neural Networks for Real-Time Nonlinear Adaptive Filtering. Proceedings of the IEEE International Conference on Neural Networks, (ICNN’93), San Francisco, USA, 2:857–862, 1993.
M. Niranjan and V. Kadirkamanathan. A Nonlinear Model for Time Series Prediction and Signal Interpolation. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, (ICASSP-91), Toronto, Canada, 3:1713–1716, 1991.
Y. Bengio. Neural Networks for Speech and Sequence Recognition. International Thomson Publishing, 1995.
R. M. Dillon and C. N. Manikopoulos. Neural Net Nonlinear Prediction for Speech Data. Electronics Letters, 27(10):824–826, 1991.
B. Townshend. Nonlinear Prediction of Speech. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, (ICASSP-91), Toronto, Canada, 1:425–428, 1991.
C. R. Gent and C. P. Sheppard. Predicting time series by a fully connected neural network trained by back propagation. Computing and Control Engineering Journal, 109-112, 1992.
S. Z. Qin, H.T. Su, and T. J. Mc-Avoy. Comparison of Four Neural Net Learning Methods for Dynamic System Identification. IEEE Transactions on Neural Networks, 3(2):122–130, 1992.
http://www.ert.rwth-aachen.de/Personen/balterse.html
J.T. Connor, R. D. Martin, and L. E. Atlas. Recurrent Neural Networks and Robust Time Series Prediction. IEEE Transactions on Neural Networks, 5(2):240–254, 1994.
R. Williams and D. Zipser. A Learning Algorithm for Continually Running Fully Recurrent Neural Networks. Neural Computation, 1:270–280, 1989.
S. Haykin. Neural Networks — A Comprehensive Foundation. Prentice Hall, 1994.
O. Nerrand, P. Roussel-Ragot, D. Urbani, L. Personnaz, and G. Dreyfus. Training Recurrent Neural Networks: Why and How? An Illustration in Dynamical Process Modelling. IEEE Transactions on Neural Networks, 5(2):178–184, 1994.
K. S. Narendra and K. Parthasarathy. Identification and Control of Dynamical Systems Using Neural Networks. IEEE Transactions on Neural Networks, 1(1):4–27, 1990.
K. S. Narendra and K. Parthasarathy. Gradient Methods for the Optimization of Dynamical Systems Containing Neural Networks. IEEE Transactions on Neural Networks, 2(2):252–262, 1991.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1998 Springer Science+Business Media New York
About this chapter
Cite this chapter
Mandic, D., Baltersee, J., Chambers, J. (1998). Non-Linear Adaptive Prediction of Speech with a Pipelined Recurrent Neural Network and Advanced Learning Algorithms. In: Procházka, A., UhlĂĹ™, J., Rayner, P.W.J., Kingsbury, N.G. (eds) Signal Analysis and Prediction. Applied and Numerical Harmonic Analysis. Birkhäuser, Boston, MA. https://doi.org/10.1007/978-1-4612-1768-8_21
Download citation
DOI: https://doi.org/10.1007/978-1-4612-1768-8_21
Publisher Name: Birkhäuser, Boston, MA
Print ISBN: 978-1-4612-7273-1
Online ISBN: 978-1-4612-1768-8
eBook Packages: Springer Book Archive