Abstract
Having a variety of good characteristics against other pattern recognition techniques, the multilayer perceptron (MLP) has been used in many applications. But, it is known that the error backpropagation (EBP) algorithm that the MLP uses in learning has the defect that requires relatively long learning time. In order to increase learning speed it is very effective to use the online-based learning methods, which update the weight vector of the MLP pattern by pattern, because the learning data for pattern recognition contain high redundancy. A typical online EBP algorithm applies the fixed learning rate for each update of the weight vector. Though a large amount of speedup with the online EBP can be obtained just by choosing the appropriate fixed rate, fixing the rate has the inefficiency that doesn’t fully utilize the instant updates of the online mode. And, although the patterns come to be divided into the learned and the unlearned during learning process and the learned have no need to go through the computation for learning, the existing online EBP uniformly computes the whole patterns during an epoch. To remedy these inefficiencies, this paper proposes a Changing rate and Omitting patterns in Instant Learning (COIL) method to apply the appropriate rate for each pattern and put only the unlearned into learning. To verify the efficiency of the COIL, experimentations are conducted for speaker verification and speech recognition as the applications of pattern recognition and the results are presented.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Morgan, N. and Bourlard, H.: Hybrid Connectionist Models for Continuous Speech Recognition. In: Lee, C. H. (eds.): Automatic Speech and Speaker Recognition. Kluwer Academic Publishers, Massachusetts (1996)
Haykin, S.: Neural Networks. 2nd edn. Prentice Hall, New Jersey (1999)
Riedmiller, M. and Braun, H.: A Direct Adaptive Method for Faster Backpropagation Learning: The RPROP Algorithm, IEEE International Conference on Neural Networks. Vol. 1 (1993) 586–591
Demuth, H. and Beale, M.: Neural Network Toolbox. The MathWorks (2001)
Fletcher, R.: Practical Methods of Optimization. 2nd edn. Wiley, New York (1987).
Moller, M.: Supervised Learning on Large Redundant Training Sets. Proceedings of the 1992 IEEE-SP Workshop Neural Networks for Signal Processing (1992) 79–89
Becker, S. and LeCun, Y.: Improving the Convergence of Back-Propagation Learning with Second-Order Methods. Proceedings of the 1988 Connectionist Models Summer School (1988) 29–37
Bengio, Y.: Neural Networks for Speech and Sequence Recognition. International Thomson Computer Press, London (1995)
Wilson, D. R. and Martinez, T. R.: The Need for Small Learning Rates on Large Problems. International Joint Conference on Neural Networks. 1 (2001) 115–119
Naik, J. M.: Speaker Verification: A Tutorial. IEEE Communications Magazine. 1 (1990) 42–48
Becchetti, C. and Ricotti, L. P.: Speech Recognition. John Wiley & Sons, Chichester (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lee, TS., Choi, HJ., Kwag, YK., Hwang, BW. (2002). A Method on Improvement of the Online Mode Error Backpropagation Algorithm for Pattern Recognition. In: Ishizuka, M., Sattar, A. (eds) PRICAI 2002: Trends in Artificial Intelligence. PRICAI 2002. Lecture Notes in Computer Science(), vol 2417. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45683-X_31
Download citation
DOI: https://doi.org/10.1007/3-540-45683-X_31
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44038-3
Online ISBN: 978-3-540-45683-4
eBook Packages: Springer Book Archive