Summary
In order to select the right-sized network, many pruning algorithms have been proposed. One may ask which of the pruning algorithms is best in terms of the generalization error of the resulting artificial neural network classifiers. In this paper, we compare the performance of four pruning algorithms in small training sample size situations. A comparative study with artificial and real data suggests that the weight-elimination method proposed by Weigend et al. is best.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Gabor, D. (1946): Theory of communication, J. Inst. Elect. Engr., 93, 429–459.
Hamamoto, Y. et al. (1996a): On the behavior of artificial neural network classifiers in high-dimensional spaces, IEEE Trans. Pattern Analysis and Machine Intelligence, 18, 5, 571–574.
Hamamoto, Y.(1996b): Recognition of handwritten numerals using Gabor features, In Proc. of 13th Int. Conf. Pattern Recognition,Vienna, in press.
Jain, A. K. and Chandrasekaran, B. (1982): Dimensionality and sample size considerations in pattern recognition practice, In Handbook of Statistics, Vol. 2, P. R. Krishnaiah and L. N. Kanal, Eds., North-Holland, 835–855.
Karnin, E. D. (1990): A simple procedure for pruning back-propagation trained neural networks, IEEE Trans. Neural Networks, 1, 2, 239–242.
Kruschke, J. K. (1988): Creating local and distributed bottlenecks in hidden layers of back-propagation networks, In Proc. 1988 Connectionist Models Summer School, 120–126.
Le Cun, Y.(1990): Optimal brain damage, In Advances in Neural Information Processing (2), Denver, 598–605.
Reed, R. (1993): Pruning algorithms–A survey, IEEE Trans. Neural Networks, 4, 5, 740–747.
Rumelhart, D. E. et al. (1986): Learning internal representations by error propagation, In D. E. Rumelhart and J. L. McClelland (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol.1: Foundations., MIT Press.
Van Ness, J. (1980): On the dominance of non-parametric Bayes rule discriminant algorithms in high dimensions, Pattern Recognition, 12, 355–368.
Weigend, A. S. et al. (1991): Generalization by weight-elimination with application to forecasting, In Advances in Neural Information Processing ( 3 ), 875–882.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1998 Springer Japan
About this paper
Cite this paper
Hamamoto, Y., Hase, T., Nakai, S., Tomita, S. (1998). Comparison of Pruning Algorithms in Neural Networks. In: Hayashi, C., Yajima, K., Bock, HH., Ohsumi, N., Tanaka, Y., Baba, Y. (eds) Data Science, Classification, and Related Methods. Studies in Classification, Data Analysis, and Knowledge Organization. Springer, Tokyo. https://doi.org/10.1007/978-4-431-65950-1_36
Download citation
DOI: https://doi.org/10.1007/978-4-431-65950-1_36
Publisher Name: Springer, Tokyo
Print ISBN: 978-4-431-70208-5
Online ISBN: 978-4-431-65950-1
eBook Packages: Springer Book Archive