Advertisement

Comparison of Pruning Algorithms in Neural Networks

  • Yoshihiko Hamamoto
  • Toshinori Hase
  • Satoshi Nakai
  • Shingo Tomita
Conference paper
Part of the Studies in Classification, Data Analysis, and Knowledge Organization book series (STUDIES CLASS)

Summary

In order to select the right-sized network, many pruning algorithms have been proposed. One may ask which of the pruning algorithms is best in terms of the generalization error of the resulting artificial neural network classifiers. In this paper, we compare the performance of four pruning algorithms in small training sample size situations. A comparative study with artificial and real data suggests that the weight-elimination method proposed by Weigend et al. is best.

Keywords

Hide Layer Training Sample Gabor Filter Generalization Error Gabor Feature 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Gabor, D. (1946): Theory of communication, J. Inst. Elect. Engr., 93, 429–459.Google Scholar
  2. Hamamoto, Y. et al. (1996a): On the behavior of artificial neural network classifiers in high-dimensional spaces, IEEE Trans. Pattern Analysis and Machine Intelligence, 18, 5, 571–574.CrossRefGoogle Scholar
  3. Hamamoto, Y.(1996b): Recognition of handwritten numerals using Gabor features, In Proc. of 13th Int. Conf. Pattern Recognition,Vienna, in press.Google Scholar
  4. Jain, A. K. and Chandrasekaran, B. (1982): Dimensionality and sample size considerations in pattern recognition practice, In Handbook of Statistics, Vol. 2, P. R. Krishnaiah and L. N. Kanal, Eds., North-Holland, 835–855.Google Scholar
  5. Karnin, E. D. (1990): A simple procedure for pruning back-propagation trained neural networks, IEEE Trans. Neural Networks, 1, 2, 239–242.CrossRefGoogle Scholar
  6. Kruschke, J. K. (1988): Creating local and distributed bottlenecks in hidden layers of back-propagation networks, In Proc. 1988 Connectionist Models Summer School, 120–126.Google Scholar
  7. Le Cun, Y.(1990): Optimal brain damage, In Advances in Neural Information Processing (2), Denver, 598–605.Google Scholar
  8. Reed, R. (1993): Pruning algorithms–A survey, IEEE Trans. Neural Networks, 4, 5, 740–747.CrossRefGoogle Scholar
  9. Rumelhart, D. E. et al. (1986): Learning internal representations by error propagation, In D. E. Rumelhart and J. L. McClelland (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol.1: Foundations., MIT Press.Google Scholar
  10. Van Ness, J. (1980): On the dominance of non-parametric Bayes rule discriminant algorithms in high dimensions, Pattern Recognition, 12, 355–368.CrossRefGoogle Scholar
  11. Weigend, A. S. et al. (1991): Generalization by weight-elimination with application to forecasting, In Advances in Neural Information Processing ( 3 ), 875–882.Google Scholar

Copyright information

© Springer Japan 1998

Authors and Affiliations

  • Yoshihiko Hamamoto
    • 1
  • Toshinori Hase
    • 1
  • Satoshi Nakai
    • 1
  • Shingo Tomita
    • 1
  1. 1.Faculty of EngineeringYamaguchi UniversityJapan

Personalised recommendations