Abstract
Weight elimination can be usefully interpreted as an assumption about the prior distribution of the weights trained in the backpropagation neural networks (BPNN). Weight elimination based on different scaling of weight parameters is of a general form, with the weight decay and subset selection methods as special cases. The applications of this method have been well developed, however, only few references provides more comprehensive theoretical analysis. To address this issue, we investigate the uniform boundedness of the trained weights based on a descriptive proof.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Werbos, P.J.: Beyond regression: new tools for prediction and analysis in the behavioral sciences. Ph.D. thesis. Harvard University, Cambridge, MA (1974)
Parker, D.B.: Learning-logic, invention report. Stanford University, Stanford (1982)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagation errors. Nature 323, 533–536 (1986)
Cherkassky, V., Mulier, F.M.: Learning from data: Concepts, Theory, and Methods. IEEE Press (2007)
Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural networks architectures. Neural Computation 7, 219–269 (1995)
Friedman, J.: An Overview of Predictive Learning and Function Approximation. In: From Statistics to Neural Networks, vol. 136, pp. 1–61. Springer, Heidelberg (1994)
Weigend, A.S., Rumelhart, D.E., Huberman, B.A.: Generalization by weight-elimination applied to currency exchange rate prediction. In: IJCNN 1991-Seattle International Joint Conference on Neural Networks, vol. 1, pp. 837–841 (1991)
Bebis, G., Georgiopoulos, M., Kaspalris, T.: Coupling weight elimination and genetic algorithms. In: IEEE International Conference on Neural Networks, vol. 2, pp. 1115–1120 (1996)
Gupta, A., Lam, M.: The weight decay backpropagation for generalizations with missing values. Annals of Operations Research 78, 165–187 (1998)
Leung, C.-S., Tsoi, A.-C., Chan, L.-W.: Two regularizers for recursive least squared algorithms in feedforward multilayered neural networks. IEEE Transactions on Neural Networks 12, 1314–1332 (2001)
Kuo, R.J., Wu, P., Wang, C.P.: An intelligent sales forecasting system through integration of artificial neural networks and fuzzy neural networks with fuzzy weight elimination. Neural Networks 15, 909–925 (2002)
Ennett, C.M., Frize, M.: Weight-elimination neural networks applied to coronary surgery mortality prediction. IEEE Transactions on Information Technology in Biomedicine 7, 86–92 (2003)
May, P., Zhou, E., Lee, C.W.: A Comprehensive Evaluation of Weight Growth and Weight Elimination Methods Using the Tangent Plane Algorithm. International Journal of Advanced Computer Sciences and Applications 4, 149–156 (2013)
Zhang, H., Wu, W., Liu, F., Yao, M.: Boundedness and Convergence of Online Gradient Method With Penalty for Feedforward Neural Networks. IEEE Transactions on Neural Networks 20, 1050–1054 (2009)
Shao, H., Zheng, G.: Boundedness and convergence of online gradient method with penalty and momentum. Neurocomputing 74, 765–770 (2011)
Wang, J., Wu, W., Zurada, J.M.: Boundedness and convergence of MPN for cyclic and almost cyclic learning with penalty. In: The 2011 International Joint Conference on Neural Networks (IJCNN), pp. 125–132 (2011)
Wang, J., Wu, W., Zurada, J.M.: Computational properties and convergence analysis of BPNN for cyclic and almost cyclic learning with penalty. Neural Networks 33, 127–135 (2012)
http://www.xieguofang.cn/Maths/Quartic/Quartic_Xie_Method_of_Determining_Root_Configuration.htm
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Wang, J., Zurada, J.M., Wang, Y., Wang, J., Xie, G. (2014). Boundedness of Weight Elimination for BP Neural Networks. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2014. Lecture Notes in Computer Science(), vol 8467. Springer, Cham. https://doi.org/10.1007/978-3-319-07173-2_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-07173-2_15
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07172-5
Online ISBN: 978-3-319-07173-2
eBook Packages: Computer ScienceComputer Science (R0)