Abstract
Among many approaches to choosing the proper size of neural networks, one popular approach is to start with an oversized network and then prune it to a smaller size so as to attain better performance with less computational complexity. In this paper, a new hidden node pruning method is proposed based on the redundancy reduction among hidden nodes. The redundancy information is given by correlation coefficients among hidden nodes and this can save computational complexity. Experimental results demonstrate the effectiveness of the proposed method.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Xu, J., Ho, D.W.C.: A new training and pruning algorithm based on node dependence and Jacobian rank deficiency. Neurocomputing 70, 544–558 (2006)
Zhang, L., Jiang, J.-H., Liu, P., Liang, Y.-Z., Yu, R.-Q.: Multivariate nonlinear modeling of fluorescence data by neural network with hidden node pruning algorithm. Analytica Chimica Acta 344, 29–39 (1997)
Engelbrecht, A.P.: A new pruning heuristic based on variance analysis of sensitivity information. IEEE Trans. Neural Networks 12, 1386–1399 (2001)
Zeng, X., Yeung, D.S.: Hidden neuron pruning of multilayer perceptrons using a quantified sensitivity measure. Neurocomputing 69, 825–837 (2006)
Lauret, P., Fock, E., Mara, T.A.: A node pruning algorithm based on a Fourier amplitude sensitivity test method. IEEE Trans. Neural Networks 17, 273–293 (2006)
Sietsma, J., Dow, R.J.F.: Creating artificial neural networks that generalize. Neural Networks 4, 67–79 (1991)
Hagiwara, M.: Removal of hidden units and weights for back propagation networks. In: Int. Joint Conf. Neural Networks, pp. 351–354 (1993)
Rumelhart, D.E., McClelland, J.L.: Parallel Distributed Processing. MIT Press, Cambridge (1986)
Hull, J.J.: A database for handwritten text recognition research. IEEE Trans. Pat. Ana. Mach. Int. 16, 550–554 (1994)
Oh, S.-H.: Improving the error back-propagation algorithm with a modified error function. IEEE Trans. Neural Networks 8, 799–803 (1997)
Lee, Y., Oh, S.-H., Kim, M.W.: An analysis of premature saturation in back propagation learning. Neural Networks 6, 719–728 (1993)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Oh, SH. (2011). Hidden Node Pruning of Multilayer Perceptrons Based on Redundancy Reduction. In: Lee, G., Howard, D., Ślęzak, D. (eds) Convergence and Hybrid Information Technology. ICHIT 2011. Lecture Notes in Computer Science, vol 6935. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24082-9_30
Download citation
DOI: https://doi.org/10.1007/978-3-642-24082-9_30
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24081-2
Online ISBN: 978-3-642-24082-9
eBook Packages: Computer ScienceComputer Science (R0)