Abstract
The multilayer perceptron (MLP) is a well established neural network model for supervised learning problems. Furthermore, it is well known that its performance for a given problem depends crucially on appropriately selecting the MLP architecture, which is typically achieved using cross-validation. In this work, we propose an incremental Bayesian methodology to address the important problem of automatic determination of the number of hidden units in MLPs with one hidden layer. The proposed methodology treats the one-hidden layer MLP as a linear model consisting of a weighted combination of basis functions (hidden units). Then an incremental method for sparse Bayesian learning of linear models is employed that effectively adjusts not only the combination weights, but also the parameters of the hidden units. Experimental results for several well-known classification data sets demonstrate that the proposed methodology successfully identifies optimal MLP architectures in terms of generalization error.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Tipping, M.E.: Sparse Bayesian learning and the relevance vector machine. Journal of Machine Learning Research 1, 211–244 (2001)
Tzikas, D., Likas, A., Galatsanos, N.: Sparse bayesian modeling with adaptive kernel learning. IEEE Transactions on Neural Networks 20(6), 926–937 (2009)
Schmolck, A., Everson, R.: Smooth relevance vector machine: a smoothness prior extension of the RVM. Machine Learning 68(2), 107–135 (2007)
Holmes, C.C., Denison, D.G.T.: Bayesian wavelet analysis with a model complexity prior. In: Bernardo, J.M., Berger, J.O., Dawid, A.P., Smith, A.F.M. (eds.) Bayesian Statistics 6: Proceedings of the Sixth Valencia International Meeting. Oxford University Press, Oxford (1999)
Tipping, M.E., Faul, A.: Fast marginal likelihood maximisation for sparse Bayesian models. In: Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics (2003)
Neal, R.M.: Bayesian Learning for Neural Networks. Lecture Notes in Statistics, vol. 118. Springer, Heidelberg (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tzikas, D., Likas, A. (2010). An Incremental Bayesian Approach for Training Multilayer Perceptrons. In: Diamantaras, K., Duch, W., Iliadis, L.S. (eds) Artificial Neural Networks – ICANN 2010. ICANN 2010. Lecture Notes in Computer Science, vol 6352. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15819-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-642-15819-3_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15818-6
Online ISBN: 978-3-642-15819-3
eBook Packages: Computer ScienceComputer Science (R0)