Abstract
The paper presents some novel methods of the activation function selection in the last hidden layer of a multilayer perceptron. For this selection, the least squares method is used. The proposed ways make it possible to decrease the cost function value. They enable achievement of a good compromise between the network complexity and the results being obtained. The methods do not require a start of learning of neural networks from the very beginning. They fit very well for improvement of the action of learnt multilayer perceptrons. They may be particularly useful for construction of the devices under microprocessor control, that have not a big memory nor computing power.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Azimi-Sadjadi, M.R., Liou, R.J.: Fast learning process of multilayer neural networks using recursive least squares method. IEEE Transactions on Signal Processing 40(2), 446–450 (1992)
Barron, A.: Universal Approximation Bounds for Superpositions of a Sigmoidal Function. IEEE Transactions on Information Theory 39(3), 930–945 (1993)
Björck, A.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)
Chyi-Tsong, C., Wei-Der, C.: A Feedforward Neural Network with Function Shape Autotuning. Neural Networks 9(4), 627–641 (1996)
Ebert, T., Bänfer, O., Nelles, O.: Multilayer Perceptron Network with Modified Sigmoid Activation Functions. In: Wang, F.L., Deng, H., Gao, Y., Lei, J. (eds.) AICI 2010. LNCS, vol. 6319, pp. 414–421. Springer, Heidelberg (2010)
Frank, A., Asuncion, A.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine, CA (2010), http://archive.ics.uci.edu/ml
Halawa, K.: A method to improve the performance of multilayer perceptron by utilizing various activation functions in the last hidden layer and the least squares method. Neural Processing Letters 34(3), 293–303 (2011)
Al-Batah, M.S., Isa, N.A.M., Zamli, K.Z., Azizli, K.A.: Modified Recursive Least Squares algorithm to train the Hybrid Multilayered Perceptron (HMLP) network. Applied Soft Computing 10, 236–244 (2010)
Nelles, O.: Nonlinear System Identification: From Classical Approaches to Neural Networks and Fuzzy Models, pp. 253–255. Springer, Berlin (2001)
Nguyen, D., Widrow, B.: Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights. In: Proceedings of the International Joint Conference on Neural Networks, vol. 3, pp. 21–26 (1990)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Halawa, K. (2012). Selection of Activation Functions in the Last Hidden Layer of the Multilayer Perceptron. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2012. Lecture Notes in Computer Science(), vol 7267. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29347-4_9
Download citation
DOI: https://doi.org/10.1007/978-3-642-29347-4_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-29346-7
Online ISBN: 978-3-642-29347-4
eBook Packages: Computer ScienceComputer Science (R0)