Abstract
The Sensitivity-Based Linear Learning Method (SBLLM) is a learning method for two-layer feedforward neural networks based on sensitivity analysis, that calculates the weights by solving a linear system of equations. Therefore, there is an important saving in computational time which significantly enhances the behavior of this method compared to other learning algorithms. In this paper a generalization of the SBLLM that includes a regularization term in the cost function is presented. The estimation of the regularization parameter is made by means of an automatic technique. The theoretical basis for the method is given and its performance is illustrated by comparing the results obtained by the automatic technique and those obtained manually by cross-validation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Rumelhart, D.E., Hinton, G.E., Willian, R.J.: Learning Representations of Back-Propagation Errors. Nature 323, 533–536 (1986)
Castillo, E., Guijarro-Berdiñas, B., Fontenla-Romero, O., Alonso-Betanzos, A.: A very fast learning method for Neural Networks Based on Sensitivity Analysis. Journal of Machine Learning Research 7, 1159–1182 (2006)
Castillo, E., Fontenla-Romero, O., Alonso-Betanzos, A., Guijarro-Berdiñas, B.: A Global Optimum Approach for One-Layer Neural Networks. Neural Computation 14(6), 1429–1449 (2002)
Erdogmus, D., Fontenla-Romero, O., Principe, J.C., Alonso-Betanzos, A., Castillo, E.: Linear-Least-Squares Initialization of Multilayer Perceptrons Through Backpropagation of the Desired Response. IEEE Transactions on Neural Network 16(2), 325–337 (2005)
Fontenla-Romero, O., Erdogmus, D., Principe, J.C., Alonso-Betanzos, A., Castillo, E.: Linear least-squares based methods for neural networks learning. In: Kaynak, O., Alpaydın, E., Oja, E., Xu, L. (eds.) ICANN 2003 and ICONIP 2003. LNCS, vol. 2714, pp. 84–91. Springer, Heidelberg (2003)
Guijarro-Berdiñas, B., Fontenla-Romero, O., Perez-Sanchez, B., Alonso-Betanzos, A.: A Regularized Learning Method for Neural Networks Based on Sensitivity Analysis. In: European Symposium on Artificial Neural Networks, pp. 289–294 (2008)
Guo, P., Lyu, M.R., Philip Chen, C.L.: Regularization Parameter Estimation for Feedforward Neural Networks. IEEE Transactions on System, Man and Cybernetics 33(1), 35–44 (2003)
Hollander, M., Wolfe, D.A.: Nonparametric Statistical Methods. Wiley, Chichester (1973)
Gibbons, J.D.: Nonparametric Statistical Inference, 2nd edn. Marcel Dekker, New York (1985)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Pérez-Sánchez, B., Fontenla-Romero, O., Guijarro-Berdiñas, B. (2009). A Supervised Learning Method for Neural Networks Based on Sensitivity Analysis with Automatic Regularization. In: Cabestany, J., Sandoval, F., Prieto, A., Corchado, J.M. (eds) Bio-Inspired Systems: Computational and Ambient Intelligence. IWANN 2009. Lecture Notes in Computer Science, vol 5517. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02478-8_20
Download citation
DOI: https://doi.org/10.1007/978-3-642-02478-8_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02477-1
Online ISBN: 978-3-642-02478-8
eBook Packages: Computer ScienceComputer Science (R0)