Abstract
Multi-objective learning has been explored in neural network because it adjusts the model capacity providing better generalization properties. It usually requires sophisticated algorithms such as ellipsoidal, sliding-mode, genetic algorithms, among others. This paper proposes an affordable algorithm that decomposes the gradient into two components and it adjusts the weights of the network separately. By doing so multi-objective learning with L 2 norm control is accomplished.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, Heidelberg (1995)
Teixeira, R.A., Braga, A.P., Takahashi, R.H.C., Saldanha, R.R.: Improving generalization of mlps with multi-objective optimization. Neurocomputing 35, 189–194 (2000)
Chankong, V., Haimes, Y.Y.: Multiobjective Decision Making: Theory and Methodology, vol. 8. North-Holland (Elsevier), New York (1983)
Braga, A.P., Takahashi, R.H.C., Teixeira, R.A., Costa, M.A.: Multi-Objective Algorithms for Neural Networks Learning. In: Jin, Y. (ed.) Multiobjective Machine Learning, pp. 151–172. Springer, Heidelberg (2006)
Costa, M.A., Braga, A.P., Menezes, B.R., Teixeira, R.A., Parma, G.G.: Training neural networks with a multi-objective sliding mode control algorithm. Neurocomputing 51, 467–473 (2003)
Liu, G.P., Kadirkamanathan, V.: Learning with multi-objective criteria. In: International Conference on Neural Networks, UK, pp. 53–58 (1995)
Bartlett, P.L.: For valid generalization, the size of the weights is more important than the size of the network. In: Proceedings of NIPS, pp. 134–140 (1997)
Boser, B., Guyon, I., Vapnik, V.: A Training algorithm for optimal margin classifiers. In: Fifth Annual Workshop on Computational Learning Theory, pp. 144–152. Morgan Kaufmann, San Mateo (1992)
Haimes, Y.Y., Lasdon, L.S., Wismer, D.A.: On a Bicriterion Formulation of the Problems of Integrated System Identification and System Optimization. IEEE Transactions on Systems, Man, and Cybernetics 1(3), 296–297 (1971)
Bland, R.G., Goldfarb, D., Todd, M.J.: The Ellipsoidal Method: A Survey. Operations Research 29(6), 1039–1091 (1981)
Medeiros, T., Braga, A.P.: A new decision strategy in multi-objective training of artificial neural networks. In: European Symposium on Neural Networks (ESANN 2007), pp. 555–560 (2007)
Teixeira, R.A., Braga, A.P., Saldanha, R.R., Takahashi, R.H.C., Medeiros, T.H.: The Usage of Golden Section in Calculating the Efficient Solution in Artificial Neural Networks Training by Multi-objective Optimization. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D.P. (eds.) ICANN 2007. LNCS, vol. 4668, pp. 289–298. Springer, Heidelberg (2007)
Kokshenev, I., Braga, A.P.: Complexity bounds and multi-objective learning of radial basis functions. Neurocomputing 71, 1203–1209 (2008)
Braga, A.P.: New decision strategies for multi-objective learning (2011) (in preparation)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning Representations by Back-Propagating Errors. Nature 323, 533–536 (1986)
Costa, M.A., Braga, A.P., Menezes, B.R.: Improving generalization of MLPs with Sliding Mode Control and the Levenberg-Marquadt algorithm. Neurocomputing 70, 1342–1347 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Costa, M.A., Braga, A.P. (2011). Gradient Descent Decomposition for Multi-objective Learning. In: Yin, H., Wang, W., Rayward-Smith, V. (eds) Intelligent Data Engineering and Automated Learning - IDEAL 2011. IDEAL 2011. Lecture Notes in Computer Science, vol 6936. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23878-9_45
Download citation
DOI: https://doi.org/10.1007/978-3-642-23878-9_45
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23877-2
Online ISBN: 978-3-642-23878-9
eBook Packages: Computer ScienceComputer Science (R0)