Abstract
In the search space of MLP(J), multilayer perceptron having J hidden units, there exist flat areas called singular regions created by applying reducibility mapping to the optimal solution of MLP(Jāā1). Since such singular regions cause serious slowdown for learning, a learning method for avoiding singular regions has been aspired. However, such avoiding does not guarantee the quality of the final solutions. This paper proposes a new learning method which does not avoid but makes good use of singular regions to stably and successively find solutions excellent enough for MLP(J). The potential of the method is shown by our experiments using artificial and real data sets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Amari, S.: Natural gradient works efficiently in learning. Neural ComputationĀ 10(2), 251ā276 (1998)
Amari, S., Park, H., Fukumizu, K.: Adaptive method of realizing natural gradient learning for multilayer perceptrons. Neural ComputationĀ 12(6), 1399ā1409 (2000)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern classification, 2nd edn. John Wiley & Sons, Inc. (2001)
Fukumizu, K., Amari, S.: Local minima and plateaus in hierarchical structure of multilayer perceptrons. Neural NetworksĀ 1(3), 317ā327 (2000)
Hamey, L.G.C.: XOR has no local minima: a case study in neural network error surface. Neural NetworksĀ 11(4), 669ā681 (1998)
Minnett, R.C.J., Smith, A.T., Lennon Jr., W.C., Hecht-Nielsen, R.: Neural network tomography: network replication from output surface geometry. Neural NetworksĀ 24(5), 484ā492 (2011)
Luenberger, D.G.: Linear and nonlinear programming. Addison-Wesley (1984)
Nakano, R., Saito, K.: Discovering Polynomials to Fit Multivariate Data Having Numeric and Nominal Variables. In: Arikawa, S., Shinohara, A. (eds.) Progress in Discovery Science. LNCS (LNAI), vol.Ā 2281, pp. 482ā493. Springer, Heidelberg (2002)
Nakano, R., Satoh, S., Ohwaki, T.: Learning method utilizing singular region of multilayer perceptron. In: Proc. 3rd Int. Conf. on Neural Computation Theory and Applications, pp. 106ā111 (2011)
Watanabe, S.: Algebraic geometry and statistical learning theory. Cambridge Univ. Press (2009)
Saito, K., Nakano, R.: Partial BFGS update and efficient step-length calculation for three-layer neural networks. Neural ComputationĀ 9(1), 239ā257 (1997)
Sussmann, H.J.: Uniqueness of the weights for minimal feedforward nets with a given input-output map. Neural NetworksĀ 5(4), 589ā593 (1992)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
Ā© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Satoh, S., Nakano, R. (2013). Multilayer Perceptron Learning Utilizing Reducibility Mapping. In: Madani, K., Dourado, A., Rosa, A., Filipe, J. (eds) Computational Intelligence. IJCCI 2011. Studies in Computational Intelligence, vol 465. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35638-4_17
Download citation
DOI: https://doi.org/10.1007/978-3-642-35638-4_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35637-7
Online ISBN: 978-3-642-35638-4
eBook Packages: EngineeringEngineering (R0)