Advertisement

Complex-Valued Multilayer Perceptron Search Utilizing Singular Regions of Complex-Valued Parameter Space

  • Seiya Satoh
  • Ryohei Nakano
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8681)

Abstract

In the search space of a complex-valued multilayer perceptron having J hidden units, C-MLP(J), there exist flat areas called singular regions, as is the case with a real-valued MLP. The singular regions cause serious stagnation of learning, preventing usual search methods from finding an excellent solution. However, there exist descending paths from the regions since most points in the regions are saddles. This paper proposes a completely new learning method that does not avoid but makes good use of singular regions to stably and successively find excellent solutions commensurate with C-MLP(J). Our experiments showed the proposed method worked well.

Keywords

complex-valued multilayer perceptron Wirtinger calculus search method singular region reducibility mapping 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Amari, S.: Natural gradient works efficiently in learning. Neural Computation 10(2), 251–276 (1998)CrossRefMathSciNetGoogle Scholar
  2. 2.
    Amin, M. F., Amin, M.I., Al-Nuaimi, A.Y.H., Murase, K.: Wirtinger calculus based gradient descent and Levenberg-Marquardt learning algorithms in complex-valued neural networks. In: Lu, B.-L., Zhang, L., Kwok, J. (eds.) ICONIP 2011, Part I. LNCS, vol. 7062, pp. 550–559. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  3. 3.
    Delgado, K.K.: The complex gradient operator and the CR-calculus. ECE275A-Lecture Supplement, Fall (2006)Google Scholar
  4. 4.
    Fukumizu, K., Amari, S.: Local minima and plateaus in hierarchical structure of multilayer perceptrons. Neural Networks 13(3), 317–327 (2000)CrossRefGoogle Scholar
  5. 5.
    Hirose, A.: Complex-Valued Neural Networks, 2nd edn. SCI, vol. 400. Springer, Heidelberg (2012)CrossRefzbMATHGoogle Scholar
  6. 6.
    Kim, M.S., Guest, C.C.: Modification of backpropagation networks for complex-valued signal processing in frequency domain. In: Proc. IJCNN 1990, vol. 3, pp. 27–31 (1990)Google Scholar
  7. 7.
    Leung, H., Haykin, S.: The complex backpropagation algorithm. IEEE Trans. Signal Process. 39(9), 2101–2104 (1991)CrossRefGoogle Scholar
  8. 8.
    Luenberger, D.G.: Linear and nonlinear programming. Addison-Wesley, Reading (1984)zbMATHGoogle Scholar
  9. 9.
    Nakano, R., Satoh, S., Ohwaki, T.: Learning method utilizing singular region of multilayer perceptron. In: Proc. 3rd Int. Conf. on Neural Computation Theory and Applications, pp. 106–111 (2011)Google Scholar
  10. 10.
    Nitta, T.: Reducibility of the complex-valued neural network. Neural Information Processing - Letters and Reviews 2(3), 53–56 (2004)Google Scholar
  11. 11.
    Nitta, T.: Local minima in hierarchical structures of complex-valued neural networks. Neural Networks 43, 1–7 (2013)CrossRefzbMATHGoogle Scholar
  12. 12.
    Satoh, S., Nakano, R.: Eigen vector descent and line search for multilayer perceptron. In: Proc. Int. Multi-Conf. of Eng. & Comp. Scientists, vol. 1, pp. 1–6 (2012)Google Scholar
  13. 13.
    Satoh, S., Nakano, R.: Fast and stable learning utilizing singular regions of multilayer perceptron. Neural Processing Letters 38(2), 99–115 (2013)CrossRefGoogle Scholar
  14. 14.
    Sussmann, H.J.: Uniqueness of the weights for minimal feedforward nets with a given input-output map. Neural Networks 5(4), 589–593 (1992)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Seiya Satoh
    • 1
  • Ryohei Nakano
    • 1
  1. 1.Chubu UniversityKasugaiJapan

Personalised recommendations