Complex-Valued Multilayer Perceptron Search Utilizing Singular Regions of Complex-Valued Parameter Space
In the search space of a complex-valued multilayer perceptron having J hidden units, C-MLP(J), there exist flat areas called singular regions, as is the case with a real-valued MLP. The singular regions cause serious stagnation of learning, preventing usual search methods from finding an excellent solution. However, there exist descending paths from the regions since most points in the regions are saddles. This paper proposes a completely new learning method that does not avoid but makes good use of singular regions to stably and successively find excellent solutions commensurate with C-MLP(J). Our experiments showed the proposed method worked well.
Keywordscomplex-valued multilayer perceptron Wirtinger calculus search method singular region reducibility mapping
Unable to display preview. Download preview PDF.
- 2.Amin, M. F., Amin, M.I., Al-Nuaimi, A.Y.H., Murase, K.: Wirtinger calculus based gradient descent and Levenberg-Marquardt learning algorithms in complex-valued neural networks. In: Lu, B.-L., Zhang, L., Kwok, J. (eds.) ICONIP 2011, Part I. LNCS, vol. 7062, pp. 550–559. Springer, Heidelberg (2011)CrossRefGoogle Scholar
- 3.Delgado, K.K.: The complex gradient operator and the CR-calculus. ECE275A-Lecture Supplement, Fall (2006)Google Scholar
- 6.Kim, M.S., Guest, C.C.: Modification of backpropagation networks for complex-valued signal processing in frequency domain. In: Proc. IJCNN 1990, vol. 3, pp. 27–31 (1990)Google Scholar
- 9.Nakano, R., Satoh, S., Ohwaki, T.: Learning method utilizing singular region of multilayer perceptron. In: Proc. 3rd Int. Conf. on Neural Computation Theory and Applications, pp. 106–111 (2011)Google Scholar
- 10.Nitta, T.: Reducibility of the complex-valued neural network. Neural Information Processing - Letters and Reviews 2(3), 53–56 (2004)Google Scholar
- 12.Satoh, S., Nakano, R.: Eigen vector descent and line search for multilayer perceptron. In: Proc. Int. Multi-Conf. of Eng. & Comp. Scientists, vol. 1, pp. 1–6 (2012)Google Scholar