Abstract
This paper proposes enhanced gradient descent learning algorithms for quaternion-valued feedforward neural networks. The quickprop, resilient backpropagation, delta-bar-delta, and SuperSAB algorithms are the most known such enhanced algorithms for the real- and complex-valued neural networks. They gave superior performances than the gradient descent algorithm, so it is natural to extend these learning methods to quaternion-valued neural networks, also. The quaternion variants of these four algorithms are presented, which are then used to learn various time series prediction applications. Experimental results show an important improvement in performance over the quaternion gradient descent.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arena, P., Fortuna, L., Muscato, G., Xibilia, M.: Multilayer perceptrons to approximate quaternion valued functions. Neural Netw. 10(2), 335–342 (1997)
Arena, P., Fortuna, L., Muscato, G., Xibilia, M.: Neural Networks in Multidimensional Domains Fundamentals and New Trends in Modelling and Control. Lecture Notes in Control and Information Sciences, vol. 234. Springer, London (1998)
Bishop, C.: Neural Networks for Pattern Recognition. Oxford University Press, Inc., New York (1995)
Buchholz, S., Le Bihan, N.: Polarized signal classification by complex and quaternionic multi-layer perceptrons. Int. J. Neural Syst. 18(2), 75–85 (2008)
Che Ujang, B., Took, C., Mandic, D.: Split quaternion nonlinear adaptive filtering. Neural Netw. 23(3), 426–434 (2010)
Che Ujang, B., Took, C., Mandic, D.: Quaternion-valued nonlinear adaptive filtering. IEEE Trans. Neural Netw. 22(8), 1193–1206 (2011)
Fahlman, S.: An empirical study of learning speed in backpropagation networks. Technical report 1800, Carnegie Mellon University, January 1988. http://repository.cmu.edu/compsci/1800
Isokawa, T., Kusakabe, T., Matsui, N., Peper, F.: Quaternion neural network and its application. In: Palade, V., Howlett, R., Jai, L. (eds.) Knowledge-Based Intelligent Information and Engineering Systems. Lecture Notes in Computer Science, vol. 2774, pp. 318–324. Springer, Heidelberg (2003)
Jacobs, R.: Increased rates of convergence through learning rate adaptation. Neural Netw. 1(4), 295–307 (1988)
Jahanchahi, C., Took, C., Mandic, D.: On HR calculus, quaternion valued stochastic gradient, and adaptive three dimensional wind forecasting. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–5. IEEE, July 2010
Kusamichi, H., Isokawa, T., Matsui, N., Ogawa, Y., Maeda, K.: A new scheme for color night vision by quaternion neural network. In: International Conference on Autonomous Robots and Agents, pp. 101–106, December 2004
Mandic, D., Chambers, J.: Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability. Wiley, New York (2001)
Popa, C.A.: Enhanced gradient descent algorithms for complex-valued neural networks. In: International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), pp. 272–279. IEEE, September 2014
Riedmiller, M.: Advanced supervised learning in multi-layer perceptrons - from backpropagation to adaptive learning algorithms. Comput. Stand. Interfaces 16(3), 265–278 (1994)
Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropagation learning: the RPROP algorithm. In: IEEE International Conference on Neural Networks, vol. 1, pp. 586–591. IEEE, March 1993
Tollenaere, T.: Supersab: fast adaptive back propagation with good scaling properties. Neural Netw. 3(5), 561–573 (1990)
Took, C., Mandic, D.: The quaternion LMS algorithm for adaptive filtering of hypercomplex processes. IEEE Trans. Signal Process. 57(4), 1316–1327 (2009)
Took, C., Mandic, D.: Quaternion-valued stochastic gradient-based adaptive IIR filtering. IEEE Trans. Signal Process. 58(7), 3895–3901 (2010)
Took, C., Mandic, D.: A quaternion widely linear adaptive filter. IEEE Trans. Signal Process. 58(8), 4427–4431 (2010)
Took, C., Mandic, D., Aihara, K.: Quaternion-valued short term forecasting of wind profile. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE, July 2010
Took, C., Strbac, G., Aihara, K., Mandic, D.: Quaternion-valued short-term joint forecasting of three-dimensional wind and atmospheric parameters. Renew. Energy 36(6), 1754–1760 (2011)
Xia, Y., Jahanchahi, C., Mandic, D.: Quaternion-valued echo state networks. IEEE Trans. Neural Netw. Learn. Syst. 26(4), 663–673 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Popa, CA. (2018). Enhanced Gradient Descent Algorithms for Quaternion-Valued Neural Networks. In: Balas, V., Jain, L., Balas, M. (eds) Soft Computing Applications. SOFA 2016. Advances in Intelligent Systems and Computing, vol 634. Springer, Cham. https://doi.org/10.1007/978-3-319-62524-9_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-62524-9_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-62523-2
Online ISBN: 978-3-319-62524-9
eBook Packages: EngineeringEngineering (R0)