Advertisement

Scaled Conjugate Gradient Learning for Quaternion-Valued Neural Networks

  • Călin-Adrian PopaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9949)

Abstract

This paper presents the deduction of the scaled conjugate gradient method for training quaternion-valued feedforward neural networks, using the framework of the HR calculus. The performances of the scaled conjugate algorithm in the real- and complex-valued cases lead to the idea of extending it to the quaternion domain, also. Experiments done using the proposed training method on time series prediction applications showed a significant performance improvement over the quaternion gradient descent and quaternion conjugate gradient algorithms.

Keywords

Quaternion-valued neural networks Scaled conjugate gradient algorithm Time series prediction 

References

  1. 1.
    Arena, P., Fortuna, L., Muscato, G., Xibilia, M.: Multilayer perceptrons to approximate quaternion valued functions. Neural Netw. 10(2), 335–342 (1997)CrossRefGoogle Scholar
  2. 2.
    Arena, P., Fortuna, L., Muscato, G., Xibilia, M.: Neural Networks in Multidimensional Domains Fundamentals and New Trends in Modelling and Control. Lecture Notes in Control and Information Sciences, vol. 234. Springer, London (1998)CrossRefzbMATHGoogle Scholar
  3. 3.
    Bishop, C.: Neural Networks for Pattern Recognition. Oxford University Press Inc., New York (1995)zbMATHGoogle Scholar
  4. 4.
    Buchholz, S., Le Bihan, N.: Polarized signal classification by complex and quaternionic multi-layer perceptrons. Int. J. Neural Syst. 18(2), 75–85 (2008)CrossRefGoogle Scholar
  5. 5.
    Che Ujang, B., Took, C., Mandic, D.: Split quaternion nonlinear adaptive filtering. Neural Netw. 23(3), 426–434 (2010)CrossRefGoogle Scholar
  6. 6.
    Che Ujang, B., Took, C.: Quaternion-valued nonlinear adaptive filtering. IEEE Trans. Neural Netw. 22(8), 1193–1206 (2011)CrossRefGoogle Scholar
  7. 7.
    Hestenes, M., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Nat. Bur. Stan. 49(6), 409–436 (1952)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Isokawa, T., Kusakabe, T., Matsui, N., Peper, F.: Quaternion neural network and its application. In: Palade, V., Howlett, R.J., Jain, L. (eds.) KES 2003. LNCS (LNAI), vol. 2774, pp. 318–324. Springer, Heidelberg (2003). doi: 10.1007/978-3-540-45226-3_44 CrossRefGoogle Scholar
  9. 9.
    Jahanchahi, C., Took, C., Mandic, D.: On hr calculus, quaternion valued stochastic gradient, and adaptive three dimensional wind forecasting. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–5. IEEE, July 2010Google Scholar
  10. 10.
    Johansson, E., Dowla, F., Goodman, D.: Backpropagation learning for multilayer feed-forward neural networks using the conjugate gradient method. Int. J. Neural Syst. 2(4), 291–301 (1991)CrossRefGoogle Scholar
  11. 11.
    Kusamichi, H., Isokawa, T., Matsui, N., Ogawa, Y., Maeda, K.: A new scheme for color night vision by quaternion neural network. In: International Conference on Autonomous Robots and Agents, pp. 101–106, December 2004Google Scholar
  12. 12.
    Mandic, D., Chambers, J.: Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability. Wiley, New York (2001)CrossRefGoogle Scholar
  13. 13.
    Møller, M.: A scaled conjugate gradient algorithm for fast supervised learning. Neural Netw. 6(4), 525–533 (1993)CrossRefGoogle Scholar
  14. 14.
    Polak, E., Ribiere, G.: Note sur la convergence de méthodes de directions conjuguées. Revue Française d’Informatique et de Recherche Opérationnelle 3(16), 35–43 (1969)CrossRefzbMATHGoogle Scholar
  15. 15.
    Popa, C.-A.: Scaled conjugate gradient learning for complex-valued neural networks. In: Matoušek, R. (ed.) Mendel 2015: Recent Advances in Soft Computing. AISC, vol. 378, pp. 221–233. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-19824-8_18 CrossRefGoogle Scholar
  16. 16.
    Reeves, C., Fletcher, R.: Function minimization by conjugate gradients. Comput. J. 7(2), 149–154 (1964)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Took, C., Mandic, D.: The quaternion LMS algorithm for adaptive filtering of hypercomplex processes. IEEE Trans. Sig. Process. 57(4), 1316–1327 (2009)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Took, C., Mandic, D.: Quaternion-valued stochastic gradient-based adaptive IIR filtering. IEEE Tran. Sig. Process. 58(7), 3895–3901 (2010)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Took, C., Mandic, D.: A quaternion widely linear adaptive filter. IEEE Trans. Sig. Process. 58(8), 4427–4431 (2010)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Took, C., Mandic, D., Aihara, K.: Quaternion-valued short term forecasting of wind profile. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE, July 2010Google Scholar
  21. 21.
    Took, C., Strbac, G., Aihara, K., Mandic, D.: Quaternion-valued short-term joint forecasting of three-dimensional wind and atmospheric parameters. Renew. Energy 36(6), 1754–1760 (2011)CrossRefGoogle Scholar
  22. 22.
    Xia, Y., Jahanchahi, C., Mandic, D.: Quaternion-valued echo state networks. IEEE Trans. Neural Netw. Learn. Syst. 26(4), 663–673 (2015)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Xu, D., Xia, Y., Mandic, D.: Optimization in quaternion dynamic systems: gradient, Hessian, and learning algorithms. IEEE Trans. Neural Netw. Learn. Syst. 27(2), 249–261 (2016)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Department of Computer and Software EngineeringPolytechnic University TimişoaraTimişoaraRomania

Personalised recommendations