Abstract
In this paper, we present the full deduction of the method to evaluate the Hessian matrix of a complex-valued feedforward neural network. The Hessian matrix is composed of the second derivatives of the error function of the network, and has many applications in network training and pruning algorithms, as well as in fast re-training of the network after a small change in training data. The software implementation of the presented method is straightforward.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Becker S, LeCun Y (1989) Improving the convergence of back-propagation learning with second-order methods. In: Touretzky D, Hinton G, Sejnowski T (eds) Proceedings of the 1988 connectionist models summer school. Morgan Kaufman, San Mateo, pp 29–37
Bishop CM (1991) A fast procedure for retraining the multilayer perceptron. Int J Neural Syst 2(3):229–236
Bishop CM (1992) Exact calculation of the hessian matrix for the multi-layer perceptron. Neural Comput 4(4):494–501
Bishop CM (1995) Neural networks for pattern recognition. Oxford University Press, Inc., New York
Hirose A (2013) Application fields and fundamental merits of complex-valued neural networks. In: Hirose A (ed) Complex-valued neural networks, Wiley, Hoboken, pp 1–31
Kreutz-Delgado K (2009) The complex gradient operator and the CR-calculus. arXiv preprint arXiv:0906.4835
LeCun Y, Denker JS, Solla S, Howard RE, Jackel LD (1990) Optimal brain damage. In: Touretzky D (ed) Advances in neural information processing systems (NIPS 1989), vol 2. Morgan Kaufman, Denver
Mackay DJC (1992) A practical Bayesian framework for backpropagation networks. Neural Comput 4(3):448–472
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Popa, CA. (2016). Exact Hessian Matrix Calculation for Complex-Valued Neural Networks. In: Balas, V., C. Jain, L., Kovačević, B. (eds) Soft Computing Applications. SOFA 2014. Advances in Intelligent Systems and Computing, vol 356. Springer, Cham. https://doi.org/10.1007/978-3-319-18296-4_36
Download citation
DOI: https://doi.org/10.1007/978-3-319-18296-4_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-18295-7
Online ISBN: 978-3-319-18296-4
eBook Packages: EngineeringEngineering (R0)