Numerical Algorithms

, 53:133 | Cite as

The complex step approximation to the Fréchet derivative of a matrix function

  • Awad H. Al-Mohy
  • Nicholas J. Higham
Original Paper


We show that the Fréchet derivative of a matrix function f at A in the direction E, where A and E are real matrices, can be approximated by Im f(A + ihE)/h for some suitably small h. This approximation, requiring a single function evaluation at a complex argument, generalizes the complex step approximation known in the scalar case. The approximation is proved to be of second order in h for analytic functions f and also for the matrix sign function. It is shown that it does not suffer the inherent cancellation that limits the accuracy of finite difference approximations in floating point arithmetic. However, cancellation does nevertheless vitiate the approximation when the underlying method for evaluating f employs complex arithmetic. The ease of implementation of the approximation, and its superiority over finite differences, make it attractive when specialized methods for evaluating the Fréchet derivative are not available, and in particular for condition number estimation when used in conjunction with a block 1-norm estimation algorithm.


Fréchet derivative Matrix function Complex step approximation Complex arithmetic Finite difference Matrix sign function Condition number estimation Block 1-norm estimator 

Mathematics Subject Classifications (2000)

15A60 65F30 


  1. 1.
    Al-Mohy, A.H., Higham, N.J.: Computing the Fréchet derivative of the matrix exponential, with an application to condition number estimation. SIAM J. Matrix Anal. Appl. 30(4), 1639–1657 (2009)MATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Al-Mohy, A.H., Higham, N.J.: A new scaling and squaring algorithm for the matrix exponential. MIMS EPrint 2009.9, Manchester Institute for Mathematical Sciences, The University of Manchester, UK. Revised April 2009 (2009) (To appear in SIAM J. Matrix Anal. Appl.)Google Scholar
  3. 3.
    Cox, M.G., Harris, P.M.: Numerical analysis for algorithm design in metrology. Software Support for Metrology Best Practice Guide No. 11, National Physical Laboratory, Teddington (2004)Google Scholar
  4. 4.
    Davies, P.I., Higham, N.J.: A Schur–Parlett algorithm for computing matrix functions. SIAM J. Matrix Anal. Appl. 25(2), 464–485 (2003)MATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Demmel, J.W.: On condition numbers and the distance to the nearest ill-posed problem. Numer. Math. 51, 251–289 (1987)MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Guo, C.H., Higham, N.J.: A Schur–Newton method for the matrix pth root and its inverse. SIAM J. Matrix Anal. Appl. 28(3), 788–804 (2006)MATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Higham, N.J.: The Matrix Function Toolbox.
  8. 8.
    Higham, N.J.: Exploiting fast matrix multiplication within the level 3 BLAS. ACM Trans. Math. Softw. 16(4), 352–368 (1990)MATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Higham, N.J.: Stability of a method for multiplying complex matrices with three real matrix multiplications. SIAM J. Matrix Anal. Appl. 13(3), 681–687 (1992)MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Higham, N.J.: Accuracy and Stability of Numerical Algorithms, 2nd edn. Society for Industrial and Applied Mathematics, Philadelphia (2002)MATHGoogle Scholar
  11. 11.
    Higham, N.J.: The scaling and squaring method for the matrix exponential revisited. SIAM J. Matrix Anal. Appl. 26(4), 1179–1193 (2005)MATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Higham, N.J.: Functions of Matrices: Theory and Computation. Society for Industrial and Applied Mathematics, Philadelphia (2008)MATHGoogle Scholar
  13. 13.
    Higham, N.J., Tisseur, F.: A block algorithm for matrix 1-norm estimation, with an application to 1-norm pseudospectra. SIAM J. Matrix Anal. Appl. 21(4), 1185–1201 (2000)MATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Kågström, B., Ling, P., Van Loan, C.F.: GEMM-based level 3 BLAS: high performance model implementations and performance evaluation benchmark. ACM Trans. Math. Softw. 24(3), 268–302 (1998)MATHCrossRefGoogle Scholar
  15. 15.
    Kelley, C.T.: Solving Nonlinear Equations with Newton’s Method. Society for Industrial and Applied Mathematics, Philadelphia (2003)MATHGoogle Scholar
  16. 16.
    Koikari, S.: An error analysis of the modified scaling and squaring method. Comput. Math. Appl. 53, 1293–1305 (2007)MATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Koikari, S.: Algorithm 894: On a block Schur–Parlett algorithm for ϕ-functions based on the sep-inverse estimate. ACM Trans. Math. Software 36(2), Article 12 (2009)Google Scholar
  18. 18.
    Lai, K.L., Crassidis, J.L.: Extensions of the first and second complex-step derivative approximations. J. Comput. Appl. Math. 219, 276–293 (2008)MATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    Lyness, J.N.: Numerical algorithms based on the theory of complex variable. In: Proceedings of the 1967 22nd National Conference, pp. 125–133. ACM, New York (1967)CrossRefGoogle Scholar
  20. 20.
    Lyness, J.N., Moler, C.B.: Numerical differentiation of analytic functions. SIAM J. Numer. Anal. 4(2), 202–210 (1967)MATHCrossRefMathSciNetGoogle Scholar
  21. 21.
    Martins, J.R.R.A., Sturdza, P., Alonso, J.J.: The connection between the complex-step derivative approximation and algorithmic differentiation. AIAA paper AIAA-2001-0921 (2001)Google Scholar
  22. 22.
    Martins, J.R.R.A., Sturdza, P., Alonso, J.J.: The complex-step derivative approximation. ACM Trans. Math. Softw. 29(3), 245–262 (2003)MATHCrossRefMathSciNetGoogle Scholar
  23. 23.
    Najfeld, I., Havel, T.F.: Derivatives of the matrix exponential and their computation. Adv. Appl. Math. 16, 321–375 (1995)MATHCrossRefMathSciNetGoogle Scholar
  24. 24.
    Shampine, L.F.: Accurate numerical derivatives in MATLAB. ACM Trans. Math. Softw. 33(4), Article 26, 17 pages (2007)Google Scholar
  25. 25.
    Skaflestad, B., Wright, W.M.: The scaling and modified squaring method for matrix functions related to the exponential. Appl. Numer. Math. 59, 783–799 (2009)MATHCrossRefMathSciNetGoogle Scholar
  26. 26.
    Squire, W., Trapp, G.: Using complex variables to estimate derivatives of real functions. SIAM Rev. 40(1), 110–112 (1998)MATHCrossRefMathSciNetGoogle Scholar
  27. 27.
    Sun, J.: Perturbation analysis of the matrix sign function. Linear Algebra Appl. 250, 177–206 (1997)MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC. 2009

Authors and Affiliations

  1. 1.School of MathematicsThe University of ManchesterManchesterUK

Personalised recommendations