Computation of large-scale quadratic forms and transfer functions using the theory of moments, quadrature and Padé approximation

  • Zhaojun Bai
  • Gene Golub
Part of the NATO Science Series book series (NAII, volume 75)


Large-scale problems in scientific and engineering computing often require solutions involving large-scale matrices. In this paper, we survey numerical techniques for solving a variety of large-scale matrix computation problems, such as computing the entries and trace of the inverse of a matrix, computing the determinant of a matrix, and computing the transfer function of a linear dynamical system.

Most of these matrix computation problems can be cast as problems of computing quadratic forms u T ƒ(A)u involving a matrix functional ƒ(A). It can then be transformed into a Riemann-Stieltjes integral, which brings the theory of moments, orthogonal polynomials and the Lanczos process into the picture. For computing the transfer function, we focus on numerical techniques based on Padé approximation via the Lanczos process and the moment-matching property. We will also discuss issues related to the development of efficient numerical algorithms, including using Monte Carlo simulation.


Krylov Subspace Linear Dynamical System Tridiagonal Matrix Hankel Matrix Expansion Point 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    B. D. O. Anderson, A system theory criterion for positive real matrices, SIAM J. Control. 5 (1967), 171–182.MathSciNetzbMATHCrossRefGoogle Scholar
  2. [2]
    A. C. Antoulas and D. C. Sorensen, Approximation of large-scale dynamical systems: An overview, Technical report, Electrical and Computer Engineering, Rice University, Houston, TX, Feb. 2001.Google Scholar
  3. [3]
    Z. Bai, J. Demmel, J. Dongarra, A. Ruhe, and H. van der Vorst, eds., Templates for the Solution of Algebraic Eigenvalue Problems: A Practical Guide, SIAM, Philadelphia, 2000.zbMATHGoogle Scholar
  4. [4]
    Z. Bai, Krylov subspace techniques for reduced-order modeling of large-scale dynamical systems, to appear in Appl. Numer. Math. (2002).Google Scholar
  5. [5]
    Z. Bai, M. Fahey, and G. Golub, Some large scale matrix computation problems, J. Comput. Appl. Math. 74 (1996), 71–89.MathSciNetzbMATHCrossRefGoogle Scholar
  6. [6]
    Z. Bai, M. Fahey, G. Golub, E. Menon, and M. Richter, Computing partial eigenvalue sum in electronic structure calculations, Scientific Computing and Computational Mathematics Program SCCM-98-03, Stanford University, 1998.Google Scholar
  7. [7]
    Z. Bai and G. Golub, Bounds for the trace of the inverse and the determinant of symmetric positive definite matrices, Ann. Numer. Math. 4 (1997), 29–38.MathSciNetzbMATHGoogle Scholar
  8. [8]
    Z. Bai and G. Golub, Some unusual matrix eigenvalue problems, in: Proceedings VECPAR’98-Third Internat. Conf. for Vector and Parallel Processing (J. Palma,J. Dongarra, and V. Hernandez, eds.), Lecture Notes in Comput. Sci. 1573, Springer, 1999, 4–19.CrossRefGoogle Scholar
  9. [9]
    Z. Bai, R. D. Slone, W. T. Smith, and Q. Ye, Error bound for reduced system model by Padé approximation via the Lanczos process, IEEE Trans. Computer-Aided Design 18 (1999), 133–141.CrossRefGoogle Scholar
  10. [10]
    Z. Bai and Q. Ye, Error estimation of the Padé approximation of transfer functions via the Lanczos process, Electron. Trans. Numer. Anal. 7 (1998), 1–17.MathSciNetzbMATHGoogle Scholar
  11. [11]
    G. A. Baker, Jr. and P. Graves-Morris, Padé Approximants, Cambridge University Press, 1996.zbMATHCrossRefGoogle Scholar
  12. [12]
    R. P. Barry and R. K. Pace, Monte Carlo estimates of the log determinant of large sparse matrices, Linear Algebra Appl. 289 (1999), 41–54.MathSciNetzbMATHCrossRefGoogle Scholar
  13. [13]
    D. L. Boley, Krylov subspace methods on state-space control models, Circuits Systems Signal Process. 13 (1994), 733–758.MathSciNetzbMATHCrossRefGoogle Scholar
  14. [14]
    S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan, Linear Matrix Inequalities in System and Control Theory, Classics Appl. Math. 15, SIAM, Philadelphia, 1994.CrossRefGoogle Scholar
  15. [15]
    A. Bultheel and M. van Barvel, Padé techniques for model reduction in linear system theory: a survey, J. Comput. Appl. Math. 14 (1986), 401–438.MathSciNetCrossRefGoogle Scholar
  16. [16]
    D. Calvetti, G. H. Golub, and L. Reichel, A computable error bound for matrix func-tionals, J. Comput. Appl. Math. 103 (1999), 301–306.MathSciNetzbMATHCrossRefGoogle Scholar
  17. [17]
    D. Calvetti, S. Morigi, L. Reichel, and F. Sgallari, Computable error bounds and estimates for the conjugate gradient method, Numer. Algorithms 25 (2000), 75–88.MathSciNetzbMATHCrossRefGoogle Scholar
  18. [18]
    E. Chiprout and M. S. Nakhla, Asymptotic Waveform Evaluation, Kluwer, Dordrecht, 1994.zbMATHGoogle Scholar
  19. [19]
    R. W. Clough and J. Penzien, Dynamics of Structures, McGraw-Hill, New York, 1975.zbMATHGoogle Scholar
  20. [20]
    R. R. Craig, Jr., Structural Dynamics: An Introduction to Computer Methods, Wiley, New York, 1981.Google Scholar
  21. [21]
    G. Dahlquist, S. C. Eisenstat, and G. H. Golub, Bounds for the error of linear systems of equations using the theory of moments, J. Math. Anal. Appl. 37 (1972), 151–166.MathSciNetzbMATHCrossRefGoogle Scholar
  22. [22]
    P. Davis and P. Rabinowitz, Methods of Numerical Integration, Academic Press, New York, 1984.zbMATHGoogle Scholar
  23. [23]
    C. De Villemagne and R. E. Skelton, Model reductions using a projection formulation, Internat. J. Control 46 (1987), 2141–2169.MathSciNetCrossRefGoogle Scholar
  24. [24]
    S. Dong and K. Liu, Stochastic estimation with z2 noise, Phys. Lett. B 328 (1994), 130–136.CrossRefGoogle Scholar
  25. [25]
    M. Fahey, Numerical computation of quadratic forms involving large scale matrix functions, Ph.D. thesis, University of Kentucky, 1998.Google Scholar
  26. [26]
    P. Feldman and R. W. Preund, Efficient linear circuit analysis by Padé approximation via the Lanczos process, IEEE Trans. Computer-Aided Design 14 (1995), 639–649.CrossRefGoogle Scholar
  27. [27]
    R. W. Freund, Reduced-order modeling techniques based on Krylov subspaces and their use in circuit simulation, in: Applied and Computational Control, Signals, and Circuits, Vol. 1 (B. N. Datta, ed.), Birkhäuser, Boston, 1999, 435–498.CrossRefGoogle Scholar
  28. [28]
    R. W. Freund, Krylov-subspace methods for reduced-order modeling in circuit simulation, J. Comput. Appl. Math. 123 (2000), 395–421.MathSciNetzbMATHCrossRefGoogle Scholar
  29. [29]
    R. W. Preund, M. H. Gutknecht, and N. M. Nachtigal, An implementation of the look-ahead Lanczos algorithm for non-Hermitian matrices, SIAM J. Sci. Comput. 14 (1993), 137–158.MathSciNetCrossRefGoogle Scholar
  30. [30]
    R. W. Preund and N. M. Nachtigal, QMRPACK: a package of QMR algorithms, ACM Trans. Math. Software 22 (1996), 46–77.MathSciNetCrossRefGoogle Scholar
  31. [31]
    A. Frommer, T. Lippert, B. Medeke, and K. Schilling, eds., Numerical Challenges in Lattice Quantum Chromodynamics, Lecture Notes in Comput. Sci. and Engrg. 15, Springer, Berlin, 2000.zbMATHGoogle Scholar
  32. [32]
    K. Gallivan, E. Grimme, and P. Van Dooren, Asymptotic waveform evaluation via a Lanczos method, Appl. Math. Lett. 7 (1994), 75–80.zbMATHCrossRefGoogle Scholar
  33. [33]
    W. Gautschi, A survey of Gauss-Christoffel quadrature formulae, in: E. B. Christoffel-the Influence of His Work on Mathematics and the Physical Sciences (P. L. Bultzer and F. Feher, eds.), Birkhäuser, Boston, 1981, 73–157.Google Scholar
  34. [34]
    G. Golub, Some modified matrix eigenvalue problems, SIAM Rev. 15 (1973), 318–334.MathSciNetzbMATHCrossRefGoogle Scholar
  35. [35]
    G. Golub and G. Meurant, Matrics, moments and quadrature, in: Proc. 15th Dundee Conference, June 1993 (D. F. Griffiths and G. A. Watson, eds.), Longman Sci. Tech., Harlow, 1994.Google Scholar
  36. [36]
    G. Golub and G. Meurant, Matrices, moments and quadrature II: How to compute the norm of the error iterative methods, BIT 37 (1997), 687–705.MathSciNetzbMATHCrossRefGoogle Scholar
  37. [37]
    G. Golub and Z. Strakoš, Estimates in quadratic formulas, Numer. Algorithms 8 (1994), 241–268.MathSciNetzbMATHCrossRefGoogle Scholar
  38. [38]
    G. Golub and U. Von Matt, Generalized cross-validation for large scale problems, J. Comput. Graph. Statist. 6 (1997), 1–34.MathSciNetGoogle Scholar
  39. [39]
    W. B. Gragg, Matrix interpretations and applications of the continued fraction algorithm, Rocky Mountain J. Math. 5 (1974), 213–225.MathSciNetCrossRefGoogle Scholar
  40. [40]
    W. B. Gragg and A. Lindquist, On the partial realization problem, Linear Algebra Appl. 50 (1983), 227–319.MathSciNetCrossRefGoogle Scholar
  41. [41]
    E. Grimme, Krylov projection methods for model reduction, Ph.D. thesis, University of Illinois at Urbana-Champaign, 1997.Google Scholar
  42. [42]
    M. Hutchinson, A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines, Comm. Statist. Simulation Comput. 18 (3) (1989), 1059–1076.MathSciNetzbMATHCrossRefGoogle Scholar
  43. [43]
    I. M. Jaimoukha and E. M. Kasenally, Oblique projection methods for large scale model reduction, SIAM J. Matrix Anal Appl. 16 (1997), 602–627.MathSciNetCrossRefGoogle Scholar
  44. [44]
    T. Kailath, Linear Systems, Prentice-Hall, New York, 1980.zbMATHGoogle Scholar
  45. [45]
    L. Komzsik, MSC/NASTRAN, Numerical Methods User’s Guide, Version 70.5, MacNeal-Schwendler Corp., Los Angeles, 1998.Google Scholar
  46. [46]
    C. Lanczos, An iteration method for the solution of the eigenvalue problem of linear differential and integral operators, J. Res. Nat. Bur. Standards 45 (1950), 225–280.MathSciNetCrossRefGoogle Scholar
  47. [47]
    A. J. Laub, Efficient calculation of frequency response matrices from state space models, ACM Trans. Math. Software 12 (1986), 26–33.CrossRefGoogle Scholar
  48. [48]
    H. Madrid, private communication, 1995.Google Scholar
  49. [49]
    G. Meurant, A review of the inverse of symmetric tridiagonal and block tridiagonal matrices, SIAM J. Math. Anal. Appl. 13 (1992), 707–728.MathSciNetzbMATHCrossRefGoogle Scholar
  50. [50]
    H.-D. Meyer and S. Pal, A band-Lanczos method for computing matrix elements of a resolvent, J. Chem. Phys. 91 (1989), 6195–6204.MathSciNetCrossRefGoogle Scholar
  51. [51]
    A. Odabasioglu, M. Celik, and L. T. Pileggi, Practical considerations for passive reduction of RLC circuits, in: Proc. Internat. Conf, on Computer-Aided Design, 1999, 214–219.Google Scholar
  52. [52]
    B. Parlett, Reduction to tridiagonal form and minimal realizations, SIAM J. Math. Anal. Appl. 13 (2) (1992), 567–593.MathSciNetzbMATHCrossRefGoogle Scholar
  53. [53]
    L. T. Pillage and R. A. Rohrer, Asymptotic waveform evaluation for timing analysis, IEEE Trans. Computer-Aided Design 9 (1990), 353–366.CrossRefGoogle Scholar
  54. [54]
    D. Pollard, Convergence of Stochastic Processes, Springer Ser. Statist. 15, Springer, Berlin, 1984.zbMATHCrossRefGoogle Scholar
  55. [55]
    A. E. Ruehli, Equivalent circuit models for three-dimensional multiconductor systems, IEEE Trans. Microwave Theory and Tech. 22 (1974), 216–221.CrossRefGoogle Scholar
  56. [56]
    B. Sapoval, Th. Gobron, and A. Margolina, Vibrations of fractal drums, Phys. Rev. Lett. 67 (21) (1991), 2974–2977.CrossRefGoogle Scholar
  57. [57]
    J. C. Sexton and D. H. Weingarten, The numerical estimation of the error induced by the valence approximation, Nuclear Phys. B Proc. Suppl. (1994), xx.Google Scholar
  58. [58]
    G. Szegö, Orthogonal Polynomials, 4th ed., Amer. Math. Soc. Colloq. Publ. 23, Amer. Math. Soc., Providence, RI, 1975.zbMATHGoogle Scholar
  59. [59]
    S. Van Huffel and J. Vandewalle, The Total Least Squares Problem: Computational Aspects and Analysis, SIAM, Philadelphia, PA, 1991.zbMATHCrossRefGoogle Scholar
  60. [60]
    J. Vlach and K. Singhal, Computer Methods for Circuit Analysis and Design, Van Nostrand, New York, 1994.Google Scholar
  61. [61]
    K. Willcox, J. Peraire, and J. White, An Arnoldi approach for generalization of reduced-order models for turbomachinery, FDRL TR-99-1, Fluid Dynamics Research Lab., Massachusetts Institute of Technology, 1999.Google Scholar
  62. [62]
    M. N. Wong, Finding tr(A -1) for large A by low discrepancy sampling, Presentation at the Fourth International Conference on Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing, Hong Kong, China, 2000.Google Scholar
  63. [63]
    S. Y. Wu, J. A. Cocks, and C. S. Jayanthi, An accelerated inversion algorithm using the resolvent matrix method, Comput. Phys. Comm. 71 (1992), 125–133.MathSciNetCrossRefGoogle Scholar
  64. [64]
    Q. Ye, A convergence analysis for nonsymmetric Lanczos algorithms, Math. Comp. 56 (1991), 677–691.MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2002

Authors and Affiliations

  • Zhaojun Bai
    • 1
  • Gene Golub
    • 2
  1. 1.Department of Computer ScienceUniversity of CaliforniaDavisUSA
  2. 2.Department of Computer ScienceStanford UniversityStanfordUSA

Personalised recommendations