Shifted extended global Lanczos processes for trace estimation with application to network analysis

In memory of Mohammed Bellalij

Abstract

The need to estimate upper and lower bounds for matrix functions of the form \({\mathrm{trace}}(W^Tf(A)V)\), where the matrix \(A\in {{\mathbb {R}}}^{n\times n}\) is large and sparse, \(V,W\in {{\mathbb {R}}}^{n\times s}\) are block vectors with \(1\le s\ll n\) columns, and f is a function arises in many applications, including network analysis and machine learning. This paper describes the shifted extended global symmetric and nonsymmetric Lanczos processes and how they can be applied to approximate the trace. These processes compute approximations in the union of Krylov subspaces determined by positive powers of A and negative powers of \(A-\sigma I_n\), where the shift \(\sigma\) is a user-chosen parameter. When A is nonsymmetric, transposes of these powers also are used. When A is symmetric and \(W=V\), we describe how error bounds or estimates of bounds for the trace can be computed by pairs of Gauss and Gauss-Radau quadrature rules, or by pairs of Gauss and anti-Gauss quadrature rules. These Gauss-type quadrature rules are defined by recursion coefficients for the shifted extended global Lanczos processes. Gauss and anti-Gauss quadrature rules also can be applied to give estimates of error bounds for the trace when A is nonsymmetric and \(W\ne V\). Applications to the computation of the Estrada index for networks and to the nuclear norm of a large matrix are presented. Computed examples show the shifted extended symmetric and nonsymmetric Lanczos processes to produce accurate approximations in fewer steps than the standard symmetric and nonsymmetric global Lanczos processes, respectively.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

Notes

  1. 1.

    If the matrix \(A\in {{\mathbb {R}}}^{n\times n}\) is symmetric the operator \(\backslash\) first seeks to compute the Cholesky factorization of \(A-\sigma I_n\). If this is not possible, because \(A-\sigma I_n\) is not positive definite, then an LU factorization is determined by Gaussian elimination with partial pivoting. The computed factorization is used to solve the linear system of equations with the matrix \(A-\sigma I_n\).

References

  1. 1.

    Alqahtani, H., Reichel, L.: Simplified anti-Gauss quadrature rules with applications in linear algebra. Numer. Algorithms 77, 577–602 (2018)

    MathSciNet  Article  Google Scholar 

  2. 2.

    Alqahtani, H., Reichel, L.: Generalized block anti-Gauss quadrature rules. Numer. Math. 143, 605–648 (2019)

    MathSciNet  Article  Google Scholar 

  3. 3.

    Baglama, J., Calvetti, D., Reichel, L.: IRBL: An implicitly restarted block-Lanczos method for large-scale Hermitian eigenproblems. SIAM J. Sci. Comput. 24, 1650–1677 (2003)

    MathSciNet  Article  Google Scholar 

  4. 4.

    Baglama, J., Calvetti, D., Reichel, L.: Algorithm 827: irbleigs: A MATLAB program for computing a few eigenpairs of a large sparse Hermitian matrix. ACM Trans. Math. Softw. 29, 337–348 (2003)

    MathSciNet  Article  Google Scholar 

  5. 5.

    Bai, Z., Golub, G.: Bounds for the trace of the inverse and the determinant of symmetric positive definite matrices. Ann. Numer. Math. 4, 29–38 (1997)

    MathSciNet  Google Scholar 

  6. 6.

    Baroni, S., Gebauer, R., Malcioglu, O.B., Saad, Y., Umari, P., Xian, J.: Harnessing molecular excited states with Lanczos chains. J. Phys. Condens. Mat., 22, Art. 074204 (2010)

  7. 7.

    Bellalij, M., Reichel, L., Rodriguez, G., Sadok, H.: Bounding matrix functionals via partial global block Lanczos decomposition. Appl. Numer. Math. 94, 127–139 (2015)

    MathSciNet  Article  Google Scholar 

  8. 8.

    Bentbib, A.H., El Guide, M., Jbilou, K.: The block Lanczos algorithm for linear ill-posed problems. Calcolo 54, 711–732 (2017)

    MathSciNet  Article  Google Scholar 

  9. 9.

    Bentbib, A.H., El Ghomari, M., Jagels, C., Jbilou, K., Reichel, L.: The extended global Lanczos method for matrix function approximation. Electron. Trans. Numer. Anal. 50, 144–163 (2018)

    MathSciNet  Article  Google Scholar 

  10. 10.

    Bouyouli, R., Jbilou, K., Sadaka, R., Sadok, H.: Convergence properties of some block Krylov subspace methods for multiple linear systems. J. Comput. Appl. Math. 196, 498–511 (2006)

    MathSciNet  Article  Google Scholar 

  11. 11.

    Caldarelli, G.: Scale-Free Networks. Oxford University Press, Oxford (2007)

    Google Scholar 

  12. 12.

    De la Cruz Cabrera, O., Matar, M., Reichel, L.: Analysis of directed networks via the matrix exponential. J. Comput. Appl. Math 355, 182–192 (2019)

    MathSciNet  Article  Google Scholar 

  13. 13.

    Davis, T., Hu, Y.: The SuiteSparse Matrix Collection. https://sparse.tamu.edu

  14. 14.

    Druskin, V., Knizhnerman, L.: Extended Krylov subspace approximations of the matrix square root and related functions. SIAM J. Matrix Anal. Appl. 19, 755–771 (1998)

    MathSciNet  Article  Google Scholar 

  15. 15.

    Estrada, E.: Characterization of the folding degree of proteins. Bioinformatics 18, 697–704 (2002)

    Article  Google Scholar 

  16. 16.

    Estrada, E.: The Structure of Complex Networks: Theory and Applications. Oxford University Press, Oxford (2011)

    Google Scholar 

  17. 17.

    Estrada, E., Fox, M., Higham, D., Oppo, G.L. (eds.): Network Science. Complexity in Nature and Technology, Springer, New York (2010)

  18. 18.

    Estrada, E., Hatano, N.: Statistical-mechanical approach to subgraph centrality in complex networks. Chem. Phys. Lett. 439, 247–251 (2007)

    Article  Google Scholar 

  19. 19.

    Estrada, E., Higham, D.J.: Network properties revealed through matrix functions. SIAM Rev. 52, 696–714 (2010)

    MathSciNet  Article  Google Scholar 

  20. 20.

    Estrada, E., Rodriguez-Velazquez, J.A.: Subgraph centrality in complex networks. Phys. Rev. E. 71, Art. 056103 (2005)

  21. 21.

    Fenu, C., Martin, D., Reichel, L., Rodriguez, G.: Block Gauss and anti-Gauss quadrature with application to networks. SIAM J. Matrix Anal. Appl. 34, 1655–1684 (2013)

    MathSciNet  Article  Google Scholar 

  22. 22.

    Gautschi, W.: Orthogonal Polynomials: Computation and Approximation. Oxford University Press, Oxford (2004)

    Google Scholar 

  23. 23.

    Golub, G.H., Meurant, G.: Matrices, Moments and Quadrature with Applications. Princeton University Press, Princeton (2010)

    Google Scholar 

  24. 24.

    Golub, G.H., Van Loan, C.F.: Matrix Computations, 4th edn. Johns Hopkins University Press, Baltimore (2013)

    Google Scholar 

  25. 25.

    Han, I., Malioutov, D., Shin, J.: Large-scale log-determinant computation through stochastic Chebyshev expansions. In: Bach, F., Blei, D. (Eds.) Proceedings of The 32nd International Conference on Machine Learning, Lille, France, 2015, JMLR Workshop and Conference Proceedings, 37, pp. 908–917 (2015)

  26. 26.

    Jagels, C., Jbilou, K., Reichel, L.: The extended global Lanczos method, Gauss-Radau quadrature, and matrix function approximation, J. Comput. Appl. Math. 381 (2021) Art. 113027

  27. 27.

    Jagels, C., Reichel, L.: The structure of matrices in rational Gauss quadrature. Math. Comput. 82, 2035–2060 (2013)

    MathSciNet  Article  Google Scholar 

  28. 28.

    Jbilou, K., Messaoudi, A., Sadok, H.: Global FOM and GMRES algorithms for matrix equations. Appl. Numer. Math. 31, 49–63 (1999)

    MathSciNet  Article  Google Scholar 

  29. 29.

    Jbilou, K., Sadok, H., Tinzefte, A.: Oblique projection methods for multiple linear systems. Electron. Trans. Num. Anal. 20, 119–138 (2005)

    Google Scholar 

  30. 30.

    Laurie, D.P.: Anti-Gaussian quadrature formulas. Math. Comp. 65, 735–747 (1996)

    MathSciNet  Article  Google Scholar 

  31. 31.

    Newman, M.E.J.: Networks: An Introduction. Oxford University Press, Oxford (2010)

    Google Scholar 

  32. 32.

    Ngo, T.T., Bellalij, M., Saad, Y.: The trace ratio optimization problem. SIAM Rev. 54, 545–569 (2012)

    MathSciNet  Article  Google Scholar 

  33. 33.

    Pozza, S., Pranić, M.S., Strakoš, Z.: The Lanczos algorithm and complex Gauss quadrature. Electron. Trans. Numer. Anal. 50, 1–19 (2018)

    MathSciNet  Article  Google Scholar 

  34. 34.

    Saad, Y., Chelikowsky, J., Shontz, S.: Numerical methods for electronic structure calculations of materials. SIAM Rev. 52, 3–54 (2010)

    MathSciNet  Article  Google Scholar 

  35. 35.

    Sorensen, D.C.: Numerical methods for large eigenvalue problems. Acta Numer. 11, 519–584 (2002)

    MathSciNet  Article  Google Scholar 

  36. 36.

    Ubaru, S., Chen, J., Saad, Y.: Fast estimation of \(tr(f(A))\) via stochastic Lanczos quadrature. SIAM J. Matrix Anal. Appl. 38, 1075–1099 (2017)

    MathSciNet  Article  Google Scholar 

  37. 37.

    Varga, R.S.: Geršgorin and His Circles. Springer, Berlin (2004)

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank Carl Jagels and Giuseppe Rodriguez for carefully reading the manuscript and for comments that lead to an improved presentation. Research by LR was supported in part by NSF Grant DMS-1720259.

Author information

Affiliations

Authors

Corresponding author

Correspondence to L. Reichel.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Bentbib, A.H., El Ghomari, M., Jbilou, K. et al. Shifted extended global Lanczos processes for trace estimation with application to network analysis. Calcolo 58, 4 (2021). https://doi.org/10.1007/s10092-020-00395-1

Download citation

Keywords

  • Shifted extended Krylov subspace
  • Block Lanczos process
  • Extended global symmetric Lanczos process
  • Extended global nonsymmetric Lanczos process
  • Gauss quadrature
  • Anti-Gauss quadrature
  • Network analysis
  • Estrada index
  • Trace estimation
  • Nuclear norm estimation

Mathematics subject classification

  • 65D32
  • 65F60