Applications of Trace Estimation Techniques

  • Shashanka UbaruEmail author
  • Yousef Saad
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11087)


We discuss various applications of trace estimation techniques for evaluating functions of the form \(\mathtt {tr}(f(A))\) where f is certain function. The first problem we consider that can be cast in this form is that of approximating the Spectral density or Density of States (DOS) of a matrix. The DOS is a probability density distribution that measures the likelihood of finding eigenvalues of the matrix at a given point on the real line, and it is an important function in solid state physics. We also present a few non-standard applications of spectral densities. Other trace estimation problems we discuss include estimating the trace of a matrix inverse \(\mathtt {tr}(A^{-1})\), the problem of counting eigenvalues and estimating the rank, and approximating the log-determinant (trace of log function). We also discuss a few similar computations that arise in machine learning applications. We review two computationally inexpensive methods to compute traces of matrix functions, namely, the Chebyshev expansion and the Lanczos Quadrature methods. A few numerical examples are presented to illustrate the performances of these methods in different applications.



This work was supported by NSF under grant CCF-1318597.


  1. 1.
    Alter, O., Brown, P.O., Botstein, D.: Singular value decomposition for genome-wide expression data processing and modeling. Proc. Nat. Acad. Sci. 97(18), 10101–10106 (2000)CrossRefGoogle Scholar
  2. 2.
    Andoni, A., Krauthgamer, R., Razenshteyn, I.: Sketching and embedding are equivalent for norms. In: Proceedings of the Forty-Seventh Annual ACM on Symposium on Theory of Computing, pp. 479–488. ACM (2015)Google Scholar
  3. 3.
    Aune, E., Simpson, D.P., Eidsvik, J.: Parameter estimation in high dimensional Gaussian distributions. Stat. Comput. 24(2), 247–263 (2014)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Avron, H., Toledo, S.: Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix. J. ACM 58(2), 8 (2011)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Bai, Z., Fahey, G., Golub, G.: Some large-scale matrix computation problems. J. Comput. Appl. Math. 74(1), 71–89 (1996)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Bai, Z., Golub, G.H.: Bounds for the trace of the inverse and the determinant of symmetric positive definite matrices. Annal. Numer. Math. 4, 29–38 (1996)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Bekas, C., Kokiopoulou, E., Saad, Y.: An estimator for the diagonal of a matrix. Appl. Numer. Math. 57(11), 1214–1229 (2007)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Boutsidis, C., Drineas, P., Kambadur, P., Kontopoulou, E.-M., Zouzias, A.: A randomized algorithm for approximating the log determinant of a symmetric positive definite matrix. Linear Algebra Appl. 533, 95–119 (2017)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Carbó-Dorca, R.: Smooth function topological structure descriptors based on graph-spectra. J. Math. Chem. 44(2), 373–378 (2008)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Davis, T.A., Hu, Y.: The University of Florida sparse matrix collection. ACM Trans. Math. Softw. (TOMS) 38(1), 1 (2011)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Di Napoli, E., Polizzi, E., Saad, Y.: Efficient estimation of eigenvalue counts in an interval. ArXiv preprint arXiv:1308.4275 (2013)
  12. 12.
    Estrada, E.: Characterization of 3D molecular structure. Chem. Phys. Lett. 319(5), 713–718 (2000)CrossRefGoogle Scholar
  13. 13.
    Golub, G.H., Meurant, G.: Matrices, Moments and Quadrature with Applications. Princeton University Press, Princeton (2009)CrossRefGoogle Scholar
  14. 14.
    Golub, G.H., Strakoš, Z.: Estimates in quadratic formulas. Numer. Algorithms 8(2), 241–268 (1994)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Golub, G.H., Welsch, J.H.: Calculation of gauss quadrature rules. Math. Comput. 23(106), 221–230 (1969)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Han, I., Malioutov, D., Avron, H., Shin, J.: Approximating spectral sums of large-scale matrices using stochastic Chebyshev approximations. SIAM J. Sci. Comput. 39(4), A1558–A1585 (2017)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Han, I., Malioutov, D., Shin, J.: Large-scale log-determinant computation through stochastic Chebyshev expansions. In: Proceedings of the 32nd International Conference on Machine Learning, pp. 908–917 (2015)Google Scholar
  18. 18.
    Higham, N.J.: Functions of Matrices: Theory and Computation. SIAM, University City (2008)CrossRefGoogle Scholar
  19. 19.
    Hutchinson, M.F.: A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines. Commun. Stat.-Simul. Comput. 19(2), 433–450 (1990)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Kalantzis, V., Bekas, C., Curioni, A., Gallopoulos, E.: Accelerating data uncertainty quantification by solving linear systems with multiple right-hand sides. Numer. Algorithms 62(4), 637–653 (2013)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Kaufman, L., Rousseeuw, P.J.: Finding Groups in Data: An Introduction to Cluster Analysis. Wiley, Hoboken (1990)CrossRefGoogle Scholar
  22. 22.
    Li, Y., Nguyên, H.L., Woodruff, D.P.: On sketching matrix norms and the top singular vector. In: Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1562–1581. SIAM (2014)Google Scholar
  23. 23.
    Lin, L., Saad, Y., Yang, C.: Approximating spectral densities of large matrices. SIAM Rev. 58(1), 34–65 (2016)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Mason, J.C., Handscomb, D.C.: Chebyshev Polynomials. CRC Press, Boca Raton (2002)CrossRefGoogle Scholar
  25. 25.
    Musco, C., Netrapalli, P., Sidford, A., Ubaru, S., Woodruff, D.P.: Spectrum approximation beyond fast matrix multiplication: algorithms and hardness. arXiv preprint arXiv:1704.04163 (2017)
  26. 26.
    Newman, M.E.J.: Finding community structure in networks using the eigenvectors of matrices. Phys. Rev. E 74(3), 036104 (2006)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Rasmussen, C., Williams, C.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)zbMATHGoogle Scholar
  28. 28.
    Roosta-Khorasani, F., Ascher, U.: Improved bounds on sample size for implicit matrix trace estimators. Found. Comput. Math. 15(5), 1187–1212 (2015)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Rue, H., Held, L.: Gaussian Markov Random Fields: Theory and Applications. CRC Press, Boca Raton (2005)CrossRefGoogle Scholar
  30. 30.
    Silver, R., Röder, H.: Densities of states of mega-dimensional Hamiltonian matrices. Int. J. Mod. Phys. C 5(04), 735–753 (1994)CrossRefGoogle Scholar
  31. 31.
    Turek, I.: A maximum-entropy approach to the density of states within the recursion method. J. Phys. C: Solid State Phys. 21(17), 3251 (1988)CrossRefGoogle Scholar
  32. 32.
    Ubaru, S., Chen, J., Saad, Y.: Fast estimation of tr(f(A)) via stochastic Lanczos quadrature. SIAM J. Matrix Anal. Appl. 38, 1075–1099 (2017)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Ubaru, S., Saad, Y.: Fast methods for estimating the numerical rank of large matrices. In: Proceedings of the 33rd International Conference on Machine Learning, pp. 468–477 (2016)Google Scholar
  34. 34.
    Ubaru, S., Saad, Y., Seghouane, A.-K.: Fast estimation of approximate matrix ranks using spectral densities. Neural Comput. 29(5), 1317–1351 (2017)CrossRefGoogle Scholar
  35. 35.
    Wang, L.-W.: Calculating the density of states and optical-absorption spectra of large quantum systems by the plane-wave moments method. Phys. Rev. B 49(15), 10154 (1994)CrossRefGoogle Scholar
  36. 36.
    Weiße, A., Wellein, G., Alvermann, A., Fehske, H.: The kernel polynomial method. Rev. Mod. Phys. 78(1), 275 (2006)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Wu, L., Laeuchli, J., Kalantzis, V., Stathopoulos, A., Gallopoulos, E.: Estimating the trace of the matrix inverse by interpolating from the diagonal of an approximate inverse. J. Comput. Phys. 326, 828–844 (2016)MathSciNetCrossRefGoogle Scholar
  38. 38.
    Xi, Y., Li, R., Saad, Y.: Fast computation of spectral densities for generalized eigenvalue problems. arXiv preprint arXiv:1706.06610 (2017)
  39. 39.
    Zhang, Y., Leithead, W.E.: Approximate implementation of the logarithm of the matrix determinant in Gaussian process regression. J. Stat. Comput. Simul. 77(4), 329–348 (2007)MathSciNetCrossRefGoogle Scholar
  40. 40.
    Zhang, Y., Wainwright, M.J., Jordan, M.I.: Distributed estimation of generalized matrix rank: efficient algorithms and lower bounds. In: Proceedings of The 32nd International Conference on Machine Learning, pp. 457–465 (2015)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of Minnesota at Twin CitiesMinneapolisUSA

Personalised recommendations