Skip to main content
SpringerLink
Log in
Book cover

International Conference on High Performance Computing in Science and Engineering

HPCSE 2017: High Performance Computing in Science and Engineering pp 19–33Cite as

Applications of Trace Estimation Techniques

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11087))

Abstract

We discuss various applications of trace estimation techniques for evaluating functions of the form \(\mathtt {tr}(f(A))\) where f is certain function. The first problem we consider that can be cast in this form is that of approximating the Spectral density or Density of States (DOS) of a matrix. The DOS is a probability density distribution that measures the likelihood of finding eigenvalues of the matrix at a given point on the real line, and it is an important function in solid state physics. We also present a few non-standard applications of spectral densities. Other trace estimation problems we discuss include estimating the trace of a matrix inverse \(\mathtt {tr}(A^{-1})\), the problem of counting eigenvalues and estimating the rank, and approximating the log-determinant (trace of log function). We also discuss a few similar computations that arise in machine learning applications. We review two computationally inexpensive methods to compute traces of matrix functions, namely, the Chebyshev expansion and the Lanczos Quadrature methods. A few numerical examples are presented to illustrate the performances of these methods in different applications.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    The matrix product is not formed explicitly since the methods involved typically require only matrix vector products.

  2. 2.

    The matrices used in the experiments can be obtained from the SuiteSparse matrix collection: https://sparse.tamu.edu/.

References

  1. Alter, O., Brown, P.O., Botstein, D.: Singular value decomposition for genome-wide expression data processing and modeling. Proc. Nat. Acad. Sci. 97(18), 10101–10106 (2000)

    Article  Google Scholar 

  2. Andoni, A., Krauthgamer, R., Razenshteyn, I.: Sketching and embedding are equivalent for norms. In: Proceedings of the Forty-Seventh Annual ACM on Symposium on Theory of Computing, pp. 479–488. ACM (2015)

    Google Scholar 

  3. Aune, E., Simpson, D.P., Eidsvik, J.: Parameter estimation in high dimensional Gaussian distributions. Stat. Comput. 24(2), 247–263 (2014)

    Article  MathSciNet  Google Scholar 

  4. Avron, H., Toledo, S.: Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix. J. ACM 58(2), 8 (2011)

    Article  MathSciNet  Google Scholar 

  5. Bai, Z., Fahey, G., Golub, G.: Some large-scale matrix computation problems. J. Comput. Appl. Math. 74(1), 71–89 (1996)

    Article  MathSciNet  Google Scholar 

  6. Bai, Z., Golub, G.H.: Bounds for the trace of the inverse and the determinant of symmetric positive definite matrices. Annal. Numer. Math. 4, 29–38 (1996)

    MathSciNet  MATH  Google Scholar 

  7. Bekas, C., Kokiopoulou, E., Saad, Y.: An estimator for the diagonal of a matrix. Appl. Numer. Math. 57(11), 1214–1229 (2007)

    Article  MathSciNet  Google Scholar 

  8. Boutsidis, C., Drineas, P., Kambadur, P., Kontopoulou, E.-M., Zouzias, A.: A randomized algorithm for approximating the log determinant of a symmetric positive definite matrix. Linear Algebra Appl. 533, 95–119 (2017)

    Article  MathSciNet  Google Scholar 

  9. Carbó-Dorca, R.: Smooth function topological structure descriptors based on graph-spectra. J. Math. Chem. 44(2), 373–378 (2008)

    Article  MathSciNet  Google Scholar 

  10. Davis, T.A., Hu, Y.: The University of Florida sparse matrix collection. ACM Trans. Math. Softw. (TOMS) 38(1), 1 (2011)

    MathSciNet  MATH  Google Scholar 

  11. Di Napoli, E., Polizzi, E., Saad, Y.: Efficient estimation of eigenvalue counts in an interval. ArXiv preprint arXiv:1308.4275 (2013)

  12. Estrada, E.: Characterization of 3D molecular structure. Chem. Phys. Lett. 319(5), 713–718 (2000)

    Article  Google Scholar 

  13. Golub, G.H., Meurant, G.: Matrices, Moments and Quadrature with Applications. Princeton University Press, Princeton (2009)

    Book  Google Scholar 

  14. Golub, G.H., Strakoš, Z.: Estimates in quadratic formulas. Numer. Algorithms 8(2), 241–268 (1994)

    Article  MathSciNet  Google Scholar 

  15. Golub, G.H., Welsch, J.H.: Calculation of gauss quadrature rules. Math. Comput. 23(106), 221–230 (1969)

    Article  MathSciNet  Google Scholar 

  16. Han, I., Malioutov, D., Avron, H., Shin, J.: Approximating spectral sums of large-scale matrices using stochastic Chebyshev approximations. SIAM J. Sci. Comput. 39(4), A1558–A1585 (2017)

    Article  MathSciNet  Google Scholar 

  17. Han, I., Malioutov, D., Shin, J.: Large-scale log-determinant computation through stochastic Chebyshev expansions. In: Proceedings of the 32nd International Conference on Machine Learning, pp. 908–917 (2015)

    Google Scholar 

  18. Higham, N.J.: Functions of Matrices: Theory and Computation. SIAM, University City (2008)

    Book  Google Scholar 

  19. Hutchinson, M.F.: A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines. Commun. Stat.-Simul. Comput. 19(2), 433–450 (1990)

    Article  MathSciNet  Google Scholar 

  20. Kalantzis, V., Bekas, C., Curioni, A., Gallopoulos, E.: Accelerating data uncertainty quantification by solving linear systems with multiple right-hand sides. Numer. Algorithms 62(4), 637–653 (2013)

    Article  MathSciNet  Google Scholar 

  21. Kaufman, L., Rousseeuw, P.J.: Finding Groups in Data: An Introduction to Cluster Analysis. Wiley, Hoboken (1990)

    Book  Google Scholar 

  22. Li, Y., Nguyên, H.L., Woodruff, D.P.: On sketching matrix norms and the top singular vector. In: Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1562–1581. SIAM (2014)

    Google Scholar 

  23. Lin, L., Saad, Y., Yang, C.: Approximating spectral densities of large matrices. SIAM Rev. 58(1), 34–65 (2016)

    Article  MathSciNet  Google Scholar 

  24. Mason, J.C., Handscomb, D.C.: Chebyshev Polynomials. CRC Press, Boca Raton (2002)

    Book  Google Scholar 

  25. Musco, C., Netrapalli, P., Sidford, A., Ubaru, S., Woodruff, D.P.: Spectrum approximation beyond fast matrix multiplication: algorithms and hardness. arXiv preprint arXiv:1704.04163 (2017)

  26. Newman, M.E.J.: Finding community structure in networks using the eigenvectors of matrices. Phys. Rev. E 74(3), 036104 (2006)

    Article  MathSciNet  Google Scholar 

  27. Rasmussen, C., Williams, C.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)

    MATH  Google Scholar 

  28. Roosta-Khorasani, F., Ascher, U.: Improved bounds on sample size for implicit matrix trace estimators. Found. Comput. Math. 15(5), 1187–1212 (2015)

    Article  MathSciNet  Google Scholar 

  29. Rue, H., Held, L.: Gaussian Markov Random Fields: Theory and Applications. CRC Press, Boca Raton (2005)

    Book  Google Scholar 

  30. Silver, R., Röder, H.: Densities of states of mega-dimensional Hamiltonian matrices. Int. J. Mod. Phys. C 5(04), 735–753 (1994)

    Article  Google Scholar 

  31. Turek, I.: A maximum-entropy approach to the density of states within the recursion method. J. Phys. C: Solid State Phys. 21(17), 3251 (1988)

    Article  Google Scholar 

  32. Ubaru, S., Chen, J., Saad, Y.: Fast estimation of tr(f(A)) via stochastic Lanczos quadrature. SIAM J. Matrix Anal. Appl. 38, 1075–1099 (2017)

    Article  MathSciNet  Google Scholar 

  33. Ubaru, S., Saad, Y.: Fast methods for estimating the numerical rank of large matrices. In: Proceedings of the 33rd International Conference on Machine Learning, pp. 468–477 (2016)

    Google Scholar 

  34. Ubaru, S., Saad, Y., Seghouane, A.-K.: Fast estimation of approximate matrix ranks using spectral densities. Neural Comput. 29(5), 1317–1351 (2017)

    Article  Google Scholar 

  35. Wang, L.-W.: Calculating the density of states and optical-absorption spectra of large quantum systems by the plane-wave moments method. Phys. Rev. B 49(15), 10154 (1994)

    Article  Google Scholar 

  36. Weiße, A., Wellein, G., Alvermann, A., Fehske, H.: The kernel polynomial method. Rev. Mod. Phys. 78(1), 275 (2006)

    Article  MathSciNet  Google Scholar 

  37. Wu, L., Laeuchli, J., Kalantzis, V., Stathopoulos, A., Gallopoulos, E.: Estimating the trace of the matrix inverse by interpolating from the diagonal of an approximate inverse. J. Comput. Phys. 326, 828–844 (2016)

    Article  MathSciNet  Google Scholar 

  38. Xi, Y., Li, R., Saad, Y.: Fast computation of spectral densities for generalized eigenvalue problems. arXiv preprint arXiv:1706.06610 (2017)

  39. Zhang, Y., Leithead, W.E.: Approximate implementation of the logarithm of the matrix determinant in Gaussian process regression. J. Stat. Comput. Simul. 77(4), 329–348 (2007)

    Article  MathSciNet  Google Scholar 

  40. Zhang, Y., Wainwright, M.J., Jordan, M.I.: Distributed estimation of generalized matrix rank: efficient algorithms and lower bounds. In: Proceedings of The 32nd International Conference on Machine Learning, pp. 457–465 (2015)

    Google Scholar 

Download references

Acknowledgments

This work was supported by NSF under grant CCF-1318597.

Author information

Authors and Affiliations

  1. University of Minnesota at Twin Cities, Minneapolis, MN, 55455, USA

    Shashanka Ubaru & Yousef Saad

Authors
  1. Shashanka Ubaru
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yousef Saad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shashanka Ubaru .

Editor information

Editors and Affiliations

  1. VSB - Technical University of Ostrava, Ostrava-Poruba, Czech Republic

    Tomáš Kozubek

  2. VSB - Technical University of Ostrava, Ostrava-Poruba, Czech Republic

    Martin Čermák

  3. Charles University in Prague, Prague, Czech Republic

    Petr Tichý

  4. Institute of Geonics of the CAS, Ostrava-Poruba, Czech Republic

    Radim Blaheta

  5. University of Manchester, Manchester, United Kingdom

    Jakub Šístek

  6. VSB - Technical University of Ostrava, Ostrava-Poruba, Czech Republic

    Dalibor Lukáš

  7. Brno University of Technology, Brno, Czech Republic

    Jiří Jaroš

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ubaru, S., Saad, Y. (2018). Applications of Trace Estimation Techniques. In: Kozubek, T., et al. High Performance Computing in Science and Engineering. HPCSE 2017. Lecture Notes in Computer Science(), vol 11087. Springer, Cham. https://doi.org/10.1007/978-3-319-97136-0_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-97136-0_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-97135-3

  • Online ISBN: 978-3-319-97136-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation