Skip to main content
SpringerLink
Log in

Fast Algorithms for Bayesian Inversion

  • Conference paper
  • First Online:
Book cover Computational Challenges in the Geosciences

Part of the book series: The IMA Volumes in Mathematics and its Applications ((IMA,volume 156))

Abstract

In this article, we review a few fast algorithms for solving large-scale stochastic inverse problems using Bayesian methods. After a brief introduction to the Bayesian stochastic inverse methodology, we review the following computational techniques, to solve large scale problems: the fast Fourier transform, the fast multipole method (classical and a black-box version), and finallym the hierarchical matrix approach. We emphasize that this is mainly a survey paper presenting a few fast algorithms applicable to large-scale Bayesian inversion techniques, applicable to applications arising from geostatistics. The article is presented at a level accessible to graduate students and computational engineers. Hence, we mainly present the algorithmic ideas and theoretical results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ambikasaran, S., Li, J., Darve, E., Kitanidis, P.K.: Large-scale stochastic linear inversion using hierarchical matrices (2012). In review

    Google Scholar 

  2. Beatson, R., Greengard, L.: A short course on fast multipole methods. Wavelets, multilevel methods and elliptic PDEs pp. 1–37 (1997)

    Google Scholar 

  3. Beatson, R., Newsam, G.: Fast evaluation of radial basis functions: I. Computers & Mathematics with Applications 24(12), 7–19 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bebendorf, M.: Approximation of boundary element matrices. Numerische Mathematik 86(4), 565–589 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bebendorf, M.: Hierarchical Matrices: A Means to Efficiently Solve Elliptic Boundary Value Problems, Lecture Notes in Computational Science and Engineering (LNCSE), vol. 63. Springer-Verlag (2008). ISBN 978-3-540-77146-3

    Google Scholar 

  6. Bebendorf, M., Rjasanow, S.: Adaptive low-rank approximation of collocation matrices. Computing 70(1), 1–24 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  7. Beylkin, G., Coifman, R., Rokhlin, V.: Fast wavelet transforms and numerical algorithms I. Communications on Pure and Applied Mathematics 44(2), 141–183 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bjørnstad, O., Falck, W.: Nonparametric spatial covariance functions: estimation and testing. Environmental and Ecological Statistics 8(1), 53–70 (2001)

    Article  MathSciNet  Google Scholar 

  9. Börm, S., Grasedyck, L., Hackbusch, W.: Hierarchical matrices. Lecture notes 21 (2003)

    Google Scholar 

  10. Burrus, C., Gopinath, R., Guo, H.: Introduction to wavelets and wavelet transforms: A primer. Recherche 67, 02 (1998)

    Google Scholar 

  11. Chandrasekaran, S., Dewilde, P., Gu, M., Pals, T., Sun, X., van der Veen, A., White, D.: Some fast algorithms for sequentially semiseparable representations. SIAM Journal on Matrix Analysis and Applications 27(2), 341 (2006)

    Article  Google Scholar 

  12. Chandrasekaran, S., Gu, M., Pals, T.: A fast ULV decomposition solver for hierarchically semiseparable representations. SIAM Journal on Matrix Analysis and Applications 28(3), 603 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  13. Cheng, H., Greengard, L., Rokhlin, V.: A fast adaptive multipole algorithm in three dimensions. Journal of Computational Physics 155(2), 468–498 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  14. Coifman, R., Rokhlin, V., Wandzura, S.: The fast multipole method for the wave equation: A pedestrian prescription. Antennas and Propagation Magazine, IEEE 35(3), 7–12 (1993)

    Article  Google Scholar 

  15. Cooley, J., Tukey, J.: An algorithm for the machine calculation of complex fourier series. Math. Comput 19(90), 297–301 (1965)

    Article  MathSciNet  MATH  Google Scholar 

  16. Cornford, D., Csató, L., Opper, M.: Sequential, Bayesian geostatistics: a principled method for large data sets. Geographical Analysis 37(2), 183–199 (2005)

    Google Scholar 

  17. Darve, E.: The fast multipole method: numerical implementation. Journal of Computational Physics 160(1), 195–240 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  18. Darve, E.: The fast multipole method - I: Error analysis and asymptotic complexity. SIAM Journal on Numerical Analysis pp. 98–128 (2001)

    Google Scholar 

  19. Darve, E., Havé, P.: Efficient fast multipole method for low-frequency scattering. Journal of Computational Physics 197(1), 341–363 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  20. Darve, E., Havé, P.: A fast multipole method for Maxwell equations stable at all frequencies. Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences 362(1816), 603–628 (2004)

    Google Scholar 

  21. Fong, W., Darve, E.: The black-box fast multipole method. Journal of Computational Physics 228(23), 8712–8725 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  22. Fritz, J., Neuweiler, I., Nowak, W.: Application of FFT-based algorithms for large-scale universal kriging problems. Mathematical Geosciences 41(5), 509–533 (2009)

    Article  MATH  Google Scholar 

  23. Golub, G., Van Loan, C.: Matrix computations, vol. 3. Johns Hopkins Univ Press (1996)

    Google Scholar 

  24. Goreinov, S., Tyrtyshnikov, E., Zamarashkin, N.: A theory of pseudoskeleton approximations. Linear Algebra and Its Applications 261(1–3), 1–21 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  25. Grasedyck, L., Hackbusch, W.: Construction and arithmetics of \(\mathcal{H}\)-matrices. Computing 70, 2003 (2003)

    Google Scholar 

  26. Greengard, L., Rokhlin, V.: A fast algorithm for particle simulations. Journal of Computational Physics 73(2), 325–348 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  27. Greengard, L., Rokhlin, V.: A new version of the fast multipole method for the Laplace equation in three dimensions. Acta Numerica 6(1), 229–269 (1997)

    Article  MathSciNet  Google Scholar 

  28. Greengard, L., Rokhlin, V., SCIENCE., Y.U.N.H.C.D.O.C.: On the efficient implementation of the fast multipole algorithm. Defense Technical Information Center (1988)

    Google Scholar 

  29. Hackbusch, W.: A sparse matrix arithmetic based on \(\mathcal{H}\)-matrices. Part I: Introduction to \(\mathcal{H}\)-matrices. Computing 62(2), 89–108 (1999)

    Google Scholar 

  30. Hackbusch, W., Börm, S.: Data-sparse approximation by adaptive \({\mathcal{H}}^{2}\)-matrices. Computing 69(1), 1–35 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  31. Hackbusch, W., Grasedyck, L., Börm, S.: An introduction to hierarchical matrices. Max-Planck-Inst. für Mathematik in den Naturwiss. (2001)

    Google Scholar 

  32. Hackbusch, W., Khoromskij, B.: A sparse \(\mathcal{H}\)-matrix arithmetic. Computing 64(1), 21–47 (2000)

    MathSciNet  MATH  Google Scholar 

  33. Hackbusch, W., Nowak, Z.: On the fast matrix multiplication in the boundary element method by panel clustering. Numerische Mathematik 54(4), 463–491 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  34. Hrycak, T., Rokhlin, V.: An improved fast multipole algorithm for potential fields. Tech. rep., DTIC Document (1995)

    Google Scholar 

  35. Kitanidis, P.K.: Statistical estimation of polynomial generalized covariance functions and hydrologic applications. Water Resources Research 19(4), 909–921 (1983)

    Article  Google Scholar 

  36. Kitanidis, P.K.: Generalized covariance functions in estimation. Mathematical Geology 25(5), 525–540 (1993)

    Article  Google Scholar 

  37. Kitanidis, P.K.: Quasi-linear geostatistical theory for inversing. Water Resources Research 31(10), 2411–2419 (1995)

    Article  Google Scholar 

  38. Kitanidis, P.K.: On the geostatistical approach to the inverse problem. Advances in Water Resources 19(6), 333–342 (1996)

    Article  Google Scholar 

  39. Kitanidis, P.K.: Introduction to geostatistics: applications to hydrogeology. Cambridge Univ Pr (1997)

    Google Scholar 

  40. Kitanidis, P.K.: Generalized covariance functions associated with the Laplace equation and their use in interpolation and inverse problems. Water Resources Research 35(5), 1361–1367 (1999)

    Article  Google Scholar 

  41. Kitanidis, P.K.: On stochastic inverse modeling. Geophysical Monograph-American Geophysical Union 171, 19 (2007)

    Google Scholar 

  42. Kitanidis, P.K., Vomvoris, E.G.: A geostatistical approach to the inverse problem in groundwater modeling (steady state) and one-dimensional simulations. Water Resources Research 19(3), 677–690 (1983)

    Article  Google Scholar 

  43. Liu, X., Illman, W., Craig, A., Zhu, J., Yeh, T.: Laboratory sandbox validation of transient hydraulic tomography. Water Resources Research 43(5), W05,404 (2007)

    Article  Google Scholar 

  44. Mallat, S.: A theory for multiresolution signal decomposition: The wavelet representation. Pattern Analysis and Machine Intelligence, IEEE Transactions on 11(7), 674–693 (1989)

    Article  MATH  Google Scholar 

  45. Mallat, S.: A wavelet tour of signal processing. Academic Press (1999)

    Google Scholar 

  46. Nishimura, N.: Fast multipole accelerated boundary integral equation methods. Applied Mechanics Reviews 55, 299 (2002)

    Article  Google Scholar 

  47. Nowak, W., Cirpka, O.A.: Geostatistical inference of hydraulic conductivity and dispersivities from hydraulic heads and tracer data. Water Resources Research 42(8), 8416 (2006)

    Article  Google Scholar 

  48. Nowak, W., Tenkleve, S., Cirpka, O.: Efficient computation of linearized cross-covariance and auto-covariance matrices of interdependent quantities. Mathematical geology 35(1), 53–66 (2003)

    Article  MathSciNet  Google Scholar 

  49. Pollock, D., Cirpka, O.: Fully coupled hydrogeophysical inversion of synthetic salt tracer experiments. Water Resources Research 46(7), W07,501 (2010)

    Article  Google Scholar 

  50. Rjasanow, S.: Adaptive cross approximation of dense matrices. IABEM 2002, International Association for Boundary Element Methods (2002)

    Google Scholar 

  51. Rjasanow, S., Steinbach, O.: The fast solution of boundary integral equations. Mathematical and Analytical Techniques with Applications to Engineering. Springer, New York (2007)

    MATH  Google Scholar 

  52. Rokhlin, V.: Rapid solution of integral equations of classical potential theory. Journal of Computational Physics 60(2), 187–207 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  53. Saibaba, A., Ambikasaran, S., Li, J., Darve, E., Kitanidis, P.: Application of hierarchical matrices to linear inverse problems in geostatistics. Oil & Gas Science and Technology–Revue d’IFP Energies nouvelles 67(5), 857–875 (2012)

    Google Scholar 

  54. Saibaba, A., Kitanidis, P.: Efficient methods for large-scale linear inversion using a geostatistical approach. Water Resources Research 48(5), W05,522 (2012)

    Article  Google Scholar 

  55. Starks, T., Fang, J.: On the estimation of the generalized covariance function. Mathematical Geology 14(1), 57–64 (1982)

    Article  MathSciNet  Google Scholar 

  56. Vandebril, R., Barel, M., Golub, G., Mastronardi, N.: A bibliography on semiseparable matrices. Calcolo 42(3), 249–270 (2005)

    Article  MathSciNet  Google Scholar 

  57. Xia, J., Chandrasekaran, S., Gu, M., Li, X.: Fast algorithms for hierarchically semiseparable matrices. Numerical Linear Algebra with Applications 17(6), 953–976 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  58. Ying, L., Biros, G., Zorin, D.: A kernel-independent adaptive fast multipole algorithm in two and three dimensions. Journal of Computational Physics 196(2), 591–626 (2004)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors were supported by “NSF Award 0934596, Subsurface Imaging and Uncertainty Quantification,” “Army High Performance Computing Research Center” (AHPCRC, sponsored by the U.S. Army Research Laboratory under contract No. W911NF-07-2-0027) and “The Global Climate and Energy Project” (GCEP) at Stanford.

Author information

Authors and Affiliations

  1. Institute for Computational and Mathematical Engineering, Huang Engineering Center 053B, 475, Via Ortega, Stanford University, Stanford, CA, 94305-4042, USA

    Sivaram Ambikasaran & Arvind K. Saibaba

  2. Mechanical Engineering, Durand 209, 496, Lomita Mall, Stanford University, Stanford, CA, 94305-4042, USA

    Eric F. Darve

  3. Civil and Environmental Engineering, Yang & Yamazaki Environment & Energy Building - MC 4020, 473 Via Ortega, Stanford, CA, 94305-4042, USA

    Peter K. Kitanidis

Authors
  1. Sivaram Ambikasaran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arvind K. Saibaba
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eric F. Darve
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peter K. Kitanidis
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. The University of Texas at Austin Institute for Computational Engineering, Austin, USA

    Clint Dawson

  2. Stanford University Department of Energy Resources Engineeri, Stanford, USA

    Margot Gerritsen

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media New York

About this paper

Cite this paper

Ambikasaran, S., Saibaba, A.K., Darve, E.F., Kitanidis, P.K. (2013). Fast Algorithms for Bayesian Inversion. In: Dawson, C., Gerritsen, M. (eds) Computational Challenges in the Geosciences. The IMA Volumes in Mathematics and its Applications, vol 156. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-7434-0_5

Download citation

Publish with us

Policies and ethics

Search

Navigation