Weak and TV consistency in Bayesian uncertainty quantification using disintegration

Abstract

Using standard techniques in Probability theory we prove a series of results relevant in the theory of Bayesian uncertainty quantification (UQ). Using the approach, found in the Bayesian literature, of defining the posterior distribution through a disintegration argument, and using weak and total variation convergence, we are able to prove the existence and numerical consistency of the posterior measure in general functional (Banach) spaces. Relaying commonly on simpler proofs and weaker assumptions, we establish these basic results useful for the theoretical foundation of most common and current UQ problems.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

References

  1. 1.

    Berger, J.O., Wolpert, R.L.: The Likelihood Principle. Lector-Note Monograph Series. Institute of Mathematical Statistics, Purdue, West Lafayette (1988)

    Google Scholar 

  2. 2.

    Billingsley, P.: Convergence of Probability Measures. Wiley (1968). http://www.worldcat.org/isbn/0471072427

  3. 3.

    Blaauw, M., Christen, J.A.: Flexible paleoclimate age-depth models using an autoregressive gamma process. Bayesian Anal. 6(3), 457–474 (2011). https://doi.org/10.1214/11-BA618

    MathSciNet  Article  MATH  Google Scholar 

  4. 4.

    Bui-Thanh, T., Ghattas, O., Martin, J., Stadler, G.: A computational framework for infinite-dimensional Bayesian inverse problems part i: the linearized case, with application to global seismic inversion. SIAM J. Sci. Comput. 35(6), A2494–A2523 (2013). https://doi.org/10.1137/12089586X

    MathSciNet  Article  MATH  Google Scholar 

  5. 5.

    Cai, C., Mohammad-Djafari, A., Legoupil, S., Rodet, T.: Bayesian data fusion and inversion in X-ray multi-energy computed tomography. In: 2011 18th IEEE International Conference on Image Processing, Brussels, pp. 1377–1380 (2011). https://doi.org/10.1109/ICIP.2011.6115694

  6. 6.

    Capistán, M., Christen, J.A., Daza-Torres, M., Flores-Arguedas, H., Montesinos, C.: Error control of the numerical posterior with bayes factors in bayesian uncertainty quantification (2020) (Submitted)

  7. 7.

    Capistrán, M., Christen, J.: A Generic Multivariate Distribution for Counting Data. ArXiv e-prints (2011)

  8. 8.

    Capistrán, M., Christen, J., Velasco-Hernández, J.: Towards uncertainty quantification and inference in the stochastic sir epidemic model. Math. Biosci. 240(2), 250–259 (2012). https://doi.org/10.1016/j.mbs.2012.08.005

    MathSciNet  Article  MATH  Google Scholar 

  9. 9.

    Chama, Z., Mansouri, B., Anani, M., Mohammad-Djafari, A.: Image recovery from Fourier domain measurements via classification using Bayesian approach and total variation regularization. AEU Int. J. Electron. Commun. 66(11), 897–902 (2012)

    Article  Google Scholar 

  10. 10.

    Chang, J.T., Pollard, D.: Conditioning as disintegration. Statistica Neerlandica 51(3), 287–317 (1997). https://doi.org/10.1111/1467-9574.00056

    MathSciNet  Article  MATH  Google Scholar 

  11. 11.

    Cho, K., Jacobs, B.: Disintegration and Bayesian inversion via string diagrams. Math. Struct. Comput. Sci. 29(7), 938–971 (2019). https://doi.org/10.1017/S0960129518000488

    MathSciNet  Article  MATH  Google Scholar 

  12. 12.

    Christen, J., Fox, C.: A general purpose sampling algorithm for continuous distributions (the t-walk). Bayesian Anal. 5(2), 263–282 (2010)

    MathSciNet  Article  Google Scholar 

  13. 13.

    Christen, J.A.: Stop using “subjective” to refer to Bayesian analyses (comment on articles by Berger and by Goldstein). Bayesian Anal. 1(3), 421–422 (2006). https://doi.org/10.1214/06-BA116A

    MathSciNet  Article  MATH  Google Scholar 

  14. 14.

    Christen, J.A., Sansó, B., Santana-Cibrian, M., Velasco-Hernández, J.X.: Bayesian deconvolution of oil well test data using gaussian processes. J. Appl. Stat. 43(4), 721–737 (2016). https://doi.org/10.1080/02664763.2015.1077374

    MathSciNet  Article  MATH  Google Scholar 

  15. 15.

    Cockayne, J., Oates, C.J., Sullivan, T.J., Girolami, M.: Bayesian probabilistic numerical methods. SIAM Rev. 61(4), 756–789 (2019). https://doi.org/10.1137/17M1139357

    MathSciNet  Article  MATH  Google Scholar 

  16. 16.

    Collis, J., Connor, A.J., Paczkowski, M., Kannan, P., Pitt-Francis, J., Byrne, H.M., Hubbard, M.E.: Bayesian calibration, validation and uncertainty quantification for predictive modelling of tumour growth: a tutorial. Bull. Math. Biol. 79(4), 939–974 (2017)

    MathSciNet  Article  Google Scholar 

  17. 17.

    Cui, T., Fox, C., O’Sullivan, M.J.: Bayesian calibration of a large-scale geothermal reservoir model by a new adaptive delayed acceptance metropolis hastings algorithm. Water Resour. Res. 47, 10 (2011)

    Article  Google Scholar 

  18. 18.

    Cui, T., Fox, C., O’Sullivan, M.J.: A posteriori stochastic correction of reduced models in delayed-acceptance MCMC, with application to multiphase subsurface inverse problems. Int. J. Numer. Methods Eng. 118(10), 578–605 (2019). https://doi.org/10.1002/nme.6028

    MathSciNet  Article  Google Scholar 

  19. 19.

    Culbertson, J., Sturtz, K.: A categorical foundation for Bayesian probability. Appl. Categ. Struct. 22(4), 647–662 (2014). https://doi.org/10.1007/s10485-013-9324-9

    MathSciNet  Article  MATH  Google Scholar 

  20. 20.

    Dashti, M., Stuart, A.M.: The bayesian approach to inverse problems. Handbook of Uncertainty Quantification, pp. 1–118 (2016)

  21. 21.

    De Valpine, P.: Improved estimation of normalizing constants from Markov chain Monte Carlo output. J. Comput. Graph. Stat. 17(2), 333–351 (2008)

    MathSciNet  Article  Google Scholar 

  22. 22.

    DeGroot, H.: Optimal Statistical Decisions. Wiley, New York (1970)

    Google Scholar 

  23. 23.

    Fox, C., Haario, H., Christen, J.: Inverse problems. In: Damien, P., Dellaportas, P., Polson, N., Stephens, D. (eds.) Bayesian Theory and Applications, chap. 31, pp. 619–643. Oxford University Press, Oxford (2013)

    Google Scholar 

  24. 24.

    Ghosal, S., Ghosh, J.K., van der Vaart, A.W.: Convergence rates of posterior distributions. Ann. Stat. 28(2), 500–531 (2000). http://www.jstor.org/stable/2674039

  25. 25.

    Ghosal, S., van der Vaart, A.: Fundamentals of Nonparametric Bayesian Inference. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge (2017). https://doi.org/10.1017/9781139029834

    Google Scholar 

  26. 26.

    Ghosh, J.K., Ramamoorthi, R.: Bayesian Nonparametrics. Springer, New York (2003)

    Google Scholar 

  27. 27.

    Gibbs, A.L., Su, F.E.: On choosing and bounding probability metrics. Int. Stat. Rev. 70(3), 419–435 (2002). https://doi.org/10.1111/j.1751-5823.2002.tb00178.x

    Article  MATH  Google Scholar 

  28. 28.

    Giovannelli, J.F., Idier, J.: Regularization and Bayesian Methods for Inverse Problems in Signal and Image Processing. Wiley, New York (2015). https://doi.org/10.1002/9781118827253

    Google Scholar 

  29. 29.

    Gowrisankaran, K.: Measurability of functions in product spaces. Proc. Am. Math. Soc. 31, 485–488 (1972)

    MathSciNet  Article  Google Scholar 

  30. 30.

    Green, P.J.: Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82(4), 711–732 (1995). https://doi.org/10.1093/biomet/82.4.711

    MathSciNet  Article  MATH  Google Scholar 

  31. 31.

    Hosseini, B., Nigam, N.: Well-posed Bayesian inverse problems: priors with exponential tails. SIAM/ASA J. Uncertain. Quantif. 5(1), 436–465 (2017). https://doi.org/10.1137/16M1076824

    MathSciNet  Article  MATH  Google Scholar 

  32. 32.

    Hutchinson, M., Oh, H., Chen, W.H.: A review of source term estimation methods for atmospheric dispersion events using static or mobile sensors. Inf. Fusion 36, 130–148 (2017). https://doi.org/10.1016/j.inffus.2016.11.010

    Article  Google Scholar 

  33. 33.

    James, L.F.: Bayesian Poisson process partition calculus with an application to Bayesian lévy moving averages. Ann. Stat. 33(4), 1771–1799 (2005). https://doi.org/10.1214/009053605000000336

    Article  MATH  Google Scholar 

  34. 34.

    Kahle, C., Lam, K., Latz, J., Ullmann, E.: Bayesian parameter identification in Cahn–Hilliard models for biological growth. SIAM/ASA J. Uncertain. Quantif. 7(2), 526–552 (2019). https://doi.org/10.1137/18M1210034

    MathSciNet  Article  MATH  Google Scholar 

  35. 35.

    Kaipio, J.P., Fox, C.: The Bayesian framework for inverse problems in heat transfer. Heat Transf. Eng. 32(9), 718–753 (2011)

    Article  Google Scholar 

  36. 36.

    Kaipio, J.P., Somersalo, E.: Statistical and Computational Inverse Problems. Applied Mathematical Sciences. Springer Science+Business Media Incorporated, Berlin (2005)

    Google Scholar 

  37. 37.

    Kallenberg, O.: Foundations of Modern Probability. Probability and its Applications (New York), 2nd edn. Springer, New York (2002). https://doi.org/10.1007/978-1-4757-4015-8

    Google Scholar 

  38. 38.

    Keats, A., Yee, E., Lien, F.: Information-driven receptor placement for contaminant source determination. Environ. Model. Softw. 25(9), 1000–1013 (2010)

    Article  Google Scholar 

  39. 39.

    Kolehmainen, V., Lassas, M., Niinimäki, K., Siltanen, S.: Sparsity-promoting bayesian inversion. Inverse Probl. 28(2), 025005 (2012). http://stacks.iop.org/0266-5611/28/i=2/a=025005

  40. 40.

    Kozawa, S., Takenouchi, T., Ikeda, K.: Subsurface imaging for anti-personal mine detection by Bayesian super-resolution with a smooth-gap prior. Artif. Life Robot. 16(4), 478–481 (2012)

    Article  Google Scholar 

  41. 41.

    Lassas, M., Siltanen, S.: Can one use total variation prior for edge-preserving Bayesian inversion? Inverse Probl. 20(5), 1537 (2004). http://stacks.iop.org/0266-5611/20/i=5/a=013

  42. 42.

    Leao Jr., D., Fragoso, M., Ruffino, P.: Regular conditional probaility, desintegration of probability and radon spaces. Proyecciones (Antofagasta) 23, 15–29 (2004). http://www.scielo.cl/scielo.php?script=sci_arttext&pid=S0716-09172004000100002&nrm=iso

  43. 43.

    Mohammad-Djafari, A.: Bayesian inference for inverse problems in signal and image processing and applications. Int. J. Imaging Syst. Technol. 16(5), 209–214 (2006). https://doi.org/10.1002/ima.20081

    Article  Google Scholar 

  44. 44.

    Nissinen, A., Kolehmainen, V.P., Kaipio, J.P.: Compensation of modelling errors due to unknown domain boundary in electrical impedance tomography. IEEE Trans. Med. Imaging 30(2), 231–242 (2011)

    Article  Google Scholar 

  45. 45.

    Osthus, D., Vander Wiel, S., Hoffman, N., Wysocki, F.: Prediction uncertainties beyond the range of experience: a case study in inertial confinement fusion implosion experiments. SIAM/ASA J. Uncertain. Quantif. 7(2), 604–633 (2019). https://doi.org/10.1137/17M1158860

    MathSciNet  Article  MATH  Google Scholar 

  46. 46.

    Palafox, A., Capistrán, M., Christen, J.A.: Effective parameter dimension via Bayesian model selection in effective parameter dimension via Bayesian model selection in the inverse acoustic scattering problem. Math. Probl. Eng. 2014(427203), 12 (2014)

    MATH  Google Scholar 

  47. 47.

    Petra, N., Martin, J., Stadler, G.O.G.: A computational framework for infinite-dimensional Bayesian inverse problems, part ii: Stochastic newton MCMC with application to ice sheet flow inverse problems. SIAM J. Sci. Comput. 36(4), A1525–A1555 (2014). https://doi.org/10.1137/130934805

    MathSciNet  Article  MATH  Google Scholar 

  48. 48.

    Pillonetto, G., Sparacino, G., Cobelli, C.: Handling non-negativity in deconvolution of physiological signals: a nonlinear stochastic approach. Ann. Biomed. Eng. 30(8), 1077–1087 (2002). https://doi.org/10.1114/1.1510449

    Article  Google Scholar 

  49. 49.

    Rosalsky, A., Rosenblatt, J.: On the rate of convergence of series of banach space valued random elements. Nonlinear Anal. Theory Methods Appl. 30, 4237–4248 (1997)

    MathSciNet  Article  Google Scholar 

  50. 50.

    Scheichl, R., Stuart, A.M., Teckentrup, A.L.: Quasi-Monte Carlo and multilevel Monte Carlo methods for computing posterior expectations in elliptic inverse problems. SIAM/ASA J. Uncertain. Quantif. 5(1), 493–518 (2017). https://doi.org/10.1137/16M1061692

    MathSciNet  Article  MATH  Google Scholar 

  51. 51.

    Schervish, M.J.: Theory of Statistics (Springer Series in Statistics), 1st ed. 1995. corr. 2nd printing edn. Springer (1997). http://www.amazon.com/exec/obidos/redirect?tag=citeulike07-20&path=ASIN/0387945466

  52. 52.

    Shen, X., Wasserman, L.: Rates of convergence of posterior distributions. Anna. Stat. 29(3), 687–714 (2001). http://www.jstor.org/stable/2673968

  53. 53.

    Sierpiński, W.: Sur un problème concernant les ensembles mesurables superficiellement. Fundamenta Mathematicae 1(1), 112–115 (1920). http://eudml.org/doc/212592

  54. 54.

    Stuart, A.M.: Inverse problems: a Bayesian perspective. Acta Numerica 19, 451–559 (2010). https://doi.org/10.1017/S0962492910000061

    MathSciNet  Article  MATH  Google Scholar 

  55. 55.

    Swartz, C.: Measure, Integration and Function Spaces. World Scientific Publishing Co. Pte. Ltd, Singapore (1994)

    Google Scholar 

  56. 56.

    Zhu, S., You, P., Wang, H., Li, X., Mohammad-Djafari, A.: Recognition-oriented bayesian SAR imaging. In: 2011 3rd International Asia-Pacific Conference on Synthetic Aperture Radar (APSAR), Seoul, pp. 1–4 (2011)

Download references

Acknowledgements

We thank Tan Bui-Thanh (UT Austin) for prompting us to work on this generalization and for several comments on a previous draft of the paper. Also to Peter Müller (UT Austin) and Fernanda Méndez (CIMAT) for invaluable comments during the many previous drafts of the paper. This research is partially founded by CONACYT CB-2016-01-284451, RDECOMM and ONRG Grants.

Author information

Affiliations

Authors

Corresponding author

Correspondence to J. Andrés Christen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

A Auxiliary lemmas

A Auxiliary lemmas

Lemma 3

Let \(b_n, b: \Theta \rightarrow \mathbb {R}^+\) be bounded, \(\pi \)-integrable functions and let \(z_n = \int b_n(\theta ) \pi (\mathrm{{d}}\theta )\) and \(z = \int b(\theta ) \pi (\mathrm{{d}}\theta )\) and assume that \(| b_n(\theta ) - b(\theta ) | < k n^{-p}\) for all \(\theta \in \Theta , n > N\); \(k>0, p>1\) fixed. Then \(z_n \rightarrow z\) and \(\frac{b_n(\theta )}{z_n} \rightarrow \frac{b(\theta )}{z}\) with convergence rates

$$\begin{aligned} |z_n - z|< k n^{-p} ~~\text {and}~~ \left| \frac{b_n(\theta )}{z_n} - \frac{b(\theta )}{z} \right| < \frac{b(\theta )}{z} \frac{k}{z} n^{-p} + \frac{k}{z} n^{-p}, \end{aligned}$$

for all \(\theta \in \Theta \) and big enough n.

Proof

Since \(b_n(\theta ) \le l(\theta )=M\) then by dominated convergence \(z_n \rightarrow z\) (since \(M = \int l(\theta ) \pi \)). Since \(z_n, z >0\), \(b_n \rightarrow b\) already implies \(b_n/z_n \rightarrow b/z\).

Now, for the rate of convergence we have

$$\begin{aligned} |z_n - z| = \left| \int b_n(\theta ) \pi (\mathrm{{d}}\theta ) - \int b(\theta ) \pi (\mathrm{{d}}\theta ) \right|< \int \left| b_n(\theta ) - b(\theta ) \right| \pi (\mathrm{{d}}\theta ) < k n^{-p}, \end{aligned}$$

since \(\int \pi (\mathrm{{d}}\theta ) = 1\) and therefore \(\frac{| z_n - z|}{z} < \frac{k}{z} n^{-p}\).

Note that the first order Taylor series with residual of \(x^{-1}\) around \(x_0\) is \(x^{-1} = x_0^{-1} - x_0^{-2}(x - x_0) + x_1^{-3} (x - x_0)^2\), for \(x_1\) between x and \(x_0\). Assuming \(x,x_0> 0\) then

$$\begin{aligned} \frac{|x^{-1} - x_0^{-1}|}{x_0^{-1}} \le \frac{|x - x_0|}{x_0} + \left( \frac{x_0}{x_1} \right) ^{3} \left( \frac{x - x_0}{x_0} \right) ^2 . \end{aligned}$$

Let \(\frac{|x - x_0|}{x_0} < \epsilon \), then also \(\frac{|x - x_0|}{x_0} < \epsilon \) and \((1- \epsilon )< \frac{x_1}{x_0} < ( 1 + \epsilon )\). Since \(x^{-3}\) is decreasing then \(( 1 + \epsilon )^{-3}< \left( \frac{x_0}{x_1} \right) ^{3} < (1- \epsilon )^{-3}\). Therefore \(\left( \frac{x_0}{x_1} \right) ^{3} \left( \frac{x - x_0}{x_0} \right) ^2 < (1- \epsilon )^{-3} \epsilon ^2\). If the relative error \(\epsilon \) (of estimating \(x_0\) with x) is below 20%, \((1- \epsilon )^{-3} \epsilon ^2\) is already one order of magnitude smaller than \(\epsilon \). Then ignoring this last term

$$\begin{aligned} \frac{|x^{-1} - x_0^{-1}|}{x_0^{-1}} \lesssim \frac{|x - x_0|}{x_0} < \epsilon . \end{aligned}$$

Assume n is big enough such that the relative error \(\frac{| z_n - z|}{z} < \frac{k}{z} n^{-p}\) is small enough and we have \(\frac{| z_n^{-1} - z^{-1} |}{z^{-1}} \lesssim \frac{| z_n - z|}{z} < \frac{k}{z} n^{-p}\). Therefore

$$\begin{aligned} z^{-1} - \frac{k}{z^2} n^{-p}< z_n^{-1} < z^{-1} + \frac{k}{z^2} n^{-p}. \end{aligned}$$
(15)

Since \(b(\theta ) - k n^{-p}< b_n(\theta ) < b(\theta ) + k n^{-p}\), and given that we may assume \(0 < z^{-1}(1 - \frac{k}{z} n^{-p})\), then multiplying (15) with the former term we have

$$\begin{aligned}&(b(\theta ) - k n^{-p}) (z^{-1} - z^{-2} k n^{-p})< \frac{b_n(\theta )}{z_n}< (z^{-1} + z^{-2} k n^{-p}) (b(\theta ) + k n^{-p}) ,\\&\quad b(\theta )z^{-1} - b(\theta ) z^{-2} k n^{-p} - k n^{-p}z^{-1} + z^{-2} k^2 n^{-2p}< \frac{b_n(\theta )}{z_n} < \\&\quad b(\theta )z^{-1} + b(\theta ) z^{-2} k n^{-p} + k n^{-p}z^{-1} + z^{-2} k^2 n^{-2p} . \end{aligned}$$

Ignoring the two terms of 2p order, we obtain the result.

Lemma 4

With the setting of Lemma 3, let \(h(\theta )\) measurable and \(\hat{h}_n = \int h(\theta ) \frac{b_n(\theta )}{z_n} \pi (\mathrm{{d}}\theta )\) and \(\hat{h} = \int h(\theta ) \frac{b(\theta )}{z} \pi (\mathrm{{d}}\theta )\) exists. Then

$$\begin{aligned} | \hat{h}_n - \hat{h} | < E_1[| h |] \frac{k}{z} n^{-p} + E_0[| h |] \frac{k}{ z} n^{-p} \end{aligned}$$

where \(E_1[| h |] = \int | h(\theta ) | \frac{b(\theta )}{z} \pi (\mathrm{{d}}\theta )\) and \(E_0[| h |] = \int | h(\theta ) | \pi (\mathrm{{d}}\theta )\). Moreover, for all h non-negative and bounded, \(\int h(\theta ) \frac{b_n(\theta )}{z_n} \pi (\mathrm{{d}}\theta )\) and \(\int h(\theta ) \frac{b(\theta )}{z} \pi (\mathrm{{d}}\theta )\) implicitly define the probability measures \(p_n\) and p, then

$$\begin{aligned} ||p_n - p||_{TV} < \frac{k}{z} n^{-p} . \end{aligned}$$

Proof

We have

$$\begin{aligned} | \hat{h}_n - \hat{h} | \le \int |h(\theta )| \left| \frac{b_n(\theta )}{z_n} - \frac{b(\theta )}{z} \right| \pi (\mathrm{{d}}\theta ) \end{aligned}$$

and using Lemma 3 we obtain the first result. Moreover, if \(|h| \le 1\) then \(\hat{h} \frac{k}{z} n^{-p} + \hat{h}_0 \frac{k}{z} n^{-p} < 2\frac{k}{z} n^{-p}\) and, therefore,

$$\begin{aligned} \frac{1}{2} \max _{|h| \le 1} \left| \int h(\theta ) p_n(\mathrm{{d}}\theta ) - \int h(\theta ) p(\mathrm{{d}}\theta ) \right| < \frac{1}{z} k n^{-p} \end{aligned}$$

and we obtain the second result.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Christen, J.A., Pérez-Garmendia, J.L. Weak and TV consistency in Bayesian uncertainty quantification using disintegration. Bol. Soc. Mat. Mex. 27, 2 (2021). https://doi.org/10.1007/s40590-021-00317-3

Download citation

Keywords

  • Inverse problems
  • Bayesian inference
  • Disintegration
  • Weak convergence
  • Total variation
  • Discretization consistency

Mathematics Subject Classification

  • 62A99
  • 62C10
  • 35R30