Statistics and Computing

, Volume 29, Issue 6, pp 1265–1283 | Cite as

Strong convergence rates of probabilistic integrators for ordinary differential equations

  • Han Cheng LieEmail author
  • A. M. Stuart
  • T. J. Sullivan


Probabilistic integration of a continuous dynamical system is a way of systematically introducing discretisation error, at scales no larger than errors introduced by standard numerical discretisation, in order to enable thorough exploration of possible responses of the system to inputs. It is thus a potentially useful approach in a number of applications such as forward uncertainty quantification, inverse problems, and data assimilation. We extend the convergence analysis of probabilistic integrators for deterministic ordinary differential equations, as proposed by Conrad et al. (Stat Comput 27(4):1065–1082, 2017., to establish mean-square convergence in the uniform norm on discrete- or continuous-time solutions under relaxed regularity assumptions on the driving vector fields and their induced flows. Specifically, we show that randomised high-order integrators for globally Lipschitz flows and randomised Euler integrators for dissipative vector fields with polynomially bounded local Lipschitz constants all have the same mean-square convergence rate as their deterministic counterparts, provided that the variance of the integration noise is not of higher order than the corresponding deterministic integrator. These and similar results are proven for probabilistic integrators where the random perturbations may be state-dependent, non-Gaussian, or non-centred random variables.


Probabilistic numerical methods Ordinary differential equations Convergence rates Uncertainty quantification 

Mathematics Subject Classification

65L20 65C99 37H10 68W20 



  1. Briol, F.-X., Oates, C., Girolami, M., Osborne, M.A.: Frank–Wolfe Bayesian quadrature: probabilistic integration with theoretical guarantees. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 28, pp. 1162–1170. Curran Associates Inc., Red Hook (2015)Google Scholar
  2. Capistrán, M.A., Christen, J.A., Donnet, S.: Bayesian analysis of ODEs: solver optimal accuracy and Bayes factors. SIAM/ASA J. Uncertain. Quantif. 4(1), 829–849 (2016). MathSciNetCrossRefzbMATHGoogle Scholar
  3. Chkrebtii, O.A., Campbell, D.A., Calderhead, B., Girolami, M.A.: Bayesian solution uncertainty quantification for differential equations. Bayesian Anal. 11(4), 1239–1267 (2016). MathSciNetCrossRefzbMATHGoogle Scholar
  4. Christen, J.A.: Posterior distribution existence and error control in Banach spaces (2017). arXiv:1712.03299
  5. Cockayne, J., Oates, C., Sullivan, T.J., Girolami, M.: Probabilistic numerical methods for PDE-constrained Bayesian inverse problems. In: Verdoolaege, G. (ed.) Proceedings of the 36th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, volume 1853 of AIP Conference Proceedings, pp. 060001–1–060001–8 (2017a).
  6. Cockayne, J., Oates, C., Sullivan, T.J., Girolami, M.: Bayesian probabilistic numerical methods. SIAM Rev. (2017b). arXiv:1702.03673v2
  7. Conrad, P.R., Girolami, M., Särkkä, S., Stuart, A.M., Zygalakis, K.C.: Statistical analysis of differential equations: introducing probability measures on numerical solutions. Stat. Comput. 27(4), 1065–1082 (2017). ISSN 0960-3174MathSciNetCrossRefzbMATHGoogle Scholar
  8. Diaconis, P.: Bayesian numerical analysis. In: Gupta, S.S., Berger, J.O. (eds.) Statistical Decision Theory and Related Topics, IV (West Lafayette, Ind., 1986), vol. 1, pp. 163–175. Springer, New York (1988)CrossRefGoogle Scholar
  9. Fang, W., Giles, M.B.: Adaptive Euler–Maruyama method for SDEs with non-globally Lipschitz drift: part I, finite time interval (2016). arXiv:1609.08101
  10. Giles, M.B.: Multilevel Monte Carlo methods. Acta Numer. 24, 259–328 (2015). MathSciNetCrossRefzbMATHGoogle Scholar
  11. Gonzalez, J., Osborne, M., Lawrence, N.: GLASSES: Relieving the myopia of Bayesian optimisation. In: Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, pp. 790–799 (2016).
  12. Hairer, E., Nørsett, S., Wanner, G.: Solving Ordinary Differential Equations I: Nonstiff Problems, volume 8 of Springer Series in Computational Mathematics. Springer, New York (2009). CrossRefGoogle Scholar
  13. Hennig, P.: Probabilistic interpretation of linear solvers. SIAM J. Optim. 25(1), 234–260 (2015). MathSciNetCrossRefzbMATHGoogle Scholar
  14. Hennig, P., Osborne, M.A., Girolami, M.: Probabilistic numerics and uncertainty in computations. Proc. R. Soc. Lond. A Math. 471(2179), 20150142 (2015). MathSciNetCrossRefzbMATHGoogle Scholar
  15. Higham, D.J., Mao, X., Stuart, A.M.: Strong convergence of Euler-type methods for nonlinear stochastic differential equations. SIAM J. Numer. Anal. 40(3), 1041–1063 (2002). MathSciNetCrossRefzbMATHGoogle Scholar
  16. Holte, J.M.: Discrete Gronwall lemma and applications (2009). Accessed 9 Oct 2019
  17. Humphries, A.R., Stuart, A.M.: Runge–Kutta methods for dissipative and gradient dynamical systems. SIAM J. Numer. Anal. 31(5), 1452–1485 (1994). MathSciNetCrossRefzbMATHGoogle Scholar
  18. Jentzen, A., Neuenkirch, A.: A random Euler scheme for Carathéodory differential equations. J. Comput. Appl. Math. 224(1), 346–359 (2009). MathSciNetCrossRefzbMATHGoogle Scholar
  19. Kaipio, J., Somersalo, E.: Statistical and Computational Inverse Problems, volume 160 of Applied Mathematical Sciences. Springer, New York (2005). CrossRefzbMATHGoogle Scholar
  20. Knapik, B.T., van der Vaart, A.W., van Zanten, J.H.: Bayesian inverse problems with Gaussian priors. Ann. Stat. 39(5), 2626–2657 (2011). MathSciNetCrossRefzbMATHGoogle Scholar
  21. Kruse, R., Wu, Y.: Error analysis of randomized Runge–Kutta methods for differential equations with time-irregular coefficients. Comput. Methods Appl. Math. 17(3), 479–498 (2017). MathSciNetCrossRefzbMATHGoogle Scholar
  22. Law, K., Stuart, A., Zygalakis, K.: Data Assimilation: A Mathematical Introduction, volume 62 of Texts in Applied Mathematics. Springer, Berlin (2015). CrossRefzbMATHGoogle Scholar
  23. Lie, H.C., Sullivan, T.J., Teckentrup, A.L.: Random forward models and log-likelihoods in Bayesian inverse problems. SIAM/ASA J. Uncertain. Quantif. 6(4), 1600–1629 (2018). MathSciNetCrossRefzbMATHGoogle Scholar
  24. Mao, X., Szpruch, L.: Strong convergence and stability of implicit numerical methods for stochastic differential equations with non-globally Lipschitz continuous coefficients. J. Comput. Appl. Math. 238, 14–28 (2013). MathSciNetCrossRefzbMATHGoogle Scholar
  25. Novak, E.: Deterministic and Stochastic Error Bounds in Numerical Analysis, volume 1349 of Lecture Notes in Mathematics. Springer, Berlin (1988). CrossRefGoogle Scholar
  26. O’Hagan, A.: Some Bayesian numerical analysis. In: Bernardo, J.M., Berger, J.O., Dawid, A.P., Smith, A.F.M. (eds.) Bayesian Statistics, 4: Proceedings of the Fourth Valencia International Meeting: Dedicated to the Memory of Morris H. DeGroot, 1931–1989: April 15–20, 1991, pp. 345–363. Clarendon Press, Oxford (1992)Google Scholar
  27. Owhadi, H.: Bayesian numerical homogenization. Multiscale Model. Simul. 13(3), 812–828 (2015). MathSciNetCrossRefzbMATHGoogle Scholar
  28. Owhadi, H.: Multigrid with rough coefficients and multiresolution operator decomposition from hierarchical information games. SIAM Rev. 59(1), 99–149 (2017). MathSciNetCrossRefzbMATHGoogle Scholar
  29. Peškir, G.: On the exponential Orlicz norms of stopped Brownian motion. Stud. Math. 117(3), 253–273 (1996). MathSciNetCrossRefzbMATHGoogle Scholar
  30. Reich, S., Cotter, C.: Probabilistic Forecasting and Bayesian Data Assimilation. Cambridge University Press, New York (2015). CrossRefzbMATHGoogle Scholar
  31. Ritter, K.: Average-Case Analysis of Numerical Problems, volume 1733 of Lecture Notes in Mathematics. Springer, Berlin (2000). CrossRefGoogle Scholar
  32. Schober, M., Duvenaud, D.K., Hennig, P.: Probabilistic ODE solvers with Runge–Kutta means. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 739–747. Curran Associates Inc., Red Hook (2014)Google Scholar
  33. Skilling, J.: Bayesian solution of ordinary differential equations. In: Smith, C.R., Erickson, G.J., Neudorfer, P.O. (eds.) Maximum Entropy and Bayesian Methods, volume 50 of Fundamental Theories of Physics, pp. 23–37. Springer, Berlin (1992). CrossRefzbMATHGoogle Scholar
  34. Smith, R.C.: Uncertainty Quantification: Theory, Implementation, and Applications, volume 12 of Computational Science & Engineering. Society for Industrial and Applied Mathematics (SIAM), Philadelphia (2014)Google Scholar
  35. Stengle, G.: Numerical methods for systems with measurable coefficients. Appl. Math. Lett. 3(4), 25–29 (1990). MathSciNetCrossRefzbMATHGoogle Scholar
  36. Stuart, A.M.: Inverse problems: a Bayesian perspective. Acta Numer. 19, 451–559 (2010). MathSciNetCrossRefzbMATHGoogle Scholar
  37. Sullivan, T.J.: Introduction to Uncertainty Quantification, volume 63 of Texts in Applied Mathematics. Springer, Berlin (2015). CrossRefGoogle Scholar
  38. Teymur, O., Lie, H.C., Sullivan, T.J., Calderhead, B.: Implicit probabilistic integrators for ODEs. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems 31 (NIPS 2018) (2018).
  39. Traub, J.F., Woźniakowsi, H.: A General Theory of Optimal Algorithms. ACM Monograph Series. Academic Press Inc., New York (1980)Google Scholar
  40. Traub, J.F., Wasilkowski, G.W., Woźniakowski, H.: Information, Uncertainty, Complexity. Addison-Wesley, Reading (1983)zbMATHGoogle Scholar
  41. Wang, J., Cockayne, J., Oates, C.: On the Bayesian solution of differential equations (2018). arXiv:1805.07109

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Institute of MathematicsUniversität PotsdamPotsdam OT GolmGermany
  2. 2.Department of Computing and Mathematical SciencesCalifornia Institute of TechnologyPasadenaUSA
  3. 3.Institute of Mathematics, Freie Universität BerlinBerlinGermany
  4. 4.Zuse Institute BerlinBerlinGermany

Personalised recommendations