Statistics and Computing

, Volume 29, Issue 6, pp 1285–1295 | Cite as

Adaptive step-size selection for state-space probabilistic differential equation solvers

  • Oksana A. ChkrebtiiEmail author
  • David A. Campbell


When models are defined implicitly by systems of differential equations with no closed-form solution, small local errors in finite-dimensional solution approximations can propagate into deviations from the true underlying model trajectory. Some recent perspectives in quantifying this uncertainty are based on Bayesian probability modeling: a prior is defined over the unknown solution and updated by conditioning on interrogations of the forward model. Improvement in accuracy via grid refinement must be considered in order for such Bayesian numerical methods to compete with state-of-the-art numerical techniques. We review the principles of Bayesian statistical design and apply these to develop an adaptive probabilistic method to sequentially select time-steps for state-space probabilistic ODE solvers. We investigate the behavior of local error under the adaptive scheme which underlies numerical variable step-size methods. Numerical experiments are used to illustrate the performance of such an adaptive scheme, showing improved accuracy over uniform designs in terms of local error.


Uncertainty quantification Differential equations Statistical design Numerical methods Data assimilation 



This material was developed, in part, at the Prob Num 2018 workshop hosted by the Lloyd’s Register Foundation programme on Data-Centric Engineering at the Alan Turing Institute, UK, and supported by the National Science Foundation, USA. This material was based upon work partially supported by the National Science Foundation under Grant DMS-1127914 to the Statistical and Applied Mathematical Sciences Institute (SAMSI) and the Isaac Newton Institute for Mathematical Sciences’ program on Uncertainty Quantification for Complex Systems: theory and methodologies. Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the above-named funding bodies and research institutions.


  1. Alexanderian, A., Gloor, P.J., Ghattas, O.: On Bayesian A- and D-optimal experimental designs in infinite dimensions. Bayesian Anal. 11(3), 671 (2016). Google Scholar
  2. Bernardo, J.M., Smith, A.F.M.: Bayesian Theory. Wiley, New York (1994)Google Scholar
  3. Butcher, J.: Numerical Methods for Ordinary Differential Equations. Wiley, Hoboken (2008). Google Scholar
  4. Chaloner, K., Verdinelli, I.: Bayesian experimental design: a review. Stat. Sci. 10, 273 (1995). Google Scholar
  5. Chkrebtii, O., Campbell, D., Calderhead, B., Girolami, M.: Bayesian solution uncertainty quantification for differential equations. Bayesian Anal. 11(4), 1239 (2016a). Google Scholar
  6. Chkrebtii, O., Campbell, D., Calderhead, B., Girolami, M.: Rejoinder. Bayesian Anal. 4(11), 1295 (2016b). Google Scholar
  7. Cockayne, J., Oates, C., Sullivan, T., Girolami, M.: Bayesian probabilistic numerical methods (2017). arXiv:1702.03673
  8. Diaconis, P.: Bayesian Numerical Analysis, pp. 163–175. Springer, New York (1988)Google Scholar
  9. Hennig, P., Hauberg, S.: Probabilistic solutions to differential equations and their application to Riemannian statistics. In: Proceedings of the 17th International Conference on Artificial Intelligence and Statistics (AISTATS), vol. 33 (2014)Google Scholar
  10. Higdon, D., Gattiker, J., Lawrence, E., Jackson, C., Tobis, M., Pratola, M., Habib, S., Heitmann, K., Price, S.: Computer model calibration using the ensemble Kalman filter. Technometrics 55(4), 488 (2013). Google Scholar
  11. Jazwinski, A.: Stochastic Processes and Filtering Theory. Mathematics in Science and Engineering. Elsevier Science, Amsterdam (1970). Google Scholar
  12. Kassam, A., Trefethen, L.: Fourth-order time-stepping for stiff PDEs. SIAM J. Sci. Comput. 26(4), 1214 (2005). Google Scholar
  13. Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22, 49 (1951). Google Scholar
  14. Kuramoto, Y., Tsuzuki, T.: Persistent propagation of concentration waves in dissipative media far from thermal equilibrium. Prog. Theor. Phys. 55, 356 (1976). Google Scholar
  15. Lorenz, E.N.: Deterministic nonperiodic flow. J. Atmos. Sci. 20, 130 (1963). Google Scholar
  16. O’Hagan, A.O.: Some Bayesian numerical analysis. Bayesian Stat. 4, 345 (1992)Google Scholar
  17. Paciorek, C.J., Schervish, M.J.: In: Thrun, S., Saul, L.K., Schölkopf, B. (eds.) Advances in Neural Information Processing Systems, vol. 16, pp. 273–280. MIT Press, Cambridge (2004)Google Scholar
  18. Paciorek, C., Schervish, M.: Spatial modelling using a new class of nonstationary covariance functions. Environmetrics 17(5), 483 (2006). Google Scholar
  19. Schober, M., Duvenaud, D.K., Hennig, P.: In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., Weinberger, K. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 739–747. Curran Associates, Inc. Red Hook (2014)Google Scholar
  20. Sebastiani, P., Wynn, H.: Maximum entropy sampling and optimal Bayesian experimental design. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 62(1), 145 (2000). Google Scholar
  21. Shampine, L.F.: Error estimation and control for ODEs. J. Sci. Comput. 25(1), 3 (2005). Google Scholar
  22. Skilling, J.: Bayesian Solution of Ordinary Differential Equations, pp. 23–37. Kluwer Academic Publishers, Seattle (1991)Google Scholar
  23. Stuart, A.M.: Inverse problems: a Bayesian perspective. Acta Numer. 19, 451 (2010). Google Scholar
  24. Wang, J., Cockayne, J., Oates, C.: On the Bayesian solution of differential equations (2018). arXiv:1805.07109
  25. Wikle, C.K., Berliner, L.M.: A Bayesian tutorial for data assimilation. Physica D 230(1), 1 (2007). Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of StatisticsThe Ohio State UniversityColumbusUSA
  2. 2.School of Mathematics and StatisticsCarleton UniversityOttawaCanada

Personalised recommendations