Advertisement

Springer Nature is making Coronavirus research free. View research | View latest news | Sign up for updates

Random time step probabilistic methods for uncertainty quantification in chaotic and geometric numerical integration

  • 17 Accesses

Abstract

A novel probabilistic numerical method for quantifying the uncertainty induced by the time integration of ordinary differential equations (ODEs) is introduced. Departing from the classical strategy to randomise ODE solvers by adding a random forcing term, we show that a probability measure over the numerical solution of ODEs can be obtained by introducing suitable random time steps in a classical time integrator. This intrinsic randomisation allows for the conservation of geometric properties of the underlying deterministic integrator such as mass conservation, symplecticity or conservation of first integrals. Weak and mean square convergence analysis is derived. We also analyse the convergence of the Monte Carlo estimator for the proposed random time step method and show that the measure obtained with repeated sampling converges in the mean square sense independently of the number of samples. Numerical examples including chaotic Hamiltonian systems, chemical reactions and Bayesian inferential problems illustrate the accuracy, robustness and versatility of our probabilistic numerical method.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

References

  1. Abdulle, A.: Fourth order Chebyshev methods with recurrence relation. SIAM J. Sci. Comput. 23, 2041–2054 (2002)

  2. Abdulle, A., Medovikov, A.A.: Second order Chebyshev methods based on orthogonal polynomials. Numer. Math. 90, 1–18 (2001)

  3. Andrieu, C., Roberts, G.O.: The pseudo-marginal approach for efficient Monte Carlo computations. Ann. Stat. 37, 697–725 (2009)

  4. Benettin, G., Giorgilli, A.: On the Hamiltonian interpolation of near to the identity symplectic mappings with application to symplectic integration algorithms. J. Stat. Phys. 74, 1117–1143 (1994)

  5. Chkrebtii, O.A., Campbell, D.A., Calderhead, B., Girolami, M.A.: Bayesian solution uncertainty quantification for differential equations. Bayesian Anal. 11, 1239–1267 (2016)

  6. Cockayne, J., Oates, C.J., Sullivan, T.J., Girolami, M.: Probabilistic numerical methods for PDE-constrained Bayesian inverse problems. In: AIP Conference Proceedings, vol 1853, p. 060001 (2017)

  7. Cockayne, J., Oates, C.J., Sullivan, T.J., Girolami, M.: Bayesian probabilistic numerical methods. SIAM Rev. 61, 756–789 (2019)

  8. Conrad, P.R., Girolami, M., Särkkä, S., Stuart, A., Zygalakis, K.: Statistical analysis of differential equations: introducing probability measures on numerical solutions. Stat. Comput. 27, 1065–1082 (2017)

  9. Dashti, M., Stuart, A.M.: The Bayesian approach to inverse problems. In: Handbook of Uncertainty Quantification. Springer, pp. 1–118 (2016)

  10. Hairer, E.: Variable time step integration with symplectic methods. Appl. Numer. Math. 25, 219–227 (1997)

  11. Hairer, E., Wanner, G.: Solving Ordinary Differential Equations II. Stiff and Differential-Algebraic Problems. Springer, Berlin (1996)

  12. Hairer, E., Nørsett, S.P., Wanner, G.: Solving Ordinary Differential Equations I. Nonstiff Problems, vol. 8, Springer Verlag Series in Computational Mathematics, Berlin (1993)

  13. Hairer, E., Lubich, C., Wanner, G.: Geometric Numerical Integration. Structure-Preserving Algorithms for Ordinary Differential Equations, Springer Series in Computational Mathematics, vol. 31, 2nd edn. Springer, Berlin (2006)

  14. Hénon, M., Heiles, C.: The applicability of the third integral of motion: some numerical experiments. Astron. J. 69, 73–79 (1964)

  15. Kersting, H., Hennig, P.: Active uncertainty calibration in Bayesian ODE solvers. In: Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence (UAI 2016), pp. 309–318. AUAI Press (2016)

  16. Lie, H.C., Sullivan, T.J., Teckentrup, A.L.: Random forward models and log-likelihoods in Bayesian inverse problems. SIAM/ASA J. Uncertain. Quantif. 6, 1600–1629 (2018)

  17. Lie, H.C., Stuart, A.M., Sullivan, T.J.: Strong convergence rates of probabilistic integrators for ordinary differential equations. Stat. Comput. 29, 1265–1283 (2019)

  18. Lorenz, E.N.: Deterministic nonperiodic flow. J. Atmos. Sci. 20, 130–141 (1963)

  19. Milstein, G.N., Tretyakov, M.V.: Stochastic Numerics for Mathematical Physics, Scientific Computing. Springer, Berlin (2004)

  20. Milstein, G.N., Tretyakov, M.V.: Numerical integration of stochastic differential equations with nonglobally Lipschitz coefficients. SIAM J. Numer. Anal. 43, 1139–1154 (2005)

  21. Oates, C.J., Sullivan, T.J.: A modern retrospective on probabilistic numerics. Stat. Comput. 29, 1335–1351 (2019)

  22. Olsen, L.F.: An enzyme reaction with a strange attractor. Phys. Lett. A 94, 454–457 (1983)

  23. Pavliotis, G.A.: Stochastic Processes and Applications, vol. 60 of Texts in Applied Mathematics. Diffusion processes, the Fokker–Planck and Langevin Equations. Springer, New York (2014)

  24. Pavliotis, G.A., Stuart, A.M.: Multiscale Methods: Averaging and Homogenization. Texts in Applied Mathematics, vol. 53. Springer, New York (2008)

  25. Schober, M., Duvenaud, D., Hennig, P.: Probabilistic ODE solvers with Runge–Kutta means. In: Advances in Neural Information Processing Systems 27. Curran Associates, Inc., pp. 739–747 (2014)

  26. Skeel, R.D., Gear, C.W.: Does variable step size ruin a symplectic integrator? Physica 60, 311–313 (1992)

  27. Störmer, C.: Sur les trajectoires des corpuscules électrisés. Arch. sci. phys. nat. Genève 24, 5–18, 113–158, 221–247 (1907)

  28. Stuart, A.M.: Inverse problems: a Bayesian perspective. Acta Numer. 19, 451–559 (2010)

  29. Sullivan, T.J.: Well-posed Bayesian inverse problems and heavy-tailed stable quasi-Banach space priors. Inverse Probl. Imaging 11, 857–874 (2017)

  30. van der Houwen, P.J., Sommeijer, B.P.: On the internal stability of explicit, \(m\)-stage Runge–Kutta methods for large \(m\)-values. Z. Angew. Math. Mech. 60, 479–485 (1980)

  31. Verlet, L.: Computer “experiments” on classical fluids. I. Thermodynamical properties of Lennard–Jones molecules. Phys. Rev. 159, 98–103 (1967)

Download references

Acknowledgements

We thank the two anonymous reviewers whose comments helped improve and clarify this manuscript. This work is partially supported by the Swiss National Science Foundation, Grant No. 200020 172710.

Author information

Correspondence to Assyr Abdulle.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

A modified stochastic differential equation

In Remark 5, we claim the existence of a modified stochastic differential equation (SDE) whose solution is well approximated by the RTS-RK method. Let us denote by \({\widetilde{f}}\) the function defining the modified equation corresponding to the numerical flow \(\Psi _h\) truncated after l terms, i.e.

$$\begin{aligned} {\widetilde{f}}(y) = f(y) + h^q f_{q+1}(y) + h^{q+1} f_{q+2}(y) + \cdots + h^l f_{l+1}(y). \end{aligned}$$

Details about the construction of such a function can be found in Sect. 7.2. In particular, analyticity of the function f is needed for a rigorous backward error analysis to hold. Therefore, we will refer in this section to Assumption 5 (see Sect. 7.2). For the additive noise method presented in Conrad et al. (2017), the authors consider the SDE

$$\begin{aligned} \,\mathrm {d}Y = {\widetilde{f}}(Y) \,\mathrm {d}t + \sqrt{Q h^{2p}} \,\mathrm {d}W, \end{aligned}$$
(51)

where W is a d-dimensional standard Brownian motion. It is possible to show (Conrad et al. 2017, Theorem 2.4) that the solution of (51) satisfies

$$\begin{aligned} {|}{{\mathbb {E}}}\big (\Phi (Y_N) - \Phi (Y(T)) \mid Y_0 = y \big ){|} \le Ch^{2p}, \end{aligned}$$

where \(T = Nh\) and \(Y_N\) is the numerical solution given by the additive noise method after N steps. Here, we present a similar construction for the RTS-RK method. In particular, let us consider the modified SDE

$$\begin{aligned} \,\mathrm {d}{\widetilde{Y}}= & {} \Big ({\widetilde{f}}({\widetilde{Y}}) + \frac{1}{2}Ch^{2p}\partial _{tt}\Psi _h({\widetilde{Y}})\Big ) \,\mathrm {d}t \nonumber \\&\quad +\, \sqrt{Ch^{2p}\partial _t \Psi _h({\widetilde{Y}})\partial _t \Psi _h({\widetilde{Y}})^\top } \,\mathrm {d}W, \end{aligned}$$
(52)

where C is given in Assumption 1(iii). Let us denote by \({\widetilde{{\mathcal {L}}}}\) the generator of (52), which can be written explicitly as

$$\begin{aligned} {\widetilde{{\mathcal {L}}}} = \Big ({\widetilde{f}} + \frac{1}{2}Ch^{2p}\partial _{tt}\Psi _h\Big ) \cdot \nabla + \frac{1}{2}Ch^{2p}\partial _t \Psi _h \partial _t \Psi _h^\top : \nabla ^2, \end{aligned}$$

and, adopting the semi-group notation, it satisfies

$$\begin{aligned} {{\mathbb {E}}}\big (\Phi ({\widetilde{Y}}(h))\mid {\widetilde{Y}}(0) = y\big ) = \hbox {e}^{h{\widetilde{{\mathcal {L}}}}}\Phi (y). \end{aligned}$$

In the following lemma, we consider the error over one step between the numerical solution given by the RTS-RK method and the solution of (52) in the weak sense. The proof is inspired by the calculations presented in Conrad et al. (2017, Section 2.4).

Lemma 9

Under the assumptions of Lemma 1 and if Assumption 5 holds, then

$$\begin{aligned} {|}{{\mathbb {E}}}\big (\Phi (Y_1)- \Phi ({\widetilde{Y}}(h))\mid Y_0 = y\big ){|} \le C h^{2p+1}, \end{aligned}$$

where C is a positive constant independent of h and of y, \({\widetilde{Y}}\) is the solution of (52) and \(Y_1\) is the numerical solution given by the RTS-RK method after one step.

Proof

Let us consider the modified ODE

$$\begin{aligned} {\widehat{y}}'(t) = {\widetilde{f}}({\widehat{y}}), \end{aligned}$$
(53)

and denote its flow as \({\widehat{\varphi }}_t\). The generator \({\widehat{{\mathcal {L}}}} = {\widetilde{f}} \cdot \nabla \) satisfies, adopting the semi-group notation,

$$\begin{aligned} \Phi ({\widehat{\varphi }}_h(y)) = \hbox {e}^{h{\widehat{{\mathcal {L}}}}}\Phi (y). \end{aligned}$$

We can now compute the distance between the solution to (52) and (53) as

$$\begin{aligned} \begin{aligned}&\hbox {e}^{h {\widetilde{{\mathcal {L}}}}}\Phi (y)- \hbox {e}^{h {\widehat{{\mathcal {L}}}}}\Phi (y) = \hbox {e}^{h{\widetilde{f}} \cdot \nabla }\\&\quad \Big (\hbox {e}^{\frac{1}{2}Ch^{{2p+1}}\partial _{tt}\Psi _h \cdot \nabla + \frac{1}{2}Ch^{{2p+1}}\partial _t \Psi _h \partial _t \Psi _h^\top : \nabla ^2} - I\Big ) \Phi (y)\\&\quad = \big (1 + {\mathcal {O}}(h)\big )\Big (\frac{1}{2}Ch^{{2p+1}}\partial _{tt}\Psi _h \cdot \nabla + \frac{1}{2}Ch^{{2p+1}}\partial _t \Psi _h \partial _t \Psi _h^\top : \nabla ^2 \\&\qquad + {\mathcal {O}}\big (h^{{4p+1}}\big )\Big )\Phi (y) \\&\quad = \frac{1}{2}Ch^{2p+1}\partial _{tt}\Psi _h \cdot \nabla \Phi (y) + \frac{1}{2}Ch^{2p+1}\partial _t \Psi _h \partial _t \Psi _h^\top : \nabla ^2 \Phi (y) \\&\qquad + {\mathcal {O}}\big (h^{{4p+1}}\big ). \end{aligned} \end{aligned}$$

Let us recall that equation (10) gives

$$\begin{aligned} \begin{aligned}&\hbox {e}^{h{\mathcal {L}}_h}\Phi (y) - \Phi (\Psi _h(y)) = \frac{1}{2} Ch^{2p+1}\partial _{tt}\Psi _h(y) \cdot \nabla \Phi (y)\\&\quad +\frac{1}{2}Ch^{2p+1}\partial _t \Psi _h(y) \partial _t \Psi _h(y)^\top :\nabla ^2\Phi (y) + {\mathcal {O}}(h^{2p+1}), \end{aligned} \end{aligned}$$

which implies that

$$\begin{aligned} \hbox {e}^{h{\widetilde{{\mathcal {L}}}}}\Phi (y)- \hbox {e}^{h{\mathcal {L}}_h}\Phi (y) = \hbox {e}^{h{\widehat{{\mathcal {L}}}}}\Phi (y) - \Phi (\Psi _h(y)) + {\mathcal {O}}(h^{2p+1}). \end{aligned}$$

Now, the theory of backward error analysis (see Sect. 7.2 or e.g. Hairer et al. 2006, Chapter IX) guarantees that

$$\begin{aligned} \hbox {e}^{h{\widehat{{\mathcal {L}}}}}\Phi (y) - \Phi (\Psi _h(y)) = {\mathcal {O}}(h^{q+l+2}). \end{aligned}$$

Choosing \(l = 2p - q - 1\), we have therefore

$$\begin{aligned} \hbox {e}^{h{\widetilde{{\mathcal {L}}}}}\Phi (y)- \hbox {e}^{h{\mathcal {L}}_h}\Phi (y) = {\mathcal {O}}(h^{2p+1}), \end{aligned}$$

which is the desired result. \(\square \)

The error can be then propagated to final time as in Theorem 1, as presented in the following theorem.

Theorem 7

Under the assumptions of Lemma 9 and Theorem 1, and if there exists a constant \(L > 0\) independent of h such that for all \(\Phi \in {\mathcal {C}}^\infty _b({\mathbb {R}}^d, {\mathbb {R}})\)

$$\begin{aligned} \sup _{u\in {\mathbb {R}}^d} {|}\hbox {e}^{h{\widetilde{{\mathcal {L}}}}}\Phi (u){|} \le (1 + Lh)\sup _{u\in {\mathbb {R}}^d}{|}\Phi (u){|}, \end{aligned}$$

then it holds

$$\begin{aligned} {|}{{\mathbb {E}}}\big (\Phi (Y_N)- \Phi ({\widetilde{Y}}(T))\mid Y_0 = y\big ){|} \le Ch^{2p}, \end{aligned}$$

where \(T = Nh\) and C is a positive constant independent of h and of y, \({\widetilde{Y}}\) is the solution of (52) and \(Y_N\) is the numerical solution given by the RTS-RK method after N steps.

Proof

The proof follows by replacing \({\mathcal {L}}\) with \(\widetilde{{\mathcal {L}}}\) and Lemma 1 with Lemma 9 in the proof of Theorem 1. \(\square \)

Proof of Lemma 6

In the following, we denote by \(\llbracket a, b \rrbracket \) the interval \(\llbracket a, b \rrbracket = [a, b]\) if \(a < b\) and \(\llbracket a, b \rrbracket = [b, a]\) if \(a \ge b\). Let us first consider \(r \ge 2\) and the function \(\gamma _r(x) = x^r \hbox {e}^{-r\kappa /x}\), whose first derivative is given by

$$\begin{aligned} \gamma _r'(x) = rx^{r-2}(x + \kappa ) \hbox {e}^{-r\kappa /x}. \end{aligned}$$

Under Assumption 6, we have that \(H_j \le Mh\) almost surely, and hence for any \(t \in \llbracket h, H_j\rrbracket \)

$$\begin{aligned} {|}\gamma _r'(t){|} \le r (Mh)^{r-2} (Mh + \kappa )\hbox {e}^{-r\kappa /(Mh)}, \end{aligned}$$

where we exploited that \(\hbox {e}^{-r\kappa /x}\) is a growing function of x. The fundamental theorem of calculus gives

$$\begin{aligned} \begin{aligned} {|}\gamma _r(H_j){|}&= \Big |\gamma _r(h) + \int _{h}^{H_j} \gamma _r'(t) \,\mathrm {d}t \,\Big |\\&\le \gamma _r(h) + r (Mh)^{r-2} (Mh + \kappa )\hbox {e}^{-r\kappa /(Mh)} {|}H_j - h{|}, \quad \text {almost surely}. \end{aligned} \end{aligned}$$

Taking expectation on both sides and since by (33) it holds \({|}\eta _j{|}^r \le C\gamma _r(H_j)\), we obtain

$$\begin{aligned} {{\mathbb {E}}}{|}\eta _j{|}^r \le C\left( \gamma _r(h) + r M^{r-2} h^{p+r-3/2} (Mh + \kappa )\hbox {e}^{-r\kappa /(Mh)}\right) , \end{aligned}$$

which proves the desired inequality. This is because Assumption 6 and Assumption 1(ii) imply that \(M \ge 1\), and because Mh can be bounded by M. Let us now consider \(r = 1\). In this case, we have for \(t \in \llbracket h, H_j\rrbracket \)

$$\begin{aligned} {|}\gamma _1'(t){|} \le (mh)^{-1} (Mh + \kappa )\hbox {e}^{-\kappa /(Mh)}, \quad \text {almost surely.} \end{aligned}$$

Hence, we apply the same reasoning as above and obtain almost surely

$$\begin{aligned} {|}\gamma _1(H_j){|} \le \gamma _1(h) + (mh)^{-1} (Mh + \kappa ) \hbox {e}^{-\kappa /(Mh)} {|}H_j - h{|}, \end{aligned}$$

which implies the desired result by proceeding as above. \(\square \)

Proof of Lemma 7

We first expand the square as

$$\begin{aligned} \begin{aligned}&\left( \sum _{j=0}^{n-1} \left( \sum _{k=q}^{N-1} a_{jk} + b_j\right) \right) ^2 = \sum _{j=0}^{n-1} \left( \sum _{k=q}^{N-1} a_{jk} + b_j\right) ^2 \\&\quad + 2 \sum _{j=1}^{n-1} \sum _{i=0}^{j-1}\left( \sum _{k=q}^{N-1} a_{jk} + b_j\right) \left( \sum _{k=q}^{N-1} a_{ik} + b_i\right) . \end{aligned} \end{aligned}$$
(54)

Then, we expand the square in the first sum and obtain

$$\begin{aligned} \begin{aligned}&\left( \sum _{k=q}^{N-1} a_{jk} + b_j\right) ^2 = \left( \sum _{k=q}^{N-1} a_{jk}\right) ^2 + b_j^2 + 2 b_j \sum _{k=q}^{N-1} a_{jk}\\&\qquad = \sum _{k=q}^{N-1} a_{jk}^2 + 2\sum _{k=q+1}^{N-1}\sum _{l=q}^{k-1} a_{jk} a_{jl} + b_j^2 + 2 b_j \sum _{k=q}^{N-1} a_{jk}\\&\qquad = a_{jq}^2 + \sum _{k=q+1}^{N-1} a_{jk}^2 + 2\sum _{k=q+1}^{N-1}\sum _{l=q}^{k-1} a_{jk} a_{jl} + b_j^2 + 2 b_j \sum _{k=q}^{N-1} a_{jk}. \end{aligned} \end{aligned}$$
(55)

We then rewrite the term appearing in the double sum in (54) as

$$\begin{aligned} \begin{aligned}&\left( \sum _{k=q}^{N-1} a_{jk} + b_j\right) \left( \sum _{k=q}^{N-1} a_{ik} + b_i\right) = a_{jq}a_{iq} + \sum _{k=q}^{N-1}\sum _{\begin{array}{c} l=q \\ l+k > 2q \end{array}}^{N-1} a_{jk}a_{il} \\&\qquad + b_j \sum _{k=q}^{N-1} a_{ik} + b_i \sum _{k=q}^{N-1} a_{jk} + b_i b_j. \end{aligned} \end{aligned}$$
(56)

Substituting expressions (55) and (56) in (54), we finally get

$$\begin{aligned}&\left( \sum _{j=0}^{n-1} \left( \sum _{k=q}^{N-1} a_{jk} + b_j\right) \right) ^2 = \sum _{j=0}^{n-1} a_{jq}^2\\&\qquad + 2 \sum _{j=1}^{n-1} \sum _{i=0}^{j-1} a_{jq}a_{iq} + R(a) + S(a, b), \end{aligned}$$

where the remainder R(a) can be written as \(R = R_1 + R_2 + R_3\) where

$$\begin{aligned} \begin{aligned}&R_1(a) = \sum _{j=0}^{n-1}\sum _{k=q+1}^{N-1} a_{jk}^2, \\&R_2(a) = 2 \sum _{j=0}^{n-1}\sum _{k=q+1}^{N-1}\sum _{l=q}^{k-1} a_{jk} a_{jl},\\&R_3(a) = 2 \sum _{j=1}^{n-1} \sum _{i=0}^{j-1} \sum _{k=q}^{N-1}\sum _{\begin{array}{c} l=q \\ l+k > 2q \end{array}}^{N-1} a_{jk}a_{il}, \end{aligned} \end{aligned}$$

and the remainder S(ab) can be written as \(S = S_1 + S_2 + S_3 + S_4\) where

$$\begin{aligned}&S_1(a, b) = \sum _{j=0}^{n-1} b_j^2, \\&S_2(a, b) = 2\sum _{j=1}^{n-1} \sum _{i=0}^{j-1} b_i b_j, \\&S_3(a, b) = 2\sum _{j=1}^{n-1} \sum _{k=q}^{N-1} b_j a_{jk}, \\&S_4(a, b) = 2\sum _{j=1}^{n-1}\sum _{i=0}^{n-1}\left( b_j \sum _{k=q}^{N-1} a_{ik} + b_i \sum _{k=q}^{N-1} a_{jk}\right) .\\ \end{aligned}$$

which proves the desired result. \(\square \)

Proof of Lemma 8

In the following, all the constants are independent of h and n, but can depend on N and q. Moreover, since \(h < 1\), we often apply \(h^r \le h^s\) for \(r \ge s\). We first notice that, under Assumption 2 and Assumption 5, we get for all \(j = 0, \ldots , n-1\) and \(k = q, \ldots , N-1\)

$$\begin{aligned} \begin{aligned} {|}\Delta _{j,k}{|}&= {|}Q_{k+1}(Y_j) - Q_{k+1}(Y_{j+1}){|}\\&\le C {{\Vert }\Psi _0(Y_j) - \Psi _{H_j}(Y_j){\Vert }}\\&\le C_\Delta {|}H_j{|}, \end{aligned} \end{aligned}$$
(57)

almost surely and where \(C_\Delta \) is independent of h. Above, we exploited that \(Q_{k+1}\) is Lipschitz continuous for all \(k=q, \ldots , N+1\) due to Assumption 5. Let us now consider \(R(\Delta )\). Due to (57) and to Assumption 6, we have

$$\begin{aligned} \begin{aligned} {{\mathbb {E}}}(H_j^k - h^k)^2 \Delta _{j,k}^2&\le {C_\Delta ^2}{{\mathbb {E}}}(H_j^{k+1} - H_j h^k)^2\\&= {C_\Delta ^2}\big (h^{2(k+1)} + C_{2(k+1)}h^{2p+2(k+1)-1} \\&+ h^{2(k+1)} + C_2 h^{2p+2k+1} \\&\quad - 2h^{2k+2} - 2C_{k+2}h^{2p+2k+1}\big )\\&= {C_\Delta ^2}\big ((C_{2(k+1)} + C_2 - 2C_{k+2})h^{2p+2k+1}\big )\\&\le C h^{2p+2k+1}, \end{aligned} \end{aligned}$$
(58)

where \(C > 0\) is a positive constant. Now, since \(k \ge q+1\), we get

$$\begin{aligned} {{\mathbb {E}}}(H_j^k - h^k)^2 \Delta _{j,k}^2 \le C h^{2(p+q+1)}. \end{aligned}$$

Hence, for \(R_1(\Delta )\) there exists a constant \({\widetilde{C}}_1\) such that

$$\begin{aligned} {{\mathbb {E}}}R_1(\Delta ) \le {\widetilde{C}}_1 n h^{2(p+q+1)}. \end{aligned}$$

We now proceed to the second remainder \(R_2(\Delta )\). Applying the Cauchy–Schwarz inequality and (58), we get

$$\begin{aligned} \begin{aligned}&{{\mathbb {E}}}\left( (H_j^k - h^k)\Delta _{j,k}(H_j^l - h^l) \Delta _{j,l}\right) \le \Big ({{\mathbb {E}}}\big ((H_j^k - h^k)^2\Delta _{j,k}^2\Big )^{1/2}\\&\qquad \Big ({{\mathbb {E}}}\big ((H_j^l - h^l)^2\Delta _{j,l}^2\Big )^{1/2}\\&\quad \le C h^{2p+k+l+1}, \end{aligned} \end{aligned}$$

where \(C > 0\) is a positive constant. Now, since in the definition of \(R_2(a)\) in (57) we have \(k \ge q+1\) and \(l \ge q\), we have here \(k+l \ge 2q+1\). Therefore, there exists a constant \({\widetilde{C}}_2\) such that

$$\begin{aligned} {{\mathbb {E}}}R_2(\Delta ) \le {\widetilde{C}}_2 n h^{2(p+q+1)}. \end{aligned}$$

We now consider the term \(R_3(\Delta )\). Since \(H_i\) and \(H_j\) are independent for \(i \ne j\), we have

$$\begin{aligned} \begin{aligned}&{{\mathbb {E}}}\big ((H_j^k - h^k)\Delta _{j,k}(H_i^l - h^l) \Delta _{i,l}\big ) \\&\quad = {{\mathbb {E}}}(H_j^k - h^k)\Delta _{j,k} {{\mathbb {E}}}(H_i^l - h^l) \Delta _{i,l}. \end{aligned} \end{aligned}$$

Computing the two factors singularly, we have due to (57) and to Assumption 6

$$\begin{aligned} \begin{aligned} {{\mathbb {E}}}(H_j^k - h^k)\Delta _{j,k}&\le C_\Delta {{\mathbb {E}}}(H_j^{k+1} - H_jh^k) \\&= C_\Delta C_{k+1} h^{2p+k}, \end{aligned} \end{aligned}$$
(59)

and analogously for \({{\mathbb {E}}}(H_i^l - h^l) \Delta _{i,l}\). Then, since \(k+l \ge 2q+1\)

$$\begin{aligned}&{{\mathbb {E}}}\big ((H_j^k - h^k)\Delta _{j,k}(H_i^l - h^l) \Delta _{i,l}\big )\nonumber \\&\quad \le C_\Delta ^2 C_{k+1} C_{l+1} h^{2(2p+q+1/2)}. \end{aligned}$$
(60)

Hence, we have for a constant \({\widetilde{C}}_3 > 0\)

$$\begin{aligned} {{\mathbb {E}}}R_3(\Delta ) \le {\widetilde{C}}_3 n^2 h^{2(2p+q+1/2)}. \end{aligned}$$

Finally, replacing \(t_n = nh\), we can write for a constant \(C > 0\)

$$\begin{aligned} \begin{aligned} {{\mathbb {E}}}R(\Delta )&\le ({\widetilde{C}}_1 + {\widetilde{C}}_2) n h^{2(p+q+1)} + {\widetilde{C}}_3 n^2 h^{2(2p+q+1/2)}\\&= ({\widetilde{C}}_1 + {\widetilde{C}}_2) t_n h^{2(p + q + 1/2)} + {\widetilde{C}}_3 t_n^2 h^{2(2p + q - 1/2)}. \end{aligned} \end{aligned}$$

Let us now consider \(S(\Delta , \eta )\). First, we notice that under the assumption \(p \ge 3/2\) we have for any \(r \ge 1\), \(\min \{r, p+r-3/2\} = r\), and therefore Lemma 6 simplifies to

$$\begin{aligned} {{\mathbb {E}}}{|}\eta _j{|}^r \le Ch^r \hbox {e}^{-r\kappa /(Mh)}. \end{aligned}$$

We first consider \(S_1(\Delta , \eta )\). Applying Lemma 6 with \(r = 2\), we obtain for a constant \({\widehat{C}}_1 > 0\)

$$\begin{aligned} {{\mathbb {E}}}S_1(\Delta , \eta ) \le {\widehat{C}}_1 n h^2 \hbox {e}^{-2\kappa /(Mh)}. \end{aligned}$$

For the second term \(S_2(\Delta , \eta )\), we have by (33) that \({|}\eta _i{|}\le CH^i \hbox {e}^{-\kappa /H_i}\) and \(\eta _j\le CH^j \hbox {e}^{-\kappa /H_j}\) almost surely. These two bounds are independent for \(i \ne j\), and therefore, applying Lemma 6 with \(r = 1\), we have for a constant \({\widehat{C}}_2 > 0\)

$$\begin{aligned} {{\mathbb {E}}}S_2(\Delta , \eta ) \le {\widehat{C}}_2 n^2 h^2 \hbox {e}^{-2\kappa /(Mh)}. \end{aligned}$$

We now consider the third remainder \(S_3(\Delta , \eta )\). Applying the Cauchy–Schwarz inequality, we obtain

$$\begin{aligned} {{\mathbb {E}}}\eta _j (H_j^k - h^k) \Delta _{j,k} \le ({{\mathbb {E}}}\eta _j^2)^{1/2} ({{\mathbb {E}}}(H_j^k - h^k)^2 \Delta _{j,k}^2)^{1/2}. \end{aligned}$$

Applying Lemma 6 with \(r = 2\) to the first factor and (58) to the second, we get

$$\begin{aligned} \begin{aligned} {{\mathbb {E}}}\eta _j (H_j^k - h^k) \Delta _{j,k}&\le C h \hbox {e}^{-\kappa /(Mh)}h^{p+k+1/2}\\&= C h^{p+k+3/2}\hbox {e}^{-\kappa /(Mh)} \end{aligned} \end{aligned}$$

Now, since \(k \ge q\), we have for a constant \({\widehat{C}}_3 > 0\)

$$\begin{aligned} {{\mathbb {E}}}S_3(\Delta , \eta ) \le {\widehat{C}}_3 n h^{p+q+3/2}\hbox {e}^{-\kappa /(Mh)}. \end{aligned}$$

Finally, we consider the last term \(S_4(\Delta , \eta )\). Since by (33) it holds \({|}\eta _j{|} \le CH_j \hbox {e}^{-\kappa /H_j}\) almost surely, and this bound is independent of \(H_i\) for \(i \ne j\), applying (59) and Lemma 6 we have

$$\begin{aligned} \begin{aligned} {{\mathbb {E}}}\eta _j(H_i^k - h^k)\Delta _{i,k}&= {{\mathbb {E}}}\eta _j {{\mathbb {E}}}(H_i^k - h^k)\Delta _{i,k}\\&\le Ch \hbox {e}^{-\kappa /(Mh)}h^{2p+k}, \end{aligned} \end{aligned}$$

which, since \(k \ge q\), implies that there exists a constant \({\widehat{C}}_4 > 0\) such that

$$\begin{aligned} {{\mathbb {E}}}S_4(\Delta , \eta ) \le {\widehat{C}}_4 n^2 h^{2p+q+1}\hbox {e}^{-\kappa /(Mh)}. \end{aligned}$$

Finally, replacing \(t_n = nh\), we can write

$$\begin{aligned} \begin{aligned}&{{\mathbb {E}}}S(\Delta , \eta ) \le ({\widehat{C}}_1 n h^2 + {\widehat{C}}_2 n^2 h^2) \hbox {e}^{-2\kappa /(Mh)} \\&\qquad + ({\widehat{C}}_3 n h^{p+q+3/2} + {\widehat{C}}_4 n^2 h^{2p+q+1})\hbox {e}^{-\kappa /(Mh)}\\&\quad = ({\widehat{C}}_1 t_n h + {\widehat{C}}_2 t_n^2) \hbox {e}^{-2\kappa /(Mh)} \\&\qquad + ({\widehat{C}}_3 t_n h^{p+q+1/2} + {\widehat{C}}_4 t_n^2 h^{2p+q-1})\hbox {e}^{-\kappa /(Mh)}, \end{aligned} \end{aligned}$$

which completes the proof. \(\square \)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Abdulle, A., Garegnani, G. Random time step probabilistic methods for uncertainty quantification in chaotic and geometric numerical integration. Stat Comput (2020). https://doi.org/10.1007/s11222-020-09926-w

Download citation

Keywords

  • Probabilistic methods for ODEs
  • Random time steps
  • Uncertainty quantification
  • Chaotic systems
  • Geometric integration
  • Inverse problems

Mathematics Subject Classification

  • 65C30
  • 65F15
  • 65L09