Non-Gaussian Test Models for Prediction and State Estimation with Model Errors

Abstract

Turbulent dynamical systems involve dynamics with both a large dimensional phase space and a large number of positive Lyapunov exponents. Such systems are ubiquitous in applications in contemporary science and engineering where the statistical ensemble prediction and the real time filtering/state estimation are needed despite the underlying complexity of the system. Statistically exactly solvable test models have a crucial role to provide firm mathematical underpinning or new algorithms for vastly more complex scientific phenomena. Here, a class of statistically exactly solvable non-Gaussian test models is introduced, where a generalized Feynman-Kac formulation reduces the exact behavior of conditional statistical moments to the solution to inhomogeneous Fokker-Planck equations modified by linear lower order coupling and source terms. This procedure is applied to a test model with hidden instabilities and is combined with information theory to address two important issues in the contemporary statistical prediction of turbulent dynamical systems: the coarse-grained ensemble prediction in a perfect model and the improving long range forecasting in imperfect models. The models discussed here should be useful for many other applications and algorithms for the real time prediction and the state estimation.

Project supported by the Office of Naval Research (ONR) Grants (No. ONR DRI N00014-10-1-0554) and the DOD-MURI award “Physics Constrained Stochastic-Statistical Models for Extended Range Environmental Prediction”.

Appendices

Appendix A: The Numerical Scheme for Solving the CGFPE System (2.10)

Here, we outline the numerical method for solving the CGFPE system in (2.10) in one spatial dimension. This is achieved by combining the third-order backward differentiation formulas [17] with the method of (see [19]) and the second-order, finite-volume representation for (2.10).

Recall that the CGFPE system consists of a hierarchy of inhomogeneous Fokker-Planck equations for the conditional moments $\mathcal{M}_{N}(\gamma,t)$ with the forcing terms depending linearly on $\mathcal{M}_{N}(\gamma,t)$ and inhomogeneities depending linearly on $\mathcal{M}_{N-i}(\gamma,t)$, i>1. Thus, due to the form of (2.10), the linearity of the forcing and inhomogeneities, we outline here the present algorithm applied to the homogeneous Fokker-Planck part of (2.10), written in the conservative form

$$\begin{aligned} \frac{\partial \pi}{\partial t} = -\frac{\partial}{\partial \gamma} \biggl[ \biggl(F-\frac{1}{2}G_\gamma \biggr)\pi- \frac{1}{2} G \pi_\gamma \biggr], \end{aligned}$$

(A.1)

where $\pi(\gamma,t) = \int p(u,\gamma,t) {\rm d}u$ and $G(\gamma ,t) = \widetilde{\sigma}^{2}(\gamma,t)$. Given the spatial grid with nodes γ _i ,i=1,…,N, the uniform spacing Δγ, and the approximation

$$\begin{aligned} Q_i(t)\equiv\frac{1}{\Delta\gamma}\int _{\gamma_{i-\frac{1}{2}}}^{\gamma_{i+\frac{1}{2}}}\pi(\gamma,t) {\rm d}\gamma, \end{aligned}$$

(A.2)

we discretize (A.1) in space through the second-order finite volume formula as

$$\begin{aligned} \frac{{\rm d}Q_i}{{\rm d}t} &= - \frac{1}{\Delta\gamma} \biggl[ \biggl(F- \frac{1}{2}G_{\gamma} \biggr)_{i+\frac{1}{2}} \biggl( \frac{9}{16}Q_i + \frac{9}{16}Q_{i+1} - \frac{1}{16}Q_{i-1} - \frac{1}{16}Q_{i+2} \biggr) \\ &\quad{}- \biggl(F-\frac{1}{2}G_{\gamma} \biggr)_{i-\frac{1}{2}} \biggl(\frac {9}{16}Q_{i-1} + \frac{9}{16}Q_{i} - \frac{1}{16}Q_{i-2} - \frac {1}{16}Q_{i+1} \biggr) \biggr] \\ &\quad{}+\frac{1}{2}\frac{1}{\Delta\gamma} \biggl[G_{i+\frac{1}{2}} \biggl(- \frac{9}{8}Q_i + \frac{9}{8}Q_{i+1} + \frac{1}{24}Q_{i-1} - \frac {1}{24}Q_{i+2} \biggr) \\ &\quad{}- G_{i-\frac{1}{2}} \biggl(-\frac{9}{8}Q_{i-1} + \frac{9}{8}Q_{i} + \frac{1}{24}Q_{i-2} - \frac{1}{24}Q_{i+1} \biggr) \biggr]. \end{aligned}$$

(A.3)

The above expression is obtained by seeking higher order interpolants for $Q_{i+\frac{1}{2}}^{n+1}$ in the standard finite-volume formulation

$$\begin{aligned} \frac{{\rm d}Q_i}{{\rm d}t} &= - \frac{1}{\Delta\gamma} \biggl[ \biggl(F- \frac{1}{2}G_{\gamma} \biggr)_{i+\frac{1}{2}} Q^{n+1}_{i+\frac{1}{2}} - \biggl(F-\frac{1}{2}G_{\gamma} \biggr)_{i-\frac{1}{2}} Q^{n+1}_{i-\frac{1}{2}} \biggr] \\ &\quad{}+\frac{1}{2\Delta\gamma}\bigl[G_{i+\frac{1}{2}}Q^{n+1}_{i+\frac{1}{2}} - G_{i-\frac{1}{2}}Q^{n+1}_{i-\frac{1}{2}}\bigr]. \end{aligned}$$

(A.4)

The second order approximations for $Q_{i+\frac{1}{2}}^{n+1}$ are obtained by determining the coefficients a,b,c,d in the expansion

$$\begin{aligned} \widetilde{Q}_{i+\frac{1}{2}} = a Q_{i}+b Q_{i+1}+c Q_{i-1}+d Q_{i+2}, \end{aligned}$$

such that $\widetilde{Q}_{i+\frac{1}{2}} - Q_{i+\frac{1}{2}}$ is of order O((Δγ)³).

The time discretization of (A.1) or (2.10) is obtained by using the three-step backward differentiation formula (BDF3) (see [17]), which belongs to the family of linear multistep methods. In particular, (A.1) is discretized in time as follows

$$\begin{aligned} Q^{n+3} - \frac{18}{11}Q^{n+2} + \frac{9}{11}Q^{n+1} - \frac {2}{11}Q^{n} = \frac{6}{11}\Delta t f\bigl(Q^{n+3}\bigr). \end{aligned}$$

(A.5)

The above implicit formulation can be solved explicitly due to the linearity of (A.1), where

$$\begin{aligned} f\bigl(Q^{n+3}\bigr) = \begin{cases} MQ^{n+3} &\text{for solving } \mathcal{M}_0,\\ MQ^{n+3}+f_{Q_3} &\text{for solving } \mathcal{M}_i\ \text{with}\ i>1. \end{cases} \end{aligned}$$

Thus, (A.5) can be rewritten as

$$\begin{aligned} Q^{n+3} = \begin{cases} (I-\frac{6}{11}\Delta t M )^{-1} (\frac{18}{11}Q^{n+2} - \frac{9}{11}Q^{n+1} + \frac{2}{11}Q^{n} ) \quad\mbox{for\ solving }\ \mathcal{M}_0,\\ (I-\frac{6}{11}\Delta t M )^{-1} (\frac{18}{11}Q^{n+2} - \frac{9}{11}Q^{n+1} + \frac{2}{11}Q^{n} + \frac{6}{11}\Delta t f_{Q_3} ) \\ \quad \mbox{for\ solving }\ \mathcal{M}_i\ \text{with}\ i>1. \end{cases} \end{aligned}$$

The (local) accuracy of the temporal discretization is $\mathcal{O}((\Delta t)^{3})$. Analogous discretization is implemented for solving the inhomogeneous system (2.10).

Appendix B: Expressions for the Initial Densities

Here, we list the formulas used for generating the initial densities $\widetilde{p}_{i}(u,\gamma)$ introduced in Sect. 5.1. Recall that we chose the initial densities with uncorrelated variables,

$$\widetilde{p}_{i}(\gamma,u) =\widetilde{\pi}_{i}(\gamma) \widetilde{\pi}_{i}(u), $$

where the marginal densities $\widetilde{\pi}_{i}(\gamma)$ and $\widetilde{\pi}_{i}(u)$ are given by the mixtures

$$\widetilde{\pi}_i(\gamma) \propto\sum_n R_n(\gamma), \quad \widetilde{\pi}_i(u) \propto\sum _n Q_n(u) $$

with the identical first and second moments chosen as

$$\begin{aligned} \langle u \rangle=0,\quad \bigl\langle u^2\bigr\rangle =3,\quad \langle\gamma \rangle=1.5, \quad\bigl\langle \gamma^2\bigr\rangle =7.5, \quad\langle\gamma u\rangle=0. \end{aligned}$$

In particular, the seven initial densities in Sect. 5.1 with the same joint second-order statistics are obtained as follows (see Table 4 for the parameters used in (1)–(7)):

Table 4 The parameters used in (1)–(7)

(1) Joint density

$$\begin{aligned} \widetilde{p}_{1}(u,\gamma) = \frac{1}{2}\bigl(R_1( \gamma)+R_2(\gamma)\bigr)Q_1(u), \end{aligned}$$

where

$$\begin{aligned} R_i(\gamma) \propto\exp \biggl(-\frac{(\gamma-\overline{\gamma }_i)^2}{2\sigma^\gamma_i} \biggr), \quad\quad Q_1(u) \propto \exp \biggl(-\frac{(u-\overline{u}_1)^2}{2\sigma ^u_1} \biggr). \end{aligned}$$

(2) Joint density

$$\begin{aligned} \widetilde{p}_{2}(u,\gamma) = {\frac{1}{4}} \bigl(R_1(\gamma)+R_2(\gamma )\bigr) \bigl(Q_1(u)+Q_2(u)\bigr), \end{aligned}$$

where

$$\begin{aligned} R_i(\gamma)\propto \exp \biggl(-\frac{(\gamma-\overline{\gamma }_i)^2}{2\sigma^\gamma_i} \biggr), \quad\quad Q_i(u) \propto \exp \biggl(-\frac{(u-\overline{u}_i)^2}{2\sigma ^u_i} \biggr) \bigl(2+ \sin(u)\bigr). \end{aligned}$$

(3) Joint density

$$\begin{aligned} \widetilde{p}_{3}(u,\gamma) = {\frac{1}{4}} \bigl(R_1(\gamma)+R_2(\gamma )\bigr) \bigl(Q_1(u)+Q_2(u)\bigr), \end{aligned}$$

where

$$\begin{aligned} &R_i(\gamma) \propto \exp \biggl(-\frac{(\gamma-\overline{\gamma }_i)^2}{2\sigma^\gamma_i} \biggr) \biggl( \frac{3}{2}{+} \sin \biggl(\frac{\pi\gamma}{2} \biggr) \biggr),\\ & Q_i(u) \propto \exp \biggl(-\frac{(u-\overline{u}_i)^2}{2\sigma^u_i} \biggr) \biggl( \frac{3}{2}{+}\sin \biggl(\frac{\pi u}{2} \biggr) \biggr). \end{aligned}$$

(4) Joint density

$$\begin{aligned} \widetilde{p}_{4}(u,\gamma) = {\frac{1}{4}} \bigl(R_1(\gamma)+R_2(\gamma )\bigr) \bigl(Q_1(u)+Q_2(u)\bigr), \end{aligned}$$

where

$$\begin{aligned} R_i(\gamma) \propto \exp \biggl(-\frac{(\gamma-\overline{\gamma }_i)^2}{2\sigma^\gamma_i} \biggr) \frac{1}{ \gamma^2+1},\quad Q_i(u) \propto \exp \biggl(- \frac{(u-\overline {u}_i)^2}{2\sigma^u_i} \biggr)\frac{1}{u^2+1}. \end{aligned}$$

(5) Joint density

$$\begin{aligned} \widetilde{p}_{5}(u,\gamma) = \frac{1}{2}R_1( \gamma) \bigl(Q_1(u)+Q_2(u)\bigr), \end{aligned}$$

where

$$\begin{aligned} R_i(\gamma) \propto \exp \biggl(-\frac{(\gamma-\overline{\gamma }_i)^2}{2\sigma^\gamma_i} \biggr) \frac{1}{\gamma^2+1},\quad Q_i(u) \propto \exp \biggl(- \frac{(u-\overline {u}_i)^2}{2\sigma^u_i} \biggr). \end{aligned}$$

(6) Joint density

$$\begin{aligned} \widetilde{p}_{6}(u,\gamma) = \frac{1}{2}\bigl(R_1( \gamma)+R_2(\gamma)\bigr)Q_1(u), \end{aligned}$$

where

$$\begin{aligned} R_i(\gamma) \propto \exp \biggl(-\frac{(\gamma-\overline{\gamma }_i)^2}{2\sigma^\gamma_i} \biggr) \biggl( \frac{3}{2}+\sin \biggl(\frac{\pi\gamma}{2} \biggr) \biggr),\quad Q_1(u) \propto \exp \biggl(-\frac{(u-\overline{u}_1)^2}{2\sigma^u_1} \biggr). \end{aligned}$$

(7) Joint density

$$\begin{aligned} \widetilde{p}_{7}(u,\gamma) = \frac{1}{2}R_1( \gamma) \bigl(Q_1(u)+Q_2(u)\bigr), \end{aligned}$$

where

$$\begin{aligned} &R_1(\gamma) \propto \exp \biggl(-\frac{(\gamma-\overline{\gamma }_1)^2}{2\sigma^\gamma_1} \biggr), \\& Q_i(u) \propto \exp \biggl(-\frac{(u-\overline{u}_i)^2}{2\sigma ^u_i} \biggr) \biggl( \frac{3}{2} +\sin \biggl(\frac{\pi u}{2}-1 \biggr) \biggr). \end{aligned}$$