An accuracy comparison of polynomial chaos type methods for the propagation of uncertainties
Abstract
In (Augustin et al. in European J. Appl. Math. 19:149190, 2008) we considered the Polynomial Chaos Expansion for the treatment of uncertainties in industrial applications. For many applications the method has been proven to be a computationally superior alternative to Monte Carlo evaluations. In the current overview we compare the accuracy of Polynomial Chaos type methods for the propagation of uncertainties in nonlinear problems and verify them on two examples relevant for industry. For weakly nonlinear timedependent models, the generalized Kalman filter equations define an efficient method, yielding good approximations if the quantities of interest are restricted to the first two moments of the solution. Secondly, stochastic collocation is discussed. The method is applied to delay differential equations and random ordinary differential equations. Finally, a generalized PC method is discussed which is based on a subdivision of the random space. This approach is even suitable for highly nonlinear models.
Keywords
Kalman Filter Extended Kalman Filter Sparse Grid Unscented Kalman Filter Polynomial Chaos Expansion1 Introduction
Uncertainty quantification in industrial applications is an interesting and important field of research [1, 2, 3, 4]. Due to uncertainties in system parameters, measurements and in the modeling of physical processes itself, deterministic approaches for the simulation of those processes are not appropriate. Uncertainty quantification became a vast field of research over the past decades and many methods have been proposed to deal with random data [5, 6, 7, 8, 9, 10]. Amongst these methods, approaches based on Polynomial Chaos expansions have proven to be very efficient in dealing with uncertainties, see [3, 10, 11].
Those methods revealed to be efficient alternatives to MonteCarlo (MC) simulations. They were successfully applied to solve stationary problems e.g. in the field of stochastic finite elements [10] and to solve differential equations with uncertain parameters and initial values [12]. The suitability of PC for industrial applications was considered in [1]. Additionally, an extensive discussion of the Polynomial Chaos approaches, applied to linear differential equations, was presented. In the current paper we consider numerical methods for the propagation of uncertainties through nonlinear differential equations. The main part is concerned with a comparison of Kalman filter techniques for state estimation and Polynomial Chaos Galerkin techniques. The methods themselves are known from literature, the contribution of this work is the comparison of these methods in selected examples relevant for industry. Our main focus is on the accuracies of the methods. Based on these results, we try to answer the question what is the numerical effort in order to achieve a low, medium or high accuracy while, in terms of function evaluations, gaining at least one order compared to a crude Monte Carlo method.

The most widespread method for treating differential equations with uncertainties are techniques of Kalman filtering. While the original Kalman filter was developed to identify noisy observations with linear differential equations [13], here we use it only to integrate the underlying system with uncertainties. A direct generalization to nonlinear equations is given by the extended Kalman filter [14]; it is based on the sensitivities of the equation. The unscented Kalman filter, a nonlinear filter which does not use derivative information, was developed in [15]. It directly approximates the covariance matrix and is, in this context, used just for state estimation.

Stochastic collocation methods (SCMs) are introduced by [16, 17]. The first practical application of the methods are discussed in [10]. Here, we apply the methods to nonlinear differential equations with delay, which present a more complex behavior than the delayfree counterparts. Stochastic collocation methods have the advantage of using the differential equation as a black box.

In [1] we presented a theory of the stochastic Galerkin method (SGM), see [3, 10], for ordinary differential equations with uncertain parameters. Nevertheless, this method suffers from the lack of convergence, especially when time evolves [18]. Thus, we have to consider generalized approaches to scope with this problem. Several methods have been proposed in literature, see for example the multi resolution method [9, 19] and the MultiElement generalized Polynomial Chaos (MEgPC) method [20]. In [21] a rigorous convergence theory for MEgPC and a fully adaptive scheme in time and random space has been developed. In the current paper we compare this MEgPC approach to the other PC type methods mentioned above.
In Section 2 we introduce two examples from industrial applications, which are the base for the comparison of the discussed methods. The first example concerns crack propagation, while the second considers the quorum sensing in biofilms. Next, we present a summary of the Kalman filter equations for state estimation in Section 3. Especially, nonlinear filters in the context of uncertainty propagation of state are discussed. In Section 4 we consider the SCM, which we apply to the random delay differential equations of quorum sensing. We discuss the multielement approach of the stochastic Galerkin method in Section 5. Finally we compare the three described methods on the base of the two applications, Sections 6.2 and 6.3. In Section 7 we close this article by drawing our conclusions.
2 Test sets of (delay) differential equations
In this section, we introduce the applications considered for the comparison of the methods of the Kalman filter, the SCM and the SGM.

The first application deals with the growth of cracks, which is important in the context of life cycle analysis. Industrial applications are for example crack growth in turbine blades or in train wheelsets. Here the task is to define appropriate inspection schemes in order to prevent severe damages. The crack growth is modelled by a highly nonlinear ordinary differential equation.

The second example is a model of the quorum sensing of biofilms, a delay differential equation. The control of biofilms requires a deep understanding of the structured community of microorganisms living on inert surfaces. Industrial applications range from sewer systems, fresh water systems to corrosion prevention in cooling flow networks of (electric) power plants.
We restrict ourselves to a low number of uncertain variables. Nevertheless, due to the nonlinearities and the delay, differences between the methods with respect to their achieved accuracies will become visible. A more complex model of crack growth, for the evaluation of inspection schemes for train wheelsets, has been computed with the unscented approximation in [22]. The biofilm model, in particular the physical background, has been studied in more detail in [23].
2.1 Crack growth example
R is the stress ratio between maximum and minimum stress, h denotes the width of the plate and ${n}_{f}$, $\mathrm{\Delta}{K}_{0}$, p, q, ${\alpha}^{\prime}$, s are curvefitting constants.
2.2 The biofilm model with quorum sensing
It represents the extra increase in population after the QS mechanism has been initiated. The process is not instantaneous. Thus, assuming it takes a certain amount of time τ to produce the signal molecules, the population of bacteria at time t, $B(t)$, will receive these signal molecules from the population of bacteria at time $t\tau $, i.e. $B(t\tau )$. The coefficient ${\alpha}_{2}$ accounts for the strength of the mechanism. The larger ${\alpha}_{2}$, the more efficient the QS mechanism takes place in the biofilm.
3 Solving stochastic differential equations by Kalman filters
For real processes, white noise is a somewhat simplified assumption. There are extensions of Kalman filtering to colored noise (e.g. [27]) but this will not be picked out here. Equations (6) and (7) describe a stochastic system which is continuous in time. In the following, we restrict ourselves to methods for discrete time systems which are obtained by time discretization of (6). Furthermore we concentrate on nonstiff systems. Of course for the numerical integration of stiff systems semiimplicit or implicit methods are required which means a much higher numerical effort. We begin with linear discrete time systems in Section 3.1. Thereafter, we discuss how nonlinear differential equations can be treated by the extended Kalman filter and the unscented Kalman filter in Sections 3.2 and 3.3. After a comparison of both filters in Section 3.4 we give a reinterpretation of the unscented Kalman filter as a sampling method, see Section 3.5.
3.1 Linear discrete time systems
Kalman filter
Model  ${\mathbf{x}}_{k}={A}_{k1}{\mathbf{x}}_{k1}+{B}_{k1}{\mathbf{u}}_{k1}+{W}_{k1}{\mathbf{w}}_{k1}$, ${\mathbf{w}}_{k1}\sim \mathcal{N}(0,{Q}_{k1})$, ${\mathbf{y}}_{k}={H}_{k}{\mathbf{x}}_{k}+{V}_{k}{\mathbf{v}}_{k}$, ${\mathbf{v}}_{k}\sim \mathcal{N}(0,{R}_{k})$ 
Initialization  ${\stackrel{\u02c6}{\mathbf{x}}}_{0}$, ${P}_{0}=E[{\stackrel{\u02c6}{\mathbf{x}}}_{0}{\stackrel{\u02c6}{\mathbf{x}}}_{0}^{T}]$ 
Predictor  ${\stackrel{\u02c6}{\mathbf{x}}}_{k}^{}={A}_{k1}{\stackrel{\u02c6}{\mathbf{x}}}_{k1}+{B}_{k1}{\mathbf{u}}_{k1}$, ${P}_{k}^{}={A}_{k1}{P}_{k1}{A}_{k1}^{T}+{W}_{k1}{Q}_{k1}{W}_{k1}^{T}$ 
Kalman gain  ${K}_{k}={P}_{k}^{}{H}_{k}^{T}{[{H}_{k}{P}_{k}^{}{H}_{k}^{T}+{V}_{k}{R}_{k}{V}_{k}^{T}]}^{1}$ 
Corrector  ${\stackrel{\u02c6}{\mathbf{x}}}_{k}={\stackrel{\u02c6}{\mathbf{x}}}_{k}^{}+{K}_{k}({\mathbf{y}}_{k}{H}_{k}{\stackrel{\u02c6}{\mathbf{x}}}_{k}^{})$, ${P}_{k}=(I{K}_{k}{H}_{k}){P}_{k}^{}$ 
3.2 Extended Kalman Filter (EKF)
Extended Kalman filter
Model  ${\mathbf{x}}_{k}=\mathbf{f}({\mathbf{x}}_{k1},{\mathbf{u}}_{k1},{\mathbf{w}}_{k1})$, ${\mathbf{w}}_{k1}\sim \mathcal{N}(0,{Q}_{k1})$, ${\mathbf{y}}_{k}=\mathbf{h}({\mathbf{x}}_{k},{\mathbf{v}}_{k})$, ${\mathbf{v}}_{k}\sim \mathcal{N}(0,{R}_{k})$ 
Initialization  ${\stackrel{\u02c6}{\mathbf{x}}}_{0}$, ${P}_{0}=E[{\stackrel{\u02c6}{\mathbf{x}}}_{0}{\stackrel{\u02c6}{\mathbf{x}}}_{0}^{T}]$ 
Predictor  ${\stackrel{\u02c6}{\mathbf{x}}}_{k}^{}=\mathbf{f}({\stackrel{\u02c6}{\mathbf{x}}}_{k1},{\mathbf{u}}_{k1},0)$, ${P}_{k}^{}={A}_{k1}{P}_{k1}{A}_{k1}^{T}+{W}_{k1}{Q}_{k1}{W}_{k1}^{T}$ 
Kalman gain  ${K}_{k}={P}_{k}^{}{H}_{k}^{T}{[{H}_{k}{P}_{k}^{}{H}_{k}^{T}+{V}_{k}{R}_{k}{V}_{k}^{T}]}^{1}$ 
Corrector  ${\stackrel{\u02c6}{\mathbf{x}}}_{k}={\stackrel{\u02c6}{\mathbf{x}}}_{k}^{}+{K}_{k}({\mathbf{y}}_{k}\mathbf{h}({\stackrel{\u02c6}{\mathbf{x}}}_{k}^{},0))$, ${P}_{k}=(I{K}_{k}{H}_{k}){P}_{k}^{}$ 
3.3 Unscented Kalman Filter (UKF)
By Taylor expansion one can show second order accuracy of mean and covariance, [15, 28].
for $i=1,\dots ,n$ respectively, where ${(\sqrt{{P}^{xx}})}_{i}$ denotes the ith column of the matrix root L of P given by ${P}^{xx}=L{L}^{T}$. Thus, the corresponding weights are ${W}_{i}^{\mathrm{mean}}={W}_{i}^{\mathrm{cov}}={W}_{i}$ for $i=1,\dots ,2n$.
Subsequently, we apply the unscented Kalman filter to the recursive estimation of dynamical processes.
3.3.1 Additive process and measurement noise
Unscented Kalman filter
Model  ${\mathbf{x}}_{k}=\mathbf{f}({\mathbf{x}}_{k1},{\mathbf{u}}_{k1},{\mathbf{w}}_{k1})=\tilde{\mathbf{f}}({\mathbf{x}}_{k1},{\mathbf{u}}_{k1})+{\mathbf{w}}_{k1}$, ${\mathbf{w}}_{k1}\sim \mathcal{N}(0,{Q}_{k1})$, ${\mathbf{y}}_{k}=\mathbf{h}({\mathbf{x}}_{k},{\mathbf{v}}_{k})=\tilde{\mathbf{h}}({\mathbf{x}}_{k})+{\mathbf{v}}_{k}$, ${\mathbf{v}}_{k}\sim \mathcal{N}(0,{R}_{k})$ 
Initialization  ${\stackrel{\u02c6}{\mathbf{x}}}_{0}$, ${P}_{0}=E[{\stackrel{\u02c6}{\mathbf{x}}}_{0}{\stackrel{\u02c6}{\mathbf{x}}}_{0}^{T}]$ 
Weights  ${W}_{i}^{\mathrm{mean}}$, ${W}_{i}^{\mathrm{cov}}$, i = 0,…,p 
Sigma points  ${\mathcal{X}}_{k1}^{(i)}={\mathcal{X}}_{k1}^{(i)}({\stackrel{\u02c6}{\mathbf{x}}}_{k1},{P}_{k1})$, i = 0,…,p 
Predictor  ${{\mathcal{X}}_{k}^{}}^{(i)}=\mathbf{f}({\mathcal{X}}_{k1}^{(i)},{\mathbf{u}}_{k1},0)$, i = 0,…,p, ${\stackrel{\u02c6}{\mathbf{x}}}_{k}^{}={\sum}_{i=0}^{p}{W}_{i}^{\mathrm{mean}}{{\mathcal{X}}_{k}^{}}^{(i)}$, ${P}_{k}^{}={\sum}_{i=0}^{p}{W}_{i}^{\mathrm{cov}}({{\mathcal{X}}_{k}^{}}^{(i)}{\stackrel{\u02c6}{\mathbf{x}}}_{k}^{}){({{\mathcal{X}}_{k}^{}}^{(i)}{\stackrel{\u02c6}{\mathbf{x}}}_{k}^{})}^{T}+{Q}_{k1}$ 
Predicted observation  ${\mathcal{Y}}_{k}^{(i)}=\mathbf{h}({{\mathcal{X}}_{k}^{}}^{(i)},0)$, ${\stackrel{\u02c6}{\mathbf{y}}}_{k}^{}={\sum}_{i=0}^{p}{W}_{i}^{\mathrm{mean}}{\mathcal{Y}}_{k}^{(i)}$, i = 0,…,p, ${P}_{k}^{yy}={\sum}_{i=0}^{p}{W}_{i}^{\mathrm{cov}}({\mathcal{Y}}_{k}^{(i)}{\stackrel{\u02c6}{\mathbf{y}}}_{k}^{}){({\mathcal{Y}}_{k}^{(i)}{\stackrel{\u02c6}{\mathbf{y}}}_{k}^{})}^{T}+{R}_{k}$, ${P}_{k}^{xy}={\sum}_{i=0}^{p}{W}_{i}^{\mathrm{cov}}({{\mathcal{X}}_{k}^{}}^{(i)}{\stackrel{\u02c6}{\mathbf{x}}}_{k}^{}){({\mathcal{Y}}_{k}^{(i)}{\stackrel{\u02c6}{\mathbf{y}}}_{k}^{})}^{T}$ 
Kalman gain  ${K}_{k}={P}_{k}^{xy}{({P}_{k}^{yy})}^{1}$ 
Corrector  ${\stackrel{\u02c6}{\mathbf{x}}}_{k}={\stackrel{\u02c6}{\mathbf{x}}}_{k}^{}+{K}_{k}({\mathbf{y}}_{k}{\stackrel{\u02c6}{\mathbf{y}}}_{k}^{})$, ${P}_{k}={P}_{k}^{}{K}_{k}{P}_{k}^{yy}{K}_{k}^{T}$ 
3.3.2 Nonlinear noise
General unscented Kalman Filter
Model  ${\mathbf{x}}_{k}=\mathbf{f}({\mathbf{x}}_{k1},{\mathbf{u}}_{k1},{\mathbf{w}}_{k1})$, ${\mathbf{w}}_{k1}\sim \mathcal{N}(0,{Q}_{k1})$, ${\mathbf{y}}_{k}=\mathbf{h}({\mathbf{x}}_{k},{\mathbf{v}}_{k})$, ${\mathbf{v}}_{k}\sim \mathcal{N}(0,{R}_{k})$ 
Initialization  ${\stackrel{\u02c6}{\mathbf{x}}}_{0}$, ${P}_{0}=E[{\stackrel{\u02c6}{\mathbf{x}}}_{0}{\stackrel{\u02c6}{\mathbf{x}}}_{0}^{T}]$ 
Weights  ${W}_{i}^{\mathrm{mean}}$, ${W}_{i}^{\mathrm{cov}}$, i = 0,…,p 
Sigma points  ${\overline{\mathcal{X}}}_{k1}^{(i)}={\overline{\mathcal{X}}}_{k1}^{(i)}({\overline{\mathbf{x}}}_{k1},{\overline{P}}_{k1})$, i = 0,…,p 
Predictor  ${{\mathcal{X}}_{k}^{}}^{(i)}=\overline{\mathbf{f}}({\overline{\mathcal{X}}}_{k1}^{(i)},{\mathbf{u}}_{k1})$, i = 0,…,p, ${\stackrel{\u02c6}{\mathbf{x}}}_{k}^{}={\sum}_{i=0}^{p}{W}_{i}^{\mathrm{mean}}{{\mathcal{X}}_{k}^{}}^{(i)}$, ${P}_{k}^{}={\sum}_{i=0}^{p}{W}_{i}^{\mathrm{cov}}({{\mathcal{X}}_{k}^{}}^{(i)}{\stackrel{\u02c6}{\mathbf{x}}}_{k}^{}){({{\mathcal{X}}_{k}^{}}^{(i)}{\stackrel{\u02c6}{\mathbf{x}}}_{k}^{})}^{T}$ 
Sigma points  ${\overline{\overline{{\mathcal{X}}_{k}}}}^{(i)}={\overline{\overline{{\mathcal{X}}_{k}}}}^{(i)}({\overline{\overline{\mathbf{x}}}}_{k},{\overline{\overline{P}}}_{k})$, i = 0,…,p 
Predicted observation  ${\mathcal{Y}}_{k}^{(i)}=\overline{\mathbf{h}}({\overline{\overline{{\mathcal{X}}_{k}}}}^{(i)})$, ${\stackrel{\u02c6}{\mathbf{y}}}_{k}^{}={\sum}_{i=0}^{p}{W}_{i}^{\mathrm{mean}}{\mathcal{Y}}_{k}^{(i)}$, i = 0,…,p, ${P}_{k}^{yy}={\sum}_{i=0}^{p}{W}_{i}^{\mathrm{cov}}({\mathcal{Y}}_{k}^{(i)}{\stackrel{\u02c6}{\mathbf{y}}}_{k}^{}){({\mathcal{Y}}_{k}^{(i)}{\stackrel{\u02c6}{\mathbf{y}}}_{k}^{})}^{T}$, ${P}_{k}^{xy}={\sum}_{i=0}^{p}{W}_{i}^{\mathrm{cov}}({{\mathcal{X}}_{k}^{}}^{(i)}{\stackrel{\u02c6}{\mathbf{x}}}_{k}^{}){({\mathcal{Y}}_{k}^{(i)}{\stackrel{\u02c6}{\mathbf{y}}}_{k}^{})}^{T}$ 
Kalman gain  ${K}_{k}={P}_{k}^{xy}{({P}_{k}^{yy})}^{1}$ 
Corrector  ${\stackrel{\u02c6}{\mathbf{x}}}_{k}={\stackrel{\u02c6}{\mathbf{x}}}_{k}^{}+{K}_{k}({\mathbf{y}}_{k}{\stackrel{\u02c6}{\mathbf{y}}}_{k}^{})$, ${P}_{k}={P}_{k}^{}{K}_{k}{P}_{k}^{yy}{K}_{k}^{T}$ 
3.4 Comparison of EKF and UKF in crack growth example
3.5 Reinterpretation of UKF
In a postprocessing step, we will apply a MonteCarlo simulation based on this polynomial approximation, to obtain approximations of any quantity of interest, i.e. the moments or density of the solution.
4 Stochastic Collocation method and random delay differential equations
if $\mathbf{w}\in {L}_{2}$. Eventually, the initial history function can also exhibit a random behavior, $x(t,\mathbf{w})=\varphi (t,\mathbf{w})$, $t\in [\tau ,0]$.
In order to account for the uncertainties the stochastic collocation schema of the Polynomial Chaos Expansion is used. The schema is explained below, along with the numerical method applied to solve for the underlying deterministic equations.
4.1 Wiener expansions
with basis polynomials ${\{{\mathrm{\Psi}}_{i}\}}_{i=0}^{\mathrm{\infty}}$. A detailed discussion on this expansions can be found in [1, 4, 30]. In [11] D. Xiu and G. Karniadakis generalized this expansion to polynomials of the Askey scheme. For convergence results of these generalized expansions we refer to [31].
Thus the solution x can be approximated by computing the coefficients ${\{{\mathbf{q}}_{i}\}}_{i=0}^{\stackrel{\u02c6}{p}}$. Different methods can be used to calculate the coefficients. We opt here for the stochastic collocation.
4.2 Stochastic collocation
where ${\mathbf{w}}^{(j)}$, meant as $\mathbf{w}({\theta}^{(j)})$, and ${W}^{(j)}$ are the nodes and weights of the ${N}_{Q}$node cubature and are given for each type of cubature, see [4] for more details.
The procedure for the stochastic collocation scheme would be then as follows: For each of the given nodes calculate the solution of (31) and sum them up with the corresponding weights. The products $({\mathrm{\Psi}}_{i},{\mathrm{\Psi}}_{i})$ are independent from the operator equations and can be conveniently precomputed and stored for efficient use in different calculations.
This approach offers some advantages. Contrary to MC techniques, the sampling is not random, but obey the cubature rules to minimize the needed number of nodes for a given precision. Thus, the amount of realizations is significantly reduced. Usually, 10 nodes per dimension are accurate enough. Nevertheless, for increasing number of random variables in the parametrization, the numbers of nodes grows exponentially (curse of dimensionality), and the dimensionindependent MC becomes eventually more efficient. Sparse grid cubatures have been proposed to mitigate the curse of dimensionality [4].
Another important feature that makes the stochastic collocation schema attractive within the Wiener expansion methodology is the splitting of the uncertainty quantification and the system governing equations. Since the values of the random variables are chosen in advance, we only need to solve the underlying deterministic equations for the given value of the parameters. Hence, no modification of our existing deterministic software is needed in order to account for uncertainties. This is specially valuable in the case of commercial programs, which cannot be modified, or complex software, whose adaptation for uncertainty quantification can be errorprone. A further advantage is the modularity, making implementation of refinements in the Wiener expansion algorithms and the underlying governing equations independent.
For solving the RDDEs (30) we use the stochastic collocation schema combined with the program RADAR for the computation of the DDEs.
4.3 Numerical solution of the DDEs
The software RADAR is used as the deterministic solver for DDEs. It has been developed by Ernst Hairer and Nicola Guglielmi. For more details a manual with examples is available [32].
where ${\alpha}_{j}^{(k)}=\alpha ({t}_{k}+{c}_{j}{h}_{k},{\mathbf{Y}}_{j}^{(k)})$ and ${\mathbf{u}}_{m}$ is a polynomial approximation of the solution $\mathbf{x}(t)$ on the interval $[{t}_{m},{t}_{m+1}]$. Here, the Lagrange interpolation formula is used to build the approximation.
RADAR allows steps larger and smaller than the delay and performs stepsize control by estimating the local error at grid points and the error in the continuous numerical approximation to the solution.
5 Stochastic Galerkin method and random ordinary differential equations
with initial condition $\mathbf{x}(0,\mathbf{w})\in {L}_{2}$. The extension of the SGM including the treatment of delays is straightforward using interpolation for the approximation of the solution at past times. In Section 6 we will apply the SGM to the test example of the crack growth (an RODE) and the biofilm model (an RDDE) as stated in the introductory Section 1. Subsequently, we intend to present the main idea of the SGM and restate the main results of its convergence analysis from [21, 34].
the approximation space. Without loss of generality we assume that the orthogonal polynomials ${\{{\mathrm{\Psi}}_{i}\}}_{i=0}^{\mathrm{\infty}}$ are normalized, i.e. $({\mathrm{\Psi}}_{i},{\mathrm{\Psi}}_{i})=1$.
In [34] it is shown that the local discretization error of the SGRKM does not vanish for decreasing stepsize $h\to 0$. Thus, this global approach of the stochastic Galerkin method fails to converge to the exact solution x. To overcome this problem a generalized approach has to be applied. X. Wan and G.E. Karniadakis proposed the MultiElement generalized Polynomial Chaos (MEgPC) method to attain convergence of the SGM. We recall that this method is based on a partitioning of the parameter space Ω into subsets, so called random elements. The SGM is then applied locally on the partition of the parameter space, i.e. the problem is conditionally restricted to each random element. We refer to the literature for a detailed description [18].
where we denote the maximal diameter of the stochastic elements by Δ, see [34].
5.1 The adaptive algorithm for the solution of RODEs
We use the estimator $\mathcal{I}(\mathrm{ERR})$ to decide about the refinement of the elements ${\{{\mathrm{\Omega}}_{i}\}}_{i=1}^{\nu}$. If $\mathcal{I}(\mathrm{ERR})<{\rho}_{1}$, then the element will be refined. If $\mathcal{I}(\mathrm{ERR})$ exceeds a coarsening threshold ${\rho}_{2}>{\rho}_{1}$, then the element will be coarsened. For a more detailed description of this procedure see [34].
6 Computational aspects and numerical results
In this section, we apply the three methods, the UKF approach, the stochastic collocation and the SGRKM, to the benchmark problems stated in the introduction. From the solution we compute the densities and the first two moments and compare the results. For the numerical time integration in the SGM we use an embedded explicit RKM of order $4(3)$. After a discussion of the efficiencies of the methods, we first consider the crack growth example (1) with the initial crack size being uncertain. Thereafter, we study the setting of two uncertain parameters in the model of crack growth. We finish this section by the comparison of the three methods applied to the biofilm model (3).
6.1 Computational aspects
Number of function evaluations for UKF, SCM and MEgPC approach based on a Gaussian cubature rule of order s
UKF  2n + 1 
SCM  ${(s+1)}^{n}$ 
MEgPC  ${(s+1)}^{n}$ 
6.2 Comparison of UKF, Stochastic Collocation and SGM in crack growth example
In this section, we compare the presented methods and Monte Carlo Sampling (10^{6} samples) in the crack example (1).
6.2.1 One dimensional random space
At first, we assume only the initial crack size ${a}_{0}$ to be a random parameter following a uniform distribution with expected value 10^{−1} and variance 10^{−6}. The other parameters are set to ${K}_{{I}_{c}}={10}^{12}$, ${\mathrm{\Delta}}_{\sigma}=16$ and ${C}_{F}=0$. We remark that although the righthand side function f in (1) is not polynomial in the initial value ${a}_{0}$, the convergence results from Section 5 apply, see [21, 34] for a detailed discussion.
Setting of parameters of the adaptive MEgPC approach for the solution of RODE ( 1 )
Degree of approximation  $\stackrel{\u02c6}{p}$  3 
Order of approximation  p  3 
Tolerance (time)  TOL  10^{−7} 
Tolerance (elements)  ε  10^{−8} 
Refinement threshold  ${\rho}_{1}$  1 
Coarsening threshold  ${\rho}_{2}$  4 
Although the initial crack size varies only by a small amount, the impact on the outcome is not negligible. A large variation in the crack size occurs at the final time $t=5\text{,}420$. The dependence of the solution $a(t,{a}_{0})$ on the uncertain initial crack size ${a}_{0}$ is highly nonlinear. This is recognized by the MEgPC algorithm, which refines the parameter space where the response surface can not be represented by a polynomial of degree $\stackrel{\u02c6}{p}=3$ up to the prescribed tolerance.
The expected values and variances of $\mathit{a}\mathbf{(}\mathit{t}\mathbf{,}{\mathit{a}}_{\mathbf{0}}\mathbf{)}$ at times $\mathit{t}\mathbf{\in}\mathbf{\{}\mathbf{5}\mathbf{\text{,}}\mathbf{000}\mathbf{,}\mathbf{5}\mathbf{\text{,}}\mathbf{420}\mathbf{\}}$ computed by SCM5, UKF, MEgPC and Monte Carlo simulation
Time  Expected value  Variance  

SCM5  MEgPC  UKF  MCS  SCM5  MEgPC  UKF  MCS  
5,000  0.1.601  0.1.601  0.1.602  0.1601  4.727⋅10^{−5}  4.729⋅10^{−5}  5.113⋅10^{−5}  4.728⋅10^{−5} 
5,420  0.2.681  0.2.726  0.2.671  0.2749  5.655⋅10^{−2}  7.791⋅10^{−2}  2.449⋅10^{−2}  8.627⋅10^{−2} 
6.2.2 Two dimensional random space
In this section we consider the crack example (1) with two uncertain parameters. In addition to Section 6.2.1 we assume the parameter ${K}_{Ic}$ to be uniformly distributed with expected value 0.25 and variance 0.02.
Setting of parameters of the adaptive MEgPC approach for the solution of RODE ( 1 )
Degree of approximation  $\stackrel{\u02c6}{p}$  (3,3) 
Order of approximation  p  3 
Tolerance (time)  TOL  10^{−8} 
Tolerance (elements)  ε  10^{−8} 
Refinement threshold  ${\rho}_{1}$  1 
Coarsening threshold  ${\rho}_{2}$  4 
The expected values and variances of $\mathit{a}\mathbf{(}\mathit{t}\mathbf{,}{\mathit{a}}_{\mathbf{0}}\mathbf{,}{\mathit{K}}_{\mathit{I}\mathit{c}}\mathbf{)}$ at times $\mathit{t}\mathbf{\in}\mathbf{\{}\mathbf{3}\mathbf{\text{,}}\mathbf{000}\mathbf{,}\mathbf{3}\mathbf{\text{,}}\mathbf{500}\mathbf{,}\mathbf{3}\mathbf{\text{,}}\mathbf{550}\mathbf{\}}$ computed by SCM5, UKF, MEgPC and Monte Carlo simulation based on ${\mathbf{10}}^{\mathbf{6}}$ samples
Time  Expected value  Variance  

SCM5  MEgPC  UKF  MCS  SCM5  MEgPC  UKF  MCS  
3,000  0.1343  0.1343  0.1342  0.1343  2.557⋅10^{−5}  2.562⋅10^{−5}  2.142⋅10^{−5}  2.566⋅10^{−5} 
3,500  0.1504  0.1504  0.1497  0.1504  1.663⋅10^{−4}  1.674⋅10^{−4}  1.098⋅10^{−4}  1.667⋅10^{−4} 
3,550  0.1534  0.1534  0.1521  0.1534  2.661⋅10^{−4}  2.718⋅10^{−4}  1.404⋅10^{−4}  2.718⋅10^{−4} 
6.3 Comparison of UKF, Stochastic Collocation and SGM in biofilm example
The introductory model for the biofilm (3) with delay Quorum Sensing has been studied in [36] in great detail. Here we compare the numerical approximations of the solution by means of its densities and first two moments. Therefore, we compute them by the UKF approach, the stochastic collocation and the MEgPC method at the time $t=100$. The uncertain parameter is assumed to be the delay τ, a $beta(2,2)$ distributed random variable with expected value 4.8. Moreover, we use the choice of parameters ${\alpha}_{20}=0.25$, $\sigma ={\alpha}_{2}={\alpha}_{3}=0.6$, ${S}_{A}=0.4$ and ${\mu}_{A}=0.5$ as well as the initial populations $B(0)=1$ and $A(0)=0$.
Setting of parameters of the adaptive MEgPC approach for the biofilm model
Degree of approximation  $\stackrel{\u02c6}{p}$  5 
Order of approximation  p  3 
Tolerance (time)  TOL  5⋅10^{−6} 
Tolerance (elements)  ε  × 
Refinement threshold  ${\rho}_{1}$  × 
Coarsening threshold  ${\rho}_{2}$  × 
The expected values and variances of the biofilm model at time $\mathit{t}\mathbf{=}\mathbf{100}$ computed by the MEgPC approach, the UKF and the stochastic collocation method
Method  Expected value  Variance 

MEgPC  1.349  1.707 
UKF  1.388  1.801 
SCM order 5  1.352  1.706 
SCM order 9  1.352  1.706 
7 Conclusions
In the present article, we discussed the numerical treatment of problems with uncertain parameters. We gave a detailed introduction to the unscented Kalman filter and its application to random ordinary differential and random delay differential equations, see Section 3. We dropped the measurements in the equations and model the uncertainty by random variables. In this setting, the unscented Kalman filter can be interpreted as a stochastic collocation method of order 2, see Section 3.5. Sampling of the resulting surrogate polynomial model yields an approximation of the moments of the solution. Due to the fact that the sampling is restricted to the directions given by the columns of the root of the covariance matrix ${P}^{xx}$, only $2n+1$ realizations of the underlying random differential equation (RDE), resp. an RDDE or an RODE, are needed in the UKF approach. Here, n denotes the number of uncertain parameters. Thus, it is a cheap, in terms of computational costs, method to approximate the random behavior of the solution.
We compare this simplified UKF approach with the SCM and the MEgPC method. In the SCM more realizations, as compared to the UKF, of the solution of the RDE have to be computed. The number of realizations depends on the order of the approximation and the underlying cubature method. For most cubature rules, even in sparse grid approximations, the number of function evaluations is considerably greater than $2n+1$ and therefore the SCM is computationally more expensive. The SGM, as opposed to the SCM and the UKF, is not based on sampling, but on a spectral expansion of the solution of the RDE into orthogonal polynomials. An approximate solution is found by a Galerkin approach which results in the stochastic weak form (34). If we are able to compute it in a preprocessing step, only a system of deterministic differential equations for the coefficients of the truncated generalized Wiener expansion has to be solved. The dimension of the projected RDE is small, because we only need a low order approximation within the respective stochastic elements. The overall computational costs scale linearly with the number of stochastic elements. In the present article we did not take advantage of the preprocessing of the projection in the stochastic weak form. On the one hand, the projection in the examples from Section 6 can be efficiently performed by an adaptive Simpson cubature rule. On the other hand, preprocessing the stochastic weak form would result in an projection error, which can not be easily estimated a priori.
In Section 6 we applied the three methods to the benchmark problems of the crack growth model (1) and the biofilm model (3). In the case of the crack growth, which is an RODE, we discussed the prescription of up to two random parameters. It revealed that the UKF approach results in a nonsufficient representation of the overall density, but gives a good approximation of the first two moments. Opposed to that, the MEgPC method computes a good approximation of the density and of the first two moments. This is in agreement of the convergence results in Section 5. In the second benchmark problem, the example of the biofilm (3), we again compared the densities of the solutions computed by the three methods. The SCM, as well as the SGM method showed perfect agreement of the results  in the density and in the first two moments. Again, opposed to that the UKF approach yielded only a rough approximation of the overall density. Nevertheless, it is able to compute a good approximation of the first two moments.
From the numerical results in Section 6 we see that the UKF approach is a cheap alternative to get a rough impression of the random behavior of the solution of the RDE. If detailed information about the solution process is needed, more sophisticated methods like SCM and multielement SGM have to be applied. This is for example in failure detection the case, where the tail of the probability distribution is of great interest.
Supplementary material
References
 1.Augustin F, Gilg A, Paffrath M, Rentrop P, Wever U: Polynomial chaos for the approximation of uncertainties: chances and limits. Eur. J. Appl. Math. 2008, 19: 149–190.MATHMathSciNetCrossRefGoogle Scholar
 2.Xiu D: Numerical Methods for Stochastic Computations: A Spectral Method Approach. Princeton University Press, Princeton; 2010.Google Scholar
 3.Karniadakis GE, Su CH, Xiu D, Lucor D, Schwab C, Todor RA: Generalized polynomial chaos solution for differential equations with random inputs. Report Nr. 2005–01, ETH Zürich. Seminar für Angewandte Mathematik; 2005.Google Scholar
 4.Le Maître OP, Knio OM Scientific Computation. In Spectral Methods for Uncertainty Quantification. Springer, New York; 2010.CrossRefGoogle Scholar
 5.Fox BL: Strategies for QuasiMonte Carlo. Springer, Berlin; 1999.CrossRefGoogle Scholar
 6.Babuska I, Tempone R, Zouraris G: Galerkin finite element approximation of stochastic elliptic partial differential equations. SIAM J. Numer. Anal. 2004,42(2):800–825. 10.1137/S0036142902418680MATHMathSciNetCrossRefGoogle Scholar
 7.Pulch R: Polynomial chaos for multirate partial differential algebraic equations with random parameters. Appl. Numer. Math. 2009,59(10):2610–2624. 10.1016/j.apnum.2009.05.015MATHMathSciNetCrossRefGoogle Scholar
 8.Xiu D: Fast numerical methods for stochastic computations. Commun. Comput. Phys. 2008, 5: 242–272.MathSciNetGoogle Scholar
 9.Maître OPL, Knio OM, Najm HN, Ghanem RG: Uncertainty propagation using WienerHaar expansions. J. Comput. Phys. 2004, 197: 28–57. 10.1016/j.jcp.2003.11.033MATHMathSciNetCrossRefGoogle Scholar
 10.Ghanem R, Spanos P: Stochastic Finite Elements: A Spectral Approach. Springer, New York; 1991.MATHCrossRefGoogle Scholar
 11.Xiu D, Karniadakis G: The WienerAskey polynomial chaos for stochastic differential equations. SIAM J. Sci. Comput. 2002,24(2):619–644. 10.1137/S1064827501387826MATHMathSciNetCrossRefGoogle Scholar
 12.Schevenels M, Lombaerts G, Degrande G: Application of the stochastic finite element method for Gaussian and nonGaussian systems. ISMA2004 International Conference on Noise and Vibration Engineering Leuven, Belgien 2004.Google Scholar
 13.Kalman R: A new approach to linear filtering and prediction problems. Trans. ASME, J. Basic Eng. 1960, 82: 35–45.CrossRefGoogle Scholar
 14.Welch G, Bishop G: An introduction to the Kalman filter. Tr 95–041, UNCChapel Hill; 2006.Google Scholar
 15.Julier SJ, Uhlmann JK, DurrantWhyte HF: A new method for the nonlinear transformation of means and covariances in filters and estimators. IEEE Trans. Autom. Control 2000,45(3):477–482. 10.1109/9.847726MATHMathSciNetCrossRefGoogle Scholar
 16.Wiener N: The homogeneous chaos. Am. J. Math. 1938, 60: 897–936. 10.2307/2371268MathSciNetCrossRefGoogle Scholar
 17.Cameron R, Martin W: The orthogonal development of nonlinear functionals in series of FourierHermite functionals. Ann. Math. 1947,48(2):385–392. 10.2307/1969178MATHMathSciNetCrossRefGoogle Scholar
 18.Wan X, Karniadakis G: An adaptive multielement generalized polynomial chaos method for stochastic differential equations. J. Comput. Phys. 2005, 209: 617–642. 10.1016/j.jcp.2005.03.023MATHMathSciNetCrossRefGoogle Scholar
 19.Maître OPL, Njam HN, Ghanem RG, Knio OM: Multiresolution analysis of Wienertype uncertainty propagation schemes. J. Comput. Phys. 2004,197(2):502–531. 10.1016/j.jcp.2003.12.020MATHMathSciNetCrossRefGoogle Scholar
 20.Wan X, Karniadakis G: Multielement generalized polynomial chaos for arbitrary probability measures. SIAM J. Sci. Comput. 2006,26(3):901–928.MathSciNetCrossRefGoogle Scholar
 21.Augustin F, Rentrop P: Stochastic Galerkin techniques for random ordinary differential equations. Numer. Math. 2012, 122: 399–419. 10.1007/s0021101204668MATHMathSciNetCrossRefGoogle Scholar
 22.Paffrath M, Wever U: Stochastic integration methods and their application to reliability analysis. ASME Turbo Expo Copenhagen, Denmark 2012.Google Scholar
 23.Villegas M: Random delay differential equations: application to biofilm modeling. Phd thesis. Techn. Universität München (Fachbereich Mathematik, Prof. Rentrop); 2011.Google Scholar
 24.Harris DO: Probabilistic crack growth and modeling. In ReliabilityBased Mechanical Design. Edited by: Cruse T. Marcel Dekker, New York; 1997.Google Scholar
 25.Forman RG: Fatigue crack growth computer program, NASA/FLAGRO version 2.0. Johnson Space Center Report JSC22267A, NASA; 1993.Google Scholar
 26.Juan Z, Zhongua Z, Yaohong S: Hopf bifurcation of a bacteriaimmunity system with delayed quorum sensing. Appl. Math. Comput. 2010, 215: 3936–3949. 10.1016/j.amc.2009.11.042MATHMathSciNetCrossRefGoogle Scholar
 27.Kuhlmann H: Kalmanfiltering with coloured measurement noise for deformation analysis. Proceedings, 11th FIG Symposium on Deformation Measurements Santorini, Greece 2003.Google Scholar
 28.Weber B: Numerik nichtlinearer Systeme mit Unsicherheiten. Master thesis. Technische Universität München, Prof. Rentrop; 2008.Google Scholar
 29.Julier S, Uhlmann J: Unscented filtering and nonlinear estimation. Proc. IEEE 2004,92(3):401–422. 10.1109/JPROC.2003.823141CrossRefGoogle Scholar
 30.Janson S: Gaussian Hilbert Spaces. Cambridge University Press, Cambridge; 1997.MATHCrossRefGoogle Scholar
 31.Ernst OG, Mugler A, Starkloff HJ, Ullmann E: On the convergence of generalized polynomial chaos expansions. ESAIM Math. Model. Numer. Anal. 2012, 46: 317–339. 10.1051/m2an/2011045MATHMathSciNetCrossRefGoogle Scholar
 32.Guglielmi N, Hairer E: Users’ guide for the code RADAR5  version 2.1. Technical Report  available at http://www.unige.ch/~hairer/software.html 2005.Google Scholar
 33.Hairer E, Wanner G Stiff and DifferentialAlgebraic Problems. In Solving Ordinary Differential Equations II. Springer, Berlin; 1996.CrossRefGoogle Scholar
 34.Augustin F: Generalized Wiener Expansions for the Numerical Solution of Random Ordinary Differential Equations. Dr. Hut, München; 2012.Google Scholar
 35.Hairer E, Nørsett SP, Wanner G: Solving Ordinary Differential Equations I. Springer, Berlin; 1993.MATHGoogle Scholar
 36.Villegas M: Biofilms with quorum sensing: A numerical approach for delay differential equations with uncertainties. Berichte aus den numerischen Arbeitsgruppen. Technische Universität München; 2011, 8/2011.Google Scholar
Copyright information
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.