Modeling and Prediction Using Stochastic Differential Equations

Abstract

Pharmacokinetic/pharmakodynamic (PK/PD) modeling for a single subject is most often performed using nonlinear models based on deterministic ordinary differential equations (ODEs), and the variation between subjects in a population of subjects is described using a population (mixed effects) setup that describes the variation between subjects. The ODE setup implies that the variation for a single subject is described by a single parameter (or vector), namely the variance (covariance) of the residuals. Furthermore the prediction of the states is given as the solution to the ODEs and hence assumed deterministic and can predict the future perfectly. A more realistic approach would be to allow for randomness in the model due to e.g., the model be too simple or errors in input. We describe a modeling and prediction setup which better reflects reality and suggests stochastic differential equations (SDEs) for modeling and forecasting. It is argued that this gives models and predictions which better reflect reality. The SDE approach also offers a more adequate framework for modeling and a number of efficient tools for model building. A software package (CTSM-R) for SDE-based modeling is briefly described.

Appendix A: Extended Kalman Filtering

For nonlinear models the innovation vectors ${\mathbf {\varepsilon }_k}$ (or ${\mathbf {\varepsilon }^i_k}$) and their covariance matrices ${{\Sigma }^{yy}_{k|k-1}}$ (or ${{\Sigma }^{yy,i}_{k|k-1}}$) can be computed recursively by means of the extended Kalman filter (EKF) as outlined in the following.

Consider first the linear time-varying model

$$\begin{aligned} d\mathbf {X}_t= & {} \left( \mathbf {A}(\mathbf {u}_t,t,\mathbf {\theta })\mathbf {X}_t+ B(\mathbf {u}_t,t,\mathbf {\theta })\right) dt+ \mathbf {\sigma }(\mathbf {u}_t,t,\mathbf {\theta })d\mathbf {\omega }_t\end{aligned}$$

(46)

$$\begin{aligned} \mathbf {Y}_k= & {} \mathbf {C}(\mathbf {u}_k,t_k,\mathbf {\theta })\mathbf {X}_k+\mathbf {e}_k \end{aligned}$$

(47)

in the following we will use $\mathbf {A}(t)$, $\mathbf {B}(t)$, and $\mathbf {\sigma }(t)$ as short-hand notation for $\mathbf {A}(\mathbf {u}_t,t,\mathbf {\theta })$, $B(\mathbf {u}_t,t,\mathbf {\theta })$, and $\mathbf {\sigma }(\mathbf {u}_t,t,\mathbf {\theta })$.

We will restrict ourselves to the initial value problem; solve (46) for $t\in [t_k,t_{k+1}]$ given that the initial condition $X_{t_k}\sim N(\hat{\mathbf {x}}_{k|k}, \Sigma ^{xx}_{k|k})$. This is the kind of solution we would get from the ordinary Kalman filter in the update step.

Now if we consider, the transformation

$$\begin{aligned} \mathbf {Z}_t = e^{-\int _{t_k}^t\mathbf {A}(s)ds}\mathbf {X}_t, \end{aligned}$$

(48)

then by Itô’s Lemma, it can be shown that the process $\mathbf {Z}_t$ is governed by the Itô stochastic differential equation

$$\begin{aligned} d\mathbf {Z}_t = e^{-\int _{t_k}^t\mathbf {A}(s)ds}\mathbf {B}(t)dt+ e^{-\int _{t_k}^t\mathbf {A}(s)ds}\mathbf {\sigma }(t) d\mathbf {\omega }_t \end{aligned}$$

(49)

with initial conditions $\mathbf {Z}_{t_k}\sim N(\hat{\mathbf {x}}_{k|k}, \Sigma ^{xx}_{k|k})$. The solution to (49) is given by the integral equation

$$\begin{aligned} \mathbf {Z}_t= & {} \mathbf {Z}_{t_k} +\int _{t_k}^te^{-\int _{t_k}^u\mathbf {A}(u)du} \mathbf {B}(s)ds+ \int _{t_k}^t e^{-\int _{t_k}^s\mathbf {A}(u)du}\mathbf {\sigma }(s) d\mathbf {\omega }_s \end{aligned}$$

(50)

Now inserting the inverse of the transformation (48) gives

$$\begin{aligned} \mathbf {X}_t= & {} e^{\int _{t_k}^t\mathbf {A}(s)ds}\mathbf {X}_0 + e^{\int _{t_k}^t\mathbf {A}(s)ds} \int _{t_k}^te^{-\int _{t_k}^u\mathbf {A}(u)du}\mathbf {B}(s)ds \nonumber \\&+\; e^{\int _{t_k}^t\mathbf {A}(s)ds}\int _{t_k}^t e^{-\int _{t_k}^s\mathbf {A}(u)du}\mathbf {\sigma }(s) d\mathbf {\omega }_s \end{aligned}$$

(51)

Taking the exception and variance on both sides of (51) gives

$$\begin{aligned} E[\mathbf {X}_t]= & {} e^{\int _{t_k}^t\mathbf {A}(s)ds}E[\mathbf {X}_{t_k}] + e^{-\int _{t_k}^t\mathbf {A}(s)ds} \int _{t_k}^te^{-\int _{t_k}^u\mathbf {A}(u)du} \mathbf {B}(s)ds \end{aligned}$$

(52)

$$\begin{aligned} V[\mathbf {X}_t]= & {} e^{\int _{t_k}^t\mathbf {A}(s)ds}V[\mathbf {X}_{t_k}] e^{\int _{t_k}^t\mathbf {A}(s)^Tds}+ e^{\int _{t_k}^t\mathbf {A}(s)ds}V\left[ \int _{t_k}^t e^{-\int _{t_k}^s\mathbf {A}(u)du}\mathbf {\sigma }(s) d\mathbf {\omega }_s\right] e^{\int _{t_k}^tA(s)^Tds}\nonumber \\= & {} e^{\int _{t_k}^t\mathbf {A}(s)ds}V[\mathbf {X}_0]e^{\int _{t_k}^t\mathbf {A}(s)^Tds}\nonumber \\&+\,e^{\int _{t_k}^t\mathbf {A}(s)ds}\int _{t_k}^t e^{-\int _{t_k}^s\mathbf {A}(u)du}\mathbf {\sigma }(s)\mathbf {\sigma }(s)^T e^{-\int _{t_k}^s\mathbf {A}(u)^Tdu}ds e^{\int _{t_k}^t\mathbf {A}^T(s)ds}, \end{aligned}$$

(53)

where we have used Itô isometry in the second equation for the variance. Now differentiation the above expression w.r.t. time gives

$$\begin{aligned} \frac{dE[\mathbf {X}_t]}{dt}= & {} \mathbf {A}(t)E[\mathbf {X}_t]+\mathbf {B}(t)\end{aligned}$$

(54)

$$\begin{aligned} \frac{dV[\mathbf {X}_t]}{dt}= & {} \mathbf {A}(t)V[\mathbf {X}_t]+ V[\mathbf {X}_t]\mathbf {A}(t)^T+ \mathbf {\sigma }(t)\mathbf {\sigma }(t)^T, \end{aligned}$$

(55)

with initial conditions given by $E[\mathbf {X}_{t_k}]=\hat{\mathbf {x}}_{k|k}$ and $V[\mathbf {X}_{t_k}]=\Sigma ^{xx}_{k|k}$.

For the nonlinear case

$$\begin{aligned} d\mathbf {X}_t= & {} \mathbf {f}(\mathbf {X}_t,\mathbf {u}_t,t,\mathbf {\theta })dt+\mathbf {\sigma }(\mathbf {u}_t,t,\mathbf {\theta })d\mathbf {\omega }_t\end{aligned}$$

(56)

$$\begin{aligned} \mathbf {Y}_k= & {} \mathbf {h}(\mathbf {X}_k,\mathbf {u}_k,t_k,\mathbf {\theta })+\mathbf {e}_k, \end{aligned}$$

(57)

we introduce the Jacobian of $\mathbf {f}$ around the expectation of $\mathbf {X}_t$ ($\hat{\mathbf {x}}_t=E[\mathbf {X}_t]$), we will use the following short hand notation

$$\begin{aligned} \mathbf {A}(t)= & {} \frac{\partial \mathbf {f}(\mathbf {x},\mathbf {u}_t,t,\mathbf {\theta })}{\partial \mathbf {x}}\bigg |_{\mathbf {x}=\hat{\mathbf {x}}_{t|k}}\quad , \quad \mathbf {f}(t) = \mathbf {f}(\hat{\mathbf {x}}_{t|k},\mathbf {u}_t,t,\mathbf {\theta }) \end{aligned}$$

(58)

where $\hat{\mathbf {x}}_{t}$ is the expectation of $\mathbf {X}_t$ at time t, this implies that we can write the first-order Taylor expansion of (56) as

$$\begin{aligned} d\mathbf {X}_t\approx \left[ \mathbf {f}(t)+\mathbf {A}(t)(\mathbf {X}_t-\hat{\mathbf {x}}_{t|k})\right] dt+\mathbf {\sigma }(t)d\mathbf {\omega }_t. \end{aligned}$$

(59)

Using the results from the linear time-varying system above, we get the following approximate solution to the (59)

$$\begin{aligned} \frac{dE[\mathbf {X}_t]}{dt}\approx & {} \mathbf {f}(t)\end{aligned}$$

(60)

$$\begin{aligned} \frac{dV[\mathbf {X}_t]}{dt}\approx & {} \mathbf {A}(t)V[\mathbf {X}_t]+ V[\mathbf {X}_t]\mathbf {A}^T(t)+\mathbf {\sigma }(t)\mathbf {\sigma }^T(t), \end{aligned}$$

(61)

with initial conditions $E[\mathbf {X}_{t_k}]=\hat{\mathbf {x}}_{k|k}$ and $V[\mathbf {X}_{t_k}]=\Sigma ^{xx}_{k|k}$. Equations (60) and (61) constitute the basis of the prediction step in the Extended Kalman Filter, which for completeness is given below

Theorem 1

(Continuous-discrete time extended Kalman filter) With given initial conditions for the ${\hat{\mathbf {x}}_{1|0}=\mathbf {x}_0}$ and ${{\Sigma }^{xx}_{1|0}= {\Sigma }^{xx}_0}$ the extended Kalman filter approximations are given by; the output prediction equations:

$$\begin{aligned} \hat{\mathbf {y}}_{k|k-1}= & {} \mathbf {h}(\hat{\mathbf {x}}_{k|k-1},\mathbf {u}_k,t_k,\mathbf {\theta });\quad \Sigma ^{yy}_{k|k-1} = \mathbf {C}_k \Sigma ^{xx}_{k|k-1}\mathbf {C}_k^T+\mathbf {S}_k \end{aligned}$$

(62)

the innovation and Kalman gain equation:

$$\begin{aligned} \mathbf {\varepsilon }_k=\mathbf {y}_k-\hat{\mathbf {y}}_{k|k-1};\quad \mathbf {K}_k={\Sigma }^{xx}_{k|k-1}\mathbf {C}^T\left( {\Sigma }^{yy}_{k|k-1}\right) ^{-1} \end{aligned}$$

(63)

the updating equations:

$$\begin{aligned} \hat{\mathbf {x}}_{k|k}= & {} \hat{\mathbf {x}}_{k|k-1}+\mathbf {K}_k\mathbf {\varepsilon }_k;\quad \Sigma ^{xx}_{k|k} = \Sigma ^{xx}_{k|k-1}-\mathbf {K}_k\Sigma ^{yy}_{k|k-1}\mathbf {K}_k^T \end{aligned}$$

(64)

and the state prediction equations:

$$\begin{aligned} \frac{d\hat{\mathbf {x}}_{t|k}}{dt}= & {} \mathbf {f}(\hat{\mathbf {x}}_{t|k},\mathbf {u}_t,t,\mathbf {\theta }){\text { , }}t\in [t_k,t_{k+1}[\end{aligned}$$

(65)

$$\begin{aligned} \frac{d\Sigma ^{xx}_{t|t_k}}{dt}= & {} \mathbf {A}(t)\Sigma ^{xx}_{t|t_k}+\Sigma ^{xx}_{t|t_k}\mathbf {A}(t)^T+\mathbf {\sigma }(t)\mathbf {\sigma }(t)^T\text { , }t\in [t_k,t_{k+1}[ \end{aligned}$$

(66)

where the following short-hand notation has been applied:

$$\begin{aligned} \mathbf {A}(t)=\frac{\partial \mathbf {f}(\mathbf {x},\mathbf {u}_t,t,\mathbf {\theta })}{\partial \mathbf {x}}\bigg |_{\mathbf {x}=\hat{\mathbf {x}}_{t|k-1}}&,&\quad \mathbf {C}_k=\frac{\partial \mathbf {h}(\mathbf {x},\mathbf {u}_{t_k},t_k,\mathbf {\theta })}{\partial \mathbf {x}}\bigg |_{\mathbf {x}=\hat{\mathbf {x}}_{k|k-1}} \end{aligned}$$

(67)

$$\begin{aligned} \mathbf {\sigma }(t)=\mathbf {\sigma }(\mathbf {u}_t,t,\mathbf {\theta })&,&\quad \mathbf {S}_k=\mathbf {S}(\mathbf {u}_k,t_k,\mathbf {\theta }) \end{aligned}$$

(68)

The prediction step was covered above and the updating step can be derived from linearization of the observation equation and the projection theorem [6]. From the construction above, it is clear that the approximation is only likely to hold if the nonlinearities are not too strong. This implies that the sampling frequency is fast enough for the prediction equations to be a good approximation and that the accuracy in the observation equation is good enough for the Gaussian approximation to hold approximately. Even though “simulation” through the prediction equations is available in CTSM-R, it is recommended that simulation results are verified (or indeed performed), by real-stochastic simulations (e.g., by simple Euler simulations).