The Stochastic Quasi-chemical Model for Bacterial Growth: Variational Bayesian Parameter Update

Abstract

We develop Bayesian methodologies for constructing and estimating a stochastic quasi-chemical model (QCM) for bacterial growth. The deterministic QCM, described as a nonlinear system of ODEs, is treated as a dynamical system with random parameters, and a variational approach is used to approximate their probability distributions and explore the propagation of uncertainty through the model. The approach consists of approximating the parameters’ posterior distribution by a probability measure chosen from a parametric family, through minimization of their Kullback–Leibler divergence.

Appendix A: Gradient Computation of the Log-joint Distribution \(\log p(\mathbf {y}, \varvec{\theta })\)

In order to perform gradient-based optimization of the ELBO approximation $\mathcal {F}_2[q]$ with respect to $\{\varvec{\mu }\}_i, \{\varvec{\Sigma }_i\}_i, i= 1,\dots , L$, we need to compute its gradient vector with entries

$$\begin{aligned} \frac{\partial }{\partial \zeta }\mathcal {F}_r[q] = \frac{\partial }{\partial \zeta }\mathcal {H}_0[q] + \frac{\partial }{\partial \zeta }\mathcal {L}_r[q] \end{aligned}$$

(A.1)

where $\zeta = (\varvec{\mu }_i)_j, (\varvec{\Sigma }_i)_{jk}$, for $i= 1,\dots ,L, j,k = 1,\dots , d$ and $r = 0,2$. Below we provide the details in computing the gradient of $\mathcal {L}_r[q], r=0,2$.

1.1 A.1: Gradient of $\mathcal {L}_r[q]$

For convenience, we set $J(\varvec{\theta }) := \log p(\mathbf {y}, \varvec{\theta })$. Then for $r=0$, the derivatives of $\mathcal {L}_0[q]$ with respect to $\zeta = (\varvec{\mu }_i)_j$ are

$$\begin{aligned} \frac{\partial }{\partial \zeta }\mathcal {L}_0[q] = \frac{1}{L}\frac{\partial }{\partial \theta _j}J(\varvec{\mu }_i) \end{aligned}$$

(A.2)

and with respect to $\zeta = (\varvec{\Sigma })_{jk}$ are

$$\begin{aligned} \frac{\partial }{\partial \zeta }\mathcal {L}_0[q] = 0. \end{aligned}$$

(A.3)

For $r = 2$ and $\zeta = (\varvec{\Sigma }_i)_{jk}$, we get

$$\begin{aligned} \frac{\partial }{\partial \zeta }\mathcal {L}_2[q] = \frac{1}{2L} \frac{\partial ^2}{\partial \theta _j \partial \theta _k} J(\varvec{\mu }_i). \end{aligned}$$

(A.4)

As mentioned above, the derivatives of $\mathcal {L}_2[q]$ with respect to $(\varvec{\mu }_i)_j$ are not used in our optimization scheme and therefore are not computed here.

1.2 A.2: Derivatives of $J(\varvec{\theta })$

First we rewrite $\varvec{\theta }= (\varvec{\xi }, \omega )$ and expand

$$\begin{aligned} J(\varvec{\theta }) = J(\varvec{\xi }, \omega ) = \log p(\mathbf {y}| \mathcal {G}(\varvec{\xi }), \omega ) + \log p(\varvec{\xi }) + \log p(\omega ). \end{aligned}$$

(A.5)

Throughout our numerical examples, we work with an isotropic Gaussian likelihood ($\varvec{\epsilon }\sim \mathcal {N}(\mathbf {0}, \sigma \mathbf {I}_N)$); therefore, we set

$$\begin{aligned} L(\mathcal {G}(\varvec{\xi }),\omega ; \mathbf {y}) := \log p(\mathbf {y}|\mathcal {G}(\varvec{\xi }), \omega ) = \log \mathcal {N}(\mathbf {y}| \mathcal {G}(\varvec{\xi }), e^{2\omega }\mathbf {I}_N), \end{aligned}$$

(A.6)

and using the chain rule, we have

$$\begin{aligned} \frac{\partial J}{\partial \xi _j}= & {} \sum _{s=1}^N\frac{\partial L}{\partial \mathcal {G}_s} \frac{\partial \mathcal {G}_s}{\partial \xi _j} + \frac{1}{p(\varvec{\xi })}\frac{\partial p(\varvec{\xi })}{\partial \xi _j} \end{aligned}$$

(A.7)

$$\begin{aligned} \frac{\partial J}{\partial \omega }= & {} \frac{\partial L}{\partial \omega } + \frac{1}{p(\omega )}\frac{d p(\omega )}{d\omega }\end{aligned}$$

(A.8)

$$\begin{aligned} \frac{\partial ^2 J}{\partial \xi _j\partial \xi _k}= & {} \sum _{s,t = 1}^N\frac{\partial ^2 L}{\partial \mathcal {G}_s \partial \mathcal {G}_t}\frac{\partial \mathcal {G}_s}{\partial \xi _j}\frac{\partial \mathcal {G}_t}{\partial \xi _k} + \sum _{s=1}^N\frac{\partial L}{\partial \mathcal {G}_s}\frac{\partial ^2 \mathcal {G}_s}{\partial \xi _j\partial \xi _k} \nonumber \\&+ \frac{1}{p(\varvec{\xi })}\frac{\partial ^2 p(\varvec{\xi })}{\partial \xi _j\partial \xi _k} -\frac{1}{p(\varvec{\xi })^2}\frac{\partial p(\varvec{\xi })}{\partial \xi _j}\frac{\partial p(\varvec{\xi })}{\partial \xi _k}\end{aligned}$$

(A.9)

$$\begin{aligned} \frac{\partial ^2 J}{\partial \omega ^2}= & {} \frac{\partial ^2 L}{\partial \omega ^2} + \frac{1}{p(\omega )}\frac{d^2 p(\omega )}{d\omega ^2} -\frac{1}{p(\omega )^2}\frac{d p(\omega )}{d\omega }\end{aligned}$$

(A.10)

$$\begin{aligned} \frac{\partial ^2 J}{\partial \xi _j\partial \omega }= & {} \sum _{s=1}^N\frac{\partial ^2 L}{\partial \mathcal {G}_s \partial \omega }\frac{\partial \mathcal {G}_s}{\partial \xi _j} \end{aligned}$$

(A.11)

In the above expressions, it becomes clear that the Jacobian and Hessian of the forward model $\mathcal {G}(\varvec{\xi })$ need to be computed. As mentioned in our application, the covariance matrices of the Gaussian mixtures components are taken to be diagonal which implies that only the diagonal elements of the Hessian of $\mathcal {G}(\varvec{\xi })$ are necessary.

1.3 A.3: Log-Likelihood Derivatives

The derivatives of the log-likelihood function required for the expression in the previous subsection are given as follows:

$$\begin{aligned} \frac{\partial L}{\partial \mathcal {G}_s}= & {} e^{-2\omega }(y_s - \mathcal {G}_s(\varvec{\xi })) \end{aligned}$$

(A.12)

$$\begin{aligned} \frac{\partial L}{\partial \omega }= & {} e^{-\omega }\left( ||\mathbf {y}- \mathcal {G}(\varvec{\xi })||_2^2e^{-2\theta } - k+1\right) \end{aligned}$$

(A.13)

$$\begin{aligned} \frac{\partial ^2 L}{\partial \omega ^2}= & {} e^{-\omega }\left( k-1 - 3||\mathbf {y}- \mathcal {G}(\varvec{\xi })||_2^2e^{-2\omega }\right) \end{aligned}$$

(A.14)

$$\begin{aligned} \frac{\partial ^2 L}{\partial \mathcal {G}_s \partial \mathcal {G}_t}= & {} -e^{-2\omega } \end{aligned}$$

(A.15)

$$\begin{aligned} \frac{\partial ^2 L}{\partial \mathcal {G}_s \partial \omega }= & {} -2 e^{-3\omega }\left( y_s - \mathcal {G}_s(\varvec{\xi })\right) . \end{aligned}$$

(A.16)

1.4 A.4: Derivatives of the Quasi-chemical Model

For the sake of generality and due to the presence of a nonlinear term in the quasi-chemical model, we present the general derivation of the system of ODEs satisfied by the derivatives of a solution $\mathbf {u}(t; \varvec{\xi })$ of the QCM with respect to its parameters. Assume $\mathbf {u}(t;\varvec{\xi })$ satisfies

$$\begin{aligned} \dot{\mathbf {u}}= & {} \mathbf {g}(\mathbf {u}, t ;\varvec{\xi }) \end{aligned}$$

(A.17)

$$\begin{aligned} \mathbf {u}(0)= & {} \mathbf {u}_0, \end{aligned}$$

(A.18)

where $\varvec{\xi }\in \mathbb {R}^4$ are parameters and the initial condition is fixed and independent of $\varvec{\xi }$. By simply differentiating the above system of equations, one can derive the following initial value problem satisfied by $v_{ij} = \partial u_i/\partial \xi _j$:

$$\begin{aligned} \dot{v}_{ij}= & {} \sum _{s= 1}^4 \frac{\partial g_i}{\partial u_s} v_{sj} + \frac{\partial g_i}{\partial \xi _j} \end{aligned}$$

(A.19)

$$\begin{aligned} v_{ij}(0)= & {} 0. \end{aligned}$$

(A.20)

Similarly, for the second derivatives $w_{ijk} = \partial ^2 u_i / (\partial \xi _j\partial \xi _k)$ we get

$$\begin{aligned} \dot{w}_{ijk}= & {} \sum _{s = 1}^4\frac{\partial g_i}{\partial u_s} w_{ijk} + \sum _{s, t= 1}^4\frac{\partial ^2 g}{\partial u_s\partial u_t} v_{sj}v_{tk} + \frac{\partial ^2 g_i}{\partial \xi _j\partial \xi _k} \end{aligned}$$

(A.21)

$$\begin{aligned} w_{ijk}= & {} 0. \end{aligned}$$

(A.22)

In practice, during numerical implementation one need to first solve (A.17) and then solve (A.19) using the solution of the former as forcing. At last, (A.21) can be solved by using both the QCM solution and its gradient as forcing.