Abstract
The problem of computing the transient probability distribution of countably infinite multidimensional continuous-time Markov chains (CTMCs) arising in systems of stochastic chemical kinetics is addressed by a software tool. Starting from an initial probability distribution, time evolution of the probability distribution associated with the CTMC is described by a system of linear first-order ordinary differential equations, known as the chemical master equation (CME). The solver for the CME uses the time stepping implicit backward differentiation formulae (BDF). Solution vectors in BDF can be stored compactly during transient analysis in one of the Hierarchical Tucker Decomposition, Quantized Tensor Train, or Transposed Quantized Tensor Train formats.
1 The Problem
Letting the initial probability distribution vector of the infinitesimal generator matrix Q underlying a multidimensional continuous-time Markov chain (CTMC) be denoted by \(\varvec{\pi }_0\), the transient probability distribution vector \(\varvec{\pi }_t \in \mathbb R_{\geqslant 0}^{1 \times |\mathcal{R}|}\) of Q at time \(t \in \mathbb R_{\geqslant 0}\) satisfies [12]
Here, \(\mathcal{R}\) is the reachable state space of the CTMC and \(\varvec{e}\) is a vector of 1’s.
When the CTMC arises in the area of systems of stochastic chemical kinetics, (1) is referred to as the chemical master equation (CME) [4]. In this case, there are a finite number of dimensions and transitions, but \(\mathcal{R}\) is almost always countably infinite. Therefore, a CTMC \(\{S(t), t \geqslant 0\}\) having H dimensions such that \(S(t) = (S_1(t),\ldots ,S_H(t))\) and \(\text {Pr}(S(t) = \mathbf{i}) = \text {Pr}(S_1(t) = i_1,\ldots ,S_H(t) = i_H)\) can be used with the state vector \(\mathbf{i} = (i_1,\ldots ,i_H)\). The state space of dimension h is given by \(\mathcal{S}^{(h)} = \mathbb Z_{\geqslant 0}\) for \(h=1,\ldots ,H\), and when there are no unreachable states, we have and K transition classes in which transition class k is represented by the pair \((\alpha _k(\varvec{i}),\varvec{v}^{(k)})\) for \(k=1,\ldots ,K\). Here, \(\alpha _k(\varvec{i}) \in \mathbb R_{\geqslant 0}\) is the transition rate function specifying the transition rate from state \(\varvec{i} \in \mathcal{R}\) to state \((\varvec{i} + \varvec{v}^{(k)}) \in \mathcal{R}\) and \(\varvec{v}^{(k)} \in \mathbb Z^{1 \times H}\) is the state change vector specifying the successor state of the transition, with \(v^{(k)}_{h}\) denoting the change in state variable \(i_h \in \mathcal{S}^{(h)}\) due to a class k transition [3]. That \(\mathcal{R}\) is equal to the product state space is a property of the models under consideration; but, this can be relaxed with the help of a well known hierarchical state space structuring approach [1].
A Kronecker representation for models in this area, which has separable state dependent transition rate functions in the form
can be obtained by letting the transition matrix of dimension h with state space \(\mathcal{S}^{(h)}\) for \(h = 1,\ldots ,H\) and transition class \(k = 1,\ldots ,K\) be denoted by \(Q^{(h)}_k \in \mathbb R_{\geqslant 0}^{|\mathcal {S}^{(h)}| \times |\mathcal {S}^{(h)}|}\) and given entrywise as
Then
Next, we introduce a tool to solve the initial value problem associated with the system of linear first-order ordinary differential equations (ODEs) in (1) [12].
2 A Software Tool
We present a software tool [2] for the transient analysis of countably infinite multidimensional CTMCs introduced in the previous section in a sequential setting. Details regarding the tool may be obtained from its user manual. Time is discretized into smaller time steps and the solver for the CME [10] uses the implicit backward differentiation formulae (BDF). BDF methods are a class of implicit multistep methods to solve stiff ODEs [12]. Stiffness generally manifests itself when reaction rates occur at different time scales, and this is the case for many realistic systems. The o-step BDF method, denoted BDFo, keeps approximations of solutions at o previous time steps and computes the solution at the current time step by solving a linear system. BDFo methods have local truncation error proportional to the oth power of the step size, and therefore, are said to be of order o. The particular solver initializes the first o backward differences with the embedded Runge–Kutta method due to Fehlberg, written RKF\(k-1(k)\), which is an order k method without the error estimate [12].
At each time step, n, the reachable state space, \(\mathcal{R}\), is truncated by using a well defined aggregation operator on the prediction vector of BDFo [11] to obtain \(\mathcal{R}_n\) [10]. Solution vectors can be stored compactly during transient analysis using one of the Hierarchical Tucker Decomposition (HTD) [5], Quantized Tensor Train (QTT) [8], or Transposed Quantized Tensor Train (QT3) [7] formats. Compact vectors in HTD format can work with a truncated generator matrix represented as a sum of Kronecker products of small molecule matrices, whereas those in QTT/QT3 format can work with a low-rank approximation of the truncated generator matrix in the same format [10].
The solution of the linear system at each time step in BDF is performed by the Jacobi iteration [12] using the Newton-Schulz method [9] to compute reciprocals of diagonal elements of the coefficient matrix for HTD and the density matrix renormalization group (DMRG) method for QTT/QT3 [7]. It is possible to use fixed and adaptive rank control strategies with compact vectors in HTD format. There are , , and variants of the BDFo solver in which (adaptive, adaptive), (fixed, adaptive), and (fixed, fixed) rank bounds are used in (Jacobi, Newton-Schulz) methods [10].
Next we show examples of results that can be obtained with the tool.
3 An Example
We consider a cascade model [6] that has five molecules each corresponding to a different dimension with the transition classes in Table 1. Here, \(H=5\), \(\varvec{i} = (i_1,i_2,i_3,i_4,i_5)\), \(K = 10\), a, b, c, \(\mu \in \mathbb R_{> 0}\), and \(\mathbf{e}_h\) is the hth principal axis vector. We let \(a = 0.7\), \(b = 1\), \(c = 5\), and \(\mu = 0.07\) as in [6].
The cascade model is analyzed using BDF5 with an accuracy tolerance of \(10^{-9}\) and the indicated compact vector formats starting from the initial distribution \(\varvec{\pi }_{0}(10,10,10,10,10) = 1\) for final time values \(t \in \{1,\ldots ,10\}\). A maximum run time of 1,000 seconds is imposed on the experiments performed on an Intel Core i7 2.6 GHz processor with 16 Gigabytes main memory under Linux. We let \(\mathbf{p}_n\) denote the transient probability distribution vector computed at time step n, \(\text {max}(|\mathcal{R}_n|)\) denote the maximum truncated state space size, \(\text {max}(r(\mathbf{p}_n))\) denote the maximum rank associated with compact solution vectors, \(N_e\) denote the total number of time steps taken up to t (if t is reached within 1,000 seconds), and \(\sum _{n=1}^{N_e}|\mathbf{p}_n\mathbf{e} -\mathbf{p}_{n-1}{} \mathbf{e}|\) express the total state space truncation error, which has been shown to be in the same order as the relative error in the solution. We do not report the results with QT3 since they did not fare well. The results in Fig. 1 indicate that relative errors of at most \(10^{-7}\) and \(10^{-5}\) are obtained respectively with QTT and adaptive rank controlled HTD formats within 1,000 seconds in all problems. Furthermore, memory and time requirements of are at least an order of magnitude better than those with QTT.
We depict in Fig. 2 the mean number of molecules when BDF5 with is used to analyze the cascade model starting from the initial distribution \(\varvec{\pi }_0(0,0,0,0,0) = 1\). We remark that all results are obtained in at most 3,783 seconds with relative errors in \([5 \times 10^{-7},10^{-3}]\) using a maximum truncated state space size of 58, 786, 560 and a maximum of 2, 301, 678 nonzeros.
References
Buchholz, P., Dayar, T., Kriege, J., Orhan, M.C.: On compact solution vectors in Kronecker-based Markovian analysis. Perform. Eval. 115, 132–149 (2017)
CompactTransientSolver software (2017). http://www.cs.bilkent.edu.tr/~tugrul/software.html
Dayar, T.: Analyzing Markov Chains using Kronecker Products: Theory and Applications. Springer, New York (2012). https://doi.org/10.1007/978-1-4614-4190-8
Goutsias, J., Jenkinson, G.: Markovian dynamics on complex reaction networks. Phys. Rep. 529(2), 199–264 (2013)
Hackbusch, W.: Tensor Spaces and Numerical Tensor Calculus. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-28027-6
Hegland, M., Burden, C., Santoso, L., MacNamara, S., Booth, H.: A solver for the stochastic master equation applied to gene regulatory networks. J. Comput. Appl. Math. 205(2), 708–724 (2007)
Kazeev, V., Khammash, M., Nip, M., Schwab, C.: Direct solution of the chemical master equation using quantized tensor trains. PLoS Comput. Biol. 10(3), e1003359 (2014)
Khoromskij, B.N.: \(O(d \text{ log }N)\)-Quantics approximation of \(N-d\) tensors in high-dimensional numerical modeling. Constructive Approximation 34(2), 257–280 (2011)
Kressner, D., Tobler, C.: htucker – A Matlab toolbox for tensors in hierarchical Tucker format. Technical Report 2012–02, Mathematics Institute of Computational Science and Engineering, Lausanne (2012)
Orhan, M.C.: On the Numerical Analysis of Infinite Multi-dimensional Markov Chains. PhD Thesis, Department of Computer Engineering, Bilkent University, Ankara (2017)
Shampine, L.F., Reichelt, M.W.: The MATLAB ODE suite. SIAM J. Sci. Comput. 18(1), 1–22 (1997)
Stewart, W.J.: Introduction to the Numerical Solution of Markov Chains. Princeton University Press, Princeton (1994)
Acknowledgement
Part of this work is supported by the Alexander von Humboldt Foundation through the Research Group Linkage Programme. The research of M. Can Orhan is carried out during his PhD studies at Bilkent University and supported by The Scientific and Technological Research Council of Turkey under grant 2211-A. We thank the referees whose comments led to an improved manuscript.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Dayar, T., Orhan, M.C. (2018). A Software Tool for the Compact Solution of the Chemical Master Equation. In: German, R., Hielscher, KS., Krieger, U. (eds) Measurement, Modelling and Evaluation of Computing Systems. MMB 2018. Lecture Notes in Computer Science(), vol 10740. Springer, Cham. https://doi.org/10.1007/978-3-319-74947-1_24
Download citation
DOI: https://doi.org/10.1007/978-3-319-74947-1_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-74946-4
Online ISBN: 978-3-319-74947-1
eBook Packages: Computer ScienceComputer Science (R0)