Examples of Continuous-Time Markov Chains

Abstract

In this case we have a finite state space E which we can take to be $\left\{ {1,2,3,...,n} \right\}$. Given any q-matrix Q, which need not be conservative, there is a unique Q-function, and we can solve either the backward or forward equations to find it. Let us consider, for example, the backward equations

$$P'\left( t \right) = QP\left( t \right),P\left( 0 \right) = I.$$

((1.1))

Suppose that Q is similar to the n x n matrix J; that is, that

$$Q = MJ{M^{ - 1}}$$

((1.2))

for some n x n invertible matrix M. Then $T\left( t \right) = {M^{ - 1}}P\left( t \right)$ satisfies

$$Y'\left( t \right) = JY\left( t \right),Y\left( 0 \right) = {M^{ - 1}}$$

((1.3))

The point is J can in general be a simpler matrix than Q, with the result that the equation in (1.3) is straightforward to solve. The extreme case is when Q has n independent eigenvectors and thus J can be taken to be the diagonal matrix whose diagonal components are the eigenvalues of Q, and M is the matrix whose columns are the corresponding eigenvectors.