Preliminaries

Abstract

By a Markov process we shall understand a continuous time stochastic process {X(t): 0 ≤ t < ∞} which has a denumerable state space S and which possesses the Markov property, i.e., for every n ≥ 2, 0 ≤ t₁ <.....< t_n and any i₁,...., i_n in S one has

$$\Pr \left\{ {X\left( {{t_{n}}} \right) = \left| {X\left( {{t_{1}}} \right) = {i_{{1,....,}}}} \right.X\left( {{t_{{n - 1}}}} \right) = {i_{{n - 1}}}} \right\} = \Pr \left\{ {X\left( {{t_{n}}} \right) = {i_{n}}\left| {X\left( {{t_{{n - 1}}}} \right)} \right.{i_{{n - 2}}}} \right\}$$

((1.1.1))

, The process is supposed to be temporally homogeneous, i.e., for every i, j in S the conditional probability Pr{X(t+s) = j| X(s) = i} does not depend on s. In this case we may put

$$Pi\left( t \right) = \Pr \left\{ {X\left( t \right) = i} \right\},\sum\limits_{i} {{P_{{i\left( t \right)}}}} = 1.$$

((1.1.2))

t ≥ 0.