Stochastic Processes

Abstract

One important type of stochastic process is a Markov process, a stochastic process that has a limited form of “historical” dependency. To precisely define this dependency, let $\{ {X_t},t \in \mathcal{T}\}$ be a stochastic process defined on the parameter set $\mathcal{T}$. We think of $\mathcal{T} \subset {\text{[0,}}\infty {\text{)}}$ in terms of time, and the values that X _t can assume are called the states which are elements of a state space S ⊂ ℝ. A stochastic process is called a Markov process if it satisfies

$$\begin{array}{*{20}{c}} {\Pr ({X_{{t_0} + {t_1}}} \leqslant x\mid {X_{{t_0}}} = {x_0},{X_\tau },0 \leqslant \tau < {t_0})} \\ { = \Pr ({X_{{t_0} + {t_1}}} \leqslant x\mid {X_{{t_0}}} = {x_0}),} \end{array}$$

(2.1)

for any value of t0, t₁> 0. To interpret (2.1), we think of t_o as being the present time. Equation (2.1) states that the evolution of a Markov process at a future time, conditioned on its present and past values, depends only on its present value. Expressed differently, the present value of X _t0 contains all the information about the past evolution of the process that is needed to determine the future distribution of the process. The condition (2.1) that defines a Markov process is sometimes termed the Markov property.