Markov chains: Discrete and continuous time

Abstract

Let phase space $\Bbb{X}$ of a random sequence $\{X_n,\,n\in\Bbb{Z}^+\}$ be enumerable. The sequence $\{X_n,\,n\in\Bbb{Z}^+\}$ is called a Markov chain if

$$\forall n\in\Bbb{N}\ \forall i_1,\ldots,i_n,i_{n+1}\in \Bbb{X} \ \forall t_1\leq\cdots\leq t_n\leq t_{n+1}\in\Bbb{Z}_+:$$

$$\mathsf{P}(X_{t_{n+1}}=i_{n+1} / X_{t_{1}}=i_{1},\ldots,X_{t_{n}}=i_{n})=\mathsf{P}(X_{t_{n+1}}=i_{n+1} / X_{t_{n}}=i_{n}).$$