Markov Chains on a Discrete State Space

Abstract

In this chapter we will discuss the case in which the state space $\mathsf {X}$ is discrete, which means either finite or countably infinite. In this case, it will always be assumed that $\mathscr {X}= \mathscr {P}(\mathsf {X})$, the set of all subsets of $\mathsf {X}$. Since every state is an atom, we will first apply the results of Chapter 6 and then highlight the specificities of Markov chains on countable state spaces. In particular, in Section 7.5 we will obtain simple drift criteria for transience and recurrence, and in Section 7.6 we will make use for the first time of coupling arguments to prove the convergence of the iterates of the kernel to the invariant probability measure.