Atomic Chains

Abstract

In this chapter, we enter the core of the theory of Markov chains. We will encounter for the first time the fundamental notions of state classification, dichotomy between transience and recurrence, period, existence, uniqueness (up to scale), and characterization of invariant measures, as well as the classical limit theorems: the law of large numbers and the central limit theorem. These notions will be introduced, and the results will be obtained by means of the simplifying assumption that the state space contains an accessible atom. An atom is a set of states out of which the chain exits under a distribution common to all its individual states. A singleton is thus an atom, but if the state space is not discrete, it will in most cases be useless by failing to be accessible. Let us recall that a set is accessible if the chains eventually enter this set wherever it starts from with positive probability.