Advertisement

Markov Chains pp 145-164 | Cite as

Markov Chains on a Discrete State Space

  • Randal DoucEmail author
  • Eric Moulines
  • Pierre Priouret
  • Philippe Soulier
Chapter
Part of the Springer Series in Operations Research and Financial Engineering book series (ORFE)

Abstract

In this chapter we will discuss the case in which the state space \(\mathsf {X}\) is discrete, which means either finite or countably infinite. In this case, it will always be assumed that \(\mathscr {X}= \mathscr {P}(\mathsf {X})\), the set of all subsets of \(\mathsf {X}\). Since every state is an atom, we will first apply the results of Chapter  6 and then highlight the specificities of Markov chains on countable state spaces. In particular, in Section 7.5 we will obtain simple drift criteria for transience and recurrence, and in Section 7.6 we will make use for the first time of coupling arguments to prove the convergence of the iterates of the kernel to the invariant probability measure.

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Randal Douc
    • 1
    Email author
  • Eric Moulines
    • 2
  • Pierre Priouret
    • 3
  • Philippe Soulier
    • 4
  1. 1.Département CITITelecom SudParisÉvryFrance
  2. 2.Centre de Mathématiques AppliquéesEcole PloytechniquePalaiseauFrance
  3. 3.Université Pierre et Marie CurieParisFrance
  4. 4.Université Paris NanterreNanterreFrance

Personalised recommendations