Abstract
The chapter opens with in Sect. 13.1 presenting the key definitions and first examples of countable Markov chains. The section also contains the classification of states of the chain. Section 13.2 contains necessary and sufficient conditions for recurrence of states, the Solidarity Theorem for irreducible Markov chains and a theorem on the structure of a periodic Markov chain. Key theorems on random walks on lattices are presented in Sect. 13.3, along with those for a general symmetric random walk on the real line. The ergodic theorem for general countable homogeneous chains is established in Sect. 13.4, along with its special case for finite Markov chains and the Law of Large Numbers and the Central Limit Theorem for the number of visits to a given state. This is followed by a short Sect. 13.5 detailing the behaviour of transition probabilities for reducible chains. The last three sections are devoted to Markov chains with arbitrary state spaces. First the ergodicity of such chains possessing a positive atom is proved in Sect. 13.6, then the concept of Harris Markov chains is introduced and conditions of ergodicity of such chains are established in Sect. 13.7. Finally, the Laws of Large Numbers and the Central Limit Theorem for sums of random variables defined on a Markov chain are obtained in Sect. 13.8.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Borovkov, A.A.: Ergodicity and Stability of Stochastic Processes. Wiley, Chichester (1998)
Grenander, U.: Probabilities on Algebraic Structures. Almqvist & Wiskel, Stockholm (1963)
Kolmogorov, A.N.: Markov chains with a countable number of possible states. In: Shiryaev, A.N. (ed.) Selected Works of A.N. Kolmogorov, vol. 2, pp. 193–208. Kluwer Academic, Dordrecht (1986)
Meyn, S.P., Tweedie, R.L.: Markov Chains and Stochastic Stability. Springer, New York (1993)
Nummelin, E.: General Irreducible Markov Chains and Nonnegative Operators. Cambridge University Press, New York (1984)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag London
About this chapter
Cite this chapter
Borovkov, A.A. (2013). Sequences of Dependent Trials. Markov Chains. In: Probability Theory. Universitext. Springer, London. https://doi.org/10.1007/978-1-4471-5201-9_13
Download citation
DOI: https://doi.org/10.1007/978-1-4471-5201-9_13
Publisher Name: Springer, London
Print ISBN: 978-1-4471-5200-2
Online ISBN: 978-1-4471-5201-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)