Abstract
Applied probability thrives on models. Markov chains are one of the richest sources of good models for capturing dynamical behavior with a large stochastic component [23, 24, 59, 80, 106, 107, 118]. In this chapter we give a few examples and a quick theoretical overview of discrete-time Markov chains. The highlight of our theoretical development, Proposition 7.4.1, relies on a coupling argument. Because coupling is one of the most powerful and intuitively appealing tools available to probabilists, we examine a few of its general applications as well. We also stress reversible Markov chains. Reversibility permits explicit construction of the long-run or equilibrium distribution of a chain when such a distribution exists. Chapter 8 will cover continuous-time Markov chains.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2010 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Lange, K. (2010). Discrete-Time Markov Chains. In: Applied Probability. Springer Texts in Statistics, vol 0. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-7165-4_7
Download citation
DOI: https://doi.org/10.1007/978-1-4419-7165-4_7
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4419-7164-7
Online ISBN: 978-1-4419-7165-4
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)