Abstract
The first mathematical results for discrete-time Markov chains with a finite state space were generated by Andrey Markov [71] whose motivation was to extend the law of large numbers to sequences of nonindependent random variables. The first important results in the more difficult context of countably infinite Markov chains were established 30 years later by Andrey Kolmogorov
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
P. Baldi, L. Mazliak, and P. Priouret. Martingales and Markov chains. Chapman & Hall/CRC, Boca Raton, FL, 2002. Solved exercises and elements of theory, Translated from the 1998 French original.
L. Breiman. Probability. Addison-Wesley, Reading, MA, 1968.
R. Durrett. Probability: theory and examples. Duxbury Press, Belmont, CA, second edition, 1996.
R. Durrett. Essentials of stochastic processes. Springer Texts in Statistics. Springer-Verlag, New York, second edition, 2012.
S. N. Ethier and T. G. Kurtz. Markov processes. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. John Wiley & Sons, Inc., New York, 1986. Characterization and convergence.
W. Feller. An introduction to probability theory and its applications. Vol. I. Third edition. John Wiley & Sons, Inc., New York, 1968.
S. Karlin and H. M. Taylor. A first course in stochastic processes. Academic Press [A subsidiary of Harcourt Brace Jovanovich], New York, second edition, 1975.
A. N. Kolmogorov. Zur Theorie der Markoffschen Ketten. Math. Ann., 112(1):155–160, 1936.
N. Lanchier and S. Scarlatos. Limiting behavior for a general class of voter models with confidence threshold. Preprint. Available as arXiv:1412.4142.
A. A. Markov. Extension of the law of large numbers to dependent quantities. Izv. Fiz.-Matem. Obsch. Kazan Univ., 15:135–156, 1906.
J. R. Norris. Markov chains, volume 2 of Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge, 1998. Reprint of 1997 original.
S. M. Ross. Stochastic processes. Wiley Series in Probability and Statistics: Probability and Statistics. John Wiley & Sons, New York, second edition, 1996.
S. M. Ross. Introduction to probability models. Elsevier/Academic Press, Amsterdam, eleventh edition, 2014.
R. B. Schinazi. Classical and spatial stochastic processes. Birkhäuser Boston, Boston, MA, 1999.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Lanchier, N. (2017). Discrete-time Markov chains. In: Stochastic Modeling. Universitext. Springer, Cham. https://doi.org/10.1007/978-3-319-50038-6_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-50038-6_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-50037-9
Online ISBN: 978-3-319-50038-6
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)