Abstract
In this chapter we introduce fundamental notions of Markov chains and state the results that are needed to establish the convergence of various MCMC algorithms and, more generally, to understand the literature on this topic. Thus, this chapter, along with basic notions of probability theory, will provide enough foundation for the understanding of the following chapters. It is, unfortunately, a necessarily brief and, therefore, incomplete introduction to Markov chains, and we refer the reader to Meyn and Tweedie (1993), on which this chapter is based, for a thorough introduction to Markov chains. Other perspectives can be found in Doob (1953), Chung (1960), Feller (1970, 1971), and Billingsley (1995) for general treatments, and Norris (1997), Nummelin (1984), Revuz (1984), and Resnick (1994) for books entirely dedicated to Markov chains. Given the purely utilitarian goal of this chapter, its style and presentation differ from those of other chapters, especially with regard to the plethora of definitions and theorems and to the rarity of examples and proofs. In order to make the book accessible to those who are more interested in the implementation aspects of MCMC algorithms than in their theoretical foundations, we include a preliminary section that contains the essential facts about Markov chains.
Leaphorn never counted on luck. Instead, he expected order—the natural sequence of behavior, the cause producing the natural effect, the human behaving in the way it was natural for him to behave. He counted on that and on his own ability to sort out the chaos of observed facts and find in them this natural order.
—Tony Hillerman, The Blessing Way
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Notes
Meyn, S. and Tweedie, R. (1993). Markov Chains and Stochastic Stability. Springer-Verlag, New York.
Norris, J. (1997). Markov Chains. Cambridge University Press, Cambridge.
Mengersen, K. and Tweedie, R. (1996). Rates of convergence of the Hastings and Metropolis algorithms. Ann. Statist., 24: 101–121.
Eaton, M. (1992). A statistical dyptich: Admissible inferences–recurrence of symmetric Markov chains. Ann. Statist., 20: 1147–1179.
Brown, L. (1971). Admissible estimators, recurrent diffusions, and insoluble boundary-value problems. Ann. Mathecoat. Statist., 42: 855–903.
Brooks, S. and Roberts, G. (1999). On quantile estimation and MCMC convergence. Biometrika, 86: 710–717.
Athreya, K., Doss, H., and Sethuraman, J. (1996). On the convergence of the Markov chain simulation method. Ann. Statist., 24: 69–100.
Rosenblatt, M. (1971). Markov Processes: Structure and Asymptotic Behavior. Springer-Verlag, New York.
Billingsley, P. (1995). Probability and Measure. John Wiley, New York, third edition.
Bradley, R. (1986). Basic properties of strong mixing conditions. In Ebberlein, E. and Taqqu, M., editors, Dependence in Probability and Statistics, pages 165–192. Birkhäuser, Boston.
Tierney, L. (1994). Markov chains for exploring posterior distributions (with discussion). Ann. Statist., 22: 1701–1786.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer Science+Business Media New York
About this chapter
Cite this chapter
Robert, C.P., Casella, G. (2004). Markov Chains. In: Monte Carlo Statistical Methods. Springer Texts in Statistics. Springer, New York, NY. https://doi.org/10.1007/978-1-4757-4145-2_6
Download citation
DOI: https://doi.org/10.1007/978-1-4757-4145-2_6
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4419-1939-7
Online ISBN: 978-1-4757-4145-2
eBook Packages: Springer Book Archive