Markov Chains and Monte Carlo Markov Chains

  • Massimiliano Bonamente
Part of the Graduate Texts in Physics book series (GTP)


The theory of Markov chains is rooted in the work of Russian mathematician Andrey Markov, and has an extensive body of literature to establish its mathematical foundations. The availability of computing resources has recently made it possible to use Markov chains to analyze a variety of scientific data, and Monte Carlo Markov chains are now one of the most popular methods of data analysis. The modern-day data analyst will find that Monte Carlo Markov chains are an essential tool that permits tasks that are simply not possible with other methods, such as the simultaneous estimate of parameters for multi-parametric models of virtually any level of complexity. This chapter starts with an introduction to the mathematical properties of Markov chains necessary to understand its implementation as a Monte Carlo Markov chains. The second part is devoted to a description of the implementation of Monte Carlo Markov chains, including the Metropolis-Hasting algorithm and a few tests of convergence.


Markov Chain Posterior Distribution Monte Carlo Markov Chain Stationary Distribution Monte Carlo Markov Chain Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Abramowitz, M., Stegun, I.A.: Handbook of Mathematical Functions: With Formulas, Graphs, and Mathematical Tables. Dover, New York (1970)Google Scholar
  2. 2.
    Akritas, M.G., Bershady, M.A.: Linear regression for astronomical data with measurement errors and intrinsic scatter. Astrophys. J. 470, 706 (1996). doi:10.1086/177901ADSCrossRefGoogle Scholar
  3. 3.
    Bayes, T., Price, R.: An essay towards solving a problem in the doctrine of chances. Philos. Trans. R. Soc. Lond. 53 (1763)Google Scholar
  4. 4.
    Bonamente, M., Swartz, D.A., Weisskopf, M.C., Murray, S.S.: Swift XRT observations of the possible dark galaxy VIRGOHI 21. Astrophys. J. Lett. 686, L71–L74 (2008). doi:10.1086/592819ADSCrossRefGoogle Scholar
  5. 5.
    Bonamente, M., Hasler, N., Bulbul, E., Carlstrom, J.E., Culverhouse, T.L., Gralla, M., Greer, C., Hawkins, D., Hennessy, R., Joy, M., Kolodziejczak, J., Lamb, J.W., Landry, D., Leitch, E.M., Marrone, D.P., Miller, A., Mroczkowski, T., Muchovej, S., Plagge, T., Pryke, C., Sharp, M., Woody, D.: Comparison of pressure profiles of massive relaxed galaxy clusters using the Sunyaev–Zel’dovich and x-ray data. N. J. Phys. 14(2), 025010 (2012). doi:10.1088/1367-2630/14/2/025010CrossRefGoogle Scholar
  6. 6.
    Brooks, S.P., Gelman, A.: General methods for monitoring convergence of iterative simulations. J. Comput. Graph. Stat. 7, 434–455 (1998)MathSciNetGoogle Scholar
  7. 7.
    Bulmer, M.G.: Principles of Statistics. Dover, New York (1967)MATHGoogle Scholar
  8. 8.
    Carlin, B., Gelfand, A., Smith, A.: Hierarchical Bayesian analysis for changepoint problems. Appl. Stat. 41, 389–405 (1992)CrossRefMATHGoogle Scholar
  9. 9.
    Cash, W.: Parameter estimation in astronomy through application of the likelihood ratio. Astrophys. J. 228, 939–947 (1979). doi:10.1086/156922ADSCrossRefGoogle Scholar
  10. 10.
    Cramer, H.: Mathematical Methods of Statistics. Princeton University Press, Princeton (1946)MATHGoogle Scholar
  11. 11.
    Cowan, G.: Statistical Data Analysis, Oxford University Press (1998)Google Scholar
  12. 12.
    Emslie, A.G., Massone, A.M.: Bayesian confidence limits ETC ETC. ArXiv e-prints (2012)Google Scholar
  13. 13.
    Fisher, R.A.: On a distribution yielding the error functions of several well known statistics. Proc. Int. Congr. Math. 2, 805–813 (1924)Google Scholar
  14. 14.
    Gamerman, D.: Markov Chain Monte Carlo. Chapman and Hall CRC, London/New York (1997)MATHGoogle Scholar
  15. 15.
    Gehrels, N.: Confidence limits for small numbers of events in astrophysical data. Astrophys. J. 303, 336–346 (1986). doi:10.1086/164079ADSCrossRefGoogle Scholar
  16. 16.
    Gelman, A., Rubin, D.: Inference from iterative simulation using multiple sequences. Stat. Sci. 7, 457–511 (1992)CrossRefGoogle Scholar
  17. 17.
    Gosset, W.S.: The probable error of a mean. Biometrika 6, 1–25 (1908)Google Scholar
  18. 18.
    Hastings, W.K.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57(1), 97–109 (1970). doi:10.1093/biomet/57.1.97.
  19. 19.
    Helmert, F.R.: Die genauigkeit der formel von peters zur berechnung des wahrscheinlichen fehlers director beobachtungen gleicher genauigkeit. Astron. Nachr. 88, 192–218 (1876)CrossRefGoogle Scholar
  20. 20.
    Hubble, E., Humason, M.: The velocity-distance relation among extra-galactic nebulae. Astrophys. J. 74, 43 (1931)ADSCrossRefGoogle Scholar
  21. 21.
    Jeffreys, H.: Theory of Probability. Oxford University Press, London (1939)Google Scholar
  22. 22.
    Kelly, B.C.: Some aspects of measurement error in linear regression of astronomical data. Astrophys. J. 665, 1489–1506 (2007). doi:10.1086/519947ADSCrossRefGoogle Scholar
  23. 23.
    Kolmogorov, A.: Sulla determinazione empirica di una legge di distribuzione. Giornale dell’ Istituto Italiano degli Attuari 4, 1–11 (1933)MATHGoogle Scholar
  24. 24.
    Kolmogorov, A.N.: Foundations of the Theory of Probability. Chelsea, New York (1950)Google Scholar
  25. 25.
    Lampton, M., Margon, B., Bowyer, S.: Parameter estimation in X-ray astronomy. Astrophys. J. 208, 177–190 (1976). doi:10.1086/154592ADSCrossRefGoogle Scholar
  26. 26.
  27. 27.
    Marsaglia, G., Tsang, W., Wang, J.: Evaluating kolmogorov’s distribution. J. Stat. Softw. 8, 1–4 (2003)Google Scholar
  28. 28.
    Mendel, G.: Versuche über plflanzenhybriden (experiments in plant hybridization). Verhandlungen des naturforschenden Vereines in Brünn pp. 3–47 (1865)Google Scholar
  29. 29.
    Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1092 (1953). doi:10.1063/1.1699114ADSCrossRefGoogle Scholar
  30. 30.
    Pearson, K., Lee, A.: On the laws on inheritance in men. Biometrika 2, 357–462 (1903)Google Scholar
  31. 31.
    Plummer, M., Best, N., Cowles, K., Vines, K.: CODA: convergence diagnosis and output analysis for MCMC. R News 6(1), 7–11 (2006).
  32. 32.
    Press, W., Teukolski, S., Vetterling, W., Flannery, B.: Numerical Recipes 3rd Edition: The Art of Scientific Computing. Cambridge University Press, Cambridge/New York (2007)Google Scholar
  33. 33.
    Protassov, R., van Dyk, D.A., Connors, A., Kashyap, V.L., Siemiginowska, A.: Statistics, handle with care: detecting multiple model components with the likelihood ratio test. Astrophys. J. 571, 545–559 (2002). doi:10.1086/339856ADSCrossRefGoogle Scholar
  34. 34.
    Raftery, A., Lewis, S.: How many iterations in the gibbs sampler? Bayesian Stat. 4, 763–773 (1992)Google Scholar
  35. 35.
    Ross, S.M.: Introduction to Probability Models. Academic, San Diego (2003)MATHGoogle Scholar
  36. 36.
    Thomson, J.J.: Cathode rays. Philos. Mag. 44, 293 (1897)Google Scholar
  37. 37.
    Tremaine, S., Gebhardt, K., Bender, R., Bower, G., Dressler, A., Faber, S.M., Filippenko, A.V., Green, R., Grillmair, C., Ho, L.C., Kormendy, J., Lauer, T.R., Magorrian, J., Pinkney, J., Richstone, D.: The slope of the black hole mass versus velocity dispersion correlation. Astrophys. J. 574, 740–753 (2002). doi:10.1086/341002ADSCrossRefGoogle Scholar
  38. 38.
    Wilks, S.S.: Mathematical Statistics. Princeton University Press, Princeton (1943)MATHGoogle Scholar

Copyright information

© Springer Science+Busines Media New York 2013

Authors and Affiliations

  • Massimiliano Bonamente
    • 1
  1. 1.Department of PhysicsUniversity of AlabamaHuntsvilleUSA

Personalised recommendations