Abstract
The theory of Markov chains is rooted in the work of Russian mathematician Andrey Markov and has an extensive body of literature to establish its mathematical foundations. The availability of computing resources has recently made it possible to use Markov chains to analyze a variety of scientific data. Monte Carlo Markov chains are now one of the most popular methods of data analysis. This chapter presents the key mathematical properties of Markov chains, necessary to understand its implementation as Monte Carlo Markov chains.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abramowitz, M., Stegun, I.A.: Handbook of Mathematical Functions: With Formulas, Graphs, and Mathematical Tables. Dover, New York (1970)
Akritas, M.G., Bershady, M.A.: Linear regression for astronomical data with measurement errors and intrinsic scatter. Astrophys. J. 470, 706 (1996). doi:10.1086/177901
Bayes, T., Price, R.: An essay towards solving a problem in the doctrine of chances. Philos. Trans. R. Soc. Lond. 53 (1763)
Bonamente, M., Swartz, D.A., Weisskopf, M.C., Murray, S.S.: Swift XRT observations of the possible dark galaxy VIRGOHI 21. Astrophys. J. Lett. 686, L71–L74 (2008). doi:10.1086/592819
Bonamente, M., Hasler, N., Bulbul, E., Carlstrom, J.E., Culverhouse, T.L., Gralla, M., Greer, C., Hawkins, D., Hennessy, R., Joy, M., Kolodziejczak, J., Lamb, J.W., Landry, D., Leitch, E.M., Marrone, D.P., Miller, A., Mroczkowski, T., Muchovej, S., Plagge, T., Pryke, C., Sharp, M., Woody, D.: Comparison of pressure profiles of massive relaxed galaxy clusters using the Sunyaev–Zel’dovich and X-ray data. New. J. Phys. 14 (2), 025010 (2012). doi:10.1088/1367-2630/14/2/025010
Brooks, S.P., Gelman, A.: General methods for monitoring convergence of iterative simulations. J. Comput. Graph. Stat. 7, 434–455 (1998)
Bulmer, M.G.: Principles of Statistics. Dover, New York (1967)
Carlin, B., Gelfand, A., Smith, A.: Hierarchical Bayesian analysis for changepoint problems. Appl. Stat. 41, 389–405 (1992)
Cash, W.: Parameter estimation in astronomy through application of the likelihood ratio. Astrophys. J. 228, 939–947 (1979)
Cowan, G.: Statistical Data Analysis. Oxford University Press, Oxford (1998)
Cramer, H.: Mathematical Methods of Statistics. Princeton University Press, Princeton (1946)
Emslie, A.G., Massone, A.M.: Bayesian confidence limits of electron spectra obtained through regularized inversion of solar hard X-ray spectra. Astrophys. J. 759, 122 (2012)
Fisher, R.A.: On a distribution yielding the error functions of several well known statistics. Proc. Int. Congr. Math. 2, 805–813 (1924)
Fisher, R.A.: The use of multiple measurements in taxonomic problems. Ann. Eugenics 7, 179–188 (1936)
Gamerman, D.: Markov Chain Monte Carlo. Chapman and Hall/CRC, London/New York (1997)
Gehrels, N.: Confidence limits for small numbers of events in astrophysical data. Astrophys. J. 303, 336–346 (1986). doi:10.1086/164079
Gelman, A., Rubin, D.: Inference from iterative simulation using multiple sequences. Stat. Sci. 7, 457–511 (1992)
Gosset, W.S.: The probable error of a mean. Biometrika 6, 1–25 (1908)
Hastings, W.K.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57 (1), 97–109 (1970). doi:10.1093/biomet/57.1.97. http://biomet.oxfordjournals.org/content/57/1/97.abstract
Helmert, F.R.: Die genauigkeit der formel von peters zur berechnung des wahrscheinlichen fehlers director beobachtungen gleicher genauigkeit. Astron. Nachr. 88, 192–218 (1876)
Hubble, E., Humason, M.: The velocity-distance relation among extra-galactic nebulae. Astrophys. J. 74, 43 (1931)
Isobe, T., Feigelson, Isobe, T., Feigelson, E.D., Akritas, M.G., Babu, G.J.: Linear regression in astronomy. Astrophys. J. 364, 104 (1990)
Jeffreys, H.: Theory of Probability. Oxford University Press, London (1939)
Kelly, B.C.: Some aspects of measurement error in linear regression of astronomical data. Astrophys. J. 665, 1489–1506 (2007). doi: 10.1086/519947
Kolmogorov, A.: Sulla determinazione empirica di una legge di distribuzione. Giornale dell’ Istituto Italiano degli Attuari 4, 1–11 (1933)
Kolmogorov, A.N.: Foundations of the Theory of Probability. Chelsea, New York (1950)
Lampton, M., Margon, B., Bowyer, S.: Parameter estimation in X-ray astronomy. Astrophys. J. 208, 177–190 (1976). doi:10.1086/154592
Lewis, S.: gibbsit. http://lib.stat.cmu.edu/S/gibbsit
Madsen, R.W., Moeschberger, M.L.: Introductory Statistics for Business and Economics. Prentice-Hall, Englewood Cliffs (1983)
Marsaglia, G., Tsang, W., Wang, J.: Evaluating Kolmogorov’s distribution. J. Stat. Softw. 8, 1–4 (2003)
Mendel, G.: Versuche über plflanzenhybriden (experiments in plant hybridization). Verhandlungen des naturforschenden Vereines in Brünn, pp. 3–47 (1865)
Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1092 (1953). doi: 10.1063/1.1699114
Pearson, K., Lee, A.: On the laws on inheritance in men. Biometrika 2, 357–462 (1903)
Plummer, M., Best, N., Cowles, K., Vines, K.: CODA: convergence diagnosis and output analysis for MCMC. R News 6 (1), 7–11 (2006). http://CRAN.R-project.org/doc/Rnews/
Press, W., Teukolski, S., Vetterling, W., Flannery, B.: Numerical Recipes 3rd Edition: The Art of Scientific Computing. Cambridge University Press, Cambridge (2007)
Protassov, R., van Dyk, D.A., Connors, A., Kashyap, V.L., Siemiginowska, A.: Statistics, handle with care: detecting multiple model components with the likelihood ratio test. Astrophys. J. 571, 545–559 (2002). doi:1086/339856
Raftery, A., Lewis, S.: How many iterations in the gibbs sampler? Bayesian Stat. 4, 763–773 (1992)
Ross, S.M.: Introduction to Probability Models. Academic, San Diego (2003)
Thomson, J.J.: Cathode rays. Philos. Mag. 44, 293 (1897)
Tremaine, S., Gebhardt, K., Bender, R., Bower, G., Dressler, A., Faber, S.M., Filippenko, A.V., Green, R., Grillmair, C., Ho, L.C., Kormendy, J., Lauer, T.R., Magorrian, J., Pinkney, J., Richstone, D.: The slope of the black hole mass versus velocity dispersion correlation. Astrophys. J. 574, 740–753 (2002). doi:10.1086/341002
Von Eye, A., Schuster, C.: Regression Analysis for Social Sciences. Academic, New York (1998)
Wilks, S.S.: Mathematical Statistics. Princeton University Press, Princeton (1943)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer Science+Busines Media New York
About this chapter
Cite this chapter
Bonamente, M. (2017). Introduction to Markov Chains. In: Statistics and Analysis of Scientific Data. Graduate Texts in Physics. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-6572-4_15
Download citation
DOI: https://doi.org/10.1007/978-1-4939-6572-4_15
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4939-6570-0
Online ISBN: 978-1-4939-6572-4
eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)