Abstract
The aim of this contribution is to provide a readable account of Markov Chain Monte Carlo methods, with particular emphasis on their relations with the numerical integration of deterministic and stochastic differential equations. The exposition is largely based on numerical experiments and avoids mathematical technicalities. The presentation is largely self-contained and includes tutorial sections on stochastic processes, Markov chains, stochastic differential equations and Hamiltonian dynamics. The Metropolis Random-Walk algorithm, Metropolis adjusted Langevin algorithm and Hybrid Monte Carlo are discussed in detail, including some recent results.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We assume notions such as discrete and continuous random variables, expectation, variance, conditional probability and independence.
- 2.
Recall that (1) Ω is a set and each point ω ∈ Ω corresponds to a possible outcome of a random experiment, (2) \(\mathcal{A}\) (a σ-algebra) is the family of those subsets A ⊆ Ω called events to which a probability \(\mathbb{P}(A)\) is assigned, (3) \(\mathbb{P}\) is a probability measure, \(\mathbb{P}: \mathcal{A}\rightarrow [0, 1]\). The probability space plays very little explicit role in the study of the process; this is carried out in terms of the distributions of the X t (see the examples in this section).
- 3.
Often the dependence of X t on ω is not incorporated explicitly to the notation.
- 4.
In general, a process \(\{X_{n}\}_{n\geq 0}\) is a random walk if \(X_{n+1} = X_{n} + Z_{n}\), where Z n is independent of X n ,…, X 0.
- 5.
Note that e.g. \(\mathbb{P}(X_{n+1} = j\mid X_{n} = i)\) does not make sense if \(\mathbb{P}(X_{n} = i) = 0\). Here we shall not pay attention to the difficulties created by probabilities conditioned to events \(\{X_{n} = i\}\) of 0 probability. These difficulties are easily avoided if, as in [9], the Markov chain is defined in the first place by means of the transition probabilities rather than in terms of the variables X n .
- 6.
If E comprises an infinite number of states, this “matrix” will of course have infinitely many rows/columns. Sums like \(\sum _{j}p_{\mathit{ij}}p_{\mathit{jk}}\) that we shall find below have a finite value if P is stochastic.
- 7.
See [5], Theorem 8.1 for the construction of the X n and the underlying probability space.
- 8.
If all the rows of P are equal, X n+1 is independent of X n .
- 9.
This should not lead to the conclusion that period 2 is the rule for MCs. The three examples in the last section are not typical in this respect and were chosen in view of the fact that are very easily described—in each of them transitions may only occur between state i and states i ± 1. Any chain where the diagonal elements of P are all ≠ 0 only contains aperiodic states.
- 10.
If λ, ν are distributions the notation \({\mid \lambda }^{T} {-\nu }^{T}\mid\) means \(\sum _{i}\vert \lambda _{i} -\nu _{i}\vert\).
- 11.
Note that there is no invariant probability distribution, since the chain is null recurrent.
- 12.
Of course computing the probability of an event A is equivalent to computing the expectation of its indicator, i.e. of the random variable that takes the value 1 if ω ∈ A and 0 if \(\omega \notin A\).
- 13.
It is also necessary that the chain constructed be positive recurrent. Also not all positive recurrent chains having the target as equilibrium measure are equally efficient, as the velocity of the convergence to the limit in (7) is of course chain-dependent.
- 14.
∧ means min.
- 15.
The symbol ∝ means proportional to. To obtain a probability density it is necessary to divide exp(−β V (x)) by the normalizing constant \(\int _{\mathbb{R}}V (x)\,\mathit{dx}\). As pointed out before the Metropolis algorithm does not require the knowledge of the normalizing constant.
- 16.
For further details of the statistical analysis of the sequence of samples x i the reader is referred to [13].
- 17.
∼ means “has a distribution.”
- 18.
These references also show that (4) is essentially a consequence of (1), (2) and (3).
- 19.
The trajectories of the Wiener process are in fact complex objects. For instance, with probability 1, the set Z(ω) of values t for which a trajectory B t (ω) vanishes is closed, unbounded, without isolated points and of Lebesgue measure 0, [5], Theorem 37.4.
- 20.
The terminology Fokker-Panck is used in physics; in probability the equation is known as Kolmogorov’s forward equation, see e.g. [10], Chap. X, Sect. 5.
- 21.
- 22.
In this connection it may be worth noting that the proof of Theorem 9 demands that the mapping Ψ T is time reversible and volume preserving, but would work even if Ψ T were not an approximation to the true Φ T . However if Ψ T is not close to Φ T , the acceptance probability will be low.
- 23.
Recall that when the MALA proposal is seen as an Euler-Maruyama step for an SDE, the MALA parameter h coincides with the square of the time-step Δ t. However in the relation of MALA with HMC studied in this section, h = Δ t as we have just pointed out.
References
E. Akhmatskaya, S. Reich, GSHMC: An efficient method for molecular simulations. J. Comput. Phys. 227, 4934–4954 (2008)
V.I. Arnold, Mathematical Methods of Classical Mechanics, 2nd edn. (Springer, New York, 1989)
A. Beskos, F.J. Pinski, J.M. Sanz-Serna, A.M. Stuart, Hybrid Monte-Carlo on Hilbert spaces. Stoch. Process. Appl. 121, 2201–2230 (2011)
A. Beskos, N. Pillai, G.O. Roberts, J.M. Sanz-Serna, A.M. Stuart, Optimal tuning of the Hybrid Monte-Carlo algorithm. Bernoulli (to appear)
P. Billingsley, Probability and Measure, 3rd edn. (Wiley, New York, 1995)
P. Brémaud, Markov Chains, Gibbs Fields, Monte Carlo Simulation, and Queues (Springer, Berlin, 1999)
E. Cancès, F. Legoll, G. Stoltz, Theoretical and numerical comparison of some sampling methods for molecular dynamics. Esaim Math. Model. Numer. Anal. 41, 351–389 (2007)
S. Duane, A.D. Kennedy, B. Pendleton, R. Roweth, Hybrid Monte Carlo. Phys. Lett. B 195, 216–222 (1987)
W. Feller, An Introduction to Probability Theory and Its Applications, vol. 1, 3rd edn. (Wiley, New York, 1968)
W. Feller, An Introduction to Probability Theory and Its Applications, vol. 1, 2nd edn. (Wiley, New York, 1971)
R.P. Feynman, R.B. Leighton, M. Sands, The Feynman Lectures on Physics, vol. 1 (Addison-Wesley, Reading, 1963)
A. Friedman, Stochastic Differential Equations and Applications (Dover, Mineola, 2006)
C.J. Geyer, Practical Markov Chain Monte Carlo. Stat. Sci. 7, 473–483 (1992)
M. Girolami, B. Calderhead, Riemann manifold Langevin and Hamiltonian Monte Carlo methods. J. R. Stat. Soc. B 73, 123–214 (2011)
G. Grimmett, D. Stirzaker, Probability and Random Processes, 3rd edn. (Oxford University Press, Oxford, 2001)
E. Hairer, C. Lubich, G. Wanner, Geometric Numerical Integration, 2nd edn. (Springer, Berlin, 2006)
W. Hastings, Monte Carlo sampling methods using Markov chains and their application. Biometrika 57, 97–109 (1970)
D. Higham, An algorithmic introduction to numerical simulation of stochastic differential equations. SIAM Rev. 43, 525–546 (2001)
M.D. Hoffman, A. Gelman, The No-U-Turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo, preprint
J.A. Izaguirre, S.S. Hampton, Shadow hybrid Monte Carlo: An efficient propagator in phase space of macromolecules. J. Comput. Phys. 200, 581–604 (2004)
I.D. Lawrie, A Unified Grand Tour of Theoretical Physics (Institute of Physics Publishing, Bristol, 1990)
B. Leimkuhler, S. Reich, Simulating Hamiltonian Dynamics (Cambridge University Press, Cambridge, 2004)
T. Lelievre, M. Rousset, G. Stoltz, Free Energy Computations: A Mathematical Perspective (Imperial College Press, London, 2010)
X. Mao, Stochastic Differential Equations and Applications, 2nd edn. (Horwood, Chichester, 2008)
J.C. Mattingly, N.S. Pillai, A.M. Stuart, Diffusion limits of the random walk Metropolis algorithm in high dimensions authors. Ann. Appl. Probab. 22, 881–930 (2012)
N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, E. Teller, Equations of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1092 (1953)
S. Meyn, R. Tweedie, Markov Chains and Stochastic Stability (Springer, New York, 1993)
R. Neal, Probabilistic Inference Using Markov Chain Monte Carlo Methods. Technical Report CRG-TR-93-1, Department of Computer Science, University of Toronto, 1993
N.S. Pillai, A.M. Stuart, A.H. Thiery, Optimal scaling and diffusion limits for the Langevin algorithm in high dimensions. Ann. Appl. Probab. 22, 2320–2356 (2012)
C.P. Robert, G. Casella, Monte Carlo Statistical Methods, 2nd edn. (Springer, Berlin, 2004)
G.O. Roberts, J.S. Rosenthal, Optimal scaling of discrete approximations to Langevin diffusions. J. R. Stat. Soc. B 60, 255–268 (1998)
G.O. Roberts, R.L. Tweedie, Exponential convergence of Langevin diffusions and their discrete approximations. Bernoulli 2, 341–263 (1996)
G.O. Roberts, A. Gelman, W.R. Gilks, Weak convergence and optimal scaling of random walk Metropolis algorithms. Ann. Appl. Probab. 7, 110–120 (1997)
J.M. Sanz-Serna, Geometric integration, in The State of the Art in Numerical Analysis, ed. by I.S. Duff, A.G. Watson (Clarendon, Oxford, 1997), pp. 121–143
J.M. Sanz-Serna, M.P. Calvo, Numerical Hamiltonian Problems (Chapman & Hall, London, 1994)
T. Schilick, Molecular Modeling and Simulation: An Interdisciplinary Guide, 2nd edn. (Springer, New York, 2010)
C. Schütte, Conformational Dynamics: Modelling, Theory, Algorithmm and Application to Biomolecules. Habilitation Thesis, Free University Berlin, 1999
B. Shahbaba, S. Lan, W.O. Johnson, R.M. Neal, Split Hamiltonian Monte Carlo, preprint
Acknowledgements
This work has been supported by Project MTM2010-18246-C03-01, Ministerio de Ciencia e Innovación, Spain.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Sanz-Serna, J.M. (2014). Markov Chain Monte Carlo and Numerical Differential Equations. In: Current Challenges in Stability Issues for Numerical Differential Equations. Lecture Notes in Mathematics(), vol 2082. Springer, Cham. https://doi.org/10.1007/978-3-319-01300-8_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-01300-8_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-01299-5
Online ISBN: 978-3-319-01300-8
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)