Skip to main content
SpringerLink
Log in

Markov Model Theory

  • Chapter
An Introduction to Markov State Models and Their Application to Long Timescale Molecular Simulation

Part of the book series: Advances in Experimental Medicine and Biology ((AEMB,volume 797))

Abstract

This section reviews the relation between the continuous dynamics of a molecular system in thermal equilibrium and the kinetics given by a Markov State Model (MSM). We will introduce the dynamical propagator, an error-less, alternative description of the continuous dynamics, and show how MSMs result from its discretization. This allows for an precise understanding of the approximation quality of MSMs in comparison to the continuous dynamics. The results on the approximation quality are key for the design of good MSMs. While this section is important for understanding the theory of discretization and related systematic errors, practitioners wishing only to learn how to construct MSMs may skip directly to the discussion of Markov model estimation.

Part of this chapter is reprinted with permission from Prinz et al. (Markov models of molecular kinetics: Generation and validation. J. Chem. Phys. 134:174,105, 2011). Copyright 2011, American Institute of Physics.

A more detailed text book version is to be found in Schütte and Sarich (Metastability and Markov State Models in Molecular Dynamics: Modeling, Analysis, Algorithmic Approaches. Courant Lecture Notes, 24. American Mathematical Society, 2013).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Andersen HC (1980) Molecular dynamics simulations at constant pressure and/or temperature. J Chem Phys 72(4):2384–2393. doi:10.1063/1.439486

    Article  CAS  Google Scholar 

  2. Bieri O, Wirz J, Hellrung B, Schutkowski M, Drewello M, Kiefhaber T (1999) The speed limit for protein folding measured by triplet-triplet energy transfer. Proc Natl Acad Sci USA 96(17):9597–9601. http://www.pnas.org/content/96/17/9597.abstract

    Article  PubMed  CAS  Google Scholar 

  3. Bowman GR, Beauchamp KA, Boxer G, Pande VS (2009) Progress and challenges in the automated construction of Markov state models for full protein systems. J Chem Phys 131(12):124,101+. doi:10.1063/1.3216567

    Article  Google Scholar 

  4. Buchete NV, Hummer G (2008) Coarse Master Equations for Peptide Folding Dynamics. J Phys Chem B 112:6057–6069

    Article  PubMed  CAS  Google Scholar 

  5. Chan CK, Hu Y, Takahashi S, Rousseau DL, Eaton WA, Hofrichter J (1997) Submillisecond protein folding kinetics studied by ultrarapid mixing. Proc Natl Acad Sci USA 94(5):1779–1784. http://www.pnas.org/content/94/5/1779.abstract

    Article  PubMed  CAS  Google Scholar 

  6. Chodera JD, Dill KA, Singhal N, Pande VS, Swope WC, Pitera JW (2007) Automatic discovery of metastable states for the construction of Markov models of macromolecular conformational dynamics. J Chem Phys 126:155,101

    Article  Google Scholar 

  7. Chodera JD, Noé F (2010) Probability distributions of molecular observables computed from Markov models, II: uncertainties in observables and their time-evolution. J Chem Phys 133:105,102

    Article  Google Scholar 

  8. Chodera JD, Swope WC, Noé F, Prinz JH, Pande VS (2010) Dynamical reweighting: improved estimates of dynamical properties from simulations at multiple temperatures. J Phys Chem. doi:10.1063/1.3592152

    Google Scholar 

  9. Chodera JD, Swope WC, Pitera JW, Dill KA (2006) Long-time protein folding dynamics from short-time molecular dynamics simulations. Multiscale Model Simul 5:1214–1226

    Article  Google Scholar 

  10. Cooke B, Schmidler SC (2008) Preserving the Boltzmann ensemble in replica-exchange molecular dynamics. J Chem Phys 129:164,112

    Article  Google Scholar 

  11. Deuflhard P, Huisinga W, Fischer A, Schütte C (2000) Identification of almost invariant aggregates in reversible nearly uncoupled Markov chains. Linear Algebra Appl 315:39–59

    Article  Google Scholar 

  12. Deuflhard P, Weber M (2005) Robust Perron cluster analysis in conformation dynamics. Linear Algebra Appl 398:161–184

    Article  Google Scholar 

  13. Djurdjevac N, Sarich M, Schütte C (2012) Estimating the eigenvalue error of Markov state models. Multiscale Model Simul 10(1):61–81

    Article  Google Scholar 

  14. Duane S (1987) Hybrid Monte Carlo. Phys Lett B 195(2):216–222. doi:10.1016/0370-2693(87)91197-X

    Article  CAS  Google Scholar 

  15. Ermak DL (1975) A computer simulation of charged particles in solution, I: technique and equilibrium properties. J Chem Phys 62(10):4189–4196

    Article  CAS  Google Scholar 

  16. Ermak DL, Yeh Y (1974) Equilibrium electrostatic effects on the behavior of polyions in solution: polyion-mobile ion interaction. Chem Phys Lett 24(2):243–248

    Article  CAS  Google Scholar 

  17. Golub GH, van Loan CF (1996) Matrix computations, 3rd edn. John Hopkins University Press, Baltimore

    Google Scholar 

  18. Herau R, Hitrik M, Sjoestrand J (2010) Tunnel effect and symmetries for Kramers Fokker-Planck type operators. arXiv:1007.0838v1

  19. Huisinga W (2001) Metastability of Markovian systems: a transfer operator based approach in application to molecular dynamics. PhD thesis, Fachbereich Mathematik und Informatik, FU Berlin

    Google Scholar 

  20. Huisinga W, Meyn S, Schütte C (2004) Phase transitions and metastability for Markovian and molecular systems. Ann Appl Probab 14:419–458

    Article  Google Scholar 

  21. Jäger M, Nguyen H, Crane JC, Kelly JW, Gruebele M (2001) The folding mechanism of a beta-sheet: the WW domain. J Mol Biol 311(2):373–393. doi:10.1006/jmbi.2001.4873

    Article  PubMed  Google Scholar 

  22. van Kampen NG (2006) Stochastic processes in physics and chemistry, 4th edn. Elsevier, Amsterdam

    Google Scholar 

  23. Keller B, Hünenberger P, van Gunsteren W (2010) An analysis of the validity of Markov state models for emulating the dynamics of classical molecular systems and ensembles. J Chem Theor Comput. doi:10.1021/ct200069c

    Google Scholar 

  24. Kube S, Weber M (2007) A coarse graining method for the identification of transition rates between molecular conformations. J Chem Phys 126:024,103–024,113

    Article  Google Scholar 

  25. Meerbach E, Schütte C, Horenko I, Schmidt B (2007) Metastable conformational structure and dynamics: peptides between gas phase and aqueous solution. Series in chemical physics, vol 87. Springer, Berlin, pp 796–806

    Google Scholar 

  26. Metzner P, Horenko I, Schütte C (2007) Generator estimation of Markov jump processes based on incomplete observations nonequidistant in time. Phys Rev E 76(6):066,702+. doi:10.1103/PhysRevE.76.066702

    Article  Google Scholar 

  27. Metzner P, Schütte C, Vanden-Eijnden E (2009) Transition path theory for Markov jump processes. Multiscale Model Simul 7(3):1192–1219

    Article  CAS  Google Scholar 

  28. Neuweiler H, Doose S, Sauer M (2005) A microscopic view of miniprotein folding: enhanced folding efficiency through formation of an intermediate. Proc Natl Acad Sci USA 102(46):16,650–16,655. doi:10.1073/pnas.0507351102

    Article  CAS  Google Scholar 

  29. Noé F, Horenko I, Schütte C, Smith JC (2007) Hierarchical analysis of conformational dynamics in biomolecules: transition networks of metastable states. J Chem Phys 126:155,102

    Article  Google Scholar 

  30. Noé F, Schütte C, Vanden-Eijnden E, Reich L, Weikl TR (2009) Constructing the full ensemble of folding pathways from short off-equilibrium simulations. Proc Natl Acad Sci USA 106:19,011–19,016

    Article  Google Scholar 

  31. Prinz JH et al. (2011) Markov models of molecular kinetics: generation and validation. J Chem Phys 134:174,105

    Article  Google Scholar 

  32. Sarich M (2011) Projected transfer operators. PhD thesis, Freie Universität Berlin

    Google Scholar 

  33. Sarich M, Noé F, Schütte C (2010) On the approximation error of Markov state models. Multiscale Model Simul 8:1154–1177

    Article  Google Scholar 

  34. Sarich M, Schütte C (2012) Approximating selected non-dominant timescales by Markov state models. Comm Math Sci (in press)

    Google Scholar 

  35. Schütte C (1998) Conformational dynamics: modelling, theory, algorithm, and applications to biomolecules. Habilitation thesis, Fachbereich Mathematik und Informatik, FU Berlin

    Google Scholar 

  36. Schütte C, Huisinga W (2003) Biomolecular conformations can be identified as metastable sets of molecular dynamics. In: Handbook of numerical analysis. Elsevier, Amsterdam, pp 699–744

    Google Scholar 

  37. Schütte C, Sarich M (2013) Metastability and Markov state models in molecular dynamics: modeling, analysis, algorithmic approaches. Courant lecture notes, vol 24. American Mathematical Society, Providence

    Google Scholar 

  38. Schütte C, Fischer A, Huisinga W, Deuflhard P (1999) A direct approach to conformational dynamics based on hybrid Monte Carlo. J Comput Phys 151:146–168

    Article  Google Scholar 

  39. Schütte C, Noé F, Meerbach E, Metzner P, Hartmann C (2009) Conformation dynamics. In: Jeltsch RGW (ed) Proceedings of the international congress on industrial and applied mathematics (ICIAM). EMS Publishing House, New York, pp 297–336

    Google Scholar 

  40. Schütte C, Noé F, Lu J, Sarich M, Vanden-Eijnden E (2011) Markov state models based on milestoning. J Chem Phys 134(19)

    Google Scholar 

  41. Sriraman S, Kevrekidis IG, Hummer G (2005) Coarse master equation from Bayesian analysis of replica molecular dynamics simulations. J Phys Chem B 109:6479–6484

    Article  PubMed  CAS  Google Scholar 

  42. Swope WC, Andersen HC, Berens PH, Wilson KR (1982) A computer simulation method for the calculation of equilibrium constants for the formation of physical clusters of molecules: application to small water clusters. J Chem Phys 76(1):637–649

    Article  CAS  Google Scholar 

  43. Swope WC, Pitera JW, Suits F (2004) Describing protein folding kinetics by molecular dynamics simulations, 1: theory. J Phys Chem B 108:6571–6581

    Article  CAS  Google Scholar 

  44. Swope WC, Pitera JW, Suits F, Pitman M, Eleftheriou M (2004) Describing protein folding kinetics by molecular dynamics simulations, 2: example applications to alanine dipeptide and beta-hairpin peptide. J Phys Chem B 108:6582–6594

    Article  CAS  Google Scholar 

  45. Voronoi MG (1908) Nouvelles applications des parametres continus a la theorie des formes quadratiques. J Reine Angew Math 134:198–287

    Google Scholar 

  46. Weber M (2006) Meshless methods in conformation dynamics. PhD thesis, Freie Universität Berlin

    Google Scholar 

  47. Weber M (2010) A subspace approach to molecular Markov state models via an infinitesimal generator. ZIB report 09-27-rev

    Google Scholar 

  48. Weber M (2011) A subspace approach to molecular Markov state models via a new infinitesimal generator. Habilitation thesis, Fachbereich Mathematik und Informatik, Freie Universität Berlin

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Freie Universität Berlin, Arnimallee 6, 14195, Berlin, Germany

    Marco Sarich, Jan-Hendrik Prinz & Christof Schütte

Authors
  1. Marco Sarich
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan-Hendrik Prinz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christof Schütte
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christof Schütte .

Editor information

Editors and Affiliations

  1. University of California, Berkeley, California, USA

    Gregory R. Bowman

  2. Department of Chemistry, Stanford University, Stanford, California, USA

    Vijay S. Pande

  3. Freie Universität Berlin, Berlin, Germany

    Frank Noé

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Sarich, M., Prinz, JH., Schütte, C. (2014). Markov Model Theory. In: Bowman, G., Pande, V., Noé, F. (eds) An Introduction to Markov State Models and Their Application to Long Timescale Molecular Simulation. Advances in Experimental Medicine and Biology, vol 797. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-7606-7_3

Download citation

Publish with us

Policies and ethics

Search

Navigation