Skip to main content
SpringerLink
Log in
Book cover

Molecular Dynamics Simulations in Statistical Physics: Theory and Applications pp 343–369Cite as

Information Theory and Statistical Mechanics

  • Chapter
  • First Online:

Part of the book series: Scientific Computation ((SCIENTCOMP))

Abstract

In this chapter, we will discuss some of the elements of the information theory measures. In particular, we will introduce the so-called Shannon and relative entropy of a discrete random process and Markov process. Then, we will discuss the relationship between the entropy using the thermodynamic view and information theory view.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • Abarbanel, H.D.I.: Analysis of Observed Chaotic Data. Springer, New York (1996)

    Book  MATH  Google Scholar 

  • Abarbanel, H.D.I., Kennel, M.B.: Local false nearest neighbors and dynamical dimensions from observed chaotic data. Phys. Rev. E 47(5), 3057–3068 (1993)

    Article  ADS  Google Scholar 

  • Arkhipov, A., Yin, Y., Schulten, K.: Four-scale description of membrane sculpting by BARdomains. Biophys. J. 95, 2806 (2008)

    Article  ADS  Google Scholar 

  • Bahar, I., Jernigan, R.L.: Inter-residue potentials in globular proteins and the dominance of highly specific hydrophilic interactions at close separation. J. Mol. Biol. 266, 195–214 (1997)

    Article  Google Scholar 

  • Bandt, C., Pompe, B.: Permutation entropy: a natural complexity measure for time series. Phys. Rev. Lett. 88, 174102 (2002)

    Article  ADS  Google Scholar 

  • Bonanno, C., Mega, M.: Toward a dynamical model for prime numbers. Chaos Solitons Fractals 20, 107–118 (2004)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  • Canutescu, A.A., Shelenkov, A.A., Dunbrack, R.L. Jr.: A graph-theory algorithm for rapid protein side-chain prediction. Protein Sci. 12, 2001–2014 (2003)

    Article  Google Scholar 

  • Cellucci, C.J., Albano, A.M., Rapp, P.E.: Comparative study of embedding methods. Phys. Rev. E 67, 066210–066213 (2003)

    Article  ADS  MathSciNet  Google Scholar 

  • Cellucci, C.J., Albano, A.M., Rapp, P.E.: Statistical validation of mutual information calculations: comparison of alternative numerical algorithms. Phys. Rev. E 71, 066208–066214 (2005)

    Article  ADS  Google Scholar 

  • Dama, J.F., Sinitskiy, A.V., McCullagh, M., Weare, J., Roux, B., Dinner, A.R., Voth, G.A.: J. Chem. Theory Comput. 9, 2466 (2013)

    Article  Google Scholar 

  • Gay, J.G., Berne, B.J.: Modification of the overlap potential to mimic a linear site-site potential. J. Chem. Phys. 74, 3316 (1981)

    Article  ADS  Google Scholar 

  • Gohlke, H., Thorpe, M.F.: A natural coarse graining for simulating large biomolecular motion. Biophys. J. 91, 2115–2120 (2006)

    Article  ADS  Google Scholar 

  • Goldstein, H.: Classical Mechanics, 2nd edn. Addison-Wesley, San Francisco (2002)

    MATH  Google Scholar 

  • Gopal, S.M., Mukherjee, S., Cheng, Y.M., Feig, M.: PRIMO/PRIMONA: A coarse-grained model for proteins and nucleic acids that preserves near-atomistic accuracy. Proteins 78, 1266–1281 (2010)

    Article  Google Scholar 

  • Gourévitch, B., Eggermont, J.: Evaluating information transfer between auditory cortical neurons. J. Neurophysiol. 97, 2533–2543 (2007)

    Article  Google Scholar 

  • Granger, J.: Investigating causal relations by econometric models and crossspectral methods. Acta Physica Polonica B 37, 424–438 (1969)

    MATH  Google Scholar 

  • Grassberger, P.: Finite sample corrections to entropy and dimension estimates. Phys. Lett. A 128, 369–373 (1988)

    Article  ADS  MathSciNet  Google Scholar 

  • Grassberger, P., Procaccia, I.: Measuring the strangeness of strange attractors. Physica D 9, 189 (1983)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  • Irbäck, A., Sjunnesson, F., Wallin, S.: Hydrogen bonds, hydrophobicity forces and the character of the collapse transition. Proc. Natl. Acad. Sci. U.S.A. 97, 13614 (2000)

    Article  ADS  Google Scholar 

  • Joe, H.: Relative entropy measures of multivariate dependence. J. Am. Statist. Assoc. 84, 157–164 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  • Kamberaj, H.: A theoretical model for the collective motion of proteins by means of principal component analysis. Cent. Eur. J. Phys. 9(1), 96–109 (2011)

    Google Scholar 

  • Kamberaj, H.: Faster protein folding using enhanced conformational sampling of molecular dynamics simulation. J. Mol. Graph. Model. 81, 32–49 (2018)

    Article  Google Scholar 

  • Kamberaj, H., van der Vaart, A.: Extracting the causality of correlated motions from molecular dynamics simulations. Biophys. J. 97, 1747–1755 (2009a)

    Article  ADS  Google Scholar 

  • Kamberaj, H., van der Vaart, A.: Extracting the causality of correlated motions from molecular dynamic simulations. Biophys. J. 97, 1747–1755 (2009b)

    Article  ADS  Google Scholar 

  • Kennel, M.B., Brown, R., Abarbanel, H.D.I.: Determining embedding dimension for phase-space reconstruction using a geometrical construction. Phys. Rev. A 45, 3403–3411 (1992)

    Article  ADS  Google Scholar 

  • Kraskov, A., Stögbauer, H., Grassberger, P.: Estimating mutual information. Phys. Rev. E 69(6), 066138 (2004)

    Article  ADS  MathSciNet  Google Scholar 

  • Kullback, S.: Information Theory and Statistics. Wiley, New York (1959)

    MATH  Google Scholar 

  • Kullback, S.: The Kullback-Leibler distance. Am. Stat. 41, 340–341 (1987)

    Google Scholar 

  • Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22, 79–86 (1951)

    Article  MathSciNet  MATH  Google Scholar 

  • Lehrman, M., Rechester, A.B., White, R.B.: Symbolic analysis of chaotic signals and turbulent fluctuations. Phys. Rev. Lett 78, 54–57 (1997)

    Article  ADS  Google Scholar 

  • Liang, X.S.: The Liang-Kleeman information flow: theory and applications. Entropy 15, 327–360 (2013)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  • Liang, X.S., Kleeman, R.: Information transfer between dynamical system components. Phys. Rev. Lett. 95, 244101 (2005)

    Article  ADS  Google Scholar 

  • Maciejczyk, M., Spasic, A., Liwo, A., Scheraga, H.A.: Coarse grained model of nucleic acid bases. J. Comput. Chem. 31, 1644 (2010)

    Google Scholar 

  • McCammon, J.A., Northrup, S.H., Karplus, M., Levy, R.M.: Helixcoil transitions in a simple polypeptide model. Biopolymers 19, 2033–2045 (1980)

    Article  Google Scholar 

  • Moon, Y.I., Rajagopalam, B., Lall, U.: Estimation of mutual information using kernel density estimators. Phys. Rev. E 52, 2318 (1995)

    Article  ADS  Google Scholar 

  • Murtola, T., Karttunen, M., Vattulainen, I.: Systematic coarse graining from structure using internal states: application to phospholipid/ cholesterol bilayer. J. Chem. Phys. 131, 055101 (2009)

    Article  ADS  Google Scholar 

  • Noakes, L.: The Takens embedding theorem. Int. J. Bifurcation Chaos Appl. Sci. Eng. 1, 867–872 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  • Oldziej, S., Liwo, A., Czaplewski, C., Pillardy, J., Scheraga, H.A.: Optimization of the UNRES force field by hierarchical design of the potential-energy landscape. 2. Off-lattice tests of the method with single proteins. J. Phys. Chem. B 108, 16934–16949 (2004)

    Article  Google Scholar 

  • Packard, N.H., Crutchfield, J.P., Farmer, J.D., Shaw, R.S.: Geometry from a time series. Phys. Rev. Lett. 45(9), 712–716 (1980)

    Article  ADS  Google Scholar 

  • Potestio, R., Pontiggia, F., Micheletti, C.: Biophys. J. 96, 4993 (2009)

    Article  ADS  Google Scholar 

  • Prokopenko, M., Lizier, J.T.: Transfer entropy and transient limits of computation. Sci. Rep. 4, 5394 (2014)

    Article  ADS  Google Scholar 

  • Prokopenko, M., Lizier, J.T., Price, D.C.: On thermodynamic interpretation of transfer entropy. Entropy 15: 524–543 (2013)

    Article  ADS  MathSciNet  Google Scholar 

  • Rechester, A.B., White, R.B.: Symbolic kinetic equations for a chaotic attractor. Phys. Lett. A 156, 419–424 (1991a)

    Article  ADS  MathSciNet  Google Scholar 

  • Rechester, A.B., White, R.B.: Symbolic kinetic analysis of two-dimensional maps. Phys. Lett. A 158, 51–56 (1991b)

    Article  ADS  MathSciNet  Google Scholar 

  • Rotkiewicz, P., Skolnick, J.: Fast procedure for reconstruction of full-atom protein models from reduced representations. J. Comput. Chem. 29, 1460–1465 (2008)

    Article  Google Scholar 

  • Sauer, T., Yorke, J.A., Casdagli, M.: Embedology. J. Stat. Phys. 65, 579–616 (1991)

    Article  ADS  Google Scholar 

  • Schreiber, T.: Measuring information transfer. Phys. Rev. Lett. 85, 461–464 (2000)

    Article  ADS  Google Scholar 

  • Shannon, C.E., Weaver, W.: The Mathematical Theory of Information. University of Illinois Press, Urbana (1949)

    MATH  Google Scholar 

  • Shi, Q., Izvekov, S., Voth, G.A.: Mixed atomistic and coarse-grained molecular dynamics: simulation of membrane-bound ion channel. J. Phys. Chem. B 110, 15045–15048 (2006)

    Article  Google Scholar 

  • Shih, A.Y., Arkhipov, A., Freddolino, P.L., Schulten, K.: Coarse grained protein-lipid model with application to lipidprotein particles. J. Phys. Chem. B 110, 3674–3684 (2006)

    Article  Google Scholar 

  • Sinitskiy, A.V., Saunders, M.G., Voth, G.A.: Optimal number of coarsegrained sites in different components of large biomolecular complexes. J. Phys. Chem. B 116, 8363–8374 (2012)

    Article  Google Scholar 

  • Smith, A.V., Hall, C.K.: α-helix formation: discontinuous molecular dynamics on an intermediate-resolution protein model Proteins 44, 344–360 (2001a)

    Google Scholar 

  • Smith, A.V., Hall, C.K.: Assembly of a tetrameric α-helical bundle: computer simulations on an intermediate-resolution protein model. Proteins 44, 376–391 (2001b)

    Article  Google Scholar 

  • Staniek, M., Lehnertz, K.: Symbolic transfer entropy. Phys. Rev. Lett. 100, 158101 (2008)

    Article  ADS  Google Scholar 

  • Stepanova, M.: Dynamics of essential collective motions in proteins: theory. Phys. Rev. E 76(5), 051918 (2007)

    Article  ADS  MathSciNet  Google Scholar 

  • Takens, F.: Detecting strange attractors in fluid turbulence, Dynamical Systems and Turbulence. Springer, Berlin (1981)

    MATH  Google Scholar 

  • Thomas, M.C., Joy, A.T.: Elements of Information Theory. Wiley, Hoboken (2006)

    MATH  Google Scholar 

  • Tozzini, V.: Coarse-grained models for proteins. Curr. Opin. Struct. Bio. 15, 144–150 (2005)

    Article  Google Scholar 

  • Tozzini, V., McCammon, J.: A coarse grained model for the dynamics of the early stages of the binding mechanism of HIV-1 protease. Chem. Phys. Lett. 413, 123–128 (2005)

    Article  ADS  Google Scholar 

  • Tozzini, V., Rocchia, W., McCammon, J.A.: Mapping all-atom models onto one-bead coarse-grained models: general properties and applications to a minimal polypeptide model. J. Chem. Theory Comput. 2, 667–673 (2006)

    Article  Google Scholar 

  • Tschöp, W., Kremer, K., Hahn, O., Batoulis, J., Bürger, T.: Simulation of polymer melts. II. From coarse-grained models back to atomistic description. Acta Polym. 49, 75–79 (1998)

    Google Scholar 

  • Tuckerman, M.E., Liu, Y., Ciccotti, G., Martyna, G.J.: Non-Hamiltonian molecular dynamics: generalizing Hamiltonian phase space principles to non-Hamiltonian systems. J. Chem. Phys. 115(4), 1678–1702 (2001)

    Article  ADS  Google Scholar 

  • Ueda, Y., Taketomi, H., Go, N.: Studies on protein folding, unfolding, and fluctuations by computer simulation. II. A. three-dimensional lattice model of lysozyme. Biopolymers 17, 1531–1548 (1978)

    Google Scholar 

  • Voth, G.A. (ed.): Coarse-Graining of Condensed Phase and Biomolecular Systems. CRC Press, Boca Raton (2008)

    Google Scholar 

  • Zhang, J., Muthukumar, M.: Simulations of nucleation and elongation of amyloid fibrils. J. Chem. Phys. 130, 035102 (2009)

    Article  ADS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Engineering, International Balkan University, Skopje, North Macedonia

    Hiqmet Kamberaj

  2. Advanced Computing Research Center, University of New York Tirana, Tirana, Albania

    Hiqmet Kamberaj

Authors
  1. Hiqmet Kamberaj
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Kamberaj, H. (2020). Information Theory and Statistical Mechanics. In: Molecular Dynamics Simulations in Statistical Physics: Theory and Applications. Scientific Computation. Springer, Cham. https://doi.org/10.1007/978-3-030-35702-3_9

Download citation

Publish with us

Policies and ethics

Search

Navigation