Journal of Statistical Physics

, Volume 155, Issue 1, pp 93–105 | Cite as

Gibbs–Jaynes Entropy Versus Relative Entropy



The maximum entropy formalism developed by Jaynes determines the relevant ensemble in nonequilibrium statistical mechanics by maximising the entropy functional subject to the constraints imposed by the available information. We present an alternative derivation of the relevant ensemble based on the Kullback–Leibler divergence from equilibrium. If the equilibrium ensemble is already known, then calculation of the relevant ensemble is considerably simplified. The constraints must be chosen with care in order to avoid contradictions between the two alternative derivations. The relative entropy functional measures how much a distribution departs from equilibrium. Therefore, it provides a distinct approach to the calculation of statistical ensembles that might be applicable to situations in which the formalism presented by Jaynes performs poorly (such as non-ergodic dynamical systems).


Gibbs–Jaynes entropy Kullback–Leibler divergence Relative entropy Maximum entropy formalism Nonequilibrium statistical mechanics 



We would like to express our gratitude to the anonymous reviewers of this article for their insightful comments.


  1. 1.
    Jaynes, E.T.: Information theory and statistical mechanics. Phys. Rev. 106, 620–630 (1957)ADSCrossRefMATHMathSciNetGoogle Scholar
  2. 2.
    Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423 (1948). 623–656CrossRefMATHMathSciNetGoogle Scholar
  3. 3.
    Jaynes, E.T.: Information theory and statistical mechanics. In: Statistical Physics. W. A. Benjamin Inc, New York (1963)Google Scholar
  4. 4.
    Kawasaki, K., Gunton, J.D.: Theory of nonlinear transport processes: nonlinear shear viscosity and normal stress effects. Phys. Rev. A 8, 20482064 (1973)CrossRefGoogle Scholar
  5. 5.
    Grabert, H.: Projection Operator Techniques in Nonequilibrium Statistical Mechanics, pp. 29–32. Springer, Berlin (1982)Google Scholar
  6. 6.
    Zubarev, D.: Statistical Mechanics of Nonequilibrium Processes, pp. 89–98. Wiley, Berlin (1996)MATHGoogle Scholar
  7. 7.
    Gaveau, B., Schulman, L.S.: A general framework for non-equilibrium phenomena: the master equation and its formal consequences. Phys. Lett. A 229, 347–353 (1997)ADSCrossRefMATHMathSciNetGoogle Scholar
  8. 8.
    Qian, H.: Relative entropy: free energy associated with equilibrium fluctuations and nonequilibrium deviations. Phys. Rev. E 63, 042103 (2001)ADSCrossRefGoogle Scholar
  9. 9.
    Kawai, R., Parrondo, J.M.R., Van der Broeck, C.: Dissipation: the phase-space perspective. Phys. Rev. Lett. 98, 080602 (2007)ADSCrossRefGoogle Scholar
  10. 10.
    Shell, M.S.: The relative entropy is fundamental to multiscale and inverse thermodynamic problems. J. Chem. Phys. 129, 144108 (2008)ADSCrossRefGoogle Scholar
  11. 11.
    Vaikuntanathan, S., Jarzynski, C.: Dissipation and lag in irreversible processes. Europhys. Lett. 87, 60005 (2009)ADSCrossRefGoogle Scholar
  12. 12.
    Horowitz, J., Jarzynski, C.: Illustrative example of the relationship between dissipation and relative entropy. Phys. Rev. E 79, 021106 (2009)ADSCrossRefGoogle Scholar
  13. 13.
    Roldán, E., Parrondo, J.M.R.: Entropy production and Kullback–Leibler divergence between stationary trajectories of discrete systems. Phys. Rev. E 85, 031129 (2012)ADSCrossRefGoogle Scholar
  14. 14.
    Crooks, G. E., and Sivak, D. A.: Measures of trajectory ensemble disparity in nonequilibrium statistical dynamics. J. Stat. Mech.: Theory Exp. 2011, P06003 (2011)Google Scholar
  15. 15.
    Crooks, G. E.: On thermodynamic and microscopic reversibility. J. Stat. Mech.: Theory Exp. 2011, P07008 (2011)Google Scholar
  16. 16.
    Sivak, D.A., Crooks, G.E.: Near equilibrium measurements of nonequilibrium free energy. Phys. Rev. Lett. 108, 150601 (2012)ADSCrossRefGoogle Scholar
  17. 17.
    Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22, 79–86 (1951)CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Departamento de Física FundamentalUniversidad Nacional de Educación a Distancia MadridSpain

Personalised recommendations