The Axioms of Maximum Entropy

  • John Skilling
Part of the Fundamental Theories of Physics book series (FTPH, volume 31-32)


Maximum entropy is presented as a universal method of finding a “best” positive distribution constrained by incomplete data. The generalised entropy ∑(f - m - f log(f/m))) is the only form which selects acceptable distributions f in particular cases. It holds even if f is not normalised, so that maximum entropy applies directly to physical distributions other than probabilities. Furthermore, maximum entropy should also be used to select “best” parameters if the underlying model m has such freedom.


Maximum Entropy Variational Equation Maximum Entropy Method Positive Distribution Entropy Formula 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Cox, R.T. (1961). The algebra of probable inference. Johns Hopkins Press, Baltimore, MD.zbMATHGoogle Scholar
  2. Good, I.J. (1963). Maximum entropy for hypothesis formulation, especially for multi-dimensional contingency tables. Annals. Math. Stat., 34, 911–934.MathSciNetzbMATHCrossRefGoogle Scholar
  3. Gull, S.F. & Skilling, J. (1984). The maximum entropy method. In Indirect imaging, ed. J.A. Roberts. Cambridge: Cambridge University Press.Google Scholar
  4. Jaynes, E.T. (1957a). Information theory and statistical mechanics I. Phys. Rev., 106, 620–630.MathSciNetCrossRefGoogle Scholar
  5. Jaynes, E.T. (1957b). Information theory and statistical mechanics II. Phys. Rev., 108, 171–190.MathSciNetCrossRefGoogle Scholar
  6. Jaynes, E.T. (1984). Monkeys, Kangaroos and N. Presented at fourth maximum entropy workshop, Calgary, ed. J.H. Justice, Dordrecht: Reidel.Google Scholar
  7. Kullback, S. (1959). Information theory and statistics. New York: Wiley.zbMATHGoogle Scholar
  8. Livesey, A.K. & Skilling, J. (1985). Maximum entropy theory Acta Cryst., A41, 113–122.Google Scholar
  9. Shannon, C.F. (1948). A mathematical theory of communication. Bell System Tech. J., 27, 379–423 and 623–656.MathSciNetzbMATHGoogle Scholar
  10. Shannon, C.E. & Weaver, W. (1949). The mathematical theory of communication. Urbana, Illinois: University Illinois Press.zbMATHGoogle Scholar
  11. Shore, J.E. & Johnson, R.W. (1980). Axiomatic derivation of the principle of maximum entropy and the principle of minimum cros-entropy. IEEE Trans. Info. Theory, IT-26, 26–37 and IT-29, 942–943.MathSciNetCrossRefGoogle Scholar
  12. Tikochinsky, Y., Tishby, N.Z. & Levine, R.D. (1984). Consistent inference of probabilities for reproducible experiments. Phys. Rev. Lett., 52, 1357–1360.CrossRefGoogle Scholar

Copyright information

© Kluwer Academic Publishers 1988

Authors and Affiliations

  • John Skilling
    • 1
  1. 1.Department of Applied Mathematics and Theoretical PhysicsCambridgeEngland

Personalised recommendations