Bayesian Inductive Inference and Maximum Entropy

  • Stephen F. Gull
Part of the Fundamental Theories of Physics book series (FTPH, volume 31-32)


The principles of Bayesian reasoning are reviewed and applied to problems of inference from data sampled from Poisson, Gaussian and Cauchy distributions. Probability distributions (priors and likelihoods) are assigned in appropriate hypothesis spaces using the Maximum Entropy Principle, and then manipulated via Bayes’ Theorem. Bayesian hypothesis testing requires careful consideration of the prior ranges of any parameters involved, and this leads to a quantitive statement of Occam’s Razor. As an example of this general principle we offer a solution to an important problem in regression analysis; determining the optimal number of parameters to use when fitting graphical data with a set of basis functions.


Posterior Distribution Scale Parameter BAYESIAN Inference Maximum Entropy Bayesian Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Birkinshaw, M., Gull, S.F. & Hardebeck, H. (1984). Nature, 309, 34–35.CrossRefGoogle Scholar
  2. Brans, C. & Dicke, R.H. (1961). Phys. Rev., 124, 925.MathSciNetzbMATHCrossRefGoogle Scholar
  3. Cox, R.P. (1946). Probability, Frequency and Reasonable Expectation. Am. Jour. Phys. 17, 1–13.CrossRefGoogle Scholar
  4. Gull, S.F. & Skilling, J. (1984). Maximum entropy method in image processing. IEE Proc., 131(F), 646–659.Google Scholar
  5. Jaynes, E.T. (1968). Prior probabilities. Reprinted in E.T. Jaynes: Papers on Probability, Statistics and Statistical Physics, ed. R. Rosenkrantz, 1983 Dordrecht: Reidel.Google Scholar
  6. Jaynes, E.T. (1976). Confidence intervals versus Bayesian Intervals. Reprinted in E.T. Jaynes: Papers on Probability, Statistics and Statistical Physics, ed. R. Rosenkrantz, 1983. Dordrecht: Reidel.Google Scholar
  7. Jaynes, E.T. (1986). Bayesian Methods — an Introductory Tutorial. In Maximum Entropy and Bayesian Methods in Applied Statistics. ed. J.H. Justice. Cambridge University Press.Google Scholar
  8. Jeffreys, H. (1939). Theory of Probability, Oxford University Press. Later editions 1948, 1961,1983.Google Scholar
  9. Shore, J.E. & Johnson, R.W. (1980). Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy. IEEE Trans. Info. Theory, IT-26, 26–39 and IT-29, 942–943.Google Scholar
  10. Skilling, J. (1986). The Axioms of Maximum Entropy. Presented at 1986 Maximum Entropy conference, Seattle, Washington (this volume).Google Scholar
  11. Skilling, J. & Gull, S.F. (1984). The entropy of an image. SIAM Amer. Math. Soc. proc. Appl. Math., 14, 167–189MathSciNetGoogle Scholar

Copyright information

© Kluwer Academic Publishers 1988

Authors and Affiliations

  • Stephen F. Gull
    • 1
  1. 1.Cavendish LaboratoryMullard Radio Astronomy ObservatoryCambridgeUK

Personalised recommendations