An Engineer Looks at Bayes
These notes are intended to accompany a tutorial session on the fundamental ideas behind the use of Bayesian methods and the principle of Maximum Entropy. The material on which these notes is based is scattered throughout the literature and seldom brought together in a coherent whole. In this presentation many difficult steps in mathematics are skipped so as to bring out the flow of ideas. References at the end provide the missing detail.
KeywordsStatistical Mechanic Maximum Entropy Incomplete Information Bayesian Method Inductive Logic
Unable to display preview. Download preview PDF.
- 1.Jeffreys, Harold “Theory of Probability”, Oxford Press, 1961 Third Edition. (This is the first comprehensive treatment, using a unique and powerful notation with many examples. An excellent reference for those who are mathematically able)Google Scholar
- 2.Cox, Richard “The Algebra of Probable Inference”, Johns Hopkins University Press, Baltimore, MD 1961. (This short book updates and extends Professor Cox’s original 1946 paper. It provides the basis for a system of inference devoid of the usual reliance on populations, etc. but never loosing sight of the original goal, which is to establish a system of inference.)zbMATHGoogle Scholar
- 4.Jaynes, Edwin T. “Where do we Stand on Maximum Entropy” in reference 3 above, pp 15–117. (This paper is a review of the foundations, applications and attacks upon the maximum entropy formalism. It contains numerous references worth reading in their own right.)Google Scholar
- 5.Justice, James H. (editor) “Maximum Entropy and Bayesian Methods in Applied Statistics” Cambridge University Press, England. 1986 (This book contains 17 papers in applied probability including an updating by Jaynes of attacks upon and defenses of the methods, applications to photo analysis, estimates of the Earth’s density distribution and applications in seismology. Each paper contains many references in the field)Google Scholar
- 6.Tribus, Myron “Rational Descriptions, Decisions and Designs”, Pergamon Press, 1969 (This book contains many of the derivations referred to in the body of this paper. It is out of print and hard to get, probably deservedly so.)Google Scholar