Abstract
Through completing an under specified probability model, Maximum Entropy (MaxEnt) supports non-monotonic inferences. Some major aspects of how this is done by MaxEnt can be understood from the background of two principles of rational decision: the concept of Indifference and the concept of Independence. In a formal specification MaxEnt can be viewed as (conservative) extension of these principles; so these principles shed light on the “magical” decisions of MaxEnt. But the other direction is true as well: Since MaxEnt is a “correct” representation of the set of models (Concentration Theorem), it elucidates these two principles (e.g. it can be shown, that the knowledge of independences can be of very different information-theoretic value). These principles and their calculi are not just arbitrary ideas: When extended to work with qualitative constraints which are modelled by probability intervals, each calculus can be successfully applied to V.Lifschitz’s Benchmarks of Non-Monotonic Reasoning and is able to infer some instances of them ([Lifschitz88]). Since MaxEnt is strictly stronger than the combination of the two principles, it yields a powerful tool for decisions in situations of incomplete knowledge. To give an example, a well-known problem of statistical inference (Simpson’s Paradox) will serve as an illustration throughout the paper.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
E.W. Adams, “The Logic of Conditionals”, D.Reidel Dordrecht Netherlands, 1975.
F. Bacchus, “Lp — A Logic for Statistical Information”, Uncertainty in Artificial Intelligence 5, pp. 3–14, Elsevier Science, ed.: M. Henrion, R.D. Shachter, L.N. Kanal, J.F. Lemmer, 1990.
F. Bacchus, A.J. Grove, J.Y. Halpern, D. Koller, “From Statistical Knowledge Bases to Degrees of Belief”, Technical Report(available via ftp at logos.uwaterloo. ca: /pub/bacchus), 1994.
C. Blyth, “Simpson’s Paradox und mutually favourable Events”, Journal of the American Statistical Association, Vol. 68, p. 746, 1973.
P. Cheeseman, “An Inquiry into Computer Understanding”, Computational Intelligence, Vol. 4, pp. 58 – 66, 1988.
R.T. Cox, “Of Inference and Inquiry — An Essay in Inductive Logic”, in: The Maximum Entropy Formalism, MIT Press, ed.: Levine & Tribus, pp. 119–167, 1979.
C. Howson, P. Urbach. Urbach,“Scientific Reasoning: The Bayesian Approach”, 2nd Edition, Open Court, 1993.
E.T. Jaynes, “Where do we stand on Maximum Entropy?”, 1978, in: E.T. Jaynes: Papers on Probability, Statistics and Statistical Physics, pp. 210 – 314, Kluwer Academic Publishers, ed.: R.D. Rosenkrantz, 1989.
E.T. Jaynes, “On the Rationale of Maximum-Entropy Methods”, Proceedings of the IEEE, Vol. 70, No. 9, pp. 939 – 952, 1982.
P.M. Lewis, “Approximating Probability Distributions to Reduce Storage Requirements”, Information and Control 2, pp. 214–225, 1959.
V. Lifschitz, “Benchmark Problems for Formal nonmonotonic Reasoning”, Lecture Notes in Artificial Intelligence Non-Monotonie Reasoning, Vol. 346, pp. 202–219, ed.: Reinfrank et al., 1988.
R.E. Neapolitan, “Probabilistic Reasoning in Expert Systems: Theory and Algorithms”, John Wiley & Sons, 1990.
J.B. Paris, A. Vencovska, “On the Applicability of Maximum Entropy to Inexact Reasoning”, Int. Journal of approximate reasoning, Vol. 3, pp. 1 – 34, 1989.
J.B. Paris, A. Vencovska, “A note on the Inevitability of Maximum Entropy”, Int. Journal of approximate reasoning, Vol. 4, pp. 183–223, 1990.
J. Pearl, “Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference”, Kaufmann, San Mateo, CA, 1988.
J.E. Shore, R.W. Johnson, “Axiomatic Derivation of the Principle of Maximum Entropy and the Principle of Minimum Cross Entropy”, IEEE Transactions on Information Theory, Vol. IT-26, No. 1, pp. 26– 37, 1980.
J. Skilling, “The Axioms of Maximum Entropy, Maximum-Entropy and Bayesian Methods in Science and Engineering, Vol. 1 – Foundations”, Kluwer Academic, ed.: G.J. Erickson, C.R. Smith, Seattle Univ. Washington, 1988.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1996 Kluwer Academic Publishers
About this paper
Cite this paper
Schramm, M., Greiner, M. (1996). Foundations: Indifference, Independence & MaxEnt. In: Skilling, J., Sibisi, S. (eds) Maximum Entropy and Bayesian Methods. Fundamental Theories of Physics, vol 70. Springer, Dordrecht. https://doi.org/10.1007/978-94-009-0107-0_23
Download citation
DOI: https://doi.org/10.1007/978-94-009-0107-0_23
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-6534-4
Online ISBN: 978-94-009-0107-0
eBook Packages: Springer Book Archive