Abstract
In this chapter we define the information of an event A ∈ Σ, or in our terminology the novelty of a propositionA as \({-\log }_{2}p(A)\). We further define the important new concept of a description and extend the definition of novelty from events to descriptions. Finally we introduce the notions of completeness and directedness of descriptions and thereby the distinction between surprise and information, which are opposite special cases of novelty.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
This requirement obviously implies that the propositions [d = A] are in Σ for every A ∈ R(d). It also implies that R(d) is finite or countable.
- 3.
For example, one can describe points \(x \in A \setminus C\) by d(x) = A, points x in A ∩ C by d(x) = A ∩ C, points x in \(B \setminus C\) by d(x) = B, and points x in B ∩ C by d(x) = C.
- 4.
From Definition (ii) it is obvious that the probability that ω is in two sets, e.g., A i and A j , is 0. So, disregarding propositions with probability 0, every ω ∈ Ω is in exactly one of the sets A i . We will usually disregard propositions with probability 0 and this is meant by the word “essentially”.
- 5.
- 6.
Strictly speaking, here a partition should be defined by \({A}_{i} \cap {A}_{j} = \varnothing \) instead of \(p({A}_{i} \cap {A}_{j}) = 0\) (compare Definition 2.4). If we disregard 0-probability-propositions we should interpret \(d =\tilde{ d}\) in Definition 2.6 as \(p[d =\tilde{ d}] = 1\) and we should use the weaker formulation \(p(d(\omega ) \cap d(\omega ^{\prime})) = 0\) in part (ii) of this definition.
- 7.
If \(p(\widetilde{A}) = 0\) for some \(\widetilde{A}\) in the range of \(\tilde{d}\), we set \(p(\widetilde{A})\log p(\widetilde{A}) = 0\) since \({\lim }_{x\rightarrow {0}^{+}}x\log x = 0\).
- 8.
Here we rely on the approximation argument as in Sect. 1.3 for the calculation of an expectation value. If all the finite sum approximations satisfy the same inequality (\(\leq \frac{1} {\ln 2}\)), then this inequality is also satisfied by the limit.
- 9.
See Definition 1.3.
References
Ash, R. B. (1965). Information theory. New York, London, Sidney: Interscience.
Billingsley, P. (1978). Ergodic theory and information. Huntington, NY: Robert E. Krieger Publishing Co.
Boltzmann, L. (1887). Über die mechanischen Analogien des zweiten Hauptsatzes der Thermodynamik. Journal für die reine und angewandte Mathematik (Crelles Journal), 100, 201–212.
Brush, S. G. (1966). Kinetic theory: Irreversible processes, Vol. 2. New York: Pergamon Press, Oxford.
Bapeswara-Rao, V. V., & Rao, M. B. (1992). A three-door game show and some of its variants. The Mathematical Scientist, 17, 89–94.
Clausius, R. J. E. (1865). Über verschiedenen für die Anwendung bequeme Formen der Hauptgleichungen der mechanischen Wärmetheorie. Annales de Physique, 125, 353–400.
Gallager, R. G. (1968). Information theory and reliable communication. New York, NY, USA: John Wiley & Sons, Inc.
Gardner, M. (1969). The unexpected hanging and other mathematical diversions. Simon and Schuster: New York.
Gardner, M. (1959). Mathematical games column. Scientific American.
Gillman, L. (1992). The car and the goats. American Mathematical Monthly, 99(1), 3–7.
Granberg, D., & Brown, T. A. (1995). The Monty hall Dilemma. Personality and Social Psychology Bulletin, 21(7), 711–723.
Khinchin, A. (1957). Mathematical foundations of information theory. New York: Dover Publications, Inc.
Palm, G. (1981). Evidence, information and surprise. Biological Cybernetics, 42(1), 57–68.
Selvin, S. (1975). On the Monty Hall problem [Letter to the editor]. The American Statistician, 29(3), 134.
Seymann, R. G. (1991). Comment on let’s make a deal: The player’s Dilemma. The American Statistician, 45(4), 287–288.
Shannon, C. E., & Weaver, W. (1949). The mathematical theory of communication. USA: University of Illinois Press.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Palm, G. (2012). Improbability and Novelty of Descriptions. In: Novelty, Information and Surprise. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29075-6_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-29075-6_2
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-29074-9
Online ISBN: 978-3-642-29075-6
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)