On the General Definition of the Quantity of Information

Abstract

The definitions (2), (4) and the properties I–IV of the expression I(ξ,η) presented below were given in A. N. Kolmogorov’s report at the meeting on probability theory and mathematical statistics (Leningrad, June 1954). Theorem 4 and Theorem 5 on the semicontinuity of I (ξ,η) under weak convergence of distributions were found by I. M. Gelfand and A. M. Yaglom. After that, A. N. Kolmogorov proposed the final version of this article, which stresses that basically the passage from the finite case to the general one, and the computation and estimates of the amount of information obtained by taking limits are absolutely trivial if the exposition is carried out in terms of normed Boolean algebras. It would not be difficult, of course, to reformulate more general principles of the passage to the limit: not as n → ∞, but by using some partial ordering.

Doklady Akad. Nauk SSSR, 1956, vol. 111, no. 4, pp. 745–748.