Dimensional explanation of capacity and ambiguity in information theory
- 20 Downloads
In this paper we discuss a dimensional explanation of the channel capacity and the ambiguity in information theory, which characterize the channel through which infinite sequences whose each coordinate takes values in alphabet are transmitted. Infinite sequences are put into the channel, transmitted and received as output sequences. Since the channel has memory and noise, the output sequences are not necessarily the same to the input sequences. Output sequences are ambiguous.
In this paper for the first time we discuss relationships between an ergodic channel capacity and the Hausdorff dimension of the set of all the output sequences for a noisy channel with finite memory and show that the former is less than or equal to the latter. For each output sequence the ambiguity set is defined to be the set of all the input sequences transmitted to the output sequence. The ambiguity is defined to be a quantity characterizing the complexity of an ambiguity set. Hausdorff dimension is also another quantity characterizing the complexity of the ambiguity set. In this paper we also consider relationships between an ambiguity and Hausdorff dimension of an ambiguity set for a noiseless channel with memory 2, and show that the Hausdorff dimension equals to the ambiguity.
Key wordsnoisy and noiseless channel essentially noisy channel channel capacity ambiguity set ambiguity
- B.P. Yakov, Dimension Theory in Dynamical Systems. University of Chicago Press, 1997.Google Scholar
- W. Feller, An Introduction to Probability Theory and Its Applications (Vol. II). John Wiley, 1971.Google Scholar
- K.J. Falconer, The Geometry of Fractal Sets. Cambring University Press, 1985.Google Scholar