# Dimensional explanation of capacity and ambiguity in information theory

• Tatsuya Suzuki
• Fumio Ohi
Article

## Abstract

In this paper we discuss a dimensional explanation of the channel capacity and the ambiguity in information theory, which characterize the channel through which infinite sequences whose each coordinate takes values in alphabet are transmitted. Infinite sequences are put into the channel, transmitted and received as output sequences. Since the channel has memory and noise, the output sequences are not necessarily the same to the input sequences. Output sequences are ambiguous.

In this paper for the first time we discuss relationships between an ergodic channel capacity and the Hausdorff dimension of the set of all the output sequences for a noisy channel with finite memory and show that the former is less than or equal to the latter. For each output sequence the ambiguity set is defined to be the set of all the input sequences transmitted to the output sequence. The ambiguity is defined to be a quantity characterizing the complexity of an ambiguity set. Hausdorff dimension is also another quantity characterizing the complexity of the ambiguity set. In this paper we also consider relationships between an ambiguity and Hausdorff dimension of an ambiguity set for a noiseless channel with memory 2, and show that the Hausdorff dimension equals to the ambiguity.

## Key words

noisy and noiseless channel essentially noisy channel channel capacity ambiguity set ambiguity

## References

1. [1]
P. Billingsley, Ergodic Theory and Information. Wiley, New York, 1965.
2. [2]
W.M. Conner, The capacity and ambiguity of a transducer. Ann. Math. Statis.,41, No. 6 (1970), 2093–2104.
3. [3]
H.G. Eggleston, The fractional dimension of a set defined by decimal properties. Quart. J. Math. Oxford Ser.(2),20 (1949), 31–36.
4. [4]
B.P. Yakov, Dimension Theory in Dynamical Systems. University of Chicago Press, 1997.Google Scholar
5. [5]
A.S. Besicovitch, On the sum of digits of numbers represented in the dyadic system. Math. Ann.,110 (1935), 321–330.
6. [6]
H.G. Eggleston, Set of fractal dimention which occur in some problems of number theory. Proc. London Math. Soc.,54 (1952), 42–93.
7. [7]
J.R. Kinney, Singular functions associated with Markov chains. Proc. Amer. Math. Soc.,9 (1958), 603–608.
8. [8]
H. Dym, On a class of monotone functions generated by ergodic sequences. Amer. Math. Monthly,75 (1968), 594–601.
9. [9]
W. Feller, An Introduction to Probability Theory and Its Applications (Vol. II). John Wiley, 1971.Google Scholar
10. [10]
K.J. Falconer, The Geometry of Fractal Sets. Cambring University Press, 1985.Google Scholar
11. [11]
L. Breiman, On achieving channel capacity in finite-memory channels. Illinois J. Math.,4 (1960), 246–252.