Dimensional explanation of capacity and ambiguity in information theory

  • Tatsuya Suzuki
  • Fumio Ohi
  • Masasi Kowada


In this paper we discuss a dimensional explanation of the channel capacity and the ambiguity in information theory, which characterize the channel through which infinite sequences whose each coordinate takes values in alphabet are transmitted. Infinite sequences are put into the channel, transmitted and received as output sequences. Since the channel has memory and noise, the output sequences are not necessarily the same to the input sequences. Output sequences are ambiguous.

In this paper for the first time we discuss relationships between an ergodic channel capacity and the Hausdorff dimension of the set of all the output sequences for a noisy channel with finite memory and show that the former is less than or equal to the latter. For each output sequence the ambiguity set is defined to be the set of all the input sequences transmitted to the output sequence. The ambiguity is defined to be a quantity characterizing the complexity of an ambiguity set. Hausdorff dimension is also another quantity characterizing the complexity of the ambiguity set. In this paper we also consider relationships between an ambiguity and Hausdorff dimension of an ambiguity set for a noiseless channel with memory 2, and show that the Hausdorff dimension equals to the ambiguity.

Key words

noisy and noiseless channel essentially noisy channel channel capacity ambiguity set ambiguity 


  1. [1]
    P. Billingsley, Ergodic Theory and Information. Wiley, New York, 1965.MATHGoogle Scholar
  2. [2]
    W.M. Conner, The capacity and ambiguity of a transducer. Ann. Math. Statis.,41, No. 6 (1970), 2093–2104.MATHCrossRefMathSciNetGoogle Scholar
  3. [3]
    H.G. Eggleston, The fractional dimension of a set defined by decimal properties. Quart. J. Math. Oxford Ser.(2),20 (1949), 31–36.CrossRefMathSciNetGoogle Scholar
  4. [4]
    B.P. Yakov, Dimension Theory in Dynamical Systems. University of Chicago Press, 1997.Google Scholar
  5. [5]
    A.S. Besicovitch, On the sum of digits of numbers represented in the dyadic system. Math. Ann.,110 (1935), 321–330.CrossRefMathSciNetGoogle Scholar
  6. [6]
    H.G. Eggleston, Set of fractal dimention which occur in some problems of number theory. Proc. London Math. Soc.,54 (1952), 42–93.CrossRefMathSciNetGoogle Scholar
  7. [7]
    J.R. Kinney, Singular functions associated with Markov chains. Proc. Amer. Math. Soc.,9 (1958), 603–608.MATHCrossRefMathSciNetGoogle Scholar
  8. [8]
    H. Dym, On a class of monotone functions generated by ergodic sequences. Amer. Math. Monthly,75 (1968), 594–601.MATHCrossRefMathSciNetGoogle Scholar
  9. [9]
    W. Feller, An Introduction to Probability Theory and Its Applications (Vol. II). John Wiley, 1971.Google Scholar
  10. [10]
    K.J. Falconer, The Geometry of Fractal Sets. Cambring University Press, 1985.Google Scholar
  11. [11]
    L. Breiman, On achieving channel capacity in finite-memory channels. Illinois J. Math.,4 (1960), 246–252.MATHMathSciNetGoogle Scholar

Copyright information

© JJIAM Publishing Committee 2002

Authors and Affiliations

  1. 1.Department of Systems EngineeringNagoya Institute of TechnologyNagoyaJapan
  2. 2.Department of Systems EngineeringNagoya Institute of TechnologyNagoyaJapan
  3. 3.School of Culture-Information StudiesSugiyama Jogakuen UniversityNagoyaJapan

Personalised recommendations