Ergodic Theory and Encoding of Individual Sequences

  • Rudolf AhlswedeEmail author
Part of the Foundations in Signal Processing, Communications and Networking book series (SIGNAL, volume 15)


In a famous series of papers Ziv and Lempel [1, 2, 3] studied the encoding of so-called individual sequences. In [3] Ziv gave a definition for a kind of entropy of any infinite sequence of letters drawn from a finite alphabet. The essential parameters in this definition are the numbers of different n-words occurring in the given infinite sequence. In particular, Ziv makes no use of any notions concerning probabilities. In [4] Dueck and Wolters started a second way which leads also to a notion of entropy of an individual sequence. They strongly used definitions and theorem from ergodic theory and connected thereby properties of individual sequences with the results of [5, 6]. In particular, they represented the behavior of block occurrences in a sequence \(\mathbf {u}\) by a set \(V_T(\mathbf {u})\) of shift-invariant measures . The entropy of \(\mathbf {u}\) is defined in terms of measure-theoretical entropies of the measures contained in \(V_T(\mathbf {u})\).


  1. 1.
    A. Lempel, J. Ziv, On the complexity of an individual sequence. IEEE Trans. Inf. Theory IT–22, 75–81 (1976)Google Scholar
  2. 2.
    J. Ziv, A. Lempel, A universal algorithm for sequential data compression. IEEE Trans. Inf. Theory IT-23, 337–343 (1977)MathSciNetCrossRefGoogle Scholar
  3. 3.
    J. Ziv, Coding theorems for individual sequences. IEEE Trans. Inf. Theory IT-24, 405–412 (1978)MathSciNetCrossRefGoogle Scholar
  4. 4.
    G. Dueck, L. Wolters, Ergodic theory and encoding of individual sequences. Probl. Control. Inf. Theory 14(5), 329–345 (1985)MathSciNetzbMATHGoogle Scholar
  5. 5.
    R.M. Gray, L.D. Davisson, Source coding theorems without the ergodic assumption. IEEE Trans. Inf. Theory IT-20, 502–516 (1974)MathSciNetCrossRefGoogle Scholar
  6. 6.
    R.M. Gray, L.D. Davisson, The ergodic decomposition of stationary discrete random processes. IEEE Trans. Inf. Theory IT-20, 625–636 (1974)MathSciNetCrossRefGoogle Scholar
  7. 7.
    M. Denker, C. Grillenberger, K. Sigmund, Ergodic Theory on Compact Spaces, Lecture notes in mathematics (Springer, Berlin, 1976)CrossRefGoogle Scholar
  8. 8.
    P. Billingsley, Ergodic Theory and Information (Wiley, New York, 1965)zbMATHGoogle Scholar
  9. 9.
    R. Ahlswede, Series: foundations in signal processing, communications and networking, in Storing and Transmitting Data, Rudolf Ahlswede’s Lectures on Information Theory 1, ed. by A. Ahlswede, I. Althöfer, C. Deppe, U. Tamm, vol. 10 (Springer, Berlin, 2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.BielefeldGermany

Personalised recommendations