Compression and entropy

  • Georges HanselEmail author
  • Dominique PerrinEmail author
  • Imre SimonEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 577)


The connection between text compression and the measure of entropy of a source seems to be well known but poorly documented. We try to partially remedy this situation by showing that the topological entropy is a lower bound for the compression ratio of any compressor. We show that for factorial sources the 1978 version of the Ziv-Lempel compression algorithm achieves this lower bound.


Compression Rate Topological Entropy Invariant Probability Measure Huffman Code Infinite Subset 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    R. Adler, A. Konheim, and M. McAndrew, Topological entropy. Trans. Amer. Math. Soc, 114:309–319, 1965.Google Scholar
  2. [2]
    T. C. Bell, J. G. Cleary, and I. H. Witten. Text Compression. Prentice Hall, Englewood Cliffs, N.J., 1990.Google Scholar
  3. [3]
    N. Chomsky and G. A. Miller. Finite state languages. Information and Control 1:91–112, 1958.Google Scholar
  4. [4]
    A. de Luca. On the entropy of a formal language. In H. Brakhage, editor, Automata Theory and Formal Languages, pages 103–109, Springer-Verlag, Berlin, 1975. Lecture Notes in Computer Science, 33.Google Scholar
  5. [5]
    A. V. Goldberg and M. Sipser. Compression and ranking. SIAM J. Comput., 20:524–536, 1991.Google Scholar
  6. [6]
    G. Hansel. Estimation of the entropy by the Lempel-Ziv method. In M. Gross and D. Perrin, editors, Electronic Dictionaries and Automata in Computational Linguistics, pages 51–65, Springer-Verlag, Berlin, 1989. Lecture Notes in Computer Science, 377.Google Scholar
  7. [7]
    A. I. Khinchin. Mathematical Foundations of Information Theory. Dover, New York, 1957.Google Scholar
  8. [8]
    D. E. Knuth. The Art of Computer Programming, Vol. 1, Fundamental Algorithms. Addison-Wesley Pu. Co., Reading, MA, 1968.Google Scholar
  9. [9]
    W. Kuich. On the entropy of context-free languages. Information and Control, 16:173–200, 1970.Google Scholar
  10. [10]
    M. Lothaire. Combinatorics on Words. Volume 17 of Encyclopedia of Mathematics and its Applications, Addison-Wesley Pu. Co., Reading, MA, 1983.Google Scholar
  11. [11]
    N. F. G. Martin and J. W. England. Mathematical Theory of Entropy. Volume 12 of Encyclopedia of Mathematics and its Applications, Addison-Wesley Pu. Co., Reading, MA, 1981.Google Scholar
  12. [12]
    S. E. Newhouse. Entropy in smooth dynamical systems. 1990. To appear in Proceedings of the 1990 World Congress of Mathematicians.Google Scholar
  13. [13]
    D. Ornstein and B. Weiss. Entropy and data compression schemes. 1990. manuscript.Google Scholar
  14. [14]
    D. S. Ornstein and P. C. Shields. Universal almost sure data compression. The Annals of Probability, 18:441–452, 1990.Google Scholar
  15. [15]
    A. Rényi. A Diary on Information Theory. John Wiley & Sons, New York, NY, 1984.Google Scholar
  16. [16]
    C. E. Shannon. A mathematical theory of communication. Bell System Technical J., 27:398–403, 1948.Google Scholar
  17. [17]
    J. A. Storer. Data Compression—Methods and Theory. Computer Science Press, Rockville,MD, 1988.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1992

Authors and Affiliations

  1. 1.Université de Rouen, MathématiquesMont Saint-AignanFrance
  2. 2.Université de Paris VII, LITPParis Cedex 05France
  3. 3.Universidade de SÃo Paulo, IMESÃo Paulo, SPBrasil

Personalised recommendations