Skip to main content
SpringerLink
Log in

Practical Implementations of Arithmetic Coding

  • Chapter
Image and Text Compression

Part of the book series: The Kluwer International Series in Engineering and Computer Science ((SECS,volume 176))

Abstract

We provide a tutorial on arithmetic coding, showing how it provides nearly optimal data compression and how it can be matched with almost any probabilistic model. We indicate the main disadvantage of arithmetic coding, its slowness, and give the basis of a fast, space-efficient, approximate arithmetic coder with only minimal loss of compression efficiency. Our coder is based on the replacement of arithmetic by table lookups coupled with a new deterministic, probability estimation scheme.

1

Support was provided in part by NASA Graduate Student Researchers Program grant NGT-50420 and by a National Science Foundation Presidential YOWlg Investigators Award grant with matching funds from IBM. Additional support was provided by a Universities Space Research Association/CESDIS associate membership.

2

Support was provided in part by National Science Foundation Presidential Young Investigator Award CCR-9047466 with matching funds from IBM, by NSF research grant CCR9007851, by Army Research Office grant DAAL03-91-G-0035, and by the Office of Naval Research and the Defense Advanced Research Projects Agency under contract N00014-91J- 4052, ARPA Order No. 8225.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. N. Abramson, Information Theory and Coding, McGraw-Hill, New York, NY, 1963.

    Google Scholar 

  2. R. B. Arps, T. K. Truong, D. J. Lu, R. C. Pasco, and T. D. Friedman, “A Multi-Purpose VLSI Chip for Adaptive Data Compression of Bilevel Images,” IBM J. Res. Develop. 32 (Nov. 1988), 775–795.

    Article  Google Scholar 

  3. T. Bell, “A Unifying Theory and Improvements for Existing Approaches to Text Compression,” Univ. of Canterbury, Ph.D. Thesis, 1986.

    Google Scholar 

  4. T. Bell and A. M. Moffat, “A Note on the DMC Data Compression Scheme,” Computer Journal 32 (1989), 16–20.

    Article  MathSciNet  MATH  Google Scholar 

  5. T. C. Bell, J. G. Cleary, and I. H. Witten, Text Compression, Prentice-Hall, Englewood Cliffs, NJ, 1990.

    Google Scholar 

  6. T. C. Bell, I. H. Witten, and J. G. Cleary, “Modeling for Text Compression,” Comput. Surveys 21 (Dec. 1989), 557–591.

    Article  Google Scholar 

  7. J. L. Bentley, D. D. Sleator, R. E. Tarjan, and V. K. Wei, “A Locally Adaptive Data Compression Scheme,” Comm. ACM 29 (Apr. 1986), 320–330.

    Article  MathSciNet  MATH  Google Scholar 

  8. A. C. Blumer and R. J. McEliece, “The Renyi Redundancy of Generalized Huffman Codes,” IEEE Trans. Inform. Theory IT-34 (Sept. 1988), 1242–1249.

    Article  MathSciNet  Google Scholar 

  9. R. M. Capocelli, R. Giancarlo, and I. J. Taneja, “Bounds on the Redundancy of Huffman Codes,” IEEE Trans. Inform. Theory IT-32 (Nov. 1986), 854–857.

    Article  MathSciNet  Google Scholar 

  10. D. Chevion, E. D. Karnin, and E. Walach, “High Efficiency, Multiplication Free Approximation of Arithmetic Coding,” in Proc. Data Compression Conference, J. A. Storer and J. H. Reif, eds., Snowbird, Utah, Apr. 8-11, 1991, 43–52.

    Google Scholar 

  11. G. V. Cormack and R. N. Horspool, “Data Compression Using Dynamic Markov Modelling,” Computer Journal 30 (Dec. 1987), 541–550.

    Article  MathSciNet  Google Scholar 

  12. G. V. Cormack and R. N. Horspool, “Algorithms for Adaptive Huffman Codes,” Inform. Process. Lett. 18 (Mar. 1984), 159–165.

    Article  MathSciNet  Google Scholar 

  13. P. Elias, “Interval and Recency Rank Source Coding: Two On-line Adaptive Variable Length Schemes,” IEEE Trans. Inform. Theory IT-33 (Jan. 1987), 3–10.

    Article  MathSciNet  Google Scholar 

  14. P. Elias, “Universal Codeword Sets and Representations of Integers,” IEEE Trans. Inform. Theory IT-21 (Mar. 1975), 194–203.

    Article  MathSciNet  Google Scholar 

  15. N. Faller, “An Adaptive System for Data Compression,” Record of the 7th Asilomar Conference on Circuits, Systems, and Computers, 1973.

    Google Scholar 

  16. Ph. Flajolet, “Approximate Counting: a Detailed Analysis,” BIT 25 (1985), 113.

    Article  MathSciNet  MATH  Google Scholar 

  17. Ph. Flajolet and G. N. N. Martin, “Probabilistic Counting Algorithms for Data Base Applications,” INRIA, Rapport de Recherche No. 313, June 1984.

    Google Scholar 

  18. R. G. Gallager, “Variations on a Theme by Huffman,” IEEE Trans. Inform. Theory IT-24 (Nov. 1978), 668–674.

    Article  MathSciNet  Google Scholar 

  19. M. Guazzo, “A General Minimum-Redundancy Source-Coding Algorithm,” IEEE Trans. Inform. Theory IT-26 (Jan. 1980), 15–25.

    Article  MathSciNet  Google Scholar 

  20. M. E. Hellman, “Joint Source and Channel Encoding,” Proc. Seventh Hawaii International Conf. System Sci., 1974.

    Google Scholar 

  21. R. N. Horspool, “Improving LZW,” in Proc. Data Compression Conference, J. A. Storer and J. H. Reif, eds., Snowbird, Utah, Apr. 8-11, 1991, 332–341.

    Google Scholar 

  22. P. G. Howard and J. S. Vitter, “Design and Analysis of Quasi-Arithmetic Coding,” Brown University, Department of Computer Science, Technical Report, 1992.

    Google Scholar 

  23. P. G. Howard and J. S. Vitter, “Analysis of Arithmetic Coding for Data Compression,” in Proc. Data Compression Conference, J. A. Storer and J. H. Reif, eds., Snowbird, Utah, Apr. 8-11, 1991, 3-12, invited paper, also to appear as an invited paper in the special issue of Information Processing and Management entitled “Data Compression for Images and Texts.”

    Google Scholar 

  24. P. G. Howard and J. S. Vitter, “New Methods for Lossless Image Compression Using Arithmetic Coding,” in Proc. Data Compression Conference, J. A. Storer and J. H. Reif, eds., Snowbird, Utah, Apr. 8-11, 1991, 257–266, also to appear as an invited paper in the special issue of Information Processing and Management entitled “Data Compression for Images and Texts.”.

    Chapter  Google Scholar 

  25. D. A. Huffman, “A Method for the Construction of Minimum Redundancy Codes,” Proceedings of the Institute of Radio Engineers 40 (1952), 1098–1101.

    Google Scholar 

  26. D. E. Knuth, “Dynamic Huffman Coding,” J. Algorithms 6 (June 1985), 163–180.

    Google Scholar 

  27. G. G. Langdon, “Probabilistic and Q-Coder Algorithms for Binary Source Adaptation,” in Proc. Data Compression Conference, J. A. Storer and J. H. Reif, eds., Snowbird, Utah, Apr. 8-11, 1991, 13–22.

    Google Scholar 

  28. G.G. Langdon, “A Note on the Ziv-LempeL Model for Compressing Individual Sequences,” IEEE Trans. Inform. Theory IT-29 (Mar. 1983), 284–287.

    Article  Google Scholar 

  29. G. G. Langdon and J. Rissanen, “Compression of Black-White Images with Arithmetic Coding,” IEEE Trans. Comm. COM-29 (1981), 858–867.

    Article  Google Scholar 

  30. F. T. Leighton and R. L. Rivest, “Estimating a Probability Using Finite Memory,” IEEE Trans. Inform. Theory IT-32 (Nov. 1986), 733–742.

    Article  Google Scholar 

  31. D. A. Lelewer and D. S. Hirschberg, “Streamlining Context Models for Data Compression,” in Proc. Data Compression Conference, J. A. Storer and J. H. Reif, eds., Snowbird, Utah, Apr. 8-11, 1991, 313–322.

    Google Scholar 

  32. V. S. Miller and M. N. Wegman, “Variations on a Theme by Ziv and Lempel,” in Combinatorial Algorithms on Words, A. Apostolico and Z. Galil, eds., NATO ASi Series #F12, Springer-Verlag, Berlin, 1984, 131–140.

    Google Scholar 

  33. J. L. Mitchell and W. B. Pennebaker, “Optimal Hardware and Software Arithmetic. Coding Procedures for the Q-Coder,” IBM J. Res. Develop. 32 (Nov. 1988), 727–736.

    Article  Google Scholar 

  34. A. M. Moffat, “Predictive Text Compression Based upon the Future Rather than the Past,” Australian Computer Science Communications 9 (1987), 254–261.

    Google Scholar 

  35. A. M. Moffat, “Word-Based Text Compression,” Software-Practice and Experience 19 (Feb. 1989), 185–198.

    Article  Google Scholar 

  36. A. M. Moffat, “Linear Time Adaptive Arithmetic Coding,” IEEE Trans. Inform. Theory IT-36 (Mar. 1990), 401–406.

    Article  MathSciNet  Google Scholar 

  37. A. M. Moffat, “Implementing the PPM Data Compression Scheme,” IEEE Trans. Comm. COM-38 (Nov. 1990), 1917–1921.

    Article  Google Scholar 

  38. K. Mohiuddin, J. J. Rissanen, and M. Wax, “Adaptive Model for Non-stationary Sources,” IBM Technical Disclosure Bulletin 28 (Apr. 1986), 4798–4800.

    Google Scholar 

  39. D. S. Parker, “Conditions for the Optimality of the Huffman Algorithm,” SIAM J. Comput. 9 (Aug. 1980), 470–489.

    Article  MathSciNet  Google Scholar 

  40. R. Pasco, “Source Coding Algorithms for Fast Data Compression,” Stanford Univ., Ph.D. Thesis, 1976.

    Google Scholar 

  41. W. B. Pennebaker and J. L. Mitchell, “Probability Estimation for the Q-Coder,” IBM J. Res. Develop. 32 (Nov. 1988), 737–752.

    Article  Google Scholar 

  42. W. B. Pennebaker and J. L. Mitchell, “Software Implementations of the Q-Coder,” IBM J. Res. Develop. 32 (Nov. 1988), 753–774.

    Article  Google Scholar 

  43. W. B. Pennebaker, J. L. Mitchell, G. G. Langdon, and R. B. Arps, “An Overview of the Basic Principles of the Q-Coder Adaptive Binary Arithmetic Coder,” IBM J. Res. Develop. 32 (Nov. 1988), 717–726.

    Article  Google Scholar 

  44. J. Rissanen, “Modeling by Shortest Data Description,” Automatica 14 (1978), 465–571.

    Article  MATH  Google Scholar 

  45. J. Rissanen, “A Universal Prior for Integers and Estimation by Minimum Description Length,” Ann. Statist. 11 (1983), 416–432.

    Article  MathSciNet  MATH  Google Scholar 

  46. J. Rissanen, “Universal Coding, Information, Prediction, and Estimation,” IEEE Trans. Inform. Theory IT-30 (July 1984), 629–636.

    Article  MathSciNet  Google Scholar 

  47. J. Rissanen and G. G. Langdon, “Universal Modeling and Coding,” IEEE Trans. Inform. Theory IT-27 (Jan. 1981), 12–23.

    Article  MathSciNet  Google Scholar 

  48. J. J. Rissanen, “Generalized Kraft Inequality and Arithmetic Coding,” IBM J. Res. Develop. 20 (May 1976), 198–203.

    Article  MathSciNet  MATH  Google Scholar 

  49. J. J. Rissanen and G. G. Langdon, “Arithmetic Coding,” IBM J. Res. Develop. 23 (Mar. 1979), 146–162.

    Article  MathSciNet  Google Scholar 

  50. J. J. Rissanen and K. M. Mohiuddin, “A Multiplication-Free Multialpha-bet Arithmetic Code,” IEEE Trans. Comm. 37 (Feb. 1989), 93–98.

    Article  Google Scholar 

  51. C. Rogers and C. D. Thomborson, “Enhancements to Ziv-Lempel Data Compression,” Dept. of Computer Science, Univ. of Minnesota, Technical Report TR 89-2, Duluth, Minnesota, Jan. 1989.

    Google Scholar 

  52. F. Rubin, “Arithmetic Stream Coding Using Fixed Precision Registers,” IEEE Trans. Inform. Theory IT-25 (Nov. 1979), 672–675.

    Article  Google Scholar 

  53. B. Y. Ryabko, “Data Compression by Means of a Book Stack,” Problemy Peredachi Informatsii 16 (1980).

    Google Scholar 

  54. C. E. Shannon, “A Mathematical Theory of Communication,” Bell Syst. Tech. J. 27 (July 1948), 398–403.

    MathSciNet  Google Scholar 

  55. J. S. Vitter, “Dynamic Huffman Coding,” ACM Trans. Math. Software 15 (June 1989), 158–167, also appears as Algorithm 673, Collected Algorithms of ACM, 1989.

    Article  MATH  Google Scholar 

  56. J. S. Vitter, “Design and Analysis of Dynamic Huffman Codes,” Journal of the ACM 34 (Oct. 1987), 825–845.

    Article  MathSciNet  MATH  Google Scholar 

  57. I. H. Witten and T. C. Bell, “The Zero Frequency Problem: Estimating the Probabilities of Novel Events in Adaptive Text Compression,” IEEE Trans. Inform. Theory IT-37 (July 1991), 1085–1094.

    Article  Google Scholar 

  58. I. H. Witten, R. M. Neal, and J. G. Cleary, “Arithmetic Coding for Data Compression,” Comm. ACM 30 (June 1987), 520–540.

    Article  Google Scholar 

  59. J. Ziv and A. Lempel, “A Universal Algorithm for Sequential Data Compression,” IEEE Trans. Inform. Theory IT-23 (May 1977), 337–343.

    Article  MathSciNet  Google Scholar 

  60. J. Ziv and A. Lempel, “Compression of Individual Sequences via Variable Rate Coding,” IEEE Trans. Inform. Theory IT-24 (Sept. 1978), 530–536.

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Brown University, Providence, R. I., 02912-1910, USA

    Paul G. Howard & Jeffrey Scott Vitter

Authors
  1. Paul G. Howard
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeffrey Scott Vitter
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Brandeis University, USA

    James A. Storer

Rights and permissions

Reprints and permissions

Copyright information

© 1992 Springer Science+Business Media New York

About this chapter

Cite this chapter

Howard, P.G., Vitter, J.S. (1992). Practical Implementations of Arithmetic Coding. In: Storer, J.A. (eds) Image and Text Compression. The Kluwer International Series in Engineering and Computer Science, vol 176. Springer, Boston, MA. https://doi.org/10.1007/978-1-4615-3596-6_4

Download citation

  • DOI: https://doi.org/10.1007/978-1-4615-3596-6_4

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4613-6598-3

  • Online ISBN: 978-1-4615-3596-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation