Background on Information Theory and Coding Theory

Part of the Applied and Numerical Harmonic Analysis book series (ANHA)


This chapter summarizing background information assumes that the reader has some familiarity with linear algebra and basic probability. The basic model of information theory and error-correcting block codes is introduced. The basic example of the Hamming [7,4,3] code is presented in detail.

What is ironic is that even in basic background issues, coding theory has interesting open questions. For example, for a given length and dimension, which code is the best 2-error-correcting code? Another example: see Manin’s theorem 19 and the closely related Conjecture 22 below.


Linear Code Cyclic Code Asymptotic Bound Cyclic Shift Quadratic Residue 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. [Af]
    Aftab, Cheung, Kim, Thakkar, Yeddanapudi: Information theory. Student term project in a course at MIT Preprint (2001). Available:
  2. [Ash]
    Ash, R.: Information Theory. Dover, New York (1965) MATHGoogle Scholar
  3. [Bal]
    Ball, S.: On large subsets of a nite vector space in which every subset of basis size is a basis. Preprint, Dec. (2010).
  4. [Ba]
    A. Barg’s Coding Theory webpage
  5. [BGT]
    Berrou, C., Glavieux, A., Thitimajshima, P.: Near Shannon limit error-correcting coding and decoding: Turbo-codes. Communications (1993). ICC 93. Geneva. Technical Program, Conference Record, IEEE International Conference on, vol. 2, pp. 1064–1070 (1993). Available:
  6. [Bi]
    Bierbrauer, J.: Introduction to Coding Theory. Chapman & Hall/CRC, New York (2005) MATHGoogle Scholar
  7. [CSi]
    Conway, F., Siegelman, J.: Dark Hero of the Information Age. MIT Press, New York (2005) MATHGoogle Scholar
  8. [dLG]
    de Launey, W., Gordon, D.: A remark on Plotkin’s bound. IEEE Trans. Inf. Theory 47, 352–355 (2001). Available: MATHCrossRefGoogle Scholar
  9. [GZ]
    Gaborit, P., Zemor, G.: Asymptotic improvement of the Gilbert–Varshamov bound for linear codes. IEEE Trans. Inf. Theory IT-54(9), 3865–3872 (2008). Available: MathSciNetCrossRefGoogle Scholar
  10. [Hil]
    Hill, R.: A First Course in Coding Theory. Oxford Univ Press, Oxford (1986) MATHGoogle Scholar
  11. [Hi]
    Hirschfeld, J.W.P.: The main conjecture for MDS codes. In: Cryptography and Coding. Lecture Notes in Computer Science, vol. 1025. Springer, Berlin (1995). Avaialble from: Google Scholar
  12. [HP1]
    Huffman, W.C., Pless, V.: Fundamentals of Error-Correcting Codes. Cambridge Univ. Press, Cambridge (2003) MATHGoogle Scholar
  13. [JV]
    Jiang, T., Vardy, A.: Asymptotic improvement of the Gilbert–Varshamov bound on the size of binary codes. IEEE Trans. Inf. Theory 50, 1655–1664 (2004) MathSciNetCrossRefGoogle Scholar
  14. [JKTu]
    Joyner, D., Kreminski, R., Turisco, J.: Applied Abstract Algebra. Johns Hopkins Univ. Press, Baltimore (2004) MATHGoogle Scholar
  15. [MS]
    MacWilliams, F., Sloane, N.: The Theory of Error-Correcting Codes. North-Holland, Amsterdam (1977) MATHGoogle Scholar
  16. [MN]
    MacKay, D.J.C., Radford, M.N.: Near Shannon limit performance of low density parity check codes. Electronics Letters, July (1996). Available:
  17. [Ni]
    Niven, I.: Coding theory applied to a problem of Ulam. Math. Mag. 61, 275–281 (1988) MathSciNetMATHCrossRefGoogle Scholar
  18. [S]
    The SAGE Group: SAGE: Mathematical software, Version 4.6.
  19. [SS]
    Shokranian, S., Shokrollahi, M.A.: Coding Theory and Bilinear Complexity. Scientific Series of the International Bureau, vol. 21. KFA Jülich (1994) Google Scholar
  20. [Tho]
    Thompson, T.: From Error-Correcting Codes Through Sphere Packings to Simple Groups. Cambridge Univ. Press, Cambridge (2004) Google Scholar
  21. [vL1]
    van Lint, J.: Introduction to Coding Theory, 3rd edn. Springer, Berlin (1999). MATHGoogle Scholar
  22. [VVS]
    Viterbi, A.J., Viterbi, A.M., Sindhushayana, N.T.: Interleaved concatenated codes: New perspectives on approaching the Shannon limit. Proc. Natl. Acad. Sci. USA 94, 9525–9531 (1997) MathSciNetMATHCrossRefGoogle Scholar
  23. [V3]
    Voloch, F.: Computing the minimum distance of cyclic codes. Preprint. Available:
  24. [War]
    Ward, H.N.: Quadratic residue codes and symplectic groups. J. Algebra 29, 150–171 (1974) MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Mathematics DepartmentUS Naval AcademyAnnapolisUSA
  2. 2.Department of MathematicsUniversity of LouisvilleLouisvilleUSA

Personalised recommendations