The Complexity of Decoders

  • John E. Savage
Part of the International Centre for Mechanical Sciences book series (CISM, volume 216)


The twenty-fifth anniversary of birth of Information Theory, as marked by the publication of Shannon’s seminal 1948 paper [Shannon (1948)], has just recently been celebrated. These have been a most productive and a most stimulating twenty-five years and to a large extent, the goal set out by information theorists has been achieved. That goal is to find and describe the means to obtain reliable communication on noisy channels at tolerably low reduction in data rates. Algebraic and non-algebraic, block and convolutional codes have been discovered along with algebraic and non-algebraic decoding procedures which do permit reliable communication on many types of noisy channels, radio, wire and space.


Boolean Function Turing Machine Code Rate Block Length Code Word 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Berlekamp, E.R., (1968) Algebraic Coding, McGraw-Hill, New York.MATHGoogle Scholar
  2. [2]
    Bluestein, G. and K.L. Jordan, (1963) Investigation of the Fano Sequential Decoding Algorithm by Computer Simulation, MIT Lincoln Laboratory, Group Report 62G - 5.Google Scholar
  3. [3]
    Chaitin, G.J., (1966) On the Length of Programs for Computing Finite Binary Sequences, Journal of the ACM, vol. 13, pp. 547–569.CrossRefMATHMathSciNetGoogle Scholar
  4. [4]
    Chaitin, G.L., (1970) On the Difficulty of Computations, IEEE Transactions on Information Theory, vol. IT-16, No. 1, pp. 5–9.CrossRefGoogle Scholar
  5. [5]
    Fano, R.M., (1963) A Heuristic Discussion of Probabilistic Decoding, IEEE Transactions on Information Theory, vol. IT-9, pp. 64–74.Google Scholar
  6. [6]
    Forney, G.D., (1967) Concatenated Codes, MIT Press, Cambridge.Google Scholar
  7. [7]
    Gallager, R.G., (1968) Information Theory and Reliable Communication, Wiley and Sons, New York.MATHGoogle Scholar
  8. [8]
    Harper, L.H. and J.E. Savage, (1972) On the Complexity of the Marriage Problem, Advances in Mathematics, Vol. 9, No. 3, pp. 249–312.CrossRefMathSciNetGoogle Scholar
  9. [9]
    Harper, L.H. and J.E. Savage, (1973) Complexity Made Simple, Proceedings of the International Symposium on Combinational Theory, Rome, September 2–15.Google Scholar
  10. [10]
    Harper, L.H., W.N. Hsieh and J.E. Savage, (1975) to appear in Journal of Theoretical Computer Science.Google Scholar
  11. [11]
    Hoperoft, J.E. and J.D. Ullman, (1969) Formal Languages and Their Relation to Automata, Addison-Wesley, Reading, Mass.Google Scholar
  12. [12]
    Jacobs, I.M. and E.R. Berlekamp, (1967) A lower Bound to the Distribution of Computation for Sequential Decoding, IEEE Transactions on Information Theory, Vol. IT-13, pp. 167–174.Google Scholar
  13. [13]
    Justesen, J. (1972) A class of Constructive Asymptotically Good Algebraic Codes, IEEE Transactions on Information Theory, Vol. IT-18, pp. 652–656.Google Scholar
  14. [14]
    Kolmogorov, A.N. (1968) Logical Basis for Information Theory and Probability Theory, IEEE Transactions on Information Theory, Vol. IT-14, pp. 662–664.Google Scholar
  15. [15]
    Muller, D.E. and F.P. Preparata (1973) Minimum Delay Networks for Sorting and Switching, Proceedings 7-th Princeton Conference on Information Science and Systems, pp. 138–139.Google Scholar
  16. [16]
    Reed, I.S. and G. Solomon, (1960) Polynomial Codes over Certain Finite Fields, Journal of the Society for Industrial and Applied Mathematics, Vol. 8, pp. 300–304.CrossRefMATHMathSciNetGoogle Scholar
  17. [17]
    Savage, J.E., (1966A) Sequential Decoding — The Computation Problem, Bell System Technical Journal Vol. XLV, pp. 149–175.Google Scholar
  18. [18]
    Savage, J.E., (1966B) The Distribution of the Sequential Decoding Computation Time, IEEE Transactions on Information Theory Vol. IT-12, pp. 143–147.Google Scholar
  19. [19]
    Savage, J.E., (1968) Progress in Sequential Decoding, Advances in Communication Systems, Academic Press, pp. 149–204, edited by A.V. Balakrishman.Google Scholar
  20. [20]
    Savage, J.E., (1969) The Complexity of Decoders — Part I: Classes of Decoding Rules, IEEE Transactions on Information Theory, Vol. IT-15, pp. 689–695.Google Scholar
  21. [21]
    Savage, J.E., (1970) A note on the Performance of Concatenated Codes, IEEE Transactions on Information Theory, Vol. IT-16, pp. 512–513.Google Scholar
  22. [22]
    Savage, J.E., (1971) The Complexity of Decoders — Part II: Computational Work and Decoding Time, IEEE Transactions on Information Theory, Vol. IT-17, pp. 77–85.Google Scholar
  23. [23]
    Savage, J.E., (1972) Computational Work and Time on Finite Machines, Journal of the ACM, Vol. 9, pp. 660–674.CrossRefGoogle Scholar
  24. [24]
    Savage, J.E., (1973) Bounds on the Performance of Computing Systems, Proc. 7-th Annual Symposium on Computer Science and Statistics, October 18,19, Ames, Iowa.Google Scholar
  25. [25]
    Savage, J.E., (1976) The Complexity of Computing to be published by Wiley-Intersience, New York.Google Scholar
  26. [26]
    Schalkwijk, J.P.M (1971) A Class of Simple and Optimal Strategies for Block Coding on the Binary Symmetric Channel with Noiseless Feed-back, IEEE Transactions on Information Theory Vol. IT-17, pp. 283–287.Google Scholar
  27. [27]
    Schalkwijk, J.P.M, (1974) A Coding Scheme for Duplex Channels, to appear in IEEE Transactions on Communication Technology.Google Scholar
  28. [28]
    Shannon, C.E. (1948) A Mathematical Theory of Communication, Bell System Tech. Journal, Vol. 27, pp. 379–423 (Part I), pp 623–656 (Part II).Google Scholar
  29. [29]
    Shannon, C.E., R.G. Gallager and E.R. Berlekamp, (1967) Lower Bounds to Error Probability for Coding on Discrete Memoryless Channels, Information and Control, Vol. 10, pp. 65–103 (Part I), pp. 522–552 (Part II).Google Scholar
  30. [30]
    Solomonoff, R.J., (1964) A Formal Theory of Inductive Inference, Information and Control, Vol. 7, pp. 1–22 (Part I), pp. 224–254 (Part II).Google Scholar
  31. [31]
    Viterbi, A,J. (1967) Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm, IEEE Transactions on Information Theory, Vol. IT-13, pp. 260–269.Google Scholar
  32. [32]
    Wozencraft, J.M. and B. Reiffen, (1961) Sequential Decoding, MIT Press, Cambridge.MATHGoogle Scholar
  33. [33]
    Ziv, J., (1966) Further Results on the Asymptotic Complexity of an Iterative Coding Scheme, IEEE Transactions on Information Theory, Vol. IT-12, pp. 168–171.Google Scholar
  34. [34]
    Ziv, J., 1967 ) Asymptotic Performance and Complexity of a Coding Scheme for Memoryless Channels, IEEE Transactions on Information Theory, Vol. IT-13, pp. 356–359.Google Scholar

Copyright information

© Springer-Verlag Wien 1975

Authors and Affiliations

  • John E. Savage
    • 1
  1. 1.Division of EngineeringBrown UniversityProvidenceUSA

Personalised recommendations