Good codes based on very sparse matrices
We present a new family of error-correcting codes for the binary symmetric channel. These codes are designed to encode a sparse source, and are defined in terms of very sparse invertible matrices, in such a way that the decoder can treat the signal and the noise symmetrically. The decoding problem involves only very sparse matrices and sparse vectors, and so is a promising candidate for practical decoding.
It can be proved that these codes are ‘very good’, in that sequences of codes exist which, when optimally decoded, achieve information rates up to the Shannon limit.
We give experimental results using a free energy minimization algorithm and a belief propagation algorithm for decoding, demonstrating practical performance superior to that of both Bose-Chaudhury-Hocquenghem codes and Reed-Muller codes over a wide range of noise levels.
KeywordsInformation Rate Sparse Matrice Symbol Rate Code Family Belief Propagation Algorithm
Unable to display preview. Download preview PDF.
- 1.S. Andreassen, M. Woldbye, B. Falck, and S. Andersen. MUNIN — a causal probabilistic network for the interpretation of electromyographic findings. In Proc. of the 10th National Conf. on AI, AAAI: Menlo Park CA., pages 121–123, 1987.Google Scholar
- 2.E. R. Berlekamp. Algebraic Coding Theory. McGraw-Hill, New York, 1968.Google Scholar
- 3.T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley, New York, 1991.Google Scholar
- 6.D. J. C. MacKay and R. M. Neal. Good codes based on very sparse matrices. Available from http://220.127.116.11/, 1995.Google Scholar
- 7.F. J. MacWilliams and N. J. A. Sloane. The theory of error-correcting codes. North-Holland, Amsterdam, 1977.Google Scholar
- 8.R. J. McEliece. The theory of information and coding: a mathematical framework for communication. Addison-Wesley, Reading, Mass., 1977.Google Scholar
- 9.W. Meier and O. Staffelbach. Fast correlation attacks on certain stream ciphers. J. Cryptology, 1:159–176, 1989.Google Scholar
- 10.M. J. Mihaljević and J. D. Golić. Convergence of a Bayesian iterative error-correction procedure on a noisy shift register sequence. In Advances in Cryptology — EUROCRYPT 92, volume 658, pages 124–137. Springer-Verlag, 1993.Google Scholar
- 11.J. Pearl. Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann, San Mateo, 1988.Google Scholar
- 12.W. W. Peterson and E. J. Weldon, Jr. Error-Correcting Codes. MIT Press, Cambridge, Massachusetts, 2nd edition, 1972.Google Scholar
- 13.J. Rissanen and G. G. Langdon. Arithmetic coding. IBM Journal of Research and Development, 23:149–162, 1979.Google Scholar
- 14.C. E. Shannon. A mathematical theory of communication. Bell Sys. Tech. J., 27:379–423, 623–656, 1948.Google Scholar