Binary Coding in Noisy Channels

Abstract

We have seen previously that in a communication system we may be concerned either with the speed of transmission of information, or with the accuracy of transmission. In chapter 3 we considered speed of transmission in noiseless systems, and noted that the efficiency of coding was a measure of the redundancy of the system, perfect efficiency implying zero redundancy. In this chapter we will consider the accuracy of transmission in a noisy system. Clearly, redundancy will have to be added to give protection against errors. For example, in a system with two input symbols A and B, coding A = 000 and B = 111 gives a reduction in the overall error rate (compared with A = 0, B = 1) since single errors can be tolerated. Unfortunately the redundancy is greatly increased, and it appears at first sight that there is bound to be a simple exchange between error rate and redundancy. However, it will be shown that Information Theory predicts that the situation is not as bad as this, and that, subject to certain restrictions, a low error rate can be achieved together with a high data rate (or low redundancy).