In coding we want to transmit or store sequences of elements of a finite set A = a1,..., am in terms of binary symbols 0 and 1. The set A is called an alphabet and its elements are called symbols, which can be of any kind, often numerals. The sequences of the symbols are called messages, or often just data when the symbols are numerals. We begin by defining the code as a function C : A → B*, taking each symbol in the alphabet into a finite binary string, called a codeword. These define a binary tree (see Figure 2.1), in which the codewords appear as paths starting at the root node. We are interested in codes that are one-to-one maps so that they have an inverse. The code may be extended to sequences x = x1,..., xn, also written as C, C : A* → B*, by the operation of concatenation: C(xxn+1) = C(x)C(xn+1), where xa denotes the string obtained when symbol a is appended to the end of the string x.
KeywordsChannel Capacity Relative Entropy Code Length Arithmetic Code Complete Binary Tree
Unable to display preview. Download preview PDF.