## Abstract

In coding we want to transmit or store sequences of elements of a finite set *A* = *a*_{1},..., *a*_{m} in terms of binary symbols 0 and 1. The set *A* is called an *alphabet* and its elements are called *symbols*, which can be of any kind, often numerals. The sequences of the symbols are called *messages*, or often just data when the symbols are numerals. We begin by defining the *code* as a function *C* : *A* → *B**, taking each symbol in the alphabet into a finite binary string, called a *codeword*. These define a binary tree (see Figure 2.1), in which the codewords appear as paths starting at the root node. We are interested in codes that are one-to-one maps so that they have an inverse. The code may be extended to sequences *x* = *x*_{1},..., *x*_{n}, also written as *C*, *C* : *A** → *B**, by the operation of *concatenation*: *C*(*xx*_{n+1}) = *C(x)C*(*x*_{n+1}), where *xa* denotes the string obtained when symbol *a* is appended to the end of the string *x*.

## Keywords

Channel Capacity Relative Entropy Code Length Arithmetic Code Complete Binary Tree## Preview

Unable to display preview. Download preview PDF.