Shannon-Wiener Information

Part of the Information Science and Statistics book series (ISS)

Abstract

In coding we want to transmit or store sequences of elements of a finite set A = a1,..., am in terms of binary symbols 0 and 1. The set A is called an alphabet and its elements are called symbols, which can be of any kind, often numerals. The sequences of the symbols are called messages, or often just data when the symbols are numerals. We begin by defining the code as a function C : AB*, taking each symbol in the alphabet into a finite binary string, called a codeword. These define a binary tree (see Figure 2.1), in which the codewords appear as paths starting at the root node. We are interested in codes that are one-to-one maps so that they have an inverse. The code may be extended to sequences x = x1,..., xn, also written as C, C : A*B*, by the operation of concatenation: C(xxn+1) = C(x)C(xn+1), where xa denotes the string obtained when symbol a is appended to the end of the string x.

Keywords

Channel Capacity Relative Entropy Code Length Arithmetic Code Complete Binary Tree
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.