Information Theory



Figure 7.1 presents a generic communication system. In this figure, the message is composed at the source. Then, it undergoes some coding to become suitable for transmission through the channel. Most channels are noisy and this is signified by the noise that affects the channel from the block on top. At the receiving end of the channel, a decoder must decode the encoded and noisy message into one that the recipient will understand. This is the basis for the development of the topic of Information Theory which started with the advent of the telegraph and telephone systems. Fisher [7], Nyquist [14, 15], Hartley [9], Shannon [18], Wiener [22], and Kullback [12] were among some of the early developers of Information Theory. A lot of this work was developed in response to the encryption and decryption needs for sensitive communications during the second world war.


Probability Density Function Relative Entropy Fisher Information Matrix Probability Mass Function Discrete Random Variable 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Recognition Technologies, Inc.Yorktown HeightsUSA

Personalised recommendations