A Critical Review of Shannon Information Theory
In the preceding chapter, we summarized the basic elements of information theory, and we now proceed to examine and analyze the main characteristics of this theory. The term “critical” in the title of the chapter implies simply that we shall present a review of the main features for and against the theory. To support the theory in its present form, one can mention Shannon results on the capacity of a channel, the Boltzmann equation, and the fact that one can prove the central limit theorem in probability by using the properties of entropy only. Against the present form of the theory we have the apparent discrepancy between discrete entropy and continuous entropy, the absence of a concept of negative information to describe information lost, and the fact that the model does not take explicitly into account syntax and semantics. In the present chapter, we shall review these features and one of our conclusions will be as follows: Contrary to what some scientists are inclined to believe, we maintain that the continuous entropy is soundly defined, and that it merely remains to exhibit the differences in physical nature between discrete entropy and continuous entropy.
KeywordsBoltzmann Equation Central Limit Theorem Shannon Entropy Transmission Error Negative Information
Unable to display preview. Download preview PDF.
- 3.2Jumarie. G.: Subjectivity, Information, Systems. Introduction to a Theory of Relativistic Cybernetics ( Gordon and Breach, New York 1986 )Google Scholar
- 3.4Feinstein, A.: Foundation of Information Theory ( McGraw-Hill, New York 1958 )Google Scholar
- 3.9Feller, W.: An Introduction to Probability Theory and its Application ( Wiley, New York 1950 )Google Scholar
- 3.16Haken, H. “Towards a Dynamic Information Theory” in Thermodynamics and Regulation of Biological Processes, ed. by I. Lamprecht and A. I. Zotin ( Walter de Gruyter, Berlin 1984 ) pp. 93–104Google Scholar