Abstract
In this chapter we want to show how, by some sort of new interpretation of probability theory, we get an insight into a seemingly quite different discipline, namely information theory. Consider again the sequence of tossing a coin with outcomes 0 and 1. Now interpret 0 and 1 as a dash and dot of a Morse alphabet. We all know that by means of a Morse alphabet we can transmit messages so that we may ascribe a certain meaning to a certain sequence of symbols. Or, in other words, a certain sequence of symbols carries information. In information theory we try to find a measure for the amount of information.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Some Basic Ideas
Monographs on this subject are: L. Brillouin: Science and Information Theory (Academic Press, New York-London 1962)
L. Brillouin: Scientific Uncertainty and Information (Academic Press, New York-London 1964)
Information theory was founded by C. E. Shannon: A mathematical theory of communication. Bell System Techn. J. 27, 370–423, (1948)
C. E. Shannon: A mathematical theory of communication. Bell System Techn. J. 27, 623–656 (1948)
C. E. Shannon: Bell System Techn. J. 30, 50 (1951)
C. E. Shannon, W. Weaver: The Mathematical Theory of Communication (Univ. of Illin. Press, Urbana 1949)
Some conceptions, related to information and information gain (H-theorem!) were introduced by L. Boltzmann: Vorlesungen über Gastheorie, 2 Vols. (Leipzig 1896, 1898)
Information Gain: An Illustrative Derivation
For a detailed treatment and definition see S. Kullback: Ann. Math. Statist. 22, 79 (1951)
S. Kullback: Information Theory and Statistics (Wiley, New York 1951)
Here we follow our lecture notes. 3.3 Information Entropy and Constraints
We follow in this chapter essentially E. T. Jaynes: Phys. Rev. 106, 4 (1957)
E. T. Jaynes: Phys. Rev. 106, 620 (1957)
E. T. Jaynes: Phys. Rev. 108, 171 (1957)
E. T. Jaynes: In Delaware Seminar in the Foundations of Physics (Springer, Berlin-Heidelberg-New York 1967)
Early ideas on this subject are presented in W. Elsasser: Phys. Rev. 52, 987 (1937)
W. Elsasser: Z. Phys. 171, 66 (1968)
An Example from Physics: Thermodynamics
The approach of this chapter is conceptually based on Jaynes’ papers, I.c. Section 3.3. For textbooks giving other approaches to thermodynamics see Landau-Lifshitz: In Course of Theoretical Physics, Vol. 5: Statistical Physics (Pergamon Press, London-Paris 1952)
R. Becker: Theory of Heat (Springer, Berlin-Heidelberg-New York 1967)
A. Münster: Statistical Thermodynamics, Vol. 1 (Springer, Berlin-Heidelberg-New York 1969)
H. B. Callen: Thermodynamics (Wiley, New York 1960)
P. T. Landsberg: Thermodynamics (Wiley, New York 1961)
R. Kubo: Thermodynamics (North Holland, Amsterdam 1968)
W. Brenig: Statistische Theorie der Wärme (Springer, Berlin-Heidelberg-New York 1975)
W. Weidlich: Thermodynamik und statistische Mechanik (Akademische Verlagsgesellschaft, Wiesbaden 1976)
An Approach to Irreversible Thermodynamics
An interesting and promising link between irreversible thermodynamics and network theory has been established by A. Katchalsky, P. F. Curran: Nonequilibrium Thermodynamics in Biophysics (Harvard University Press, Cambridge Mass. 1967)
For a recent representation including also more current results see J. Schnakenberg: Thermodynamic Network Analysis of Biological Systems, Universitext (Springer, Berlin-Heidelberg-New York 1977)
For detailed texts on irreversible thermodynamics see I. Prigogine: Introduction to Thermodynamics of Irreversible Processes (Thomas, New York 1955)
I. Prigogine: Non-equilibrium Statistical Mechanics (Interscience, New York 1962)
S. R. De Groot, P. Mazur: Non-equilibrium Thermodynamics (North Holland, Amsterdam 1962)
R. Haase: Thermodynamics of Irreversible Processes (Addison-Wesley, Reading, Mass. 1969)
D. N. Zubarev: Non-equilibrium Statistical Thermodynamics (Consultants Bureau, New York-London 1974)
Here, we present a hitherto unpublished treatment by the present author. 3.6 Entropy—Curse of Statistical Mechanics?
For the problem subjectivistic-objectivistic see for example E. T. Jaynes: Information Theory. In Statistical Physics, Brandeis Lectures, Vol. 3 (W. A. Benjamin, New York 1962)
Coarse graining is discussed by A. Münster: In Encyclopedia of Physics, ed. by S. Flügge, Vol. III/2: Principles of Thermodynamics and Statistics (Springer, Berlin-Göttingen-Heidelberg 1959) The concept of entropy is discussed in all textbooks on thermodynamics, cf. references to Section 3.4.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 1978 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Haken, H. (1978). Information. In: Synergetics. Springer Series in Synergetics, vol 1. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-96469-5_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-96469-5_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-96471-8
Online ISBN: 978-3-642-96469-5
eBook Packages: Springer Book Archive