In this chapter we want to show how, by some sort of new interpretation of probability theory, we get an insight into a seemingly quite different discipline, namely information theory. Consider again the sequence of tossing a coin with outcomes 0 and 1. Now interpret 0 and 1 as a dash and dot of a Morse alphabet. We all know that by means of a Morse alphabet we can transmit messages so that we may ascribe a certain meaning to a certain sequence of symbols. Or, in other words, a certain sequence of symbols carries information. In information theory we try to find a measure for the amount of information.
KeywordsPartition Function Internal Energy Information Gain Information Entropy Entropy Density
Unable to display preview. Download preview PDF.
Some Basic Ideas
- Some conceptions, related to information and information gain (H-theorem!) were introduced by L. Boltzmann: Vorlesungen über Gastheorie, 2 Vols. (Leipzig 1896, 1898)Google Scholar
Information Gain: An Illustrative Derivation
- S. Kullback: Information Theory and Statistics (Wiley, New York 1951)Google Scholar
Here we follow our lecture notes. 3.3 Information Entropy and Constraints
- E. T. Jaynes: In Delaware Seminar in the Foundations of Physics (Springer, Berlin-Heidelberg-New York 1967)Google Scholar
An Example from Physics: Thermodynamics
- The approach of this chapter is conceptually based on Jaynes’ papers, I.c. Section 3.3. For textbooks giving other approaches to thermodynamics see Landau-Lifshitz: In Course of Theoretical Physics, Vol. 5: Statistical Physics (Pergamon Press, London-Paris 1952)Google Scholar
- R. Becker: Theory of Heat (Springer, Berlin-Heidelberg-New York 1967)Google Scholar
- R. Kubo: Thermodynamics (North Holland, Amsterdam 1968)Google Scholar
- W. Brenig: Statistische Theorie der Wärme (Springer, Berlin-Heidelberg-New York 1975)Google Scholar
- W. Weidlich: Thermodynamik und statistische Mechanik (Akademische Verlagsgesellschaft, Wiesbaden 1976)Google Scholar
An Approach to Irreversible Thermodynamics
- An interesting and promising link between irreversible thermodynamics and network theory has been established by A. Katchalsky, P. F. Curran: Nonequilibrium Thermodynamics in Biophysics (Harvard University Press, Cambridge Mass. 1967)Google Scholar
- For detailed texts on irreversible thermodynamics see I. Prigogine: Introduction to Thermodynamics of Irreversible Processes (Thomas, New York 1955)Google Scholar
- S. R. De Groot, P. Mazur: Non-equilibrium Thermodynamics (North Holland, Amsterdam 1962)Google Scholar
- R. Haase: Thermodynamics of Irreversible Processes (Addison-Wesley, Reading, Mass. 1969)Google Scholar
- D. N. Zubarev: Non-equilibrium Statistical Thermodynamics (Consultants Bureau, New York-London 1974)Google Scholar
Here, we present a hitherto unpublished treatment by the present author. 3.6 Entropy—Curse of Statistical Mechanics?
- For the problem subjectivistic-objectivistic see for example E. T. Jaynes: Information Theory. In Statistical Physics, Brandeis Lectures, Vol. 3 (W. A. Benjamin, New York 1962)Google Scholar
- Coarse graining is discussed by A. Münster: In Encyclopedia of Physics, ed. by S. Flügge, Vol. III/2: Principles of Thermodynamics and Statistics (Springer, Berlin-Göttingen-Heidelberg 1959) The concept of entropy is discussed in all textbooks on thermodynamics, cf. references to Section 3.4.Google Scholar