Classical Dynamics, Measurement, and Information
In this chapter, we review the basic concepts in the classical information theory [1, 2], which is needed to quantitatively discuss the relationship between thermodynamics and information. First, we formulate stochastic dynamics in classical systems. Second, we introduce the basic quantities in the classical information theory: the Shannon information, the Kullback–Leibler divergence (the relative entropy), and the mutual information. Third, we discuss classical measurement theory with stochastic errors by using the information theory. We illustrate three typical examples of classical measurements.
KeywordsMutual Information Discrete Variable Relative Entropy Shannon Entropy Stochastic Dynamic
- 1.C. Shannon, Bell Syst. Tech. J. 27, 379–423 and 623–656 (1948).Google Scholar