Statistical Mechanics and the Maximum Entropy Method

Abstract

This course reviews the foundations and methods of statistical mechanics in their relation to the maximum entropy principle. After having introduced the main tools of information theory and of statistical physics, we turn to the problem of assignment of probabilities. We show that two methods currently used in this respect, based on the principle of indifference and on the principle of maximum statistical entropy, respectively, are equivalent. The general occurrence of the resulting canonical distributions, and of the partition function technique, is stressed. We introduce the relevant entropy relative to some set of data, which measures the uncertainty associated with the knowledge of these sole data. This quantity is identified with the entropy of thermodynamics when the system is in equilibrium or quasi-equilibrium, in which case the Second Law appears as a consequence of the maximization of the statistical entropy. In non-equilibrium statistical mechanics, the introduction of several relevant entropies, associated with different levels of description, helps in discussing the irreversibility problem and understanding dissipation as a loss of information towards irrelevant, microscopic variables. The elimination of these variables, which allows us to derive the dynamics of a macroscopic set of variables from the microscopic equations of motion, is achieved by means of the projection method. The latter is shown to rely on the determination, at each time, of the state with maximum entropy which accounts for the macroscopic data. A natural metric structure, arising from the existence of the entropy of von Neumann, is introduced in the space of states; the projection method is then interpreted as an orthogonal projection over the subspace of generalized canonical states. Examples are given throughout, and casual remarks are made about prior probabilities, ergodicity, the thermodynamic limit, the choice of relevant variables, the memory kernel, short-memory approximations, spin echoes. Changes in entropy caused by quantum measurements are also discussed; in particular, the reduction of the wave packet amounts to a maximization of entropy. Finally we mention the existence of other non-probabilistic entropies.