Kolmogorov’s Legacy about Entropy, Chaos, and Complexity

Abstract

The concept of entropy was initially introduced in thermodynamics in a phenomenological context. Later, mainly by the contribution of Boltzmann, a probabilistic interpretation of the entropy was developed in order to clarify its deep relation with the microscopic structure underlying the macroscopic bodies. In the unrelated field of communication systems, Shannon showed that, with a suitable generalization of the entropy concept, one can establish a mathematically self-consistent information theory. Kolmogorov realized, just after Shannon’s work, the conceptual relevance of information theory and the importance of its ideas for the characterization of irregular behaviors in dynamical systems. Going beyond a probabilistic point of view, the Algorithmic Complexity (introduced by Chaitin, Kolmogorov and Solomonoff) allows a formalization of the intuitive notion of randomness of a sequence. In this chapter we discuss the connections among entropy, chaos and algorithmic complexity; in addition we briefly discuss how these concepts and methods can be successfully applied to other unrelated fields: linguistics, bioinformatics, finance.