Foundations and Formalizations of Self-Organization

Abstract

In view of the various streams and directions of the field of self-organization, it is beyond the present introductory chapter to review all the currents of research in the field. Rather, the aim of the present section is to address some of the points judged as most relevant and to provide a discussion of suitable candidate formalisms for the treatment of self-organization. In the author’s opinion, discussing formalisms is not just a vain exercise, but allows one to isolate the essence of the notion one wishes to develop. Thus even if one disagrees with the path taken (as is common in the case of not yet universally agreed upon formal notions), starting from operational formalisms helps to serve as a compass guiding one towards notions suitable for one’s purposes. This is the philosophy of the present chapter. The chapter is structured as follows: in Sect. 2.2, we will present several central conceptual issues relevant in the context of self-organization. Some historical remarks about related relevant work are then done in Sect. 2.3. To illustrate the setting, a brief overview over some classical examples for self-organizing processes is given in Sect. 2.4. In Sects. 2.5 and 2.6, introduces the two main information-theoretic concepts of self-organization that the present chapter aims to discuss. One concept, based on the ϵ-machine formalism by Crutchfield and Shalizi, introduces self-organization as an increase of (statistical) complexity with time. The other concept will suggest measuring self-organization as an increase of mutual correlations (measured by multiinformation) between different components of a system. In Sect. 2.7, finally, important properties of these two measures as well as their distinctive characteristics (namely their power to identify temporal versus compositional self-organization) will be discussed, before Sect. 2.8 gives some conclusive remarks.

Appendix: Proof of Relation Between Fine and Coarse-Grained Multi-Information

Proof

First, note that a different way to write the composite random variables $\Tilde X_{j}$ is $\Tilde X_{j} = (X_{k_{j-1}+1},\dots,X_{k_{j}})$ for $j=1\dots\Tilde k$, giving

$$ H(\Tilde X_j) = H(X_{k_{j-1}+1},\dots,X_{k_j})\;. $$

(2.6)

Similarly, the joint random variable $(\Tilde X_{1},\dots,\Tilde X_{\Tilde k})$ consisting of the composite random variables $\Tilde X_{j}$ can be seen as a regrouping of the elementary random variables X ₁,…,X _k .Therefore the joint random variable constructed from the $\Tilde X_{j}$ and that constructed from the X _i have both the same entropy:

$$ H(\Tilde X_1,\dots,\Tilde X_{\Tilde k}) = H(X_1,\dots,X_k)\;. $$

(2.7)

For consistency of notation, write k ₀=0 and $k_{\Tilde k} = k$. One then obtains

where the first term results from a regrouping of summands, the second term results from Eq. (2.6) and the third from rewriting the whole set of random variables from the coarse-grained to the fine-grained notation, thus giving

which proves the equation. □