Twisting Features with Properties

Abstract

We provide three steps in the direction of shifting probability from a descriptive tool of unpredictable events to a way of understanding them. At a very elementary level we state an operational definition of probability based solely on symmetry assumptions about observed data. This definition converges, however, to the Kolmogorov one within a special large number law fashion that represents a first way of twisting features observed in the data with properties expected in the next observations. Within this probability meaning we fix a general sampling mechanism to generate random variables and extend our twisting device to computing probability distributions on population properties on the basis of the likelihood of the observed features. Here the randomness core translates from the above symmetry assumptions in a generator of unitary uniform random variables. Willing discovering suitable features (which are classically defined as sufficient statistics), we refer directly to the notions of Kolmogorov complexity and coding theorem in particular. This is to connect the features to the inner structure of the observed data in terms of concise computer codes describing them in a well equipped computational framework.

This new statistical framework allows us to recover and improve results on computational learning at both subsymbolic and symbolic stages, figuring a unique shell where the full trip from sensory data to their conceptual management might occur.