We have mentioned the MDL principle on several occasions somewhat loosely as the principle that calls for finding the model and model class with which the data together with the model and model class, respectively, can be encoded with the shortest code length. Actually to apply the principle we must distinguish between two types of models — those for data compression and others for general statistical purposes such as prediction. In data compression, we apply the models to the same data from which the models are determined. Hence these models need not have any predictive power; and, in fact, to get the shortest code length we do not even need to fit models in the class considered, say, \( \mathcal{M}_\gamma \)γ. This is because the universal NML model gives a code length, which we called the stochastic complexity and which we consider to be the shortest for all intents and purposes.


Data Compression Code Length Prefix Code Metic Code Decodable Code 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer Science+Business Media, LLC 2007

Personalised recommendations