Stochastic Complexity and the Maximum Entropy Principle
This is an outline of a modeling principle based upon the search for the shortest code length of the data, defined to be the stochastic complexity. This principle is generally applicable to statistical problems, and when restricted to the special exponential family, arising in the maximum entropy formalism with a set of moment constraints, it provides a generalization which permits the set of the constraints or their number to be optimized as well.
KeywordsPrediction Error Maximum Entropy Nuisance Parameter Code Length Minimum Description Length
Unable to display preview. Download preview PDF.
- Hoel, A.E. and Kennard, R.W. (1970), “Ridge regression: Biased estimation for nonorthogonal problems’, Technometrics, 12, 55–68.Google Scholar
- James, W. and Stein, C. (1961), ‘Estimation with quadratic loss’, Proc. 4th Berkeley Symp. 1, 363–379.Google Scholar
- Jaynes, E. T. (1982), Papers on Probability, Statistics, and Statistical Physics, a reprint collection, D. Reidel, Dordrecht-Holland.Google Scholar
- Rao, C.R. (1981), ‘Prediction of future observations in polynomial growth curve models’, Proc. Indian Stat. Inst. Golden Jubilee Int. Con! on Statistics: Applications and New Directions., 512–520.Google Scholar
- Rissanen, J. (1987c), ‘Complexity and Information in Contingency Tables’, Proceedings of The Second International Tampere Conference in Statistics, June 1–4, Tampere, Finland.Google Scholar