Stochastic Complexity and Statistical Inference
We regard the fundamental task in statistics to be to “understand” a given set of observations. By “understanding” we mean in broad terms the discovery of the various constraints and regu-larities that restrict the data. An attempt to “understand” the data presents us with a dilemma. As our main source of information we only have the observed data, to which we, in the final analysis, end up in fitting a model, rather like passing a smooth curve through a scatter of data points. At the same time we realize that too good a fit is not what we want; after all, we can always get a perfect fit by just adding enough parameters to the model. Instead, intuitively, we want a model which captures the vaguely defined underlying regular features in the data, which we hope will hold even in the future and hence will enable us to make reliable predictions.
KeywordsPrediction Error Code Length Parent Distribution Binary Digit Traditional Statistic
Unable to display preview. Download preview PDF.
- 1.Akaike, H. (1974) “A new look at the statistical model identification’’, IEEE Trans. AC-19, 716–723.Google Scholar
- 2.Dawid, A.P. (1984), ’’Present Position and Potential Developments: Some Personal Views, Statistical Theory, The Prequential Approach’’, J. Royal Stat. Soc. Series A, Vol. 147, Part 2, 278–292.Google Scholar
- 5.Hjorth, U. (1982), “Model Selection and Forward Validation”, Scand. J. Stat. 9, 95–105. 416–431.Google Scholar
- 6.Kalman, R.E., (1983), Identifiability and Modeling in Econometrics’’, in Developments in Statistics, 4, P. Krishnaiah ed., 97–134, Academic Press.Google Scholar
- 8.Kullback, S. (1959), Information Theory and Statistics. John Wiley & Sons.Google Scholar
- 10.Rao, C.R. (1981), “Prediction of future observations in polynomial growth curve models’’, Proc. Indian Stat. Inst. Golden Jubilee Int. Conf. on Statistics: Applications and New Di-rections., 512–520.Google Scholar
- 12.Rissanen, J. (1983), “A Universal Prior for Integers and Estimation by Minimum De-scription Length’’, Ann. of Statistics, Vol. 11, No. 2Google Scholar
- 14.Rissanen, J. (1985), “Minimum Description Length Principle’’, Encyclopedia of Statistical Sciences, Vol. V, (S. Kotz & N. L. Johnson eds.), pp. 523–527. John Wiley and Sons, New York.Google Scholar
- 15.Rissanen, J. (1986), ’’Stochastic Complexity and Modeling’’, Ann. of Statistics, September (1986) (to appear).Google Scholar
- 16.Rissanen, J. (1986), ’’The Selection-of-Variables Problem’’, (to appear).Google Scholar
- 17.Solomonoff, R.J. (1964), “A Formal Theory of Inductive Inference”. Part I, Information and Control 7, 1–22; Part II, Information and Control 7, 224–254.Google Scholar
- 18.Stone, M. (1974), “Cross-Validatory Choice and Assessment of Statistical Predictions’’, J. Royal Stat. Soc., Ser. B, 36, 111–147.Google Scholar