Skip to main content
SpringerLink
Log in

Stochastic Complexity and Statistical Inference

  • Conference paper
Analysis and Optimization of Systems

Part of the book series: Lecture Notes in Control and Information Sciences ((LNCIS,volume 83))

Abstract

We regard the fundamental task in statistics to be to “understand” a given set of observations. By “understanding” we mean in broad terms the discovery of the various constraints and regu-larities that restrict the data. An attempt to “understand” the data presents us with a dilemma. As our main source of information we only have the observed data, to which we, in the final analysis, end up in fitting a model, rather like passing a smooth curve through a scatter of data points. At the same time we realize that too good a fit is not what we want; after all, we can always get a perfect fit by just adding enough parameters to the model. Instead, intuitively, we want a model which captures the vaguely defined underlying regular features in the data, which we hope will hold even in the future and hence will enable us to make reliable predictions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 74.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Akaike, H. (1974) “A new look at the statistical model identification’’, IEEE Trans. AC-19, 716–723.

    Google Scholar 

  2. Dawid, A.P. (1984), ’’Present Position and Potential Developments: Some Personal Views, Statistical Theory, The Prequential Approach’’, J. Royal Stat. Soc. Series A, Vol. 147, Part 2, 278–292.

    Google Scholar 

  3. Geisser, S. and Eddy, W. (1979), “A Predictive Approach to Model Selection’’, J. American Stat. Ass., Vol. 74, Nr. 365, 153–160.

    Article  MATH  MathSciNet  Google Scholar 

  4. Chaitin, G.J. (1975), ’’A Theory of Program Size Formally Identical to Information Theory’’, JACM, Vol. 22, No. 3, 329–340.

    Article  MATH  MathSciNet  Google Scholar 

  5. Hjorth, U. (1982), “Model Selection and Forward Validation”, Scand. J. Stat. 9, 95–105. 416–431.

    Google Scholar 

  6. Kalman, R.E., (1983), Identifiability and Modeling in Econometrics’’, in Developments in Statistics, 4, P. Krishnaiah ed., 97–134, Academic Press.

    Google Scholar 

  7. Kolmogorov, A.N. (1965), “Three Approaches to the Quantitative Definition of Informa-tion’’, Problems of Information Transmission 1, 4–7.

    MathSciNet  Google Scholar 

  8. Kullback, S. (1959), Information Theory and Statistics. John Wiley & Sons.

    Google Scholar 

  9. Quenouille, M.H. (1956), “Notes on Bias in Estimation’’, Biometrika, 43, 353–360.

    Article  MATH  MathSciNet  Google Scholar 

  10. Rao, C.R. (1981), “Prediction of future observations in polynomial growth curve models’’, Proc. Indian Stat. Inst. Golden Jubilee Int. Conf. on Statistics: Applications and New Di-rections., 512–520.

    Google Scholar 

  11. Rissanen, J. (1978), “Modeling by shortest data description’’, Automatica, Vol. 14, pp. 465–471.

    Article  MATH  Google Scholar 

  12. Rissanen, J. (1983), “A Universal Prior for Integers and Estimation by Minimum De-scription Length’’, Ann. of Statistics, Vol. 11, No. 2

    Google Scholar 

  13. Rissanen, J. (1984), “Universal Coding, Information, Prediction, and Estimation”, IEEE Trans. Inf. Theory, Vol. IT-30, Nr. 4, 629–636.

    Article  MathSciNet  Google Scholar 

  14. Rissanen, J. (1985), “Minimum Description Length Principle’’, Encyclopedia of Statistical Sciences, Vol. V, (S. Kotz & N. L. Johnson eds.), pp. 523–527. John Wiley and Sons, New York.

    Google Scholar 

  15. Rissanen, J. (1986), ’’Stochastic Complexity and Modeling’’, Ann. of Statistics, September (1986) (to appear).

    Google Scholar 

  16. Rissanen, J. (1986), ’’The Selection-of-Variables Problem’’, (to appear).

    Google Scholar 

  17. Solomonoff, R.J. (1964), “A Formal Theory of Inductive Inference”. Part I, Information and Control 7, 1–22; Part II, Information and Control 7, 224–254.

    Google Scholar 

  18. Stone, M. (1974), “Cross-Validatory Choice and Assessment of Statistical Predictions’’, J. Royal Stat. Soc., Ser. B, 36, 111–147.

    Google Scholar 

  19. Stone, M. (1977), “Asymptotics for and against Cross-Validation”, Biometrika 64, 29–35.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IBM Res., K54/802, San Jose, Ca, 95120-6099, USA

    Jorma Rissanen

Authors
  1. Jorma Rissanen
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institut National de Recherche en Informatique et en Automatique, INRIA, Domaine de Voluceau, Rocquencourt, B.P. 105, 78153, Le Chesnay, France

    A. Bensoussan  & J. L. Lions  & 

Rights and permissions

Reprints and permissions

Copyright information

© 1986 Springer Science+Business Media Dordrecht

About this paper

Cite this paper

Rissanen, J. (1986). Stochastic Complexity and Statistical Inference. In: Bensoussan, A., Lions, J.L. (eds) Analysis and Optimization of Systems. Lecture Notes in Control and Information Sciences, vol 83. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0007575

Download citation

  • DOI: https://doi.org/10.1007/BFb0007575

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-16729-7

  • Online ISBN: 978-3-540-39856-1

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation