Skip to main content
SpringerLink
Log in

Towards an Algorithmic Statistics

(Extended Abstract)

  • Conference paper
  • First Online:
Book cover Algorithmic Learning Theory (ALT 2000)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 1968))

Included in the following conference series:

  • 433 Accesses

Abstract

While Kolmogorov complexity is the accepted absolute measure of information content of an individual finite object, a similarly absolute notion is needed for the relation between an individual data sample and an individual model summarizing the information in the data, for example, a finite set where the data sample typically came from. The statistical theory based on such relations between individual objects can be called algorithmic statistics, in contrast to ordinary statistical theory that deals with relations between probabilistic ensembles. We develop a new algorithmic theory of typical statistic, sufficient statistic, and minimal suffcient statistic.

The paper was partly written during this author’s visit at CWI.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. A.R. Barron, J. Rissanen, and B. Yu, The minimum description length principle in coding and modeling, IEEE Trans. Inform. Theory, IT-44:6(1998), 2743-2760.

    Google Scholar 

  2. T.M. Cover, Kolmogorov complexity, data compression, and inference, pp. 23–33 in: The Impact of Processing Techniques on Communications, J.K. Skwirzynski, Ed., Martinus Nijhoff Publishers, 1985.

    Google Scholar 

  3. T.M. Cover and J.A. Thomas, Elements of Information Theory, Wiley, New York, 1991.

    MATH  Google Scholar 

  4. R. A. Fisher, On the mathematical foundations of theoretical statistics, Philosophical Transactions of the Royal Society of London, Ser. A, 222(1922), 309–368.

    Google Scholar 

  5. Q. Gao, M. Li and P.M.B. Vitányi, Applying MDL to learn best model granularity, Artificial Intelligence, To appear. http://xxx.lanl.gov/abs/physics/0005062

  6. P. Gács, J. Tromp, P. Vitányi, Algorithmic statistics, Submitted. http://xxx.lanl.gov/abs/math.PR/0006233

  7. M. Li and P.M.B. Vitányi, An Introduction to Kolmogorov Complexity and its Applications, Springer-Verlag, New York, 2nd Edition, 1997.

    MATH  Google Scholar 

  8. A.N. Kolmogorov, Three approaches to the quantitative definition of information, Problems Inform. Transmission 1:1 (1965) 1–7.

    MathSciNet  Google Scholar 

  9. A.N._Kolmogorov, On logical foundations of probability theory, Pp. 1–5 in: Probability Theory and Mathematical Statistics, Lect. Notes Math., Vol. 1021, K. Itô and Yu.V. Prokhorov, Eds., Springer-Verlag, Heidelberg, 1983.

    Chapter  Google Scholar 

  10. A.N. Kolmogorov and V.A. Uspensky, Algorithms and Randomness, SIAM Theory Probab. Appl., 32:3(1988), 389–412.

    Article  Google Scholar 

  11. L.A. Levin, Laws of information conservation (nongrowth) and aspects of the foundation of probability theory, Problems Inform. Transmission 10:3(1974), 206–210.

    Google Scholar 

  12. L.A. Levin Randomness conservation inequalities: information and independence in mathematical theories, Information and Control 61 (1984) 15–37.

    Article  MATH  MathSciNet  Google Scholar 

  13. P. Martin-Löf, The definition of random sequences, Inform. Contr., 9(1966), 602–619.

    Article  Google Scholar 

  14. A.Kh. Shen, The concept of (α, β)-stochasticity in the Kolmogorov sense, and its properties, Soviet Math. Dokl., 28:1(1983), 295–299.

    MATH  Google Scholar 

  15. A.Kh. Shen, Discussion on Kolmogorov complexity and statistical analysis, The Computer Journal, 42:4(1999), 340–342.

    Article  MATH  Google Scholar 

  16. P.M.B. Vitányi and M. Li, Minimum Description Length Induction, Bayesianism, and Kolmogorov Complexity, IEEE Trans. Inform. Theory, IT-46:2(2000), 446–464.

    Article  Google Scholar 

  17. V.V. V’yugin, On the defect of randomness of a finite object with respect to measures with given complexity bounds, SIAM Theory Probab. Appl., 32:3(1987), 508–512.

    Article  MATH  MathSciNet  Google Scholar 

  18. V.V. V’yugin, Algorithmic complexity and stochastic properties of finite binary sequences, The Computer Journal, 42:4(1999), 294–317.

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, Boston University, 02215, MA, Boston, USA

    Peter Gács

  2. CWI, Kruislaan 413, 1098, SJ, Amsterdam, The Netherlands

    John Tromp & Paul Vitányi

Authors
  1. Peter Gács
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John Tromp
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paul Vitányi
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Informatics, Kyushu University, Hakozaki 6-10-1, 812-8581, Fukuoka, Japan

    Hiroki Arimura

  2. School of Computing, National University of Singapore, 3 Science Drive 2, 117543, Singapore, Singapore

    Sanjay Jain

  3. School of Computer Science and Engineering, The University of New South Wales, 2052, Sydney, Australia

    Arun Sharma

Rights and permissions

Reprints and permissions

Copyright information

© 2000 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Gács, P., Tromp, J., Vitányi, P. (2000). Towards an Algorithmic Statistics. In: Arimura, H., Jain, S., Sharma, A. (eds) Algorithmic Learning Theory. ALT 2000. Lecture Notes in Computer Science(), vol 1968. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-40992-0_4

Download citation

  • DOI: https://doi.org/10.1007/3-540-40992-0_4

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-41237-3

  • Online ISBN: 978-3-540-40992-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation