Algorithmic Probability

  • Ming Li
  • Paul Vitányi
Part of the Graduate Texts in Computer Science book series (TCS)


P.S. Laplace (1749–1827) has pointed out the following reason why intuitively a regular outcome of a random event is unlikely:

“We arrange in our thought all possible events in various classes; and we regard as extraordinary those classes which include a very small number. In the game of heads and tails, if heads comes up a hundred times in a row then this appears to us extraordinary, because the almost infinite number of combinations that can arise in a hundred throws are divided in regular sequences, or those in which we observe a rule that is easy to grasp, and in irregular sequences, that are incomparably more numerous.”


Recursive Function Infinite Sequence Code Word Kolmogorov Complexity Algorithmic Probability 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

History and References

  1. [A.N. Kolmogorov, Grundbegriffe der Wahrscheinlichkeitsrechnung, Springer-Verlag, 1933; English translation published by Chelsea, 1956].Google Scholar
  2. L.A. Levin and A.K. Zvonkin [Russ. Math. Surveys, 25(1970), 83–124].MathSciNetzbMATHCrossRefGoogle Scholar
  3. W. Feller, An Introduction to Probability Theory and Its Applications, vol. II, Wiley, 1970.Google Scholar
  4. [Russ. Math. Surveys, 25(1970), 83-124].Google Scholar
  5. [L.A. Levin and V.V. V’yugin, Lecture Notes in Computer Science, vol. 53, Springer-Verlag, 1977, pp. 359-364].Google Scholar
  6. Levin and Zvonkin [Russ. Math. Surveys, 25(1970), 83–124].MathSciNetzbMATHCrossRefGoogle Scholar
  7. D.G. Willis [J. ACM, 17(1970), 241–259].MathSciNetzbMATHCrossRefGoogle Scholar
  8. R.J. Solomonoff [Inform. Contr., 7(1964), 1–22].MathSciNetzbMATHCrossRefGoogle Scholar
  9. [Logical Foundations of Probability, Univ. Chicago Press, 1950].Google Scholar
  10. M. Minsky, J. McCarthy, and C.E. Shannon, and in fact stayed on to spend the whole summer there.Google Scholar
  11. N. Chomsky, [IRE Trans. Inform. Theory, IT-2(1956), 113–126].zbMATHCrossRefGoogle Scholar
  12. [Tech. Rept. ZTB-138, Zator Company, Cambridge, Mass.]Google Scholar
  13. [A formal theory of inductive inference, Part 1, Inform. Contr., 7(1964), 1-22].Google Scholar
  14. [M. Li and P.M.B. Vitányi, Inform. Process. Lett, 42(1992), 145–149].MathSciNetzbMATHCrossRefGoogle Scholar
  15. [P.B. Miltersen, SIAM J. Comput., 22:1(1993), 147–156MathSciNetzbMATHCrossRefGoogle Scholar
  16. K. Kobayashi, IEICE Trans. Inform. Systems, E76-D:6(1993), 634–640; K. Kobayashi, Transformations that preserve malignness of universal distributions, Theoret. Comput. Sci., to appear; A. Jakoby and R. Reis-chuk, and C. Schindelhauer, Proc. 12th Symp. Theoret. Aspects Comput. Sci., 1995, pp. 628-639].Google Scholar
  17. A.K. Jagota and K.W. Regan [‘Performance of MAX-CLIQUE approximation heuristics under description-length weighed distributions,’ UB-CS-TR 92-24, SUNY at Buffalo, 1992].Google Scholar
  18. [A.K. Zvonkin and L.A. Levin, Russ. Math. Surveys, 25(1970), 83–124].MathSciNetzbMATHCrossRefGoogle Scholar
  19. [L.A. Levin, Soviet Math. Dokl, 14(1973), 1477–1480].Google Scholar
  20. [L.A. Levin, Problems Inform. Transmission, 10:3(1974), 206–210Google Scholar
  21. P. Gács, Soviet Math. Dokl, 15(1974), 1477–1480].zbMATHGoogle Scholar
  22. G.J. Chaitin [J. ACM, 22(1975), 329–340].MathSciNetzbMATHCrossRefGoogle Scholar
  23. [L.A. Levin, Inform. Contr., 61(1984), 15–37].zbMATHCrossRefGoogle Scholar
  24. C.P. Schnorr [Lecture Notes in Mathematics, vol. 218, Springer-Verlag, 1971].Google Scholar
  25. J. Ville in 1939 [Étude Critique de la Notion de Collectif, Gauthier-Villars, 1939].Google Scholar
  26. C.P. Schnorr [J. Comput. System Sci., 7(1973), 376–388]MathSciNetzbMATHCrossRefGoogle Scholar
  27. L.A. Levin [Sov. Math. Dokl, 14(1973), 1413–1416]zbMATHGoogle Scholar
  28. [M. van Lambalgen, Random Sequences, Ph.D. Thesis, University of Amsterdam, 1987; J. Symb. Logic, 54(1989), 1389-1400].Google Scholar
  29. L.A. Levin [Soviet Math. Dokl, 14(1973), 1413–1416]zbMATHGoogle Scholar
  30. [J. Comput. System Sci., 7(1973), 376-388].Google Scholar
  31. [V.V. V’yugin, Semiotika i Informatika, 16(1981), 14–43 (p. 35).MathSciNetGoogle Scholar
  32. English translation: Selecta Mathematica formerly Sovietica, 13:4(1994), 357-389].Google Scholar
  33. C.P. Schnorr in [Basic Problems in Methodology and Linguistics, R.E. Butts and J. Hintikka, Eds., Reidel, 1977, pp. 193-210].Google Scholar
  34. [A.N. Kolmogorov and V.A. Uspensky, SIAM J. Theory Probab. Appl., 32(1987), 387–412].CrossRefGoogle Scholar
  35. [C.P. Schnorr, J. Cornput Syst. Sci., 7(1973), 376–388.MathSciNetzbMATHCrossRefGoogle Scholar
  36. A.K. Zvonkin and L.A. Levin, Russ. Math. Surveys, 25:6(1970), 83–124.MathSciNetzbMATHCrossRefGoogle Scholar
  37. L.A. Levin; L.A. Levin, Sov. Math. Dokl., 14(1973), 1413–1416.zbMATHGoogle Scholar
  38. P. Gács, Theoret. Comput. Sci., 22(1983), 71–93]MathSciNetzbMATHCrossRefGoogle Scholar
  39. [A.Kh. Shen’, Sov. Math. Dokl., 29:3(1984), 569–573].zbMATHGoogle Scholar
  40. [Kolmogorov Complexity and Computational Complexity, O. Watanabe (Ed.), Springer-Verlag, 1992, pp. 85-101].Google Scholar
  41. [V.A. Uspensky and A.Kh. Shen’, Math. Systems Theory, 29(1996), 271–292].MathSciNetzbMATHGoogle Scholar
  42. P. Gács [Theoret. Comput. Sei., 22(1983), 71–93].zbMATHCrossRefGoogle Scholar
  43. [J. Laurie Snell, Mathematical Intelligencer, 4:3(1982)].Google Scholar
  44. P. Levy and continued by J. Ville [Etude Critique de la Concept du Collectif, Gauthier-Villars, 1939].Google Scholar
  45. C.P. Schnorr [Lecture Notes in Mathematics, vol. 218, Springer-Verlag, 1971].Google Scholar
  46. [pp. 193-210 in: Basic Problems in Methodology and Linguistics, R.E. Butts, J. Hintikka (Eds.), D. Reidel, 1977].Google Scholar
  47. [R. Heim, IEEE Trans. Inform. Theory, IT-25(1979), 558–566]MathSciNetCrossRefGoogle Scholar
  48. [P. Gács, Lecture Notes on Descriptional Complexity and Randomness, Manuscript, Boston University, 1987; T.M. Cover, P. Gács and R.M. Gray, Ann. Probab., 17:3(1989), 840-865].Google Scholar
  49. [T.M. Cover and J.A. Thomas, Elements of Information Theory, Wiley, 1991].Google Scholar
  50. [P. Gács, Z. Math. Logik Grandl. Math., 28(1980), 385–394CrossRefGoogle Scholar
  51. Theoret. Comput. Sci., 22(1983), 71-93]Google Scholar

Copyright information

© Springer Science+Business Media New York 1997

Authors and Affiliations

  • Ming Li
    • 1
  • Paul Vitányi
    • 2
  1. 1.Department of Computer ScienceUniversity of WaterlooWaterlooCanada
  2. 2.Centrum voor Wiskunde en InformaticaSJ AmsterdamThe Netherlands

Personalised recommendations