Advertisement

Prediction and Dimension

  • Lance Fortnow
  • Jack H. Lutz
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2375)

Abstract

Given a set X of sequences over a finite alphabet, we investigate the following three quantities.
  1. (i)

    The feasible predictability of X is the highest success ratio that a polynomial-time randomized predictor can achieve on all sequences in X.

     
  2. (ii)

    The deterministic feasible predictability of X is the highest success ratio that a polynomial-time deterministic predictor can achieve on all sequences in X.

     
  3. (iii)

    The feasible dimension of X is the polynomial-time effectivization of the classical Hausdorff dimension (“fractal dimension”) of X.

     

Predictability is known to be stable in the sense that the feasible predictability of XY is always the minimum of the feasible predictabilities of X and Y. We show that deterministic predictability also has this property if X and Y are computably presentable. We show that deterministic predictability coincides with predictability on singleton sets. Our main theorem states that the feasible dimension of X is bounded above by the maximum entropy of the predictability of X and bounded below by the segmented self-information of the predictability of X, and that these bounds are tight.

Keywords

Fractal Dimension Polynomial Time Binary Sequence Kolmogorov Complexity Bell System Technical Journal 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    J. L. Balcázar, J. Díaz, and J. Gabarró. Structural Complexity I (second edition). Springer-Verlag, Berlin, 1995.Google Scholar
  2. 2.
    N. Cesa-Bianchi, Y. Freund, D. Haussler, D. P. Helmbold, R. E. Schapire, and M. K. Warmuth. How to use expert advice. Journal of the ACM, 44(3):427–485, May 1997.Google Scholar
  3. 3.
    N. Cesa-Bianchi and G. Lugosi. On prediction of individual sequences. Annals of Statistics, 27(6):1865–1895, 1999.zbMATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    T. M. Cover and J. A. Thomas. Elements of Information Theory. John Wiley & Sons, Inc., New York, N.Y., 1991.zbMATHGoogle Scholar
  5. 5.
    J. J. Dai, J. I. Lathrop, J. H. Lutz, and E. Mayordomo. Finite-state dimension. In Proceedings of the Twenty-Eighth International Colloquium on Automata, Languages, and Programming, pages 1028–1039. Springer-Verlag, 2001.Google Scholar
  6. 6.
    K. Falconer. Fractal Geometry: Mathematical Foundations and Applications. John Wiley & Sons, 1990.Google Scholar
  7. 7.
    M. Feder. Gambling using a finite state machine. IEEE Transactions on Information Theory, 37:1459–1461, 1991.CrossRefMathSciNetGoogle Scholar
  8. 8.
    M. Feder, N. Merhav, and M. Gutman. Universal prediction of individual sequences. IEEE Transations on Information Theory, 38:1258–1270, 1992.zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    F. Hausdorff. Dimension und äusseres Mass. Math. Ann., 79:157–179, 1919.CrossRefMathSciNetGoogle Scholar
  10. 10.
    J. Kelly. A new interpretation of information rate. Bell Systems Technical Journal, 35:917–926, 1956.Google Scholar
  11. 11.
    A. N. Kolmogorov. On tables of random numbers. Sankhyā, Series A, 25:369–376, 1963.zbMATHMathSciNetGoogle Scholar
  12. 12.
    D. W. Loveland. A new interpretation of von Mises’ concept of a random sequence. Zeitschrift für Mathematische Logik und Grundlagen der Mathematik, 12:279–294, 1966.zbMATHMathSciNetCrossRefGoogle Scholar
  13. 13.
    J. H. Lutz. Dimension in complexity classes. In Proceedings of the Fifteenth Annual IEEE Conference on Computational Complexity, pages 158–169. IEEE Computer Society Press, 2000. Updated version appears as Technical report cs.CC/0203016, ACM Computing Research Repository, 2002.Google Scholar
  14. 14.
    J. H. Lutz. Gales and the constructive dimension of individual sequences. In Proceedings of the Twenty-Seventh International Colloquium on Automata, Languages, and Programming, pages 902–913. Springer-Verlag, 2000. Updated version appears as Technical report cs.CC/0203017, ACM Computing Research Repository, 2002.Google Scholar
  15. 15.
    D. A. Martin. Classes of recursively enumerable sets and degrees of unsolvability. Zeitschrift für Mathematische Logik und Grundlagen der Mathematik, 12:295–310, 1966.zbMATHCrossRefGoogle Scholar
  16. 16.
    B. Ya. Ryabko. Noiseless coding of combinatorial sources. Problems of Information Transmission, 22:170–179, 1986.zbMATHMathSciNetGoogle Scholar
  17. 17.
    B. Ya. Ryabko. Algorithmic approach to the prediction problem. Problems of Information Transmission, 29:186–193, 1993.MathSciNetGoogle Scholar
  18. 18.
    B. Ya. Ryabko. The complexity and effectiveness of prediction problems. Journal of Complexity, 10:281–295, 1994.zbMATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    C. P. Schnorr. A unified approach to the definition of random sequences. Mathematical Systems Theory, 5:246–258, 1971.zbMATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    C. P. Schnorr. Zufälligkeit und Wahrscheinlichkeit. Lecture Notes in Mathematics, 218, 1971.Google Scholar
  21. 21.
    C. E. Shannon. Certain results in coding theory for noisy channels. Bell Systems Technical Journal, 30:50–64, 1951.Google Scholar
  22. 22.
    A. Kh. Shen’. On relations between different algorithmic definitions of randomness. Soviet Mathematics Doklady, 38:316–319, 1989.zbMATHMathSciNetGoogle Scholar
  23. 23.
    L. Staiger. Kolmogorov complexity and Hausdorff dimension. Information and Control, 103:159–94, 1993.zbMATHMathSciNetGoogle Scholar
  24. 24.
    L. Staiger. A tight upper bound on Kolmogorov complexity and uniformly optimal prediction. Theory of Computing Systems, 31:215–29, 1998.zbMATHCrossRefMathSciNetGoogle Scholar
  25. 25.
    V. Vovk. A game of prediction with expert advice. Journal of Computer and System Sciences, pages 153–173, 1998.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Lance Fortnow
    • 1
  • Jack H. Lutz
    • 2
  1. 1.NEC Research InstitutePrinceton
  2. 2.Department of Computer ScienceIowa State UniversityAmes

Personalised recommendations