# Predictive Complexity and Information

## Abstract

A new notion of predictive complexity and corresponding amount of information are considered. Predictive complexity is a generalization of Kolmogorov complexity which bounds the ability of any algorithm to predict elements of a sequence of outcomes. We consider predictive complexity for a wide class of bounded loss functions which are generalizations of square-loss function. Relations between unconditional *KG*(*x*) and conditional *KG*(*xy*) predictive complexities are studied. We define an algorithm which has some “expanding property”. It transforms with positive probability sequences of given predictive complexity into sequences of essentially bigger predictive complexity. A concept of amount of predictive information *IG*(*y*: *x*) is studied. We show that this information is non-commutative in a very strong sense and present asymptotic relations between values *IG*(*y*: *x*), *IG*(*x*: *y*), *KG*(*x*) and *KG*(*y*).

## Keywords

Loss Function Binary Tree Computable Mapping Expert Advice Prediction Strategy## Preview

Unable to display preview. Download preview PDF.

## References

- 1.Haussler, D., Kivinen, J., Warmuth, M. K. (1994) Tight worst-case loss bounds for predicting with expert advice. Technical Report UCSC-CRL-94-36, University of California at Santa Cruz, revised December 1994. Short version in P. Vitányi, editor,
*Computational Learning Theory*, Lecture Notes in Computer Science, volume 904, pages 69–83, Springer, Berlin, 1995.Google Scholar - 2.Cesa-Bianchi, N., Freund, Y., Helmbold, D. P., Haussler, D., Schapire, R. E., Warmuth, M. K. (1997) How to use expert advice.
*Journal of the ACM*, 44, 427–485zbMATHCrossRefMathSciNetGoogle Scholar - 3.Cormen, H., Leiserson, E., Rivest, R (1990)
*Introduction to Algorithms*. New York: McGraw Hill.Google Scholar - 4.Kalnishkan, Y. (1999) General linear relations among different types of predictive complexity. In.
*Proc. 10th international Conference on Algorithmic Learning Theory-ALT’ 99*,*v. 1720 of Lecture Notes in Artificial Intelligence*, pp. 323–334, Springer-Verlag.Google Scholar - 5.Li, M., Vitányi, P. (1997)
*An Introduction to Kolmogorov Complexity and Its Applications*. Springer, New York, 2nd edition.zbMATHGoogle Scholar - 6.Rogers, H. (1967) Theory of recursive functions and effective computability, New York: McGraw Hill.zbMATHGoogle Scholar
- 7.Vovk, V. (1990) Aggregating strategies. In M. Fulk and J. Case, editors,
*Proceedings of the 3rd Annual Workshop on Computational Learning Theory*, pages 371–383, San Mateo, CA, 1990. Morgan Kaufmann.Google Scholar - 8.Vovk, V. (1998) A game of prediction with expert advice.
*J. Comput. Syst. Sci.*, 56:153–173.zbMATHCrossRefMathSciNetGoogle Scholar - 9.Vovk, V., Gammerman, A. (1999) Complexity estimation principle,
*The Computer Journal*,*42:4*, 318–322.zbMATHCrossRefGoogle Scholar - 10.Vovk, V., Watkins, C. J. H. C. (1998) Universal portfolio selection,
*Proceedings of the 11th Annual Conference on Computational Learning Theory*, 12–23.Google Scholar - 11.V’yugin, V. V. (1999) Does snooping help? Technical report No CLRC-TR-99-06, December 1999, Computer Learning Research Centre, Royal Holloway, University of London.Google Scholar
- 12.Vyugin, M. V., V’yugin, V. V. (2001) Non-linear inequalities between Kolmogorov and predictive complexities, In Proc. Twelfth International Conference on Algorithmic Learning Theory—ALT’ 01, 190–204.Google Scholar
- 13.Zvonkin, A. K., Levin, L. A. (1970) The complexity of finite objects and the algorithmic concepts of information and randomness,
*Russ. Math. Surv.***25**, 83–124.zbMATHCrossRefMathSciNetGoogle Scholar - 14.Yamanishi, K. (1995) Randomized approximate aggregating strategies and their applications to prediction and discrimination, in
*Proceedings, 8th Annual ACM Conference on Computational Learning Theory*, 83–90, Assoc. Comput. Mach., New York.CrossRefGoogle Scholar