Application of kolmogorov complexity to inductive inference with limited memory
We consider inductive inference with limited memory.
U can be learned with linear long-term memory (and no short-term memory);
U can be learned with logarithmic long-term memory (and some amount of short-term memory);
if U is learned with sublinear long-term memory, then the short-term memory exceeds arbitrary recursive function.
Thus an open problem posed by Freivalds, Kinber and Smith is solved. To prove our result, we use Kolmogorov complexity.
Unable to display preview. Download preview PDF.
- 1. [FKS93]R. Freivalds, E. Kinber and C. Smith, On the impact of forgetting on learning machines, Proceedings of the 6-th ACM COLT, 1993, pp. 165–174. To appear in Information and Computation.Google Scholar
- 2. [Gol67]
- 3. [Kol65]A. N. Kolmogorov, Three approaches to the quantitative definition of’ information', Problems of Information Transmission, vol. 1 (1965), pp. 1–7Google Scholar
- 4. [LV93]M. Li, P.Vitanyi, Introduction to Kolmogorov complexity and its applications, Springer, 1993Google Scholar
- 5. [MY78]M. Machtey and P. Young, An Introduction to the General Theory of Algorithms, North-Holland, New York, 1978Google Scholar
- 6. [Rog67]H. Rogers, Theory of Recursive Functions and Effective Computability, McGraw-Hill, New York, 1967. Reprinted by MIT Press, Cambridge, MA, 1987.Google Scholar