Lower Bounds on Expected Redundancy for Classes of Continuous Markov Sources
A nonasymptotic lower bound is derived for the per symbol expected redundancy based on n observations from a continuous dth order Markov source. The bound is minimax over a Lipschitz class of such sources. The constant in the lower bound is explicitly described in terms of d. By making d go to infinity with n at an appropriate rate, it is shown that no universal rate of expected redundancy exists for the class of Markov sources of all orders, and this provides an alternative and simpler derivation of a similar result by Shields. Similar results are obtained for the Kullback Leibler estimation error for the joint density of d-tuples based on n observations from a continuous (d — 1)st order Markov source.
KeywordsJoint Density Predictive Density Finite Alphabet Prefix Code Lipschitz Class
Unable to display preview. Download preview PDF.
- Devroye, L. (1987). A course in density estimation. Progress in probability and statistics 14, Birkhauser.Google Scholar
- Shields, P. C. (1991). Universal redundancy rates don’t exist. Preprint.Google Scholar
- Yu, B. (1992). A note on the nonexistence of universal redundancy rates. (in preparation)Google Scholar