Lower Bounds on Expected Redundancy for Classes of Continuous Markov Sources

Conference paper

Abstract

A nonasymptotic lower bound is derived for the per symbol expected redundancy based on n observations from a continuous dth order Markov source. The bound is minimax over a Lipschitz class of such sources. The constant in the lower bound is explicitly described in terms of d. By making d go to infinity with n at an appropriate rate, it is shown that no universal rate of expected redundancy exists for the class of Markov sources of all orders, and this provides an alternative and simpler derivation of a similar result by Shields. Similar results are obtained for the Kullback Leibler estimation error for the joint density of d-tuples based on n observations from a continuous (d — 1)st order Markov source.

Keywords

Entropy Neral Pyramid Prefix Estima 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Assouad, P. (1983). Deux remarques sur l’estimation. Comptes Rendus de l’Academie des Sciences de Paris. 296, 1021–1024.MathSciNetMATHGoogle Scholar
  2. Birge, L. (1985). Non-asymptotic minimax risk for Hellinger balls. Probability and Mathematical Statistics 5, 21–29.MathSciNetMATHGoogle Scholar
  3. Clarke, B. S. and Barron, A. R. (1990). Information theoretic asymptotics of Bayes methods. IEEE Trans. Infor. Th. 36, 453–471.MathSciNetMATHCrossRefGoogle Scholar
  4. Devroye, L. (1983). On arbitrary slow rates of global convergence in density estimation. Z. Wahrsch. Verw. Gebiete 62, 475–483.MathSciNetMATHCrossRefGoogle Scholar
  5. Devroye, L. (1987). A course in density estimation. Progress in probability and statistics 14, Birkhauser.Google Scholar
  6. Donoho, D. L., Liu, R. C., and MacGibbon, B. (1990). Minimax risk over hyperrectangles, and implications. Ann. Statist. 18, 1416–1437.MathSciNetMATHCrossRefGoogle Scholar
  7. Doob, J. L. (1953). Stochastic Processes. Wiley, New York.MATHGoogle Scholar
  8. Rissanen, J. (1986). Stochastic complexity and modeling. Ann. Statist. 14, 1080–1100.MathSciNetMATHCrossRefGoogle Scholar
  9. Shields, P. C. (1991). Universal redundancy rates don’t exist. Preprint.Google Scholar
  10. Yu, B. and Speed, T. P. (1992). Data compression and histograms. Probab. Th. Rel. Fields 2, 195–229.MathSciNetCrossRefGoogle Scholar
  11. Yu, B. (1992). A note on the nonexistence of universal redundancy rates. (in preparation)Google Scholar

Copyright information

© Springer-Verlag New York, Inc. 1994

Authors and Affiliations

  • Bin Yu
    • 1
  1. 1.University of Wisconsin-MadisonUSA

Personalised recommendations