Abstract
A nonasymptotic lower bound is derived for the per symbol expected redundancy based on n observations from a continuous dth order Markov source. The bound is minimax over a Lipschitz class of such sources. The constant in the lower bound is explicitly described in terms of d. By making d go to infinity with n at an appropriate rate, it is shown that no universal rate of expected redundancy exists for the class of Markov sources of all orders, and this provides an alternative and simpler derivation of a similar result by Shields. Similar results are obtained for the Kullback Leibler estimation error for the joint density of d-tuples based on n observations from a continuous (d — 1)st order Markov source.
Research supported in part by ARO Grant DAAL03-91-G-0107 and by NSF Grant DMS-8505550 to MSRI at Berkeley through a postdoc-fellowship.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Assouad, P. (1983). Deux remarques sur l’estimation. Comptes Rendus de l’Academie des Sciences de Paris. 296, 1021–1024.
Birge, L. (1985). Non-asymptotic minimax risk for Hellinger balls. Probability and Mathematical Statistics 5, 21–29.
Clarke, B. S. and Barron, A. R. (1990). Information theoretic asymptotics of Bayes methods. IEEE Trans. Infor. Th. 36, 453–471.
Devroye, L. (1983). On arbitrary slow rates of global convergence in density estimation. Z. Wahrsch. Verw. Gebiete 62, 475–483.
Devroye, L. (1987). A course in density estimation. Progress in probability and statistics 14, Birkhauser.
Donoho, D. L., Liu, R. C., and MacGibbon, B. (1990). Minimax risk over hyperrectangles, and implications. Ann. Statist. 18, 1416–1437.
Doob, J. L. (1953). Stochastic Processes. Wiley, New York.
Rissanen, J. (1986). Stochastic complexity and modeling. Ann. Statist. 14, 1080–1100.
Shields, P. C. (1991). Universal redundancy rates don’t exist. Preprint.
Yu, B. and Speed, T. P. (1992). Data compression and histograms. Probab. Th. Rel. Fields 2, 195–229.
Yu, B. (1992). A note on the nonexistence of universal redundancy rates. (in preparation)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1994 Springer-Verlag New York, Inc.
About this paper
Cite this paper
Yu, B. (1994). Lower Bounds on Expected Redundancy for Classes of Continuous Markov Sources. In: Gupta, S.S., Berger, J.O. (eds) Statistical Decision Theory and Related Topics V. Springer, New York, NY. https://doi.org/10.1007/978-1-4612-2618-5_35
Download citation
DOI: https://doi.org/10.1007/978-1-4612-2618-5_35
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4612-7609-8
Online ISBN: 978-1-4612-2618-5
eBook Packages: Springer Book Archive