Skip to main content
SpringerLink
Log in
Book cover

Statistical Decision Theory and Related Topics V pp 303–326Cite as

Minimax Bayes, Asymptotic Minimax and Sparse Wavelet Priors

  • Conference paper

Abstract

Pinsker (1980) gave a precise asymptotic evaluation of the minimax mean squared error of estimation of a signal in Gaussian noise when the signal is known a priori to lie in a compact ellipsoid in Hilbert space. This ‘Minimax Bayes’ method can be applied to a variety of global non-parametric estimation settings with parameter spaces far from ellipsoidal. For example it leads to a theory of exact asymptotic minimax estimation over norm balls in Besov and Triebel spaces using simple co-ordinatewise estimators and wavelet bases.

This paper outlines some features of the method common to several applications. In particular, we derive new results on the exact asymptotic minimax risk over weak lp- balls in Rn as n → ∞, and also for a class of ‘local’ estimators on the Triebel scale.

By its very nature, the method reveals the structure of asymptotically least favorable distributions. Thus we may simulate ‘least favorable’ sample paths. We illustrate this for estimation of a signal in Gaussian white noise over norm balls in certain Besov spaces. In wavelet bases, when p < 2, the least favorable priors are sparse, and the resulting sample paths strikingly different from those observed in Pinsker’s ellipsoidal setting (p = 2).

I am grateful for many conversations with David Donoho and Carl Taswell, and to a referee for helpful comments. This work was supported in part by NSF grants DMS 84-51750, 9209130, and NIH PHS grant GM21215-12.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bergh, J. and Lofstrom. (1976) Interpolation Spaces. An Introduction. New York, Springer-Verlag.

    MATH  Google Scholar 

  2. Bickel, P. J. (1983). Minimax estimation of a normal mean subject to doing well at a point. In Recent Advances in Statistics ( M. H. Rizvi, J. S. Rustagi, and D. Siegmund, eds.), Academic Press, New York, 511–528.

    Google Scholar 

  3. Chentsov, N. N. (1972). Statistical Decision Functions and Optimal Inference. Trans-lations of Mathematical Monographs, Vol. 53. American Math. Society. Providence, R. I.

    Google Scholar 

  4. Daubechies, I. (1992) Ten Lectures on Wavelets. SIAM. Philadelphia.

    Google Scholar 

  5. DeVore, R. A . (1989) Degree of Nonlinear Approximation. in Approximation Theory VI, Volume 1, C.K. Chui, L. L. Schumaker and J. D. Ward (eds.) pp. 175–201. Academic Press.

    MathSciNet  Google Scholar 

  6. Donoho, D. L. and Johnstone, I. M (1990) Muiimax risk over lp-balls. Technical Report, Department of Statistics, University of California, Berkeley.

    Google Scholar 

  7. Donoho, D. L. and Johnstone, I. M (1992a) Minimax estimation via Wavelet Shrinkage. Technical Report, Department of Statistics, Stanford University.

    Google Scholar 

  8. Donoho, D. L. and Johnstone, I. M (1992b) Ideal Spatial Adaptation via Wavelet Shrinkage. Technical Report, Department of Statistics, Stanford University. To appear, Biometrika.

    Google Scholar 

  9. Donoho, D. L., Johnstone, I. M and Taswell, C. (1992c) Wavelet software for data analysis using MATLAB. Manuscript.

    Google Scholar 

  10. Donoho, D. L. and Johnstone, I. M (1992d) Non-dassical minimax problems, thresholding, adaptation. Manuscript.

    Google Scholar 

  11. Efroimovich, S. Y. and Pinsker, M. S. (1981) Estimation of square-integrable [spectral] density based on a sequence of observations. Problemy Peredatsii Informatsii 17 50–68 (in Russian); Problems of Information Transmission (1982) 182–196 (in English).

    Google Scholar 

  12. Efroimovich, S. Y. and Pinsker, M. S. (1982) Estimation of square-integrable probability density of a random variable. Problemy Peredatsii Informatsii 18 19-38 (in Russian); Problems of Information Transmission (1983) 175–189 (in English).

    Google Scholar 

  13. Farrell, R.H. (1972) On the best obtainable asymptotic rates of convergence in estimation of a density function at a point. Ann. Math. Statist. 43. 170–180.

    Article  MathSciNet  MATH  Google Scholar 

  14. Feldman, I. (1991) Constrained minimax estimation of the mean of the normal distribution. Ann. Statist. 19. 2259–2265.

    Article  MathSciNet  MATH  Google Scholar 

  15. Frazier, M., Jawerth, B. and Weiss G. (1991) Littlewood Paley theory and the study of function spaces. CBMS — Conference Lecture Notes 79, American Mathematical Society, Providence, R. I.

    Google Scholar 

  16. Golubev, G. K. and Nussbaum, M. (1990). A Risk Bound in Sobolev class regression. Ann. Statist. 18 758–778.

    Article  MathSciNet  MATH  Google Scholar 

  17. Hodges, J. L. Jr. and Lehmann, E. L. (1952). The use of previous experience in reaching statistical decisions. Ann. Math. Statist. bf 23, 396–407.

    Article  MathSciNet  MATH  Google Scholar 

  18. Ibragimov, I. A. and Has’minskii, R. Z. (1981) Statistical Estimation, Asymptotic Theory. Translation: Springer-Verlag, New York.

    MATH  Google Scholar 

  19. Ibragimov, I. A. and Has’minskii, R. Z. (1982) Bounds for the risk of nonparametric regression estimates. Theory Probab. Appl. 27 84–99.

    Article  MATH  Google Scholar 

  20. Johnstone, I. M. and Silverman, B. W. (1990) Speed of estimation in positron emission tomography and related inverse problems. Ann. Statist. 18 251’280.

    Article  MathSciNet  MATH  Google Scholar 

  21. LeCam, L. (1986) Asymptotic methods in statistical decision theory. Springer Verlag. New York.

    Google Scholar 

  22. Morris, C. (1983) Parametric empirical Bayes inference: Theory and applications. J. Amer. Statist. Assoc. 78, 47–65.

    Article  MathSciNet  MATH  Google Scholar 

  23. Nemirovsku, A. S. (1985) Nonparametric estimation of smooth regression functions. I. zv. Akad. Nauk. SSR Teckhn. Kibernet. 3, 50–60 (in Russian). J. Cornput. Syst. Sci. 23, 6, 1–11, (1986) (in English).

    Google Scholar 

  24. Nemirovsku, A. S., Polyak, B. T. and Tsybakov, A. B. (1985) Rate of convergence of nonparametric estimates of maximum-likelihood type. Problems of Information Trans-mission 21, 258–272.

    Google Scholar 

  25. Nussbaum, M. (1985) Spline smoothing and asymptotic efficiency in LZ. Ann. Statist., 13, 984–997.

    Article  MathSciNet  MATH  Google Scholar 

  26. Peetre, J. (1976) New Thoughts on Besov Spaces. Duke University Mathematics Series. Number 1.

    Google Scholar 

  27. Pinsker, M. S. (1980) Optimal filtering of square integrable signals in Gaussian white noise. Problemy Peredatsii Informatsii 16 52–68 (in Russian); Problems of Information Transmission (1980) 120–133 (in English).

    Google Scholar 

  28. Shorack, G. R. and Wellner, J. A. (1986) Empirical Processes with Applications to Statistics. Wiley, New York.

    MATH  Google Scholar 

  29. Sion, M. On general minimax theorems. Pacific J. Math. 8, 171–176.

    Google Scholar 

  30. Speckman, P. (1985) Spline smoothing and optimal rates of convergence in nonpararmetric regression models. Ann. Statist., 13, 970–983.

    Article  MathSciNet  MATH  Google Scholar 

  31. Stone, C. (1982). Optimal global rates of convergence for nonparametric estimators. Ann. Statist., 10, 1040–1053.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Stanford University, Germany

    Iain M. Johnstone

Authors
  1. Iain M. Johnstone
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Statistics, Purdue University, 47907, West Lafayette, IN, USA

    Shanti S. Gupta  & James O. Berger  & 

Rights and permissions

Reprints and permissions

Copyright information

© 1994 Springer-Verlag New York, Inc.

About this paper

Cite this paper

Johnstone, I.M. (1994). Minimax Bayes, Asymptotic Minimax and Sparse Wavelet Priors. In: Gupta, S.S., Berger, J.O. (eds) Statistical Decision Theory and Related Topics V. Springer, New York, NY. https://doi.org/10.1007/978-1-4612-2618-5_23

Download citation

  • DOI: https://doi.org/10.1007/978-1-4612-2618-5_23

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4612-7609-8

  • Online ISBN: 978-1-4612-2618-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation