Wavelet Shrinkage: From Sparsity and Robust Testing to Smooth Adaptation

Part of the Applied and Numerical Harmonic Analysis book series (ANHA)


Wavelet transforms are said to be sparse in that they represent smooth and piecewise regular signals by coefficients that are mostly small except for a few that are significantly large. The WaveShrink or wavelet shrinkage estimators introduced by Donoho and Johnstone in their seminal work exploit this sparsity to estimate or reconstruct a deterministic function from the observation of its samples corrupted by independent and additive white Gaussian noise AWGN. After a brief survey on wavelet shrinkage, this chapter presents several theoretical results, from which we derive new adaptable WaveShrink estimators that overcome the limitations of standard ones, have an explicit close form and can apply to any wavelet transform (orthogonal, redundant, multi-wavelets, complex wavelets, among others) and to a large class of estimation problems. These WaveShrink estimators do not induce additional computational cost that could depend on the application and let the user choose freely the wavelet transform suitable for a given application, in contrast to all parametric methods and also in contrast to some non-parametric methods. In addition, our estimators can be adapted to each decomposition level thanks to known properties of the wavelet transform. The two theoretical results on which our WaveShrink estimators rely are, fist, a new measure of sparsity for sequences of noisy random signals and, second, the construction of a family of smooth shrinkage functions, the so-called Smooth Sigmoid Based Shrinkage (SSBS) functions. The measure of sparsity is based on recent results in non-parametric statistics for the detection of signals with unknown distributions and unknown probabilities of presence in independent AWGN. The SSBS functions allow for a flexible control of the shrinkage thanks to parameters that directly relate to the attenuation wanted for the small, median and large coefficients. The relevance of the approach is illustrated in image denoising, a typical application field for WaveShrink estimators.


Additive White Gaussian Noise Decomposition Level Image Denoising Constant False Alarm Rate Wavelet Shrinkage 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Antoniadis, A., Fan, J.: Regularization of Wavelet Approximations. Journal of the American Statistical Association, 96(455), 939–955 (2001)MATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Atto, A.M., Pastor, D., Mercier, G.: Detection Threshold for Non-Parametric Estimation, Signal, Image and Video Processing, 2, 207–223 (2008)MATHCrossRefGoogle Scholar
  3. 3.
    Atto, A.M., Pastor, D., Mercier, G.: Smooth Sigmoid Wavelet Shrinkage for Non-Parametric Estimation. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Las Vegas, NV, USA (2008)Google Scholar
  4. 4.
    Berman, S.: Sojourns and Extremes of Stochastic Processes. Wadsworth and Brooks/Cole, Belmont, CA (1992)MATHGoogle Scholar
  5. 5.
    Bruce, A.G., Gao, H.Y.: Understanding WaveShrink: Variance and Bias Estimation. Biometrika, 83(4), 727–745 (1996)MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Coifman, R.R., Donoho, D.L.: Translation invariant de-noising. Lecture Notes in Statistics: Wavelets and Statistics, 125–150, Springer, New York (1995)Google Scholar
  7. 7.
    Do, M.N., Vetterli, M.: Wavelet-Based Texture Retrieval Using Generalized Gaussian Density and Kullback-Leibler Distance. IEEE Transactions on Image Processing, 11(2), 146–158 (2002)CrossRefMathSciNetGoogle Scholar
  8. 8.
    Donoho, D.L., Johnstone, I.M.: Ideal Spatial Adaptation by Wavelet Shrinkage. Biometrika, 81(3), 425–455 (1994)MATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Donoho, D.L.: De-Noising by Soft-Thresholding. IEEE Transactions on Information Theory, 41(3), 613–627 (1995)MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Donoho, D.L., Johnstone, I.M.: Adapting to Unknown Smoothness via Wavelet Shrinkage. Journal of the American Statistical Association, 90(432), 1200–1224 (1995)MATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Gao, H.Y.: Wavelet Shrinkage Denoising Using the Non-Negative Garrote. Journal of Computational and Graphical Statistics, 7(4), 469–488 (1998)CrossRefMathSciNetGoogle Scholar
  12. 12.
    Holschneider, M., Kronland-Martinet, R., Morlet, J., Tchamitchian, P.: A Real-Time Algorithm for Signal Analysis with the Help of the Wavelet Transform. In: Wavelets: Time-Frequency Methods and Phase-space. Springer, Berlin (1989)Google Scholar
  13. 13.
    Johnstone, I.M.: Wavelets and the Theory of Non-parametric Function Estimation. Journal of the Royal Statistical Society, A(357), 2475–2493 (1999)MathSciNetGoogle Scholar
  14. 14.
    Johnstone, I.M., Silverman, B.W.: Wavelet Threshold Estimators for Data with Correlated Noise. Journal of the Royal Statistical Society, Series B, 59(2) 319–351 (1997)MATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Johnstone, I.M., Silverman, B.W.: Empirical Bayes Selection of Wavelet Thresholds. Annals of Statistics, 33(4), 1700–1752 (2005)MATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Kailath, T., Poor, H.V.: Detection of Stochastic Processes. IEEE Transactions on Information Theory, 44(6), 2230–2259 (1998)MATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Kay, S.M.: Fundamentals of Statistical Signal Processing, Volume II, Detection Theory. Prentice-Hall, Upper Saddle River, NJ (1993)MATHGoogle Scholar
  18. 18.
    Lebedev, N.N.: Special Functions and Their Applications, Prentice-Hall, Englewood Cliffs, NJ (1965)MATHGoogle Scholar
  19. 19.
    Lehmann, E.L., Romano, J.P.: Testing Statistical Hypotheses, third edition. Springer, New York (2005)MATHGoogle Scholar
  20. 20.
    Luisier, F., Blu, T., Unser, M.: A New SURE Approach to Image Denoising: Interscale Orthonormal Wavelet Thresholding. IEEE Transactions on Image Processing, 16(3), 593–606 (2007)CrossRefMathSciNetGoogle Scholar
  21. 21.
    Mallat, S.: A Wavelet Tour of Signal Processing, second edition. Academic, New York (1999)MATHGoogle Scholar
  22. 22.
    Minkler, G., Minkler, J.: The Principles of Automatic Radar Detection in Clutter, CFAR. Magellan Book Company, Baltimore, MD (1990)Google Scholar
  23. 23.
    Pastor, D., Amehraye, A.: Algorithms and Applications for Estimating the Standard Deviation of AWGN When Observations Are Not Signal-Free. Journal of Computers, 2(7), 1–10 (2007)CrossRefGoogle Scholar
  24. 24.
    Pastor, D., Gay, R., Groenenboom, A.: A Sharp Upper Bound for the Probability of Error of Likelihood Ratio Test for Detecting Signals in White Gaussian Noise. IEEE Transactions on Information Theory, 48(1), 228–238 (2002)MATHCrossRefMathSciNetGoogle Scholar
  25. 25.
    Pastor, D., Socheleau, F.-X., Aïssa-El-Bey, A.: Sparsity Hypotheses for Robust Estimation of the Noise Standard Deviation in Various Signal Processing Applications. Second International Workshop on Signal Processing with Adaptive Sparse Structured Representations, SPARS’09, Saint-Malo, France (2009)Google Scholar
  26. 26.
    Poor, H.V.: An Introduction to Signal Detection and Estimation, second edition. Springer, New York (1994)MATHGoogle Scholar
  27. 27.
    Portilla, J., Strela, V., Wainwright, M.J., Simoncelli, E.P.: Image Denoising Using Scale Mixtures of Gaussians in the Wavelet Domain. IEEE Transactions on Image Processing, 12(11), 1338–1351 (2003).CrossRefMathSciNetGoogle Scholar
  28. 28.
    Şendur, L., Selesnick, I.V.: Bivariate Shrinkage Functions for Wavelet-Based Denoising Exploiting Interscale Dependency. IEEE Transactions on Signal Processing, 50(11), 2744–2756 (2002)CrossRefGoogle Scholar
  29. 29.
    Serfling, R.J.: Approximation Theorems of Mathematical Statistics. Wiley, New York (1980)MATHCrossRefGoogle Scholar
  30. 30.
    Shensa, M.J.: The Discrete Wavelet Transform: Wedding the à Trous and Mallat Algorithms. IEEE Transactions on Signal Processing, 40(10), 2464–2482 (1992)MATHCrossRefGoogle Scholar
  31. 31.
    Simoncelli, E.P., Adelson, E.H.: Noise Removal via Bayesian Wavelet Coring. IEEE International Conference on Image Processing (ICIP), 379–382 (1996)CrossRefGoogle Scholar
  32. 32.
    Socheleau, F.-X., Pastor, D., Aïssa-El-Bey, A., Houcke, S.: Blind Noise Variance Estimation for OFDMA Signals. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Taipei, Taiwan (2009)Google Scholar
  33. 33.
    Stein, C.: Estimation of the Mean of a Multivariate Normal Distribution. The Annals of Statistics, 9(6), 1135–1151 (1981)MATHCrossRefMathSciNetGoogle Scholar
  34. 34.
    ter Braak, C.J.F.: Bayesian Sigmoid Shrinkage with Improper Variance Priors and an Application to Wavelet Denoising. Computational Statistics and Data Analysis, 51(2), 1232–1242 (2006)MATHCrossRefMathSciNetGoogle Scholar
  35. 35.
    Van-Trees, H.: Detection, Estimation, and Modulation Theory. Part I. Wiley, New York (1968)MATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Institut TELECOM, TELECOM Bretagne, Lab-STICC, CNRS UMR 3192Technopôle Brest-IroiseBrest Cedex 3France
  2. 2.Université Européenne de BretagneRennesFrance

Personalised recommendations