Bayesian Denoising of Visual Images in the Wavelet Domain

  • Eero P. Simoncelli
Part of the Lecture Notes in Statistics book series (LNS, volume 141)


The use of multi-scale decompositions has led to significant advances in representation, compression, restoration, analysis, and synthesis of signals. The fundamental reason for these advances is that the statistics of many natural signals, when decomposed in such bases, are substantially simplified. Choosing a basis that is adapted to statistical properties of the input signal is a classical problem. The traditional solution is principal components analysis (PCA), in which a linear decomposition is chosen to diagonalize the covariance structure of the input. The most well-known description of image statistics is that their Fourier spectra take the form of a power law [e.g., 1, 2, 3]. Coupled with a constraint of translation-invariance, this suggests that the Fourier transform is an appropriate PCA representation. Fourier and related representations are widely used in image processing applications. For example, the classical solution to the noise removal problem is the Wiener filter, which can be derived by assuming a signal model of decorrelated Gaussian-distributed coefficients in the Fourier domain.

Recently a number of authors have noted that statistics of order greater than two can be utilized in choosing a basis for images. Field [2, 4] noted that the coefficients of frequency subbands of natural scenes have much higher kurtosis than a Gaussian density. Recent work on so-called “independent components analysis” (ICA) has sought linear bases that optimize higher-order statistical measures [e.g., 5, 6]. Several authors have constructed optimal bases for images by optimizing such information-theoretic criterion [7, 8]. The resulting basis functions are oriented and have roughly octave bandwidth, similar to many of the most common multi-scale decompositions. A number of authors have explored the optimal choice of a basis from a library of functions based on entropy or other statistical criterion [e.g. 9, 10, 11, 12, 13].

In this chapter, we examine the empirical statistical properties of visual images within two fixed multi-scale bases, and describe two statistical models for the coefficients in these bases. The first is a non-Gaussian marginal model, previously described in [14]. The second is a joint non-Gaussian Markov model for wavelet subbands, previous versions of which have been described in [15, 16]. We demonstrate the use of each of these models in Bayesian estimation of an image contaminated by additive Gaussian white noise.


Independent Component Analysis Coarse Scale Wavelet Domain Marginal Model Orthonormal Wavelet 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Alex Pentland. Fractal based description of natural scenes. IEEE Trans. PAMI, 6(6):661–674, 1984.CrossRefGoogle Scholar
  2. [2]
    D J Field. Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A, 4(12):2379–2394, 1987.CrossRefGoogle Scholar
  3. [3]
    D L Ruderman and W Bialek. Statistics of natural images: Scaling in the woods. Phys. Rev. Letters, 73(6), 1994.Google Scholar
  4. [4]
    D J Field. What is the goal of sensory coding? Neural Computation, 6:559–601, 1994.CrossRefGoogle Scholar
  5. [5]
    J F Cardoso. Source separation using higer order moments. In ICASSP, pages 2109-2112, 1989.Google Scholar
  6. [6]
    P Common. Independent component analysis, a new concept? Signal Process., 36:314–387, 1994.CrossRefGoogle Scholar
  7. [7]
    B A Olshausen and D J Field. Natural image statistics and efficient coding. Network: Computation in Neural Systems, 7:333–339, 1996.CrossRefGoogle Scholar
  8. [8]
    A J Bell and T J Sejnowski. Learning the higher-order structure of a natural sound. Network: Computation in Neural Systems, 7:261–266, 1996.MATHCrossRefGoogle Scholar
  9. [9]
    R R Coifman and M V Wickerhauser. Entropy-based algorithms for best basis selection. IEEE Trans. Info. Theory, IT-38:713–718, March 1992.CrossRefGoogle Scholar
  10. [10]
    Stephane Mallat and Zhifeng Zhang. Matching pursuits with time-frequency dictionaries. IEEE Trans. Signal Proc, December 1993.Google Scholar
  11. [11]
    D L Donoho and I M Johnstone. Ideal denoising in an orthogonal basis chosen from a library of bases. C.R. Acad. Sci., 319:1317–1322, 1994.MathSciNetMATHGoogle Scholar
  12. [12]
    M V Wickerhauser. Adapted Wavelet Analysis: From Theory to Software. A K Peters, Wellesley, MA, 1994.MATHGoogle Scholar
  13. [13]
    J C Pesquet, H Krim, D Leporini, and E Hamman. Bayesian approach to best basis selection. In Proc Int’l Conf Acoustics, Speech and Signal Proc, pages 2634-2638, Atlanta, May 1996.Google Scholar
  14. [14]
    E P Simoncelli and E H Adelson. Noise removal via Bayesian wavelet coring. In Third Int’l Conf on Image Proc, volume I, pages 379-382, Lausanne, September 1996. IEEE Sig Proc Society.Google Scholar
  15. [15]
    R W Buccigrossi and E P Simoncelli. Image compression via joint statistical characterization in the wavelet domain. Technical Report 414, GRASP Laboratory, University of Pennsylvania, May 1997. Accepted (3/99) for publication in IEEE Trans Image Processing.Google Scholar
  16. [16]
    E P Simoncelli. Statistical models for images: Compression, restoration and synthesis. In 31st Asilomar Conf on Signals, Systems and Computers, pages 673-678, Pacific Grove, CA, November 1997. IEEE Computer Society. Available at: Scholar
  17. [17]
    S G Mallat. A theory for multiresolution signal decomposition: The wavelet representation. IEEE Pat Anal Mach. Intell., 11:674–693, July 1989.MATHCrossRefGoogle Scholar
  18. [18]
    D Donoho and I Johnstone. Adapting to unknown smootness via wavelet shrinkage. J American Stat Assoc, 90(432), December 1995.Google Scholar
  19. [19]
    F Abramovich, T Sapatinas, and B W Silverman. Wavelet thresholding via a bayesian approach. J R Stat Soc B, 60:725–749, 1998.MathSciNetMATHCrossRefGoogle Scholar
  20. [20]
    D Leporini and J C Pesquet. Multiscale regularization in besov spaces. In 31st Asilomar Conf on Signals, Systems and Computers, Pacific Grove, CA, November 1998.Google Scholar
  21. [21]
    J P Rossi. JSMPTE, 87:134–140, 1978.Google Scholar
  22. [22]
    B. E. Bayer and P. G. Powell. A method for the digital enhancement of unsharp, grainy photographic images. Adv in Computer Vision and Im Proc, 2:31–88, 1986.Google Scholar
  23. [23]
    J. M. Ogden and E. H. Adelson. Computer simulations of oriented multiple spatial frequency band coring. Technical Report PRRL-85-TR-012, RCA David Sarnoff Research Center, April 1985.Google Scholar
  24. [24]
    E Simoncelli and J Portilla. Texture characterization via joint statistics of wavelet coefficient magnitudes. In Fifth IEEE Int’l Conf on Image Proc, volume I, Chicago, October 4-7 1998. IEEE Computer Society.Google Scholar
  25. [25]
    E P Simoncelli and E H Adelson. Subband transforms. In John W Woods, editor, Subband Image Coding, chapter 4, pages 143–192. Kluwer Academic Publishers, Norwell, MA, 1990.Google Scholar
  26. [26]
    E P Simoncelli, W T Freeman, E H Adelson, and D J Heeger. Shiftable multi-scale transforms. IEEE Trans Information Theory, 38(2):587–607, March 1992. Special Issue on Wavelets.MathSciNetCrossRefGoogle Scholar
  27. [27]
    R R Coifman and D L Donoho. Translation-invariant de-noising. Technical Report 475, Statistics Department, Stanford University, May 1995.Google Scholar

Copyright information

© Springer Science+Business Media New York 1999

Authors and Affiliations

  • Eero P. Simoncelli

There are no affiliations available

Personalised recommendations