Skip to main content

Iterative-Shrinkage Algorithms

  • Chapter
  • First Online:
Sparse and Redundant Representations

Abstract

In this chapter, our goal is the minimization of a function of the form

$$f\left({\mathbf{x}}\right)=\lambda \mathbf{1}^T \rho{\left({\mathbf{x}}\right)}+\frac{1}{2}\parallel \mathbf{b}-\mathbf{A}\mathbf{x}{\parallel^{2}_{2}},$$

which we have seen before in several forms. The function ρ(x) operates entry-wise on the vector x. As an example, for ρ(x) = |x|p we get that \(\mathbf{1}^T \rho\left(\mathbf{X}\right)=\parallel \mathbf{X}{\parallel^{p}_{p}},\) which gives us the freedom to choose any P we find as fit. In this chapter we shall keep the discussion general, and allow any “sparsity-promoting” function ρ(·). In particular, the function posed above is a generalized version of what we have defined as (Q λ 1).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 49.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 99.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Further Reading

  1. S. Becker, J. Bobin and E. Candes, NESTA: A fast and accurate first-order method for sparse recovery, Preprint, 2009.

    Google Scholar 

  2. J. Bioucas-Dias, Bayesian wavelet-based image deconvolution: a GEM algorithm exploiting a class of heavy-tailed priors, IEEE Trans. on Image processing, 15(4):937–951, April 2006.

    Article  MathSciNet  Google Scholar 

  3. T. Blumensath and M.E. Davies, Iterative thresholding for sparse approximations, Journal of Fourier Analysis and Applications, 14(5):629–654, 2008.

    Article  MATH  MathSciNet  Google Scholar 

  4. J. Bobin, Y. Moudden, J.-L. Starck, and M. Elad, Morphological diversity and source separation, IEEE Signal Processing Letters, 13(7):409–412, July 2006.

    Article  Google Scholar 

  5. J. Cai, S. Osher, and Z. Shen, Convergence of the linearized Bregman iteration for l1-norm minimization, Mathematics of Computation, 78:2127–2136, 2009.

    Article  MathSciNet  Google Scholar 

  6. J. Cai, S. Osher, and Z. Shen, Linearized Bregman iterations for compressed sensing, Mathematics of Computation, 78:1515–1536, 2009.

    Article  MathSciNet  Google Scholar 

  7. J. Cai, S. Osher, and Z. Shen, Linearized Bregman iteration for frame based image deblurring, SIAM Journal on Imaging Sciences, 2(1):226–252, 2009.

    Article  MATH  MathSciNet  Google Scholar 

  8. P.L. Combettes and V.R. Wajs, Signal recovery by proximal forward-backward splitting, Multiscale Modeling and Simulation, 4(4):1168–1200, November 2005.

    Article  MATH  MathSciNet  Google Scholar 

  9. I. Daubechies, M. Defrise, and C. De-Mol, An iterative thresholding algorithm for linear inverse problems with a sparsity constraint, Communications on Pure and Applied Mathematics, LVII:1413–1457, 2004.

    Article  MathSciNet  Google Scholar 

  10. D.L. Donoho, De-noising by soft thresholding, IEEE Trans. on Information Theory, 41(3):613–627, 1995.

    Article  MATH  MathSciNet  Google Scholar 

  11. D.L. Donoho and I.M. Johnstone, Ideal spatial adaptation by wavelet shrinkage, Biometrika, 81(3):425–455, 1994.

    Article  MATH  MathSciNet  Google Scholar 

  12. D.L. Donoho, I.M. Johnstone, G. Kerkyacharian, and D. Picard,Wavelet shrinkage - asymptopia, Journal of the Royal Statistical Society Series B - Methodological, 57(2):301–337, 1995.

    MATH  MathSciNet  Google Scholar 

  13. D.L. Donoho and I.M. Johnstone, Minimax estimation via wavelet shrinkage, Annals of Statistics, 26(3):879–921, 1998.

    Article  MATH  MathSciNet  Google Scholar 

  14. D.L. Donoho, Y. Tsaig, I. Drori, and J.-L. Starck, Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit, Technical Report, Stanford University, 2006.

    Google Scholar 

  15. M. Elad, Why simple shrinkage is still relevant for redundant representations?, to appear in the IEEE Trans. on Information Theory.

    Google Scholar 

  16. M. Elad, B. Matalon, and M. Zibulevsky, Image denoising with shrinkage and redundant representations, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), NY, June 17–22, 2006.

    Google Scholar 

  17. M. Elad, B. Matalon, and M. Zibulevsky, On a Class of Optimization methods for Linear Least Squares with Non-Quadratic Regularization, to appear in Applied and Computational Harmonic Analysis.

    Google Scholar 

  18. M. Elad, J.-L. Starck, P. Querre, and D.L. Donoho, Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA), Journal on Applied and Comp. Harmonic Analysis, 19:340–358, 2005.

    Article  MATH  MathSciNet  Google Scholar 

  19. M.J. Fadili, J.-L. Starck, Sparse representation-based image deconvolution by iterative thresholding, Astronomical Data Analysis ADA’06, Marseille, France, September, 2006.

    Google Scholar 

  20. M.A. Figueiredo and R.D. Nowak, An EM algorithm for wavelet-based image restoration, IEEE Trans. Image Processing, 12(8):906–916, 2003.

    Article  MathSciNet  Google Scholar 

  21. M.A. Figueiredo, and R.D. Nowak, A bound optimization approach to waveletbased image deconvolution, IEEE International Conference on Image Processing - ICIP 2005, Genoa, Italy, 2:782–785, September 2005.

    Google Scholar 

  22. M.A. Figueiredo, J.M. Bioucas-Dias, and R.D. Nowak, Majorization-minimization algorithms for wavelet-based image restoration, to appear in the IEEE Trans. On Image Processing.

    Google Scholar 

  23. D. Needell and J. A. Tropp, CoSaMP: Iterative signal recovery from incomplete and inaccurate samples. to appear in Appl. Comp. Harmonic Anal..

    Google Scholar 

  24. Y. Nesterov, Smooth minimization of non-smooth functions, Math. Program., Serie A., 103:127–152, 2007.

    Article  MathSciNet  Google Scholar 

  25. S. Sardy, A.G. Bruce, and P. Tseng, Block coordinate relaxation methods for nonparametric signal denoising with wavelet dictionaries, Journal of computational and Graphical Statistics, 9:361–379, 2000.

    Article  MathSciNet  Google Scholar 

  26. J.-L. Starck, M. Elad, and D.L. Donoho, Image decomposition via the combination of sparse representations and a variational approach. IEEE Trans. on Image Processing, 14(10):1570–1582, 2005.

    Article  MathSciNet  Google Scholar 

  27. J.-L. Starck, F. Murtagh, and A. Bijaoui, Multiresolution support applied to image filtering and restoration, Graphical Models and Image Processing, 57:420–431, 1995.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Michael Elad .

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Elad, M. (2010). Iterative-Shrinkage Algorithms. In: Sparse and Redundant Representations. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-7011-4_6

Download citation

Publish with us

Policies and ethics