Skip to main content

Pursuit Algorithms – Practice

  • Chapter
  • First Online:
Sparse and Redundant Representations

Abstract

It is now time to consider reliable and effcient methods for solving (P 0), as a straightforward approach seems hopeless.We now discuss methods which, it seems, have no hope of working – but which, under specific conditions, will work. Looking at the problem (P 0),

$${\left(P_o\right):\quad \min\limits_X \parallel \mathbf{X}\parallel_0 \,{\rm subject\,\, to} \mathbf \quad \mathbf{b}=\mathbf{A\mathbf{x}}},$$

one observes that the unknown X is composed of two effective parts to be found – the support of the solution, and the non-zero values over this support. Thus, one way to attack the numerical solution of (P 0) is to focus on the support, with the understanding that once found, the non-zero values of X are easily detected by plain Least-Squares. As the support is discrete in nature, algorithms that seek it are discrete as well. This line of reasoning leads to the family of greedy algorithms that will be presented hereafter.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 49.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 99.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Further Reading

  1. S.S. Chen, D.L. Donoho, and M.A. Saunders, Atomic decomposition by basis pursuit, SIAM Journal on Scientific Computing, 20(1):33–61 (1998).

    Article  MathSciNet  Google Scholar 

  2. S.S. Chen, D.L. Donoho, and M.A. Saunders, Atomic decomposition by basis pursuit, SIAM Review, 43(1):129–159, 2001.

    Article  MATH  MathSciNet  Google Scholar 

  3. A. Cohen, R.A. DeVore, P. Petrushev, and H. Xu, Nonlinear approximation and the space BV(ℝ2), American Journal of Mathematics, 121(3):587–628, June, 1999.

    Article  MATH  MathSciNet  Google Scholar 

  4. G. Davis, S. Mallat, and M. Avellaneda, Adaptive greedy approximations, Journal of Constructive Approximation, 13:57–98, 1997.

    MATH  MathSciNet  Google Scholar 

  5. G. Davis, S. Mallat, and Z. Zhang, Adaptive time-frequency decompositions, Optical-Engineering, 33(7):2183–91, 1994.

    Article  Google Scholar 

  6. R.A. DeVore and V. Temlyakov, Some Remarks on Greedy Algorithms, Advances in Computational Mathematics, 5:173–187, 1996.

    Article  MATH  MathSciNet  Google Scholar 

  7. I.F. Gorodnitsky and B.D. Rao, Sparse signal reconstruction from limited data using FOCUSS: A re-weighted norm minimization algorithm, IEEE Trans. On Signal Processing, 45(3):600–616, 1997.

    Article  Google Scholar 

  8. R. Gribonval and P. Vandergheynst, On the exponential convergence of Matching Pursuits in quasi-incoherent dictionaries, IEEE Trans. Information Theory, 52(1):255–261, 2006.

    Article  MathSciNet  Google Scholar 

  9. L.A. Karlovitz, Construction of nearest points in the ℓ p , p even and ℓ norms, Journal of Approximation Theory, 3:123–127, 1970.

    Article  MATH  MathSciNet  Google Scholar 

  10. S. Mallat, A Wavelet Tour of Signal Processing, Academic-Press, 1998.

    MATH  Google Scholar 

  11. S. Mallat and Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Trans. Signal Processing, 41(12):3397–3415, 1993.

    Article  MATH  Google Scholar 

  12. D. Needell and J.A. Tropp, CoSaMP: Iterative signal recovery from incomplete and inaccurate samples, Applied Computational Harmonic Analysis, 26:301– 321, May 2009.

    Article  MATH  MathSciNet  Google Scholar 

  13. B.D. Rao and K. Kreutz-Delgado, An a_ne scaling methodology for best basis selection, IEEE Trans. on signal processing, 47(1):187–200, 1999.

    Article  MATH  MathSciNet  Google Scholar 

  14. V.N. Temlyakov, Greedy algorithms and m-term approximation, Journal of Approximation Theory, 98:117–145, 1999.

    Article  MATH  MathSciNet  Google Scholar 

  15. V.N. Temlyakov, Weak greedy algorithms, Advances in Computational Mathematics, 5:173–187, 2000.

    MathSciNet  Google Scholar 

  16. J.A. Tropp, Greed is good: Algorithmic results for sparse approximation, IEEE Trans. On Information Theory, 50(10):2231–2242, October 2004.

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Michael Elad .

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Elad, M. (2010). Pursuit Algorithms – Practice. In: Sparse and Redundant Representations. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-7011-4_3

Download citation

Publish with us

Policies and ethics