Skip to main content

Pursuit Algorithms – Guarantees

  • Chapter
  • First Online:
Sparse and Redundant Representations

Abstract

Assume that the linear system A x = b has a sparse solution with k0 non-zeros, i.e., \(\parallel \mathbf{X}\parallel_{0}=k_0.\) Furthermore, assume that k 0 < spark(A)/2. Will matching pursuit or Basis-Pursuit succeed in recovering the sparsest solution? Clearly, such success cannot be expected for all k 0 and for all matrices A, since this would conflict with the known NP-hardness of the problem in the general case. However, if the equation actually has a "suffciently sparse" solution, the success of these algorithms in addressing the original objective (P 0) can be guaranteed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 49.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 99.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Further Reading

  1. J.H. Ahlberg and E.N. Nilson, Convergence properties of the spline fit, J. SIAM, 11:95–104, 1963.

    MATH  MathSciNet  Google Scholar 

  2. S.S. Chen, D.L. Donoho, and M.A. Saunders, Atomic decomposition by basis pursuit, SIAM Journal on Scientific Computing, 20(1):33–61 (1998).

    Article  MathSciNet  Google Scholar 

  3. S.S. Chen, D.L. Donoho, and M.A. Saunders, Atomic decomposition by basis pursuit, SIAM Review, 43(1):129–159, 2001.

    Article  MATH  MathSciNet  Google Scholar 

  4. C. Couvreur and Y. Bresler, On the optimality of the Backward Greedy Algorithm for the subset selection problem, SIAM Journal on Matrix Analysis and Applications, 21(3):797–808, 2000.

    Article  MATH  MathSciNet  Google Scholar 

  5. D.L. Donoho and M. Elad, Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization, Proc. of the National Academy of Sciences, 100(5):219227–02, 2003.

    Article  MathSciNet  Google Scholar 

  6. D.L. Donoho and X. Huo, Uncertainty principles and ideal atomic decomposition, IEEE Trans. on Information Theory, 47(7):2845–2862, 1999.

    Article  MathSciNet  Google Scholar 

  7. D.L. Donoho and J. Tanner, Sparse nonnegative solutions of underdetermined linear equations by linear programming, Proceedings of the National Academy of Sciences, 102(27):9446–9451, July 2005.

    Article  MathSciNet  Google Scholar 

  8. D.L. Donoho and J. Tanner, Neighborliness of randomly-projected Simplices in high dimensions, Proceedings of the National Academy of Sciences, 102(27):9452– 9457, July 2005.

    Google Scholar 

  9. M. Elad and A.M. Bruckstein, A generalized uncertainty principle and sparse representation in pairs of bases, IEEE Trans. on Information Theory, 48:2558– 2567, 2002.

    Article  MATH  MathSciNet  Google Scholar 

  10. A. Feuer and A. Nemirovsky, On sparse representation in pairs of bases, IEEE Trans. On Information Theory, 49:1579–1581, June 2002.

    Article  Google Scholar 

  11. J.J. Fuchs, On sparse representations in arbitrary redundant bases, IEEE Trans. On Information Theory, 50:1341–1344, 2004.

    Article  Google Scholar 

  12. R. Gribonval and M. Nielsen, Sparse decompositions in unions of bases, IEEE Trans. On Information Theory, 49(12):3320–3325, 2003.

    Article  MathSciNet  Google Scholar 

  13. X. Huo, Sparse Image representation Via Combined Transforms, PhD thesis, Stanford, 1999.

    Google Scholar 

  14. S. Mallat and Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Trans. Signal Processing, 41(12):3397–3415, 1993.

    Article  MATH  Google Scholar 

  15. N. Morača, Bounds for norms of the matrix inverse and the smallest singular value, Linear Algebra and its Applications, 429:2589–2601, 2008.

    Article  MATH  MathSciNet  Google Scholar 

  16. J.A. Tropp, Greed is good: Algorithmic results for sparse approximation, IEEE Trans. On Information Theory, 50(10):2231–2242, October 2004.

    Article  MathSciNet  Google Scholar 

  17. J.M. Varah, A lower bound for the smallest singular value of a matrix, Linear Algebra Appl. 11:3–5, 1975.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Michael Elad .

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Elad, M. (2010). Pursuit Algorithms – Guarantees. In: Sparse and Redundant Representations. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-7011-4_4

Download citation

Publish with us

Policies and ethics