Skip to main content

From Exact to Approximate Solutions

  • Chapter
  • First Online:
Sparse and Redundant Representations

Abstract

The exact constraint A x = b is often relaxed, with an approximated equality measured using the quadratic penalty function \(Q\left(\mathbf{X}\right)=\parallel \mathbf{A}\mathbf{x}-\mathbf{b}{\parallel^{2}_{2}}.\) Such relaxation allows us to (i) define a quasi-solution in case no exact solution exists (even in cases where A has more rows than columns); (ii) exploit ideas from optimization theory; (iii) measure the quality of a candidate solution; and more.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 49.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 99.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Further Reading

  1. Z. Ben-Haim, Y.C. Eldar, and M. Elad, Coherence-based performance guarantees for estimating a sparse vector under random noise, submitted to IEEE Trans. on Signal Processing, 2009.

    Google Scholar 

  2. S.S. Chen, D.L. Donoho, and M.A. Saunders, Atomic decomposition by basis pursuit, SIAM Journal on Scientific Computing, 20(1):33–61 (1998).

    Article  MathSciNet  Google Scholar 

  3. S.S. Chen, D.L. Donoho, and M.A. Saunders, Atomic decomposition by basis pursuit, SIAM Review, 43(1):129–159, 2001.

    Article  MATH  MathSciNet  Google Scholar 

  4. G. Davis, S. Mallat, and M. Avellaneda, Adaptive greedy approximations, Journal of Constructive Approximation, 13:57–98, 1997.

    MATH  MathSciNet  Google Scholar 

  5. G. Davis, S. Mallat, and Z. Zhang, Adaptive time-frequency decompositions, Optical-Engineering, 33(7):2183–91, 1994.

    Article  Google Scholar 

  6. D.L. Donoho and M. Elad, On the stability of the basis pursuit in the presence of noise, Signal Processing, 86(3):511–532, March 2006.

    Article  MATH  Google Scholar 

  7. D.L. Donoho, M. Elad, and V. Temlyakov, Stable recovery of sparse overcomplete representations in the presence of noise, IEEE Trans. on Information Theory, 52(1):6–18, 2006.

    Article  MathSciNet  Google Scholar 

  8. B. Efron, T. Hastie, I.M. Johnstone, and R. Tibshirani, Least angle regression, The Annals of Statistics, 32(2):407–499, 2004.

    Article  MATH  MathSciNet  Google Scholar 

  9. A.K. Fletcher, S. Rangan, V.K. Goyal, and K. Ramchandran, Analysis of denoising by sparse approximation with random frame asymptotics, IEEE Int. Symp. on Inform. Theory, 2005.

    Google Scholar 

  10. A.K. Fletcher, S. Rangan, V.K. Goyal, and K. Ramchandran, Denoising by sparse approximation: error bounds based on rate-distortion theory, EURASIP Journal on Applied Signal Processing, Paper No. 26318, 2006.

    Google Scholar 

  11. J.J. Fuchs, Recovery of exact sparse representations in the presence of bounded noise, IEEE Trans. on Information Theory, 51(10):3601–3608, 2005.

    Article  Google Scholar 

  12. A.C. Gilbert, S. Muthukrishnan, and M.J. Strauss, Approximation of functions over redundant dictionaries using coherence, 14th Ann. ACM-SIAM Symposium Discrete Algorithms, 2003.

    Google Scholar 

  13. I.F. Gorodnitsky and B.D. Rao, Sparse signal reconstruction from limited data using FOCUSS: A re-weighted norm minimization algorithm, IEEE Trans. On Signal Processing, 45(3):600–616, 1997.

    Article  Google Scholar 

  14. R. Gribonval, R. Figueras, and P. Vandergheynst, A simple test to check the optimality of a sparse signal approximation, Signal Processing, 86(3):496–510, March 2006.

    Article  MATH  Google Scholar 

  15. T. Hastie, R. Tibshirani, and J.H. Friedman, Elements of Statistical Learning. New York: Springer, 2001.

    MATH  Google Scholar 

  16. L.A. Karlovitz, Construction of nearest points in the ℓ p , p even and ℓ norms, Journal of Approximation Theory, 3:123–127, 1970.

    Article  MATH  MathSciNet  Google Scholar 

  17. S. Mallat, A Wavelet Tour of Signal Processing, Academic Press, 1998.

    MATH  Google Scholar 

  18. M.R. Osborne, B. Presnell, and B.A. Turlach, A new approach to variable selection in least squares problems, IMA J. Numerical Analysis, 20:389–403, 2000.

    Article  MATH  MathSciNet  Google Scholar 

  19. V.N. Temlyakov, Greedy algorithms and m-term approximation, Journal of Approximation Theory, 98:117–145, 1999.

    Article  MATH  MathSciNet  Google Scholar 

  20. V.N. Temlyakov, Weak greedy algorithms, Advances in Computational Mathematics, 5:173–187, 2000.

    MathSciNet  Google Scholar 

  21. J.A. Tropp, Just relax: Convex programming methods for subset selection and sparse approximation, IEEE Trans. on Information Theory, 52(3):1030–1051, March 2006.

    Article  MathSciNet  Google Scholar 

  22. J.A. Tropp, A.C. Gilbert, S. Muthukrishnan, and M.J. Strauss, Improved sparse approximation over quasi-incoherent dictionaries, IEEE International Conference on Image Processing, Barcelona, September 2003.

    Google Scholar 

  23. B. Wohlberg, Noise sensitivity of sparse signal representations: Reconstruction error bounds for the inverse problem. IEEE Trans. on Signal Processing, 51(12):3053–3060, 2003.

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Michael Elad .

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Elad, M. (2010). From Exact to Approximate Solutions. In: Sparse and Redundant Representations. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-7011-4_5

Download citation

Publish with us

Policies and ethics