Abstract
The exact constraint A x = b is often relaxed, with an approximated equality measured using the quadratic penalty function \(Q\left(\mathbf{X}\right)=\parallel \mathbf{A}\mathbf{x}-\mathbf{b}{\parallel^{2}_{2}}.\) Such relaxation allows us to (i) define a quasi-solution in case no exact solution exists (even in cases where A has more rows than columns); (ii) exploit ideas from optimization theory; (iii) measure the quality of a candidate solution; and more.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Further Reading
Z. Ben-Haim, Y.C. Eldar, and M. Elad, Coherence-based performance guarantees for estimating a sparse vector under random noise, submitted to IEEE Trans. on Signal Processing, 2009.
S.S. Chen, D.L. Donoho, and M.A. Saunders, Atomic decomposition by basis pursuit, SIAM Journal on Scientific Computing, 20(1):33–61 (1998).
S.S. Chen, D.L. Donoho, and M.A. Saunders, Atomic decomposition by basis pursuit, SIAM Review, 43(1):129–159, 2001.
G. Davis, S. Mallat, and M. Avellaneda, Adaptive greedy approximations, Journal of Constructive Approximation, 13:57–98, 1997.
G. Davis, S. Mallat, and Z. Zhang, Adaptive time-frequency decompositions, Optical-Engineering, 33(7):2183–91, 1994.
D.L. Donoho and M. Elad, On the stability of the basis pursuit in the presence of noise, Signal Processing, 86(3):511–532, March 2006.
D.L. Donoho, M. Elad, and V. Temlyakov, Stable recovery of sparse overcomplete representations in the presence of noise, IEEE Trans. on Information Theory, 52(1):6–18, 2006.
B. Efron, T. Hastie, I.M. Johnstone, and R. Tibshirani, Least angle regression, The Annals of Statistics, 32(2):407–499, 2004.
A.K. Fletcher, S. Rangan, V.K. Goyal, and K. Ramchandran, Analysis of denoising by sparse approximation with random frame asymptotics, IEEE Int. Symp. on Inform. Theory, 2005.
A.K. Fletcher, S. Rangan, V.K. Goyal, and K. Ramchandran, Denoising by sparse approximation: error bounds based on rate-distortion theory, EURASIP Journal on Applied Signal Processing, Paper No. 26318, 2006.
J.J. Fuchs, Recovery of exact sparse representations in the presence of bounded noise, IEEE Trans. on Information Theory, 51(10):3601–3608, 2005.
A.C. Gilbert, S. Muthukrishnan, and M.J. Strauss, Approximation of functions over redundant dictionaries using coherence, 14th Ann. ACM-SIAM Symposium Discrete Algorithms, 2003.
I.F. Gorodnitsky and B.D. Rao, Sparse signal reconstruction from limited data using FOCUSS: A re-weighted norm minimization algorithm, IEEE Trans. On Signal Processing, 45(3):600–616, 1997.
R. Gribonval, R. Figueras, and P. Vandergheynst, A simple test to check the optimality of a sparse signal approximation, Signal Processing, 86(3):496–510, March 2006.
T. Hastie, R. Tibshirani, and J.H. Friedman, Elements of Statistical Learning. New York: Springer, 2001.
L.A. Karlovitz, Construction of nearest points in the ℓ p , p even and ℓ∞ norms, Journal of Approximation Theory, 3:123–127, 1970.
S. Mallat, A Wavelet Tour of Signal Processing, Academic Press, 1998.
M.R. Osborne, B. Presnell, and B.A. Turlach, A new approach to variable selection in least squares problems, IMA J. Numerical Analysis, 20:389–403, 2000.
V.N. Temlyakov, Greedy algorithms and m-term approximation, Journal of Approximation Theory, 98:117–145, 1999.
V.N. Temlyakov, Weak greedy algorithms, Advances in Computational Mathematics, 5:173–187, 2000.
J.A. Tropp, Just relax: Convex programming methods for subset selection and sparse approximation, IEEE Trans. on Information Theory, 52(3):1030–1051, March 2006.
J.A. Tropp, A.C. Gilbert, S. Muthukrishnan, and M.J. Strauss, Improved sparse approximation over quasi-incoherent dictionaries, IEEE International Conference on Image Processing, Barcelona, September 2003.
B. Wohlberg, Noise sensitivity of sparse signal representations: Reconstruction error bounds for the inverse problem. IEEE Trans. on Signal Processing, 51(12):3053–3060, 2003.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2010 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Elad, M. (2010). From Exact to Approximate Solutions. In: Sparse and Redundant Representations. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-7011-4_5
Download citation
DOI: https://doi.org/10.1007/978-1-4419-7011-4_5
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4419-7010-7
Online ISBN: 978-1-4419-7011-4
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)