Skip to main content
SpringerLink
Log in

Algorithms for ℓ1-Minimization

  • Chapter
  • First Online:
A Mathematical Introduction to Compressive Sensing

Part of the book series: Applied and Numerical Harmonic Analysis ((ANHA))

  • 14k Accesses

Abstract

This chapter presents a selection of three algorithms designed specifically to compute solutions of ℓ 1-minimization problems. The algorithms, chosen with simplicity of analysis and diversity of techniques in mind, are the homotopy method, Chambolle and Pock’s primal–dual algorithm, and the iteratively reweighted least squares algorithm. Other algorithms are also mentioned but discussed in less detail.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 49.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 89.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. M. Afonso, J. Bioucas Dias, M. Figueiredo, Fast image recovery using variable splitting and constrained optimization. IEEE Trans. Image Process. 19(9), 2345–2356 (2010)

    Article  MathSciNet  Google Scholar 

  2. K. Arrow, L. Hurwicz, H. Uzawa, Studies in Linear and Non-linear Programming (Stanford University Press, Stanford, California, 1958)

    MATH  Google Scholar 

  3. S. Bartels, Total variation minimization with finite elements: convergence and iterative solution. SIAM J. Numer. Anal. 50(3), 1162–1180 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  4. H. Bauschke, P. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Spaces. CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC (Springer, New York, 2011)

    Google Scholar 

  5. A. Beck, M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  6. A. Beck, M. Teboulle, Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans. Image Process. 18(11), 2419–2434 (2009)

    Article  MathSciNet  Google Scholar 

  7. S. Becker, J. Bobin, E.J. Candès, NESTA: A fast and accurate first-order method for sparse recovery. SIAM J. Imaging Sci. 4(1), 1–39 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  8. D. Bertsekas, J. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods (Athena Scientific, Cambridge, MA, 1997)

    Google Scholar 

  9. E. Birgin, J. Martínez, M. Raydan, Inexact spectral projected gradient methods on convex sets. IMA J. Numer. Anal. 23(4), 539–559 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  10. S. Boyd, L. Vandenberghe, Convex Optimization (Cambridge University Press, Cambridge, 2004)

    Book  MATH  Google Scholar 

  11. K. Bredies, D. Lorenz, Linear convergence of iterative soft-thresholding. J. Fourier Anal. Appl. 14(5–6), 813–837 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  12. M. Burger, M. Moeller, M. Benning, S. Osher, An adaptive inverse scale space method for compressed sensing. Math. Comp. 82(281), 269–299 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  13. A. Chambolle, T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imag. Vis. 40, 120–145 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  14. R. Chartrand, Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Process. Lett. 14(10), 707–710 (2007)

    Article  Google Scholar 

  15. R. Chartrand, V. Staneva, Restricted isometry properties and nonconvex compressive sensing. Inverse Probl. 24(3), 1–14 (2008)

    Article  MathSciNet  Google Scholar 

  16. R. Chartrand, W. Yin, Iteratively reweighted algorithms for compressive sensing. In 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP, Las Vegas, Nevada, USA, pp. 3869–3872, 2008

    Google Scholar 

  17. G. Chen, R. Rockafellar, Convergence rates in forward-backward splitting. SIAM J. Optim. 7(2), 421–444 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  18. A. Cline, Rate of convergence of Lawson’s algorithm. Math. Commun. 26, 167–176 (1972)

    MathSciNet  MATH  Google Scholar 

  19. P. Combettes, J.-C. Pesquet, A Douglas-Rachford Splitting Approach to Nonsmooth Convex Variational Signal Recovery. IEEE J. Sel. Top. Signal Process. 1(4), 564–574 (2007)

    Article  Google Scholar 

  20. P. Combettes, J.-C. Pesquet, Proximal splitting methods in signal processing. In Fixed-Point Algorithms for Inverse Problems in Science and Engineering, ed. by H. Bauschke, R. Burachik, P. Combettes, V. Elser, D. Luke, H. Wolkowicz (Springer, New York, 2011), pp. 185–212

    Google Scholar 

  21. P. Combettes, V. Wajs, Signal recovery by proximal forward-backward splitting. Multiscale Model. Sim. 4(4), 1168–1200 (electronic) (2005)

    Google Scholar 

  22. I. Daubechies, M. Defrise, C. De Mol, An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Comm. Pure Appl. Math. 57(11), 1413–1457 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  23. I. Daubechies, R. DeVore, M. Fornasier, C. Güntürk, Iteratively re-weighted least squares minimization for sparse recovery. Comm. Pure Appl. Math. 63(1), 1–38 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  24. I. Daubechies, M. Fornasier, I. Loris, Accelerated projected gradient methods for linear inverse problems with sparsity constraints. J. Fourier Anal. Appl. 14(5–6), 764–792 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  25. D.L. Donoho, Y. Tsaig, Fast solution of l1-norm minimization problems when the solution may be sparse. IEEE Trans. Inform. Theor. 54(11), 4789–4812 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  26. J. Douglas, H. Rachford, On the numerical solution of heat conduction problems in two or three space variables. Trans. Am. Math. Soc. 82, 421–439 (1956)

    Article  MathSciNet  MATH  Google Scholar 

  27. B. Efron, T. Hastie, I. Johnstone, R. Tibshirani, Least angle regression. Ann. Stat. 32(2), 407–499 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  28. H.W. Engl, M. Hanke, A. Neubauer, Regularization of Inverse Problems (Springer, New York, 1996)

    Book  MATH  Google Scholar 

  29. E. Esser, Applications of Lagrangian-based alternating direction methods and connections to split Bregman. Preprint (2009)

    Google Scholar 

  30. M.A. Figueiredo, R.D. Nowak, An EM algorithm for wavelet-based image restoration. IEEE Trans. Image Process. 12(8), 906–916 (2003)

    Article  MathSciNet  Google Scholar 

  31. M.A. Figueiredo, R.D. Nowak, S. Wright, Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Proces. 1(4), 586–598 (2007)

    Article  Google Scholar 

  32. M. Fornasier, Numerical methods for sparse recovery. In Theoretical Foundations and Numerical Methods for Sparse Recovery, ed. by M. Fornasier. Radon Series on Computational and Applied Mathematics, vol. 9 (de Gruyter, Berlin, 2010), pp. 93–200

    Google Scholar 

  33. M. Fornasier, H. Rauhut, Iterative thresholding algorithms. Appl. Comput. Harmon. Anal. 25(2), 187–208 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  34. M. Fornasier, H. Rauhut, Recovery algorithms for vector valued data with joint sparsity constraints. SIAM J. Numer. Anal. 46(2), 577–613 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  35. M. Fornasier, H. Rauhut, R. Ward, Low-rank matrix recovery via iteratively reweighted least squares minimization. SIAM J. Optim. 21(4), 1614–1640 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  36. D. Gabay, Applications of the method of multipliers to variational inequalities. In Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, ed. by M. Fortin, R. Glowinski (North-Holland, Amsterdam, 1983), pp. 299–331

    Chapter  Google Scholar 

  37. D. Gabay, B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite elements approximations. Comput. Math. Appl. 2, 17–40 (1976)

    Article  MATH  Google Scholar 

  38. R. Glowinski, T. Le, Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics (SIAM, Philadelphia, 1989)

    Book  MATH  Google Scholar 

  39. D. Goldfarb, S. Ma, Convergence of fixed point continuation algorithms for matrix rank minimization. Found. Comput. Math. 11(2), 183–210 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  40. T. Goldstein, S. Osher, The split Bregman method for L1-regularized problems. SIAM J. Imag. Sci. 2(2), 323–343 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  41. E. Hale, W. Yin, Y. Zhang, Fixed-point continuation for ℓ 1-minimization: methodology and convergence. SIAM J. Optim. 19(3), 1107–1130 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  42. E. Hale, W. Yin, Y. Zhang, Fixed-point continuation applied to compressed sensing: implementation and numerical experiments. J. Comput. Math. 28(2), 170–194 (2010)

    MathSciNet  MATH  Google Scholar 

  43. B. He, X. Yuan, Convergence analysis of primal-dual algorithms for a saddle-point problem: from contraction perspective. SIAM J. Imag. Sci. 5(1), 119–149 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  44. S. Kim, K. Koh, M. Lustig, S. Boyd, D. Gorinevsky, A method for large-scale l1-regularized least squares problems with applications in signal processing and statistics. IEEE J. Sel. Top. Signal Proces. 4(1), 606–617 (2007)

    Article  Google Scholar 

  45. J. King, A minimal error conjugate gradient method for ill-posed problems. J. Optim. Theor. Appl. 60(2), 297–304 (1989)

    Article  MATH  Google Scholar 

  46. C. Lawson, Contributions to the Theory of Linear Least Maximum Approximation. PhD thesis, University of California, Los Angeles, 1961

    Google Scholar 

  47. J. Lee, Y. Sun, M. Saunders, Proximal Newton-type methods for convex optimization. Preprint (2012)

    Google Scholar 

  48. B. Lemaire, The proximal algorithm. In New Methods in Optimization and Their Industrial Uses (Pau/Paris, 1987), Internationale Schriftenreihe Numerischen Mathematik, vol. 87 (Birkhäuser, Basel, 1989), pp. 73–87

    Google Scholar 

  49. P.-L. Lions, B. Mercier, Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16, 964–979 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  50. D. Lorenz, M. Pfetsch, A. Tillmann, Solving Basis Pursuit: Heuristic optimality check and solver comparison. Preprint (2011)

    Google Scholar 

  51. I. Loris, L1Packv2: a Mathematica package in minimizing an ℓ 1-penalized functional. Comput. Phys. Comm. 179(12), 895–902 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  52. B. Martinet, Régularisation d’inéquations variationnelles par approximations successives. Rev. Française Informat. Recherche Opérationnelle 4(Ser. R-3), 154–158 (1970)

    Google Scholar 

  53. A. Nemirovsky, D. Yudin, Problem Complexity and Method Efficiency in Optimization. A Wiley-Interscience Publication. (Wiley, New York, 1983)

    Google Scholar 

  54. Y. Nesterov, A method for solving the convex programming problem with convergence rate O(1 ∕ k 2). Dokl. Akad. Nauk SSSR 269(3), 543–547 (1983)

    MathSciNet  Google Scholar 

  55. Y. Nesterov, Smooth minimization of non-smooth functions. Math. Program. 103(1, Ser. A), 127–152 (2005)

    Google Scholar 

  56. J. Nocedal, S. Wright, Numerical Optimization, 2nd edn. Springer Series in Operations Research and Financial Engineering (Springer, New York, 2006)

    Google Scholar 

  57. M. Osborne, Finite Algorithms in Optimization and Data Analysis (Wiley, New York, 1985)

    MATH  Google Scholar 

  58. M. Osborne, B. Presnell, B. Turlach, A new approach to variable selection in least squares problems. IMA J. Numer. Anal. 20(3), 389–403 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  59. M. Osborne, B. Presnell, B. Turlach, On the LASSO and its dual. J. Comput. Graph. Stat. 9(2), 319–337 (2000)

    MathSciNet  Google Scholar 

  60. T. Pock, A. Chambolle, Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In IEEE International Conference Computer Vision (ICCV) 2011, pp. 1762 –1769, November 2011

    Google Scholar 

  61. T. Pock, D. Cremers, H. Bischof, A. Chambolle, An algorithm for minimizing the Mumford-Shah functional. In ICCV Proceedings (Springer, Berlin, 2009)

    Google Scholar 

  62. H. Raguet, J. Fadili, G. Peyré, A generalized forward-backward splitting. Preprint (2011)

    Google Scholar 

  63. R.T. Rockafellar, Monotone operators and the proximal point algorithm. SIAM J. Contr. Optim. 14(5), 877–898 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  64. S. Setzer, Operator splittings, Bregman methods and frame shrinkage in image processing. Int. J. Comput. Vis. 92(3), 265–280 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  65. J.-L. Starck, F. Murtagh, J. Fadili, Sparse Image and Signal Processing: Wavelets, Curvelets, Morphological Diversity (Cambridge University Press, Cambridge, 2010)

    Book  Google Scholar 

  66. E. van den Berg, M. Friedlander, Probing the Pareto frontier for basis pursuit solutions. SIAM J. Sci. Comput. 31(2), 890–912 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  67. S. Worm, Iteratively re-weighted least squares for compressed sensing. Diploma thesis, University of Bonn, 2011

    Google Scholar 

  68. X. Zhang, M. Burger, X. Bresson, S. Osher, Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM J. Imag. Sci. 3(3), 253–276 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  69. X. Zhang, M. Burger, S. Osher, A unified primal-dual algorithm framework based on Bregman iteration. J. Sci. Comput. 46, 20–46 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  70. M. Zhu, T. Chan, An efficient primal-dual hybrid gradient algorithm for total variation image restoration. Technical report, CAM Report 08–34, UCLA, Los Angeles, CA, 2008

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Drexel University, Philadelphia, PA, USA

    Simon Foucart

  2. Lehrstuhl C für Mathematik (Analysis), RWTH Aachen University, Aachen, Germany

    Holger Rauhut

Authors
  1. Simon Foucart
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Holger Rauhut
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media New York

About this chapter

Cite this chapter

Foucart, S., Rauhut, H. (2013). Algorithms for ℓ1-Minimization. In: A Mathematical Introduction to Compressive Sensing. Applied and Numerical Harmonic Analysis. Birkhäuser, New York, NY. https://doi.org/10.1007/978-0-8176-4948-7_15

Download citation

Publish with us

Policies and ethics

Search

Navigation