Compressive Sensing

  • Massimo Fornasier
  • Holger Rauhut


Compressive sensing is a new type of sampling theory, which predicts that sparse signals and images can be reconstructed from what was previously believed to be incomplete information. As a main feature, efficient algorithms such as 1-minimization can be used for recovery. The theory has many potential applications in signal processing and imaging. This chapter gives an introduction and overview on both theoretical and numerical aspects of compressive sensing.


Compressive Sense Measurement Matrix Restricted Isometry Property Homotopy Method Sparse Vector 

References and Further Reading

  1. 1.
    Achlioptas D (2001) Database-friendly random projections. In: Proceedings of the 20th annual ACM SIGACT-SIGMOD-SIGART symposium on principles of database systems, Santa Barbara, 21–23 May 2001, pp 274–281Google Scholar
  2. 2.
    Affentranger F, Schneider R (1992) Random projections of regular simplices. Discrete Comput Geom 7(3):219–226MathSciNetMATHCrossRefGoogle Scholar
  3. 3.
    Baraniuk R (2007) Compressive sensing. IEEE Signal Process Mag 24(4):118–121CrossRefGoogle Scholar
  4. 4.
    Baraniuk RG, Davenport M, DeVore RA, Wakin M (2008) A simple proof of the restricted isometry property for random matrices. Constr Approx 28(3):253–263MathSciNetMATHCrossRefGoogle Scholar
  5. 5.
    Berinde R, Gilbert A, Indyk P, Karloff H, Strauss M (2008) Combining geometry and combinatorics: a unified approach to sparse signal recovery. In: Proceedings 46th Annual Allerton Conference on Communication, Control, and Computing, pp 798–805Google Scholar
  6. 6.
    Blanchard JD, Cartis C, Tanner J, Thompson A (2009) Phase transitions for greedy sparse approximation algorithms. PreprintGoogle Scholar
  7. 7.
    Blumensath T, Davies M (2009) Iterative hard thresholding for compressed sensing. Appl Comput Harmon Anal 27(3):265–274MathSciNetMATHCrossRefGoogle Scholar
  8. 8.
    Bobin J, Starck J-L, Ottensamer R (2008) Compressed sensing in astronomy. IEEE J Sel Top Signal Process 2(5):718–726CrossRefGoogle Scholar
  9. 9.
    Boyd S, Vandenberghe L (2004) Convex Optim- ization. Cambridge University Press, CambridgeGoogle Scholar
  10. 10.
    Bungartz H-J, Griebel M (2004) Sparse grids. Acta Numerica 13:147–269MathSciNetCrossRefGoogle Scholar
  11. 11.
    Candès E, Wakin M (2008) An introduction to compressive sampling. IEEE Signal Process Mag 25(2):21–30CrossRefGoogle Scholar
  12. 12.
    Candès EJ (2006) Compressive sampling. In: Proceedings of the International congress of mathematicians, MadridGoogle Scholar
  13. 13.
    Candès EJ, Romberg J, Tao T (2006) Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans Inform Theory 52(2):489–509MathSciNetMATHCrossRefGoogle Scholar
  14. 14.
    Candès EJ, Recht B (2009) Exact matrix completion via convex optimization. Found Comput Math 9:717–772MathSciNetMATHCrossRefGoogle Scholar
  15. 15.
    Candès EJ, Romberg J, Tao T (2006) Stable signal recovery from incomplete and inaccurate measurements. Commun Pure Appl Math 59(8):1207–1223MATHCrossRefGoogle Scholar
  16. 16.
    Candès EJ, Tao T (2006) Near optimal signal recovery from random projections: universal encoding strategies? IEEE Trans Inform Theory 52(12):5406–5425MathSciNetCrossRefGoogle Scholar
  17. 17.
    Capalbo M, Reingold O, Vadhan S, Wigderson A (2002) Randomness conductors and constant-degree lossless expanders. In: Proceedings of the thirty-fourth annual ACM, ACM, Montreal, pp 659–668 (electronic)Google Scholar
  18. 18.
    Chen SS, Donoho DL, Saunders MA (1999) Atomic decomposition by basis pursuit. SIAM J Sci Comput 20(1):33–61MathSciNetMATHCrossRefGoogle Scholar
  19. 19.
    Christensen O (2003) An introduction to frames and Riesz bases: applied and numerical harmonic analysis. Birkhäuser, BostonMATHGoogle Scholar
  20. 20.
    Cline AK (1972) Rate of convergence of Lawson’s algorithm. Math Comput 26:167–176MathSciNetMATHGoogle Scholar
  21. 21.
    Cohen A, Dahmen W, DeVore RA (2009) Compressed sensing and best k-term approximation. J Am Math Soc 22(1):211–231MathSciNetMATHCrossRefGoogle Scholar
  22. 22.
    Cormode G, Muthukrishnan S (2006) Combinatorial algorithms for compressed sensing, SIROCCO, Springer, HeidelbergGoogle Scholar
  23. 23.
    Daubechies I, DeVore R, Fornasier M, Güntürk C (2010) Iteratively re-weighted least squares minimization for sparse recovery. Commun Pure Appl Math 63(1):1–38MATHCrossRefGoogle Scholar
  24. 24.
    Do Ba K, Indyk P, Price E, Woodruff D (2010) Lower bounds for sparse recovery. In: Proceedings of ACM-SIAM Symposium on Discrete Algorithms (SODA10), pp 1190–1197.Google Scholar
  25. 25.
    Donoho D, Logan B (1992) Signal recovery and the large sieve. SIAM J Appl Math 52(2): 577–591MathSciNetMATHCrossRefGoogle Scholar
  26. 26.
    Donoho DL (2006) Compressed sensing. IEEE Trans Inform Theory 52(4):1289–1306MathSciNetCrossRefGoogle Scholar
  27. 27.
    Donoho DL (2006) For most large underdetermined systems oflinear equations the minimal l 1 solution is also the sparsest solution. Commun Pure Appl Anal 59(6):797–829Google Scholar
  28. 28.
    Donoho DL (2006) High-dimensional centrally symmetric polytopes with neighborliness proportional to dimension. Discrete Comput Geom 35(4):617–652MathSciNetMATHCrossRefGoogle Scholar
  29. 29.
    Donoho DL, Elad M (2003) Optimally sparse representation in general (nonorthogonal) dictionaries via \({\ell}^{1}\) minimization. Proc Natl Acad Sci USA 100(5):2197–2202MathSciNetMATHCrossRefGoogle Scholar
  30. 30.
    Donoho DL, Huo X (2001) Uncertainty principles and ideal atomic decompositions. IEEE Trans Inform Theory 47(7):2845–2862MathSciNetMATHCrossRefGoogle Scholar
  31. 31.
    Donoho DL, Tanner J (2005) Neighborliness of randomly projected simplices in high dimensions. Proc Natl Acad Sci USA 102(27): 9452–9457MathSciNetMATHCrossRefGoogle Scholar
  32. 32.
    Donoho DL, Tanner J (2009) Counting faces of randomly-projected polytopes when the projection radically lowers dimension. J Am Math Soc 22(1):1–53MathSciNetMATHCrossRefGoogle Scholar
  33. 33.
    Donoho DL, Tsaig Y (2008) Fast solution of l1-norm minimization problems when the solution may be sparse. IEEE Trans Inform Theory 54(11):4789–4812MathSciNetCrossRefGoogle Scholar
  34. 34.
    Dorfman R (1943) The detection of defective members of large populations. Ann Statist 14:436–440CrossRefGoogle Scholar
  35. 35.
    Duarte M, Davenport M, Takhar D, Laska J, Ting S, Kelly K, Baraniuk R (2008) Single-pixel imaging via compressive sampling. IEEE Signal Process Mag 25(2):83–91CrossRefGoogle Scholar
  36. 36.
    Efron B, Hastie T, Johnstone I, Tibshirani R (2004) Least angle regression. Ann Statist 32(2):407–499MathSciNetMATHCrossRefGoogle Scholar
  37. 37.
    Elad M, Bruckstein AM (2002) A generalized uncertainty principle and sparse representation in pairs of bases. IEEE Trans Inform Theory 48(9):2558–2567MathSciNetMATHCrossRefGoogle Scholar
  38. 38.
    Fannjiang A, Yan P, Strohmer T (in press), Compressed remote sensing of sparse object. SIAM J Imaging SciGoogle Scholar
  39. 39.
    Fornasier M (2010) Numerical methods for sparse recovery. In: Fornasier M (ed) Theoretical foundations and numerical methods for sparse recovery. Radon Series on Computational and Applied Mathematics, de Gruyter, Berlin, pp 218–236Google Scholar
  40. 40.
    Fornasier M, Langer A, Schönlieb CB (to appear, A convergent overlapping domain decomposition method for total variation minimization. Numer MathGoogle Scholar
  41. 41.
    Fornasier M, March R (2007) Restoration of color images by vector valued BV functions and variational calculus. SIAM J Appl Math 68(2):437–460MathSciNetMATHCrossRefGoogle Scholar
  42. 42.
    Fornasier M, Ramlau R, Teschke G (2009) The application of joint sparsity and total variation minimization algorithms to a real-life art restoration problem. Adv Comput Math 31(1–3):157–184MathSciNetMATHCrossRefGoogle Scholar
  43. 43.
    Foucart S (to appear) A note on guaranteed sparse recovery via \({\ell}_{1}\)-minimization. Appl Comput Harmon AnalGoogle Scholar
  44. 44.
    Foucart S, Pajor A, Rauhut H, Ullrich T (to appear) The Gelfand widths of \({\ell}_{\mathrm{p}}\)-balls for \(0 < \mathrm{p} \leq 1\). J Complexity. doi:10.1016/j.jco.2010.04.004Google Scholar
  45. 45.
    Foucart S, Rauhut H (in preparation) A mathematical introduction to compressive sensing. Applied and Numerical Harmonic Analysis, Birkhäuser, BostonGoogle Scholar
  46. 46.
    Fuchs JJ (2004) On sparse representations in arbitrary redundant bases. IEEE Trans Inform Theory 50(6):1341–1344MathSciNetCrossRefGoogle Scholar
  47. 47.
    Garnaev A, Gluskin E (1984) On widths of the Euclidean ball. Sov Math Dokl 30: 200–204MATHGoogle Scholar
  48. 48.
    Gilbert AC, Muthukrishnan S, Guha S, Indyk P, Strauss M (2002) Near-optimal sparse Fourier representations via sampling. In: Proceedings of the STOC’02, Association for Computing Machinery, New York, pp 152–161Google Scholar
  49. 49.
    Gilbert AC, Muthukrishnan S, Strauss MJ (2003) Approximation of functions over redundant dictionaries using coherence. In: Proceedings of the fourteenth annual ACM-SIAM symposium on discrete algorithms, SIAM and Association for Computing Machinery, Baltimore, 12–14 January 2003, pp 243–252Google Scholar
  50. 50.
    Gilbert AC, Strauss M, Tropp JA, Vershynin R (2007) One sketch for all: fast algorithms for compressed sensing. In: Proceedings of the ACM Symposium on the Theory of Computing (STOC), San DiegoGoogle Scholar
  51. 51.
    Gluskin E (1984) Norms of random matrices and widths of finite-dimensional sets. Math USSR-Sb 48:173–182MATHCrossRefGoogle Scholar
  52. 52.
    Gorodnitsky I, Rao B (1997) Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm. IEEE Trans Signal Process 45(3): 600–616CrossRefGoogle Scholar
  53. 53.
    Gribonval R, Nielsen M (2003) Sparse representations in unions of bases. IEEE Trans Inform Theory 49(12):3320–3325MathSciNetCrossRefGoogle Scholar
  54. 54.
    Horn R, Johnson C (1990) Matrix analysis. Cambridge University Press, CambridgeMATHGoogle Scholar
  55. 55.
    Johnson WB, Lindenstrauss J (eds) (2001) Handbook of the geometry of Banach spaces, vol 1. North-Holland, AmsterdamMATHGoogle Scholar
  56. 56.
    Kashin B (1977) Diameters of some finite-dimensional sets and classes of smooth functions. Math USSR, Izv 11:317–333MATHCrossRefGoogle Scholar
  57. 57.
    Lawson C (1961) Contributions to the theory of linear least maximum approximation. PhD thesis, University of California, Los AngelesGoogle Scholar
  58. 58.
    Ledoux M, Talagrand M (1991) Probability in Banach spaces. Springer, Berlin/Heidelberg/ New YorkMATHGoogle Scholar
  59. 59.
    Logan B (1965) Properties of high-pass signals. PhD thesis, Columbia University, New YorkGoogle Scholar
  60. 60.
    Lorentz GG, von Golitschek M, Makovoz Y (1996) Constructive approximation: advanced problems. Springer, BerlinMATHCrossRefGoogle Scholar
  61. 61.
    Mallat SG, Zhang Z (1993) Matching pursuits with time-frequency dictionaries. IEEE Trans Signal Process 41(12):3397–3415MATHCrossRefGoogle Scholar
  62. 62.
    Marple S (1987) Digital spectral analysis with applications. Prentice-Hall, Englewood CliffsGoogle Scholar
  63. 63.
    Mendelson S, Pajor A, Tomczak Jaegermann N (2009) Uniform uncertainty principle for Bernoulli and subgaussian ensembles. Constr Approx 28(3):277–289MathSciNetCrossRefGoogle Scholar
  64. 64.
    Natarajan BK (1995) Sparse approximate solutions to linear systems. SIAM J Comput 24:227–234MathSciNetMATHCrossRefGoogle Scholar
  65. 65.
    Nesterov Y, Nemirovskii A (1994) Interior-point polynomial algorithms in convex programming. In: Volume 13 of SIAM studies in applied mathematics, Society for Industrial and Applied Mathematics (SIAM), PhiladelphiaGoogle Scholar
  66. 66.
    Novak E (1995) Optimal recovery and n-widths for convex classes of functions. J Approx Theory 80(3):390–408MathSciNetMATHCrossRefGoogle Scholar
  67. 67.
    Osborne M, Presnell B, Turlach B (2000) A new approach to variable selection in least squares problems. IMA J Numer Anal 20(3): 389–403MathSciNetMATHCrossRefGoogle Scholar
  68. 68.
    Osborne M, Presnell B, Turlach B (2000) On the LASSO and its dual. J Comput Graph Stat 9(2):319–337MathSciNetCrossRefGoogle Scholar
  69. 69.
    Pfander GE, Rauhut H (2010) Sparsity in time-frequency representations. J Fourier Anal Appl 16(2):233–260MathSciNetMATHCrossRefGoogle Scholar
  70. 70.
    Pfander GE, Rauhut H, Tanner J (2008) Identification of matrices having a sparse representation. IEEE Trans Signal Process 56(11): 5376–5388MathSciNetCrossRefGoogle Scholar
  71. 71.
    Prony R (1795) Essai expérimental et analytique sur les lois de la Dilatabilité des uides élastique et sur celles de la Force expansive de la vapeur de leau et de la vapeur de lalkool, à différentes températures. J École Polytech 1:24–76Google Scholar
  72. 72.
    Rauhut H (2007) Random sampling of sparse trigonometric polynomials. Appl Comput Harmon Anal 22(1):16–42MathSciNetMATHCrossRefGoogle Scholar
  73. 73.
    Rauhut H (2008) Stability results for random sampling of sparse trigonometric polynomials. IEEE Trans Inform Theory 54(12): 5661–5670MathSciNetCrossRefGoogle Scholar
  74. 74.
    Rauhut H (2009) Circulant and Toeplitz matrices in compressed sensing. In: Proceedings of the SPARS’09, Saint-MaloGoogle Scholar
  75. 75.
    Rauhut H (2010) Compressive sensing and structured random matrices. In: Fornasier M (ed) Theoretical foundations and numerical methods for sparse recovery. Radon Series on Computational and Applied Mathematics, de Gruyter, BerlinGoogle Scholar
  76. 76.
    Recht B, Fazel M, Parillo P (to appear) Guaranteed minimum rank solutions to linear matrix equations via nuclear norm minimization. SIAM RevGoogle Scholar
  77. 77.
    Romberg J (2008) Imaging via compressive sampling. IEEE Signal Process Mag 25(2): 14–20CrossRefGoogle Scholar
  78. 78.
    Rudelson M, Vershynin R (2008) On sparse reconstruction from Fourier and Gaussian measurements. Commun Pure Appl Math 61:1025–1045MathSciNetMATHCrossRefGoogle Scholar
  79. 79.
    Rudin L, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Physica D 60(1–4):259–268MATHCrossRefGoogle Scholar
  80. 80.
    Santosa F, Symes W (1986) Linear inversion of band-limited reflection seismograms. SIAM J Sci Stat Comput 7(4):1307–1330MathSciNetMATHCrossRefGoogle Scholar
  81. 81.
    Schnass K, Vandergheynst P (2008) Dictionary preconditioning for greedy algorithms. IEEE Trans Signal Process 56(5):1994–2002MathSciNetCrossRefGoogle Scholar
  82. 82.
    Strohmer T, Heath RW Jr (2003) Grassmannian frames with applications to coding and communication. Appl Comput Harmon Anal 14(3):257–275MathSciNetMATHCrossRefGoogle Scholar
  83. 83.
    Strohmer T, Hermann M (2008) Compressed sensing radar. In: IEEE proceedings of the international conference on acoustic, speech, and signal processing, pp 1509–1512Google Scholar
  84. 84.
    Tadmor E (2009) Numerical methods for nonlinear partial differential equations. In: Meyers R (ed) Encyclopedia of complexity and systems science, SpringerGoogle Scholar
  85. 85.
    Talagrand M (1998) Selecting a proportion of characters. Israel J Math 108:173–191MathSciNetMATHCrossRefGoogle Scholar
  86. 86.
    Tauboeck G, Eiwen D, Hlawatsch F, Rauhut H (2010) Compressive estimation of doubly selective channels: exploiting channel sparsity to improve spectral efficiency in multicarrier transmissions. J Sel Topics Signal Process 4(2): 255–271CrossRefGoogle Scholar
  87. 87.
    Taylor H, Banks S, McCoy J (1979) Deconvolution with the \({\ell}_{1}\)-norm. Geophys J Int 44(1):39–52Google Scholar
  88. 88.
    Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B 58(1):267–288MathSciNetMATHGoogle Scholar
  89. 89.
    Traub J, Wasilkowski G, Woniakowski H (1988) Information-based complexity. Computer Science and Scientific Computing. Academic, New YorkMATHGoogle Scholar
  90. 90.
    Tropp JA Needell D (2008) CoSaMP: Iterative signal recovery from incomplete and inaccurate samples. Appl Comput Harmon Anal 26: 301–321MathSciNetGoogle Scholar
  91. 91.
    Tropp JA (2004) Greed is good: algorithmic results for sparse approximation. IEEE Trans Inform Theory 50(10):2231–2242MathSciNetCrossRefGoogle Scholar
  92. 92.
    Tropp JA (2006) Just relax: convex programming methods for identifying sparse signals in noise. IEEE Trans Inform Theory 51(3): 1030–1051MathSciNetCrossRefGoogle Scholar
  93. 93.
    Tropp JA, Laska JN, Duarte MF, Romberg JK, Baraniuk RG (2010) Beyond Nyquist: efficient sampling of sparse bandlimited signals. IEEE Trans Inform Theory 56(1):520–544MathSciNetCrossRefGoogle Scholar
  94. 94.
    Unser M (2000) Sampling – 50 years after shannon. Proc IEEE 88(4):569–587CrossRefGoogle Scholar
  95. 95.
    Vybiral J (2008) Widths of embeddings in function spaces. J Complexity 24(4):545–570MathSciNetMATHCrossRefGoogle Scholar
  96. 96.
    Wagner G, Schmieder P, Stern A, Hoch J (1993) Application of nonlinear sampling schemes to cosy-type spectra. J Biomol NMR 3(5):569Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Massimo Fornasier
  • Holger Rauhut

There are no affiliations available

Personalised recommendations