Skip to main content
SpringerLink
Log in

Compressed Sensing and Sparse Recovery

  • Chapter
  • First Online:
Cognitive Networked Sensing and Big Data

  • 2247 Accesses

Abstract

The central mathematical tool for algorithm analysis and development is the concentration of measure for random matrices. This chapter is motivated to provide applications examples for the theory developed in Part I. We emphasize the central role of random matrices.

Compressed sensing is a recent revolution. It is built upon the observation that sparsity plays a central role in the structure of a vector. The unexpected message here is that for a sparse signal, the relevant “information” is much less that what we thought previously. As a result, to recover the sparse signal, the required samples are much less than what is required by the traditional Shannon’s sampling theorem.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    A point in a vector space is a vector.

Bibliography

  1. G. Bennett, “Probability inequalities for the sum of independent random variables,” Journal of the American Statistical Association, vol. 57, no. 297, pp. 33–45, 1962.

    Article  MATH  Google Scholar 

  2. M. Ledoux and M. Talagrand, Probability in Banach spaces. Springer, 1991.

    Google Scholar 

  3. J. A. Tropp, J. N. Laska, M. F. Duarte, J. K. Romberg, and R. G. Baraniuk, “Beyond nyquist: Efficient sampling of sparse bandlimited signals,” Information Theory, IEEE Transactions on, vol. 56, no. 1, pp. 520–544, 2010.

    Article  MathSciNet  Google Scholar 

  4. J. Nelson, “Johnson-lindenstrauss notes,” tech. rep., Technical report, MIT-CSAIL, Available at http://web.mit.edu/minilek/www/jl_notes.pdf, 2010.

  5. H. Rauhut, “Compressive sensing and structured random matrices,” Theoretical foundations and numerical methods for sparse recovery, vol. 9, pp. 1–92, 2010.

    MathSciNet  Google Scholar 

  6. F. Krahmer, S. Mendelson, and H. Rauhut, “Suprema of chaos processes and the restricted isometry property,” arXiv preprint arXiv:1207.0235, 2012.

    Google Scholar 

  7. V. De la Peña and E. Giné, Decoupling: from dependence to independence. Springer Verlag, 1999.

    Google Scholar 

  8. D. L. Hanson and F. T. Wright, “A bound on tail probabilities for quadratic forms in independent random variables,” The Annals of Mathematical Statistics, pp. 1079–1083, 1971.

    Google Scholar 

  9. S. Boucheron, G. Lugosi, and P. Massart, “Concentration inequalities using the entropy method,” The Annals of Probability, vol. 31, no. 3, pp. 1583–1614, 2003.

    Article  MathSciNet  MATH  Google Scholar 

  10. M. Talagrand, The generic chaining: upper and lower bounds of stochastic processes. Springer Verlag, 2005.

    Google Scholar 

  11. R. M. Dudley, “The sizes of compact subsets of hilbert space and continuity of gaussian processes,” J. Funct. Anal, vol. 1, no. 3, pp. 290–330, 1967.

    Article  MathSciNet  MATH  Google Scholar 

  12. X. Fernique, “Régularité des trajectoires des fonctions aléatoires gaussiennes,” Ecole d’Eté de Probabilités de Saint-Flour IV-1974, pp. 1–96, 1975.

    Google Scholar 

  13. M. Talagrand, “Regularity of gaussian processes,” Acta mathematica, vol. 159, no. 1, pp. 99–149, 1987.

    Article  MathSciNet  MATH  Google Scholar 

  14. G. Pisier, The volume of convex bodies and Banach space geometry, vol. 94. Cambridge Univ Pr, 1999.

    Google Scholar 

  15. M. Talagrand, “New concentration inequalities in product spaces,” Inventiones Mathematicae, vol. 126, no. 3, pp. 505–563, 1996.

    Article  MathSciNet  MATH  Google Scholar 

  16. G. E. Pfander, H. Rauhut, and J. A. Tropp, “The restricted isometry property for time–frequency structured random matrices,” Probability Theory and Related Fields, pp. 1–31, 2011.

    Google Scholar 

  17. T. T. Cai, L. Wang, and G. Xu, “Shifting inequality and recovery of sparse signals,” Signal Processing, IEEE Transactions on, vol. 58, no. 3, pp. 1300–1308, 2010.

    Article  MathSciNet  Google Scholar 

  18. S. Foucart, “A note on guaranteed sparse recovery via ℓ 1-minimization,” Applied and Computational Harmonic Analysis, vol. 29, no. 1, pp. 97–103, 2010.

    Article  MathSciNet  MATH  Google Scholar 

  19. S. Foucart, “Sparse recovery algorithms: sufficient conditions in terms of restricted isometry constants,” Approximation Theory XIII: San Antonio 2010, pp. 65–77, 2012.

    Article  Google Scholar 

  20. D. Needell and J. A. Tropp, “Cosamp: Iterative signal recovery from incomplete and inaccurate samples,” Applied and Computational Harmonic Analysis, vol. 26, no. 3, pp. 301–321, 2009.

    Article  MathSciNet  MATH  Google Scholar 

  21. T. Blumensath and M. E. Davies, “Iterative hard thresholding for compressed sensing,” Applied and Computational Harmonic Analysis, vol. 27, no. 3, pp. 265–274, 2009.

    Article  MathSciNet  MATH  Google Scholar 

  22. S. Foucart, “Hard thresholding pursuit: an algorithm for compressive sensing,” SIAM Journal on Numerical Analysis, vol. 49, no. 6, pp. 2543–2563, 2011.

    Article  MathSciNet  MATH  Google Scholar 

  23. R. Baraniuk, M. Davenport, R. DeVore, and M. Wakin, “A Simple Proof of the Restricted Isometry Property for Random Matrices.” Submitted for publication, January 2007.

    Google Scholar 

  24. M. Rudelson and R. Vershynin, “On sparse reconstruction from fourier and gaussian measurements,” Communications on Pure and Applied Mathematics, vol. 61, no. 8, pp. 1025–1045, 2008.

    Article  MathSciNet  MATH  Google Scholar 

  25. H. Rauhut, J. Romberg, and J. A. Tropp, “Restricted isometries for partial random circulant matrices,” Applied and Computational Harmonic Analysis, vol. 32, no. 2, pp. 242–254, 2012.

    Article  MathSciNet  MATH  Google Scholar 

  26. G. E. Pfander and H. Rauhut, “Sparsity in time-frequency representations,” Journal of Fourier Analysis and Applications, vol. 16, no. 2, pp. 233–260, 2010.

    Article  MathSciNet  MATH  Google Scholar 

  27. G. Pfander, H. Rauhut, and J. Tanner, “Identification of Matrices having a Sparse Representation,” in Preprint, 2007.

    Google Scholar 

  28. E. Candes and T. Tao, “Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies?,” IEEE Transactions on Information Theory, vol. 52, no. 12, pp. 5406–5425, 2006.

    Article  MathSciNet  Google Scholar 

  29. S. Mendelson, A. Pajor, and N. Tomczak-Jaegermann, “Uniform uncertainty principle for bernoulli and subgaussian ensembles,” Constructive Approximation, vol. 28, no. 3, pp. 277–289, 2008.

    Article  MathSciNet  MATH  Google Scholar 

  30. A. Cohen, W. Dahmen, and R. DeVore, “Compressed Sensing and Best k-Term Approximation,” in Submitted for publication, July, 2006.

    Google Scholar 

  31. A. Y. Garnaev and E. D. Gluskin, “The widths of a euclidean ball,” in Dokl. Akad. Nauk SSSR, vol. 277, pp. 1048–1052, 1984.

    Google Scholar 

  32. W. B. Johnson and J. Lindenstrauss, “Extensions of lipschitz mappings into a hilbert space,” Contemporary mathematics, vol. 26, no. 189–206, p. 1, 1984.

    Google Scholar 

  33. F. Krahmer and R. Ward, “New and improved johnson-lindenstrauss embeddings via the restricted isometry property,” SIAM Journal on Mathematical Analysis, vol. 43, no. 3, pp. 1269–1281, 2011.

    Article  MathSciNet  MATH  Google Scholar 

  34. N. Alon, “Problems and results in extremal combinatorics-i,” Discrete Mathematics, vol. 273, no. 1, pp. 31–53, 2003.

    Article  MathSciNet  MATH  Google Scholar 

  35. N. Ailon and E. Liberty, “An almost optimal unrestricted fast johnson-lindenstrauss transform,” in Proceedings of the Twenty-Second Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 185–191, SIAM, 2011.

    Google Scholar 

  36. D. Achlioptas, “Database-friendly random projections,” in Proceedings of the twentieth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, pp. 274–281, ACM, 2001.

    Google Scholar 

  37. D. Achlioptas, “Database-friendly random projections: Johnson-lindenstrauss with binary coins,” Journal of computer and System Sciences, vol. 66, no. 4, pp. 671–687, 2003.

    Article  MathSciNet  MATH  Google Scholar 

  38. N. Ailon and B. Chazelle, “Approximate nearest neighbors and the fast johnson-lindenstrauss transform,” in Proceedings of the thirty-eighth annual ACM symposium on Theory of computing, pp. 557–563, ACM, 2006.

    Google Scholar 

  39. N. Ailon and B. Chazelle, “The fast johnson-lindenstrauss transform and approximate nearest neighbors,” SIAM Journal on Computing, vol. 39, no. 1, pp. 302–322, 2009.

    Article  MathSciNet  MATH  Google Scholar 

  40. G. Lorentz, M. Golitschek, and Y. Makovoz, “Constructive approximation, volume 304 of grundlehren math. wiss,” 1996.

    Google Scholar 

  41. R. I. Arriaga and S. Vempala, “An algorithmic theory of learning: Robust concepts and random projection,” Machine Learning, vol. 63, no. 2, pp. 161–182, 2006.

    Article  MATH  Google Scholar 

  42. J. Matoušek, “On variants of the johnson–lindenstrauss lemma,” Random Structures & Algorithms, vol. 33, no. 2, pp. 142–156, 2008.

    Article  MathSciNet  MATH  Google Scholar 

  43. M. Rudelson and R. Vershynin, “Geometric approach to error-correcting codes and reconstruction of signals,” International Mathematics Research Notices, vol. 2005, no. 64, p. 4019, 2005.

    Google Scholar 

  44. J. Vybíral, “A variant of the johnson–lindenstrauss lemma for circulant matrices,” Journal of Functional Analysis, vol. 260, no. 4, pp. 1096–1105, 2011.

    Article  MathSciNet  MATH  Google Scholar 

  45. A. Hinrichs and J. Vybíral, “Johnson-lindenstrauss lemma for circulant matrices,” Random Structures & Algorithms, vol. 39, no. 3, pp. 391–398, 2011.

    MathSciNet  MATH  Google Scholar 

  46. H. Rauhut, K. Schnass, and P. Vandergheynst, “Compressed Sensing and Redundant Dictionaries,” IEEE Transactions on Information Theory, vol. 54, no. 5, pp. 2210–2219, 2008.

    Article  MathSciNet  Google Scholar 

  47. W. U. Bajwa, J. Haupt, A. M. Sayeed, and R. Nowak, “Compressed channel sensing: A new approach to estimating sparse multipath channels,” Proceedings of the IEEE, vol. 98, no. 6, pp. 1058–1076, 2010.

    Article  Google Scholar 

  48. J. Chiu and L. Demanet, “Matrix probing and its conditioning,” SIAM Journal on Numerical Analysis, vol. 50, no. 1, pp. 171–193, 2012.

    Article  MathSciNet  MATH  Google Scholar 

  49. G. E. Pfander, “Gabor frames in finite dimensions,” Finite Frames, pp. 193–239, 2012.

    Google Scholar 

  50. J. Haupt, W. U. Bajwa, G. Raz, and R. Nowak, “Toeplitz compressed sensing matrices with applications to sparse channel estimation,” Information Theory, IEEE Transactions on, vol. 56, no. 11, pp. 5862–5875, 2010.

    Article  MathSciNet  Google Scholar 

  51. K. Gröchenig, Foundations of time-frequency analysis. Birkhäuser Boston, 2000.

    Google Scholar 

  52. F. Krahmer, G. E. Pfander, and P. Rashkov, “Uncertainty in time–frequency representations on finite abelian groups and applications,” Applied and Computational Harmonic Analysis, vol. 25, no. 2, pp. 209–225, 2008.

    Article  MathSciNet  MATH  Google Scholar 

  53. J. Lawrence, G. E. Pfander, and D. Walnut, “Linear independence of gabor systems in finite dimensional vector spaces,” Journal of Fourier Analysis and Applications, vol. 11, no. 6, pp. 715–726, 2005.

    Article  MathSciNet  MATH  Google Scholar 

  54. B. M. Sanandaji, T. L. Vincent, and M. B. Wakin, “Concentration of measure inequalities for compressive toeplitz matrices with applications to detection and system identification,” in Decision and Control (CDC), 2010 49th IEEE Conference on, pp. 2922–2929, IEEE, 2010.

    Google Scholar 

  55. B. M. Sanandaji, T. L. Vincent, and M. B. Wakin, “Concentration of measure inequalities for toeplitz matrices with applications,” arXiv preprint arXiv:1112.1968, 2011.

    Google Scholar 

  56. B. M. Sanandaji, M. B. Wakin, and T. L. Vincent, “Observability with random observations,” arXiv preprint arXiv:1211.4077, 2012.

    Google Scholar 

  57. M. Meckes, “On the spectral norm of a random toeplitz matrix,” Electron. Comm. Probab, vol. 12, pp. 315–325, 2007.

    MathSciNet  MATH  Google Scholar 

  58. R. Adamczak, “A few remarks on the operator norm of random toeplitz matrices,” Journal of Theoretical Probability, vol. 23, no. 1, pp. 85–108, 2010.

    Article  MathSciNet  MATH  Google Scholar 

  59. R. Calderbank, S. Howard, and S. Jafarpour, “Construction of a large class of deterministic sensing matrices that satisfy a statistical isometry property,” Selected Topics in Signal Processing, IEEE Journal of, vol. 4, no. 2, pp. 358–374, 2010.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tennessee Technological University, Cookeville, Tennessee, USA

    Robert Qiu

  2. Utica, NY, USA

    Michael Wicks

Authors
  1. Robert Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Wicks
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media New York

About this chapter

Cite this chapter

Qiu, R., Wicks, M. (2014). Compressed Sensing and Sparse Recovery. In: Cognitive Networked Sensing and Big Data. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-4544-9_7

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-4544-9_7

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-4543-2

  • Online ISBN: 978-1-4614-4544-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation