Compressive Sensing and Sparse Coding

  • Kevin ChenEmail author
  • H. T. Kung
Part of the Springer Handbooks of Computational Statistics book series (SHCS)


Compressive sensing is a technique to acquire signals at rates proportional to the amount of information in the signal, and it does so by exploiting the sparsity of signals. This section discusses the fundamentals of compressive sensing, and how it is related to sparse coding.


Compressive sensing Sparse coding 


  1. Baraniuk R, Davenport M, DeVore R, Wakin M (2008) A simple proof of the restricted isometry property for random matrices. Constr Approx 28(3):253–263MathSciNetCrossRefGoogle Scholar
  2. Beck A, Teboulle M (2009) A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imag Sci 2(1):183–202MathSciNetCrossRefGoogle Scholar
  3. Blumensath T, Davies ME (2009) Iterative hard thresholding for compressed sensing. Appl Comput Harmon Anal 27(3):265–274MathSciNetCrossRefGoogle Scholar
  4. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Learn 3(1):1–122CrossRefGoogle Scholar
  5. Candes EJ, Tao T (2005) Decoding by linear programming. IEEE Trans Inf Theory 51(12):4203–4215MathSciNetCrossRefGoogle Scholar
  6. Candes EJ, Tao T (2006) Near-optimal signal recovery from random projections: universal encoding strategies? IEEE Trans Inf Theory 52(12):5406–5425MathSciNetCrossRefGoogle Scholar
  7. Candès EJ, Romberg J, Tao T (2006) Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans Inf Theory 52(2):489–509MathSciNetCrossRefGoogle Scholar
  8. Coates A, Ng AY, Lee H (2011) An analysis of single-layer networks in unsupervised feature learning. In: International conference on artificial intelligence and statistics, pp 215–223Google Scholar
  9. Comiter M, Chen H-C, Kung HT (2017) Nonlinear compressive sensing for distorted measurements and application to improving efficiency of power amplifiers. In: IEEE international conference on communicationsGoogle Scholar
  10. Donoho DL (2006) Compressed sensing. IEEE Trans Inf Theory 52(4):1289–1306MathSciNetCrossRefGoogle Scholar
  11. Efron B, Hastie T, Johnstone I, Tibshirani R, et al (2004) Least angle regression. Ann Stat 32(2):407–499Google Scholar
  12. Gkioulekas IA, Zickler T (2011) Dimensionality reduction using the sparse linear model. In: Advances in neural information processing systems, pp 271–279Google Scholar
  13. Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. In: 2009 IEEE 12th international conference on computer vision. IEEE, New York, pp 349–356Google Scholar
  14. Jacques L, Laska JN, Boufounos PT, Baraniuk RG (2013) Robust 1-bit compressive sensing via binary stable embeddings of sparse vectors. IEEE Trans Inf Theory 59(4):2082–2102MathSciNetCrossRefGoogle Scholar
  15. Lee H, Battle A, Raina R, Ng AY (2006) Efficient sparse coding algorithms. In: Advances in neural information processing systems, pp 801–808Google Scholar
  16. Lin T-H, Kung HT (2014) Stable and efficient representation learning with nonnegativity constraints. In: Proceedings of the 31st international conference on machine learning (ICML-14), pp 1323–1331Google Scholar
  17. Mairal J, Bach F, Ponce J, Sapiro G (2009) Online dictionary learning for sparse coding. In: Proceedings of the 26th annual international conference on machine learning. ACM, New York, pp 689–696Google Scholar
  18. Mairal J, Bach F, Ponce J, Sapiro G, Zisserman A (2009) Non-local sparse models for image restoration. In: 2009 IEEE 12th international conference on computer vision. IEEE, New York, pp 2272–2279CrossRefGoogle Scholar
  19. Needell D, Tropp JA (2009) Cosamp: iterative signal recovery from incomplete and inaccurate samples. Appl Comput Harmon Anal 26(3):301–321MathSciNetCrossRefGoogle Scholar
  20. Olshausen BA, Field DJ (1997) Sparse coding with an overcomplete basis set: a strategy employed by v1? Vision Res 37(23):3311–3325CrossRefGoogle Scholar
  21. Pati YC, Rezaiifar R, Krishnaprasad PS (1993) Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: 1993 conference record of the twenty-seventh Asilomar conference on signals, systems and computers. IEEE, New York, pp 40–44Google Scholar
  22. Plan Y, Vershynin R (2013) Robust 1-bit compressed sensing and sparse logistic regression: a convex programming approach. IEEE Trans Inf Theory 59(1):482–494MathSciNetCrossRefGoogle Scholar
  23. Romberg J (2009) Compressive sensing by random convolution. SIAM J Imag Sci 2(4):1098–1128MathSciNetCrossRefGoogle Scholar
  24. Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B Methodol 58(1):267–288MathSciNetzbMATHGoogle Scholar
  25. Yan M, Yang Y, Osher S (2012) Robust 1-bit compressive sensing using adaptive outlier pursuit. IEEE Trans Signal Process 60(7):3868–3875MathSciNetCrossRefGoogle Scholar
  26. Yang J, Yu K, Gong Y, Huang T (2009) Linear spatial pyramid matching using sparse coding for image classification. In: IEEE conference on computer vision and pattern recognition, CVPR 2009. IEEE, New York, pp 1794–1801CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Engineering and Applied SciencesHarvard UniversityCambridgeUSA

Personalised recommendations