Skip to main content
SpringerLink
Log in

Randomization of data acquisition and ℓ1-optimization (recognition with compression)

  • Reviews
  • Published:
Automation and Remote Control Aims and scope Submit manuscript

Abstract

A new paradigm for processing signals with sparse representation in some basis is actively developed for some time past. It relies largely on the ideas of measurement randomization and ℓ1-optimization. The recent methods of acquisition and representation of the compressed data were christened compressive sensing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Granichin, O.N. and Kiyaev, V.I., Informatsionnye tekhnologii v upravlenii (Information Technologies in Control), Moscow: Binom, 2008.

    Google Scholar 

  2. Ermakov, S.M. and Zhiglyavskii, A.A., Matematicheskaya teoriya optimal’nogo eksperimenta (Mathematical Theory of Optimal Experiments), Moscow: Nauka, 1987.

    Google Scholar 

  3. Granichin, O.N. and Polyak, B.T., Randomizirovannye algoritmy otsenivaniya i optimizatsii pri pochti proizvol’nykh pomekhakh (Randomized Algorithms of Estimation and Optimization under Almost Arbitrary Noise), Moscow: Nauka, 2003.

    Google Scholar 

  4. Granichin, O.N., Linear Regression and Filtering under Nonstandard Assumptions (Arbitrary Noise), IEEE Trans. Automat. Control, 2004, vol. 49, no. 10, pp. 1830–1835.

    Article  MathSciNet  Google Scholar 

  5. Fisher, R.A., The Design of Experiments, Edinburgh: Oliver and Boyd, 1935.

    Google Scholar 

  6. Ljung, L., System Identification: Theory for the User, Upper Saddle River: PTR Prentice Hall, 2nd ed., 1999.

    Google Scholar 

  7. Tsypkin, Ya.Z., Informatsionnaya teoriya identifikatsii (Informational Identification Theory), Moscow: Nauka, 1995.

    MATH  Google Scholar 

  8. Györfi, L., Stochastic Approximation from Ergodic Sample for Linear Regression, Z. Wahrscti. Verw. Geb., 1980, vol. 54, pp. 47–55.

    Article  MATH  Google Scholar 

  9. Krieger, A. and Masry, E., Convergence Analysis of Adaptive Linear Estimation for Dependent Stationary Processes, IEEE Trans. Inform. Theory, 1988, vol. 34, pp. 177–182.

    Article  MathSciNet  Google Scholar 

  10. Young, R.S., Recursive Estimation and Time-Series Analysis. An Introduction, Berlin: Springer, 1984.

    MATH  Google Scholar 

  11. Albert, A., Regression and the Moore-Penrose Pseudoinverse, New York: Academic, 1972.

    MATH  Google Scholar 

  12. Polyak, B.T. and Tsypkin, Ya.Z., Adaptive Estimation Algorithms (Converhence, Optimality, Stability), Autom. Remote Control, 1979, no. 3, pp. 378–389.

  13. Polyak, B.T., New Method of Stochastic Approximation Type, Autom. Remote Control, 1990, no. 7, pp. 937–946.

  14. Candes, E., Romberg, J., and Tao, T., Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information, IEEE Trans. Inform. Theory, 2006, vol. 52, no. 2, pp. 489–509.

    Article  MathSciNet  Google Scholar 

  15. Donoho, D., Compressed Sensing, IEEE Trans. Inform. Theory, 2006, vol. 52, no. 4, pp. 1289–1306.

    Article  MathSciNet  Google Scholar 

  16. Kashin, B.S. and Temlyakov, V.N., Remark on the Problem of Compressed Measurement, Mat. Zametki, 2007, vol. 82, no. 6, pp. 829–837.

    MathSciNet  Google Scholar 

  17. Claerbout, J.F. and Muir, F., Robust Modeling with Erratic Data, Geophysics, 1973, vol. 38, no. 5, pp. 826–844.

    Article  Google Scholar 

  18. Santosa, F. and Symes, W.W., Linear Inversion of Band-limited Reflection Seismograms, SIAM J. Sci. Statist. Comput., 1986, vol. 7, no. 4, pp. 1307–1330.

    Article  MATH  MathSciNet  Google Scholar 

  19. Granichin, O.N., Optimal Control of Linear Plant with Nonregular Bounded Noise, in Abstracts of All-Union Conf. “Theory of Adaptive Systems and Its Applications,” Moscow: VINITI, 1983, p. 26.

    Google Scholar 

  20. Barabanov, A.E. and Granichin, O.N., Optimal Controller for a Linear Plant with Bounded Noise, Autom. Remote Control, 1984, no. 5, pp. 578–584.

  21. Dahlem, M. and Pearson, J.B., ℓ-1 optimal Feedback Controllers for MIMO Discrete Systems, IEEE Trans. Automat. Control, 1987, vol. 32, no. 4, pp. 314–322.

    Article  Google Scholar 

  22. Baraniuk, R.G., Compressive Sensing, IEEE Signal Process. Magaz., 2007, vol. 52, no. 2, pp. 118–120, 124.

    Article  MathSciNet  Google Scholar 

  23. Wiener, N., Cybernetics or Control and Communication in the Animals and the Machines, New York: MIT Press, 1961. Translated under the title Kibernetika ili upravlenie i svyaz’ v zhibotnom i mashine, Moscow: Nauka, 1983.

    Google Scholar 

  24. Kotel’nikov, V.A., On the Throughput of Ether and Wire in Electrical Communication, in All-Union Energy Committee. Materials for the First All-Union Conference on Technical Reconstruction of Communication and Development of Low-current Industry, 1933. Paper reprinted in Usp. Phys. Nauk, vol. 176, no. 7, pp. 762–770.

  25. Kupfmuller, K., Uber die Dynamik der Selbsttatigen Verstarkungsregler, Elektr. Nachrichtentechnik, 1928, vol. 5, no. 11, pp. 459–467.

    Google Scholar 

  26. Nyquist, H., Certain Topics in Telegraph Transmission Theory, Trans. AIEE, 1928, vol. 47, pp. 617–644.

    Google Scholar 

  27. Shannon, C.E., Communication in the Presence of Noise, Proc. IRE, 1949, vol. 37, no. 1, pp. 10–21.

    Article  MathSciNet  Google Scholar 

  28. Wakin, M., Manifold-based Signal Recovery and Parameter Estimation from Compressive Measurements, Preprint of Colorado School of Mines, 2008.

  29. Rauhut, H., Schnass, K., and Vandergheynst, P., Compressed Sensing and Redundant Dictionaries, IEEE Trans. Inf. Theory, 2008, vol. 54, no. 5, pp. 2210–2219.

    Article  MathSciNet  Google Scholar 

  30. Cevher, V., Duarte, M., Hegde, C., and Baraniuk, R., Sparse Signal Recovery Using Markov Random Fields, in Proc. Workshop Neural Information Process. Systems (NIPS), 2008.

  31. Mishali, M. and Eldar, Y.C., Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors, IEEE Trans. Signal Process., 2008, vol. 56, no. 10, pp. 4692–4702.

    Article  MathSciNet  Google Scholar 

  32. Taubman, D.S. and Marcellin, M.V., JPEG 2000: Image Compression Fundamentals, Standards and Practice, Norwell: Kluwer, 2001.

    Google Scholar 

  33. Boufounos, P.T. and Baraniuk, R.G., 1-Bit Compressive Sensing, in 42nd Conf. Inf. Sc. Systems (CISS), Princeton, 2008, pp. 19–21.

  34. Takhar, D., Bansal, V., Wakin, M., et al., A Compressed Sensing Camera: New Theory and an Implementation Using Digital Micromirrors, in Proc. Comput. Imaging IV SPIE Electronic Imaging, San Jose, 2006.

  35. Duarte, M.F., Davenport, M.A., Takhar, D., et al., Single-Pixel Imaging via Compressive Sampling, IEEE Signal Process. Magaz., 2008, vol. 25, no. 2, pp. 83–91.

    Article  Google Scholar 

  36. Candes, E.J., The Restricted Isometry Property and Its Implications for Compressed Sensing, Comptes Rendus de l’Acad. des Sci., Serie I, 2008, vol. 346, pp. 589–592.

    MATH  MathSciNet  Google Scholar 

  37. Candes, E. and Romberg, J., Sparsity and Incoherence in Compressive Sampling, Inverse Problems, 2007, vol. 23, no. 3, pp. 969–985.

    Article  MATH  MathSciNet  Google Scholar 

  38. Candes, E. and Wakin, M., People Hearing without Listening: An Introduction to Compressive Sampling, IEEE Signal Process. Magaz., 2008, vol. 25, no. 2, pp. 21–30.

    Article  Google Scholar 

  39. Kashin, B.S., Diameters of Some Finite-dimensional Sets and Classes of Smooth Functions, Izv. Akad. Nauk SSSR, Mat., 1977, vol. 42, no. 2, pp. 334–351.

    Google Scholar 

  40. Garnaev, A.Yu. and Gluskin, E.D., On Diameters of the Euclidean Sphere, Dokl. Akad. Nauk SSSR, 1984, vol. 277, no. 5, pp. 200–204.

    MathSciNet  Google Scholar 

  41. Baraniuk, R., Davenport, M., DeVore, R., and Wakin, M., A Simple Proof of the Restricted Isometry Property for Random Matrices (aka “The Johnson-Lindenstrauss Lemma Meets Compressed Sensing”), Constr. Approx., 2007, vol. 28, no. 3, pp. 253–263.

    Article  MathSciNet  Google Scholar 

  42. Johnson, W. and Lindenstrauss, J., Extensions of Lipschitz Mapping into a Hilbert Space, in Conf. Modern Anal. Probabil., 1984, pp. 189–206.

  43. Achlioptas, D., Database-friendly Random Projections, in Proc. ACM SIGACT-SIGMOD-SIGART Symp. Principles Database Systems, 2001, pp. 274–281.

  44. Juditsky, A. and Nemirovski, A., On Verifiable Sufficient Conditions for Sparse Signal Recovery via ℓ1 Minimization, Rreprint, 2010, Available at http://arxiv.org/abs/0809.2650.

  45. Baron, D., Wakin, M.B., Duarte, M., Sarvotham, S., and Baraniuk, R.G., Distributed Compressed Sensing, 2005, Available at: http://dsp.rice.edu/cs/DCS112005.pdf.

  46. Chen, S.S., Donoho, D.L., and Saunders, M.A., Atomic Decomposition by Basis Pursuit, SIAM J. Sci. Comput., 2001, vol. 43, no. 1, pp. 129–159.

    MATH  MathSciNet  Google Scholar 

  47. Tibshirani, R., Regression Shrinkage and Selection via the Lasso, J. Royal. Statist. Soc. B, 1996, vol. 58, no. 1, pp. 267–288.

    MATH  MathSciNet  Google Scholar 

  48. Chretien, S., An Alternating l 1 Approach to the Compressed Sensing Problem, IEEE Signal Process. Lett., 2010, vol. 17, no. 2, pp. 181–184.

    Article  Google Scholar 

  49. Mohimani, H., Babaie-Zadeh, M., and Jutten, C., A Fast Approach for Overcomplete Sparse Decomposition Based on Smoothed ℓ0 Norm, IEEE Trans. Signal Process., 2009, vol. 57, no. 1, pp. 289–301.

    Article  Google Scholar 

  50. Polyak, B.T., Vvedenie v optimizatsiyu, Moscow: Nauka, 1983. Translated into English under the title Introduction to Optimization, New York: Optimization Software, 1987.

    MATH  Google Scholar 

  51. Chartrand, R., Exact Reconstructions from Surprisingly Little Data, IEEE Signal Process. Lett., 2007, vol. 14, pp. 707–710.

    Article  Google Scholar 

  52. Chartrand, R., Fast Algorithms for Nonconvex Compressive Sensing: MRI Reconstruction from Very Few Data, IEEE Int. Sympos. Biomedical Imaging (ISBI), 2009, pp. 1349–1360.

  53. Saab, R., Chartrand, R., and Yilmaz, O., Stable Sparse Approximations via Nonconvex Optimization, in Proc. 33rd Int. Conf. Acoustics, Speech, Signal Process. (ICASSP), 2008, pp. 3885–3888.

  54. Chartrand, R. and Yin, W., Iteratively Reweighted Algorithms for Compressive Sensing, in IEEE Int. Conf. Acoustics, Speech, Signal Process. (ICASSP), Las Vegas, 2008.

  55. Daubechies, I., DeVore, R., Fornasier, M., and Gunturk, S., Iteratively Re-weighted Least Squares Minimization: Proof of Faster than Linear Rate for Sparse Recovery, in 42nd Ann. Conf. Inf. Sci. Systems, 19–21 March 2008, pp. 26–29.

  56. Wipf, D. and Nagarajan, S., Iterative Reweighted 1 and 2 Methods for Finding Sparse Solutions, UC San Francisco, Technical Report, 2008.

  57. Bredies, K. and Lorenz, D.A., Iterated Hard Shrinkage for Minimization Problems with Sparsity Constraints, SIAM J. Sci. Comput., 2008, vol. 30, no. 2, pp. 657–683.

    Article  MATH  MathSciNet  Google Scholar 

  58. Herrity, K.K., Gilbert, A.C., and Tropp, J.A., Sparse Approximation via Iterative Thresholding, in ICASSP 2006 Proc., 14–16 May 2006, vol. 3, pp. 624–627.

  59. Ma, J., Improved Iterative Curvelet Thresholding for Compressed Sensing, Preprint, 2009, Available at: http://dsp.rice.edu/sites/dsp.rice.edu/files/cs/ISTcs2.pdf.

  60. Beck, A. and Teboulle, M., A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems, SIAM J. Imaging Sci., 2009, vol. 2, no. 1, pp. 183–202.

    Article  MATH  MathSciNet  Google Scholar 

  61. Bioucas-Dias, J. and Figueiredo, M., A New TwIST: Two-step Iterative Shrinkage/Thresholding Algorithms for Image Restoration, IEEE Trans. Image Process., 2007, vol. 16, no. 12, pp. 2992–3004

    Article  MathSciNet  Google Scholar 

  62. Combettes, P.L. and Wajs, V.R., Signal Recovery by Proximal Forward-Backward Splitting, SIAM J. Multiscale Modeling Simulation, 2005, vol. 4, no. 4, pp. 1168–1200.

    Article  MATH  MathSciNet  Google Scholar 

  63. Combettes, P.L. and Pesquet, J.-Ch., Proximal Splitting Methods in Signal Processing, Preprint, 2010, Available at: http://arxiv4.library.cornell.edu/abs/0912.3522.

  64. Egiazarian, K., Foi, A., and Katkovnik, V., Compressed Sensing Image Reconstruction via Recursive Spatially Adaptive Filtering, in Proc. IEEE Int. Conf. Image Process., ICIP 2007, San Antonio, 16–19 September 2007, pp. 549–552.

  65. Mallat, S. and Zhang, Z., Matching Pursuit with Time-Frequency Dictionaries, IEEE Trans. Signal Process., 1993, vol. 41, no. 12, pp. 3397–3415.

    Article  MATH  Google Scholar 

  66. Tropp, J. and Gilbert, A.C., Signal Recovery from Partial Information via OrthogonalMatching Pursuit, 2005, Available at: www-personal.umich.edu/~jtropp/papers/TG06-Signal-Recovery.pdf.

  67. Davenport, M.A. and Wakin, M.B., Analysis of Orthogonal Matching Pursuit Using the Restricted Isometry Property, Preprint, 2010, Available at: http://arxiv.org/abs/0909.0083.

  68. Needell, D. and Vershynin, R., Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit, Foundat. Comput. Math., 2009, vol. 9, pp. 317–334.

    Article  MATH  MathSciNet  Google Scholar 

  69. Needell, D. and Tropp, J.A., CoSaMP: Iterative Signal Recovery from Incomplete and Inaccurate Samples, Appl. Comp. Harmonic Anal., 2008, vol. 26, pp. 301–321.

    Article  MathSciNet  Google Scholar 

  70. Dai, W. and Milenkovic, O., Subspace Pursuit for Compressive Sensing Signal Reconstruction, IEEE Trans. Inf. Theory, 2009, vol. 55, no. 5, pp. 2230–2249.

    Article  Google Scholar 

  71. Donoho, D.L., Tsaig, Y., Drori, I., and Starck, J.-L., Sparse Solution of Underdetermined Linear Equations by Stagewise Orthogonal Matching Pursuit, Stanford Univ. Tech. Rep., 2006, Available at: http://stat.stanford.edu/idrori/StOMP.pdf.

  72. Cohen, A., Dahmen, W., and DeVore, R., Instance Optimal Decoding by Thresholding in Compressed Sensing, Preprint, 2008, Available: http://www.igpm.rwth-aachen.de/Download/reports/pdf/IGPM289.pdf.

  73. La, C. and Minh, N.D., Tree-based Orthogonal Matching Pursuit Algorithm for Signal Reconstruction, in IEEE Int. Conf. Image Process., 8–11 Oct., 2006, pp. 1277–1280.

  74. Blumensath, T. and Davies, M.E., Gradient Pursuits, IEEE Trans. Signal Process., 2008, vol. 56, no. 6, pp. 2370–2382.

    Article  MathSciNet  Google Scholar 

  75. Berinde, R. and Indyk, P., Sequential Sparse Matching Pursuit, Allerton: MIT, 2009, Available at: http://people.csail.mit.edu/indyk/ssmp.pdf.

    Google Scholar 

  76. Baron, D., Sarvotham, S., and Baraniuk, R.G., Bayesian Compressive Sensing via Belief Propagation, 2008, Available at: http://arxiv.org/abs/0812.4627.

  77. Cormode, G. and Muthukrishnan, S., Combinatorial Algorithms for Compressed Sensing, Berlin: Springer, 2006.

    Google Scholar 

  78. Jafarpour, S., Xu, W., Hassibi, B., and Calderbank, R., Efficient and Robust Compressed Sensing Using Optimized Expander Graphs, IEEE Trans. Info. Theory, 2009, vol. 55, no. 9, pp. 4299–4308.

    Article  MathSciNet  Google Scholar 

  79. Berinde, R., Indyk, P., and Ruzic, M., Practical Near-optimal Sparse Recovery in the L1 Norm, in 46th Annual Allerton Conf. Communication, Control, Computing, Allerton, 2008, pp. 198–205.

  80. Lustig, M., Donoho, D.L., Santos, J.M., and Pauly, J.M., Compressed Sensing MRI, IEEE Signal Process. Magaz., 2008, vol. 25, no. 2, pp. 72–82.

    Article  Google Scholar 

  81. Candes, E.J. and Romberg, J., Practical Signal Recovery from Random Projections, in Proc. SPIE Computational Imaging, 2005, vol. 5674, pp. 76–86.

    Google Scholar 

  82. Wakin, M., Laska, J., Duarte, M., Baron, D., Sarvotham, S., Takhar, D., Kelly, K., and Baraniuk, R., Compressive Imaging for Video Representation and Coding, in Proc. Picture Coding Sympos.—PCS 2006, Beijing, 2006, Available at: http://www.mines.edu/mwakin/papers/pcs-camera.pdf.

  83. Ma, J. and Le Dimet, F.X., Deblurring from Highly Incomplete Measurements for Remote Sensing, IEEE Trans. Geoscience Remote Sensing, 2009, vol. 47, no. 3, pp. 792–802.

    Article  Google Scholar 

  84. Katkovnik, V. and Egiazarian, K., Nonlocal Image Deblurring: Variational Formulation with Nonlocal Collaborative L0 Norm Prior, in Int. Workshop Local Non-Local Approximation in Image Process., 19–21 Aug., 2009, pp. 46–53.

  85. Calderbank, R., Jafarpour, S., and Robert Schapire, R., Compressed Learning: Universal Sparse Dimensionality Reduction and Learning in the Measurement Domain, Preprint, 2009, Available at: http://dsp.rice.edu/files/cs/cl.pdf.

  86. Hsu, D., Langford, J., Kakade, S., and Zhang, T., Multi-label Prediction via Compressed Sensing, Preprint, 2009, Available at: http://arxiv.org/abs/0902.1284.

  87. Jiyoung, C., Minwoo, K., Won, S., and Jong, C.Y., Compressed Sensing Metal Artifact Removal in Dental CT, in IEEE Int. Sympos. Biological Imaging, 2009, pp. 334–337.

  88. Kirolos, S., Laska, J., Wakin, M., et al., Analog-to-Information Conversion via Random Demodulation, in Proc. IEEE Dallas Circuits Systems Workshop, 2006, pp. 71–74.

  89. Baraniuk, R. and Steeghs, P., Compressive Radar Imaging, in IEEE Radar Conf., 2007, pp. 128–133.

  90. Vetterli, M., Marziliano, P., and Blu, T., Sampling Signals with Finite Rate of Innovation, IEEE Trans. Signal Process., 2002, vol. 50, no. 6, pp. 1417–1428.

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. St. Petersburg State University, St. Petersburg, Russia

    O. N. Granichin & D. V. Pavlenko

Authors
  1. O. N. Granichin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. D. V. Pavlenko
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Original Russian Text © O.N. Granichin, D.V. Pavlenko, 2010, published in Avtomatika i Telemekhanika, 2010, No. 11, pp. 3–28.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Granichin, O.N., Pavlenko, D.V. Randomization of data acquisition and ℓ1-optimization (recognition with compression). Autom Remote Control 71, 2259–2282 (2010). https://doi.org/10.1134/S0005117910110019

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0005117910110019

Keywords

Search

Navigation