Skip to main content
SpringerLink
Log in

Compressive Algorithms—Adaptive Solutions of PDEs and Variational Problems

  • Conference paper
Mathematics of Surfaces XIII (Mathematics of Surfaces 2009)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 5654))

Included in the following conference series:

Abstract

This paper is concerned with an overview of the main concepts and a few significant applications of a class of adaptive iterative algorithms which allow for dimensionality reductions when used to solve large scale problems. We call this class of numerical methods Compressive Algorithms. The introduction of this paper presents an historical excursus on the developments of the main ideas behind compressive algorithms and stresses the common features of diverse applications. The first part of the paper is addressed to the optimal performances of such algorithms when compared with known benchmarks in the numerical solution of elliptic partial differential equations. In the second part we address the solution of inverse problems both with sparsity and compressibility constraints. We stress how compressive algorithms can stem from variational principles. We illustrate the main results and applications by a few significant numerical examples. We conclude by pointing out future developments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alexeev, B., Ward, R.: On the complexity of free-discontinuity and ℓ0 regularization in image processing (preprint) (2009)

    Google Scholar 

  2. Ambrosio, L., Fusco, N., Pallara, D.: Functions of Bounded Variation and Free Discontinuity Problems. Oxford Mathematical Monographs, xviii. Clarendon Press, Oxford (2000)

    MATH  Google Scholar 

  3. Aubert, G., Kornprobst, P.: Mathematical Problems in Image Processing. In: Partial Differential Equations and the Calculus of Variation. Springer, Heidelberg (2002)

    Google Scholar 

  4. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  5. Binev, P., Dahmen, W., DeVore, R.: Adaptive finite element methods with convergence rates. Numer. Math. 97(2), 219–268 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  6. Blake, A., Zisserman, A.: Visual Reconstruction. MIT Press, Cambridge (1987)

    Google Scholar 

  7. Bourdin, B., Chambolle, A.: Implementation of an adaptive finite-element approximation of the Mumford-Shah functional. Numer. Math. 85(4), 609–646 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  8. Candès, E.J.: Compressive sampling. In: International Congress of Mathematicians. Eur. Math. Soc., Zürich, vol. III, pp. 1433–1452 (2006)

    Google Scholar 

  9. Candès, E., Romberg, J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Comm. Pure Appl. Math. 59(8), 1207–1223 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  10. Candès, E., Tao, T.: Near optimal signal recovery from random projections: universal encoding strategies? IEEE Trans. Inform. Theory 52(12), 5406–5425 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  11. Candès, E.J., Romberg, J., Tao, T.: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)

    Article  MATH  Google Scholar 

  12. Candès, E.J., Tao, T.: Decoding by linear programming. IEEE Trans. Inform. Theory 51(12), 4203–4215 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  13. Candès, E.J., Wakin, M., Boyd, S.: Enhancing sparsity by reweighted l1 minimization (Technical Report, California Institute of Technology)

    Google Scholar 

  14. Chambolle, A.: Image segmentation by variational methods: Mumford and Shah functional and the discrete approximations. SIAM J. Appl. Math. 55(3), 827–863 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  15. Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vision 20(1-2), 89–97 (2004)

    MathSciNet  MATH  Google Scholar 

  16. Chambolle, A., DeVore, R.A., Lee, N.-Y., Lucier, B.J.: Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage. IEEE Trans. Image Process. 7(3), 319–335 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  17. Chambolle, A., Lions, P.-L.: Image recovery via total variation minimization and related problems. Numer. Math. 76(2), 167–188 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  18. Chambolle, A., Dal Maso, G.: Discrete approximation of the Mumford-Shah functional in dimension two. M2AN Math. Model. Numer. Anal. 33(4), 651–672 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  19. Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 1, 33–61 (1998) (Reprint)

    Article  MathSciNet  MATH  Google Scholar 

  20. Christensen, O.: An Introduction to Frames and Riesz Bases. Birkhäuser, Boston (2003)

    Google Scholar 

  21. Claerbout, J.F., Muir, F.: Robust modeling with erratic data. Geophysics 38(5), 826–844 (1973)

    Article  Google Scholar 

  22. Cohen, A., Dahmen, W., DeVore, R.: Adaptive wavelet methods for elliptic operator equations — Convergence rates. Math. Comp. 70, 27–75 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  23. Cohen, A., Dahmen, W., DeVore, R.: Adaptive wavelet methods II: Beyond the elliptic case. Found. Comput. Math. 2(3), 203–245 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  24. Cohen, A., Dahmen, W., DeVore, R.: Adaptive wavelet schemes for nonlinear variational problems. SIAM J. Numer. Anal. 41(5), 1785–1823 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  25. Cohen, A., Hoffmann, M., Reiss, M.: Adaptive wavelet Galerkin methods for linear inverse problems. SIAM J. Numer. Anal. 42(4), 1479–1501 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  26. Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simul. 4(4), 1168–1200 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  27. Dahlke, S.: Besov regularity for elliptic boundary value problems on polygonal domains. Applied Mathematics Letters 12, 31–38 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  28. Dahlke, S., Dahmen, W., DeVore, R.: Nonlinear approximation and adaptive techniques for solving elliptic operator equations. In: Dahmen, W., Kurdila, A., Oswald, P. (eds.) Multiscale Wavelet Methods for Partial Differential Equations, pp. 237–283. Academic Press, San Diego (1997)

    Chapter  Google Scholar 

  29. Dahlke, S., Dahmen, W., Hochmuth, R., Schneider, R.: Stable multiscale bases and local error estimation for elliptic problems. Appl. Numer. Math. 23, 21–48 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  30. Dahlke, S., Dahmen, W., Urban, K.: Adaptive wavelet methods for saddle point problems — Optimal convergence rates. SIAM J. Numer. Anal. 40(4), 1230–1262 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  31. Dahlke, S., DeVore, R.: Besov regularity for elliptic boundary value problems. Commun. Partial Differ. Equations 22(1&2), 1–16 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  32. Dahlke, S., Fornasier, M., Primbs, M., Raasch, T., Werner, M.: Nonlinear and adaptive frame approximation schemes for elliptic pdes: Theory and numerical experiments. To appear in Numerical Methods for Partial Differential Equations (2009)

    Google Scholar 

  33. Dahlke, S., Fornasier, M., Raasch, T.: Adaptive frame methods for elliptic operator equations. Adv. Comput. Math. 27(1), 27–63 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  34. Dahlke, S., Fornasier, M., Raasch, T., Stevenson, R., Werner, M.: Adaptive frame methods for elliptic operator equations: The steepest descent approach. IMA J. Numer. Anal. 27(4), 717–740 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  35. Dahlke, S., Hochmuth, R., Urban, K.: Adaptive wavelet methods for saddle point problems. M2AN Math. Model. Numer. Anal. 34, 1003–1022 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  36. Dahmen, W., Schneider, R.: Wavelets with complementary boundary conditions — Function spaces on the cube. Result. Math. 34(3–4), 255–293 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  37. Dahmen, W., Schneider, R.: Composite wavelet bases for operator equations. Math. Comp. 68, 1533–1567 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  38. Dahmen, W., Schneider, R.: Wavelets on manifolds I. Construction and domain decomposition. SIAM J. Math. Anal. 31, 184–230 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  39. Dal Maso, G.: An Introduction to Γ-Convergence. Birkhäuser, Boston (1993)

    Book  Google Scholar 

  40. Daubechies, I.: Ten Lectures on Wavelets. SIAM, Philadelphia (1992)

    Book  MATH  Google Scholar 

  41. Daubechies, I., Defrise, M., DeMol, C.: An iterative thresholding algorithm for linear inverse problems. Comm. Pure Appl. Math. 57(11), 1413–1457 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  42. Daubechies, I., DeVore, R., Fornasier, M., Güntürk, S.: Iteratively re-weighted least squares minimization for sparse recovery. To appear in Commun. Pure Appl. Math. (2009); arXiv:0807.0575

    Google Scholar 

  43. Daubechies, I., Fornasier, M., Loris, I.: Accelerated projected gradient methods for linear inverse problems with sparsity constraints. J. Fourier Anal. Appl. 14(5-6), 764–792 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  44. Daubechies, I., Teschke, G.: Variational image restoration by means of wavelets: Simultaneous decomposition, deblurring, and denoising. Appl. Comput. Harmon. Anal. 19(1), 1–16 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  45. Daubechies, I., Teschke, G., Vese, L.: Iteratively solving linear inverse problems under general convex constraints. Inverse Problems and Imaging 1, 29–46 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  46. Daudet, L., Torrésani, B.: Hybrid representations for audiophonic signal encoding. Signal Processing 82(11), 1595–1617 (2002)

    Article  MATH  Google Scholar 

  47. DeVore, R.: Nonlinear approximation. Acta Numerica 7, 51–150 (1998)

    Article  MATH  Google Scholar 

  48. Donoho, D.L.: Superresolution via sparsity constraints. SIAM J. Math. Anal. 23(5), 1309–1331 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  49. Donoho, D.L.: De-noising by soft-thresholding. IEEE Trans. Inf. Theory 41(3), 613–627 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  50. Donoho, D.L.: Nonlinear solution of linear inverse problems by wavelet-vaguelette decomposition. Appl. Comput. Harmon. Anal. 2(2), 101–126 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  51. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  52. Donoho, D.L.: High-dimensional centrally symmetric polytopes with neighborliness proportional to dimension. Discrete Comput. Geom. 35(4), 617–652 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  53. Donoho, D.L., Johnstone, I.M.: Ideal spatial adaptation by wavelet shrinkage. Biometrika 81(3), 425–455 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  54. Donoho, D.L., Johnstone, I.M., Kerkyacharian, G., Picard, D.: Wavelet shrinkage: asymptopia? J. Roy. Statist. Soc. Ser. B 57(2), 301–369 (1995)

    MathSciNet  MATH  Google Scholar 

  55. Donoho, D.L., Logan, B.F.: Signal recovery and the large sieve. SIAM J. Appl. Math. 52(2), 557–591 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  56. Donoho, D.L., Stark, P.B.: Uncertainty principles and signal recovery. SIAM J. Appl. Math. 49(3), 906–931 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  57. Donoho, D.L., Tanner, J.: Sparse nonnegative solutions of underdetermined linear equations by linear programming. Proc. Nat. Acad. Sci. 102(27), 9446–9451 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  58. Donoho, D.L., Tanner, J.: Counting faces of randomly-projected polytopes when the projection radically lowers dimension (preprint) (2006)

    Google Scholar 

  59. Duffin, R.J., Schaeffer, A.C.: A class of nonharmonic Fourier series. Trans. Amer. Math. Soc. 72, 341–366 (1952)

    Article  MathSciNet  MATH  Google Scholar 

  60. Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. In: Mathematics and its Applications (Dordrecht), p. 375. Kluwer Academic Publishers, Dordrecht (1996)

    Google Scholar 

  61. Figueiredo, M., Nowak, R., Wright, S.J.: Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems. IEEE J. Selected Topics in Signal Process 4(1), 586–597 (2007)

    Article  Google Scholar 

  62. Fornasier, M.: Quasi-orthogonal decompositions of structured frames. J. Math. Anal. Appl. 289(1), 180–199 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  63. Fornasier, M.: Domain decomposition methods for linear inverse problems with sparsity constraints. Inverse Problems 23, 2505–2526 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  64. Fornasier, M., Langer, A., Schönlieb, C.B.: Domain decomposition methods for compressed sensing. In: Proceedings of SampTA 2009, Marseilles (2009)

    Google Scholar 

  65. Fornasier, M., Ramlau, R., Teschke, G.: The application of joint sparsity and total variation minimization algorithms to a real-life art restoration problem. To appear in Adv. Comput. Math. (2009)

    Google Scholar 

  66. Fornasier, M., Rauhut, H.: Recovery algorithms for vector valued data with joint sparsity constraints. SIAM J. Numer. Anal. 46(2), 577–613 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  67. Fornasier, M., Rauhut, H.: Iterative thresholding algorithms. Appl. Comput. Harmon. Anal. 25(2), 187–208 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  68. Fornasier, M., Schönlieb, C.B.: Subspace correction methods for total variation and ℓ1-minimization. To appear in SIAM J. Numer. Anal. (2009)

    Google Scholar 

  69. Fornasier, M., Ward, R.: Iterative thresholding meets free-discontinuity problems, arXiv:0901.2605 (2009)

    Google Scholar 

  70. Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell 6, 721–741 (1984)

    Article  MATH  Google Scholar 

  71. Gilbert, A.C., Strauss, M., Tropp, J.A., Vershynin, R.: One sketch for all: Fast algorithms for compressed sensing (preprint) (2006)

    Google Scholar 

  72. Gilbert, A.C., Tropp, J.A.: Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans. Inform. Theory 53(12), 4655–4666 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  73. Gribonval, R., Nielsen, M.: Highly sparse representations from dictionaries are unique and independent of the sparseness measure. Appl. Comput. Harmon. Anal. 22(3), 335–355 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  74. Gröchenig, K.: Foundations of Time–Frequency Analysis. Birkhäuser, Basel (2000)

    MATH  Google Scholar 

  75. Kim, S.-J., Koh, K., Lustig, M., Boyd, S., Gorinevsky, D.: A method for large-scale ℓ1-regularized least squares problems with applications in signal processing and statistics. IEEE Journal on Selected Topics in Signal Process (2007)

    Google Scholar 

  76. Loris, I.: On the performance of algorithms for the minimization of ℓ1-penalized functionals. Inverse Problems 25(3), 035008 (2009)

    Google Scholar 

  77. Loris, I., Nolet, G., Daubechies, I., Dahlen, F.A.: Tomographic inversion using l 1-norm regularization of wavelet coefficients. Geophys. J. Int. 170, 359–370 (2007)

    Article  Google Scholar 

  78. Mallat, S.: A Wavelet Tour of Signal Processing, 2nd edn. Academic Press, San Diego (1999)

    MATH  Google Scholar 

  79. Morin, P., Nochetto, R.H., Siebert, K.G.: Convergence of adaptive finite element methods. SIAM Rev. 44, 631–658 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  80. Mumford, D., Shah, K.: Optimal approximation by piecewise smooth functions and associated variational problems. Communications on Pure and Applied Mathematics 42, 577–684 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  81. Nesterov, Y.: Smooth minimization of non-smooth functions. Mathematic Programming, Ser. A 103, 127–152 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  82. Osher, S., Burger, M., Goldfarb, D., Xu, J., Yin, W.: An Iterative Regularization Method for Total Variation-Based Image Restoration. Multiscale Model. Simul. 4(2), 460–489 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  83. Oswald, P.: Frames and space splittings in Hilbert spaces, Survey lectures on multilevel schemes for elliptic problems in Sobolev spaces (1997), http://www.faculty.iu-bremen.de/poswald/bonn1.pdf

  84. Oswald, P.: Multilevel frames and Riesz bases in Sobolev spaces, Survey lectures on multilevel schemes for elliptic problems in Sobolev spaces (1997), http://www.faculty.iu-bremen.de/poswald/bonn2.pdf

  85. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1-4), 259–268 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  86. Santosa, F., Symes, W.W.: Linear inversion of band-limited reflection seismograms. SIAM J. Sci. Stat. Comput. 7(4), 1307–1330 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  87. Starck, J.-L., Candès, E.J., Donoho, D.L.: Astronomical image representation by curvelet transform. Astronomy and Astrophysics 298, 785–800 (2003)

    Article  Google Scholar 

  88. Starck, J.-L., Nguyen, M.K., Murtagh, F.: Wavelets and curvelets for image deconvolution: a combined approach. Signal Proc. 83, 2279–2283 (2003)

    Article  MATH  Google Scholar 

  89. Stevenson, R.: Adaptive solution of operator equations using wavelet frames. SIAM J. Numer. Anal. 41(3), 1074–1100 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  90. Stevenson, R.: An optimal adaptive finite element method. SIAM J. Numer. Anal. 42(5), 2188–2217 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  91. Stevenson, R.: Optimality of a standard adaptive finite element method. Found. Comput. Math. 7(2), 245–269 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  92. Stevenson, R., Werner, M.: Computation of differential operators in aggregated wavelet frame coordinates. IMA J. Numer. Anal. 28(2), 354–381 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  93. Stevenson, R., Werner, M.: A multiplicative Schwarz adaptive wavelet method for elliptic boundary value problems. Math. Comput. 78(266), 619–644 (2009)

    Article  MATH  Google Scholar 

  94. Tadmor, E.: Numerical methods for nonlinear partial differential equations (preprint) (2009)

    Google Scholar 

  95. Taylor, H.L., Banks, S.C., McCoy, J.F.: Deconvolution with the ℓ1 norm. Geophysics 44(1), 39–52 (1979)

    Article  Google Scholar 

  96. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58(1), 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  97. Vese, L.: A study in the BV space of a denoising-deblurring variational problem. Appl. Math. Optim. 44, 131–161 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  98. Weiss, P., Blanc-Féraud, L., Aubert, G.: Efficient schemes for total variation minimization under constraints in image processing. To appear in SIAM J. Sci. Comput. (2009)

    Google Scholar 

  99. Zou, J., Gilbert, A.C., Strauss, M., Daubechies, I.: Theoretical and experimental analysis of a randomized algorithm for sparse Fourier transform analysis. J. Comput. Phys. 211, 572–595 (2005)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Johann Radon Institute for Computational and Applied Mathematics, Austria

    M. Fornasier

Authors
  1. M. Fornasier
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Computer Vision and Pattern Recognition Group,Computer Science, University of York Heslington, YO10-5DD, York, United Kingdom

    Edwin R. Hancock

  2. School of Computer Science, Cardiff University, Cardiff, UK

    Ralph R. Martin

  3. Numerical Geometry Ltd, 26 Abbey Lane, Lode, CB5 9EP, Cambridge, UK

    Malcolm A. Sabin

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Fornasier, M. (2009). Compressive Algorithms—Adaptive Solutions of PDEs and Variational Problems. In: Hancock, E.R., Martin, R.R., Sabin, M.A. (eds) Mathematics of Surfaces XIII. Mathematics of Surfaces 2009. Lecture Notes in Computer Science, vol 5654. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03596-8_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-03596-8_9

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-03595-1

  • Online ISBN: 978-3-642-03596-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation