Skip to main content
SpringerLink
Log in

A new piecewise quadratic approximation approach for L0 norm minimization problem

  • Articles
  • Published:
Science China Mathematics Aims and scope Submit manuscript

Abstract

In this paper, we consider the problem of finding sparse solutions for underdetermined systems of linear equations, which can be formulated as a class of L0 norm minimization problem. By using the least absolute residual approximation, we propose a new piecewise quadratic function to approximate the L0 norm. Then, we develop a piecewise quadratic approximation (PQA) model where the objective function is given by the summation of a smooth non-convex component and a non-smooth convex component. To solve the (PQA) model, we present an algorithm based on the idea of the iterative thresholding algorithm and derive the convergence and the convergence rate. Finally, we carry out a series of numerical experiments to demonstrate the performance of the proposed algorithm for (PQA). We also conduct a phase diagram analysis to further show the superiority of (PQA) over L1 and L1/2 regularizations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Beck A, Teboulle M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imaging Sci, 2009, 2: 183–202

    Article  MathSciNet  MATH  Google Scholar 

  2. Becker S, Bobin J, Cands E J. NESTA: A fast and accurate first-order method for sparse recovery. SIAM J Imaging Sci, 2011, 4: 1–39

    Article  MathSciNet  MATH  Google Scholar 

  3. Bioucas-Dias J M, Figueiredo M A. A new TwIST: Two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans Image Process, 2007, 16: 2992–3004

    Article  MathSciNet  Google Scholar 

  4. Blumensath T, Davies M E. Iterative thresholding for sparse approximations. J Fourier Anal Appl, 2008, 14: 629–654

    Article  MathSciNet  MATH  Google Scholar 

  5. Blumensath T, Davies M E. Iterative hard thresholding for compressed sensing. Appl Comput Harmon Anal, 2008, 27: 265–274

    Article  MathSciNet  MATH  Google Scholar 

  6. Bredies K, Lorenz D A, Reiterer S. Minimization of non-smooth, non-convex functionals by iterative thresholding. J Optim Theory Appl, 2014, 165: 78–112

    Article  MathSciNet  MATH  Google Scholar 

  7. Bruckstein A M, Donoho D L, Elad M. From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev, 2009, 51: 34–81

    Article  MathSciNet  MATH  Google Scholar 

  8. Candes E J, Recht B. Exact matrix completion via convex optimization. Found Comput Math, 2008, 9: 717–772

    Article  MathSciNet  MATH  Google Scholar 

  9. Candes E J, Romberg J, Tao T. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans Inform Theory, 2006, 52: 489–509

    Article  MathSciNet  MATH  Google Scholar 

  10. Candes E J, Wakin M B, Boyd S P. Enhancing sparsity by reweighted l 1 minimization. J Fourier Anal Appl, 2008, 14: 877–905

    Article  MathSciNet  MATH  Google Scholar 

  11. Cao W F, Sun J, Xu Z B. Fast image deconvolution using closed-form thresholding formulas of L q (q = 1/2, 2/3) regularization. J Vis Comm Image Represent, 2013, 24: 1529–1542

    Article  Google Scholar 

  12. Chen S S, Donoho D L, Saunders M A. Atomic decomposition by basis pursuit. SIAM J Sci Comput, 1998, 20: 33–61

    Article  MathSciNet  MATH  Google Scholar 

  13. Daubechies I, Defrise M, Christine D M. An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Comm Pure Appl Math, 2004, 57: 1413–1457

    Article  MathSciNet  MATH  Google Scholar 

  14. Daubechies I, Devore R, Fornasier M, et al. Iteratively reweighted least squares minimization for sparse recovery. Comm Pure Appl Math, 2010, 63: 1–38

    Article  MathSciNet  MATH  Google Scholar 

  15. Devore R, Jawerth B. Image compression through wavelet transform coding. IEEE Trans Inform Theory, 1992, 38: 719–746

    Article  MathSciNet  MATH  Google Scholar 

  16. Donoho D L. De-noising by soft-thresholding. IEEE Trans Inform Theory, 1995, 41: 613–627

    Article  MathSciNet  MATH  Google Scholar 

  17. Donoho D L. Compressed sensing. IEEE Trans Inform Theory, 2006, 52: 1289–1306

    Article  MathSciNet  MATH  Google Scholar 

  18. Efron B, Hastie T. Least angle regression. Ann Statist, 2004, 32: 407–499

    Article  MathSciNet  MATH  Google Scholar 

  19. Fan J, Li R. Variable selection via nonconcave penalized likelihood and its oracle properties. J Amer Statist Assoc, 2001, 96: 1348–1360

    Article  MathSciNet  MATH  Google Scholar 

  20. Gasso G, Rakotomamonjy A, Canu S. Recovering sparse signals with a certain family of non-convex penalties and DC programming. IEEE Trans Signal Process, 2009, 57: 4686–4698

    Article  MathSciNet  MATH  Google Scholar 

  21. Geman D, Reynolds G. Constrained restoration and the recovery of discontinuities. IEEE Trans Pattern Anal Mach Intell, 1992, 14: 367–383

    Article  Google Scholar 

  22. Geman D, Yang C. Nonlinear image recovery with half-quadratic regularization. IEEE Trans Image Process, 1995, 4: 932–946

    Article  Google Scholar 

  23. Gong P H, Zhang C S, Lu Z S, et al. A general iteartive shrinkage and thresholding algorithm for non-convex regularized optimization problems. Proc Int Conf Mach Learn, 2013, 28: 37–45

    Google Scholar 

  24. Gorodnitsky I F, Rao B D. Sparse signal reconstruction from limited data using FOCUSS: A re-weighted minimum norm algorithm. IEEE Trans Signal Process, 1997, 45: 600–616

    Article  Google Scholar 

  25. Hale E T, Yin W, Zhang Y. A fixed-point continuation method for L 1-minimization: Methodology and convergence. SIAM J Optim, 2008, 19: 1107–1130

    Article  MathSciNet  MATH  Google Scholar 

  26. Lai M J, Xu Y Y, YinWT. Improved iteratively reweighted least squares for unconstrained smoothed L q minimization. SIAM J Numer Anal, 2013, 51: 927–957

    Article  MathSciNet  MATH  Google Scholar 

  27. Lai M J, Yin W T. Augmented L 1 and nuclear-norm models with a globally linearly convergent algorithm. SIAM J Imaging Sci, 2013, 6: 1059–1091

    Article  MathSciNet  MATH  Google Scholar 

  28. Moreau J J. Proximite et dualite dans un espace hilbertien. Bull Soc Math France, 1965, 93: 273–299

    Article  MathSciNet  MATH  Google Scholar 

  29. Natarajan B K. Sparse approximate solutions to linear systems. SIAM J Comput, 1995, 24: 227–234

    Article  MathSciNet  MATH  Google Scholar 

  30. Nesterov Y. Introductory Lectures on Convex Optimization: A Basic Course. Boston: Springer, 2003

    MATH  Google Scholar 

  31. Parikh N, Boyd S. Proximal algorithms. Found Trends Optim, 2013, 1: 123–231

    Google Scholar 

  32. Qian Y T, Jia S, Zhou J, et al. Hyperspectral unmixing via sparsity-constrained nonnegative matrix factorization. IEEE Trans Geosci Remote Sens, 2011, 49: 4282–4297

    Article  Google Scholar 

  33. Rakotomamonjy A, Flamary R, Gasso G, et al. L p-L q penalty for sparse linear and sparse multiple kernel multitask learning. IEEE Trans Neural Netw, 2011, 22: 1307–1320

    Article  Google Scholar 

  34. Tibshirani R. Regression shrinkage and selection via the LASSO. J R Stat Soc Ser B, 1996, 58: 267–288

    MathSciNet  MATH  Google Scholar 

  35. Wen B, Chen X J, Pong T K. Linear convergence of proximal gradient algorithm with extrapolation for a class of nonconvex nonsmooth minimizaiton porblems. SIAM J Optimi, 2017, 27: 124–145

    Article  MATH  Google Scholar 

  36. Wu B. High-dimensional analysis on matrix decomposition with applications to correlation matrix estimation in factor models. PhD Thesis. Singapore: National University of Singapore, 2014

    Google Scholar 

  37. Xu Z B. Data modeling: Visual psychology approach and L 1/2 regularization theory. In: Proceedings of the Interna-tional Congress of Mathematicians. Berlin: International Mathematical Union, 2010, 3151–3184

    Google Scholar 

  38. Xu Z B, Chang X Y, Xu F M, et al. L 1/2 regularization: A thresholding representation theory and a fast solver. IEEE Trans Neural Netw Learn Syst, 2012, 23: 1013–1027

    Article  Google Scholar 

  39. Xu Z B, Guo H L, Wang Y, et al. Representative of L 1/2 regularization among L q (0 < q < 1) regularizations: An experimental study based on phase diagram. Acta Automat Sinica, 2012, 38: 1225–1228

    MathSciNet  Google Scholar 

  40. Xu Z B, Zhang H, Wang Y, et al. L 1/2 regularization. Sci China Inf Sci, 2010, 53: 1159–1169

    Article  MathSciNet  Google Scholar 

  41. Yang A Y, Ganesh A, Ma Y. Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell, 2009, 31: 210–227

    Article  Google Scholar 

  42. Yang A Y, Ganesh A, Zhou Z H, et al. Fast L 1-minimization algorithms for robust face recognition. IEEE Trans Image Process, 2013, 22: 3234–3246

    Article  Google Scholar 

  43. Yin P H, Lou Y F, He Q, et al. Minimization of L 1–2 for compressed sensing. SIAM J Sci Comput, 2015, 37: 536–563

    Article  MathSciNet  Google Scholar 

  44. Zeng J S, Lin S B, Wang Y, et al. L 1/2 regularization: convergence of iterative half thresholding algorithm. IEEE Trans Signal Process, 2014, 62: 2317–2329

    Article  MathSciNet  MATH  Google Scholar 

  45. Zeng J S, Lin S B, Xu Z B. Sparse regularization: Convergence of iterative jumping thresholding algorithm. IEEE Trans Signal Process, 2014, 64: 5106–5118

    Article  MathSciNet  Google Scholar 

  46. Zhang C H. Nearly unbiaised variable selection under minimax concave penalty. Ann Statist, 2010, 38: 894–942

    Article  MathSciNet  MATH  Google Scholar 

  47. Zhang T. Analysis of multi-stage convex relaxation for sparse regularization. J Mach Learn Res, 2010, 11: 1081–1107

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant No. 11771275).

Author information

Authors and Affiliations

  1. Department of Mathematics, Shanghai University, Shanghai, 200444, China

    Qian Li, Yanqin Bai & Changjun Yu

  2. Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, 100190, China

    Ya-xiang Yuan

Authors
  1. Qian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanqin Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Changjun Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ya-xiang Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanqin Bai.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Q., Bai, Y., Yu, C. et al. A new piecewise quadratic approximation approach for L0 norm minimization problem. Sci. China Math. 62, 185–204 (2019). https://doi.org/10.1007/s11425-017-9315-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11425-017-9315-9

Keywords

MSC(2010)

Search

Navigation