Advertisement

Journal of Optimization Theory and Applications

, Volume 164, Issue 1, pp 109–122 | Cite as

Necessary and Sufficient Conditions of Solution Uniqueness in 1-Norm Minimization

  • Hui Zhang
  • Wotao Yin
  • Lizhi Cheng
Article

Abstract

This paper shows that the solutions to various 1-norm minimization problems are unique if, and only if, a common set of conditions are satisfied. This result applies broadly to the basis pursuit model, basis pursuit denoising model, Lasso model, as well as certain other 1-norm related models. This condition is previously known to be sufficient for the basis pursuit model to have a unique solution. Indeed, it is also necessary, and applies to a variety of 1-norm related models. The paper also discusses ways to recognize unique solutions and verify the uniqueness conditions numerically. The proof technique is based on linear programming strong duality and strict complementarity results.

Keywords

l1 minimization Basis pursuit Lasso Solution uniqueness  Strict complementarity 

Mathematics Subject Classification

65K05 90C25 

Notes

Acknowledgments

The authors thank Prof. Dirk Lorenz for bringing References [19] and [25] to their attention. The work of H. Zhang is supported by the Chinese Scholarship Council during his visit to Rice University, and in part by NUDT Funding of Innovation B110202. The work of W. Yin is supported in part by NSF Grants DMS-0748839 and ECCS-1028790. The work of L. Cheng is supported by NSFC Grants 61271014 and 61072118.

References

  1. 1.
    Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32(2), 407–499 (2004)CrossRefMATHMathSciNetGoogle Scholar
  2. 2.
    Best, M.J., Grauer, R.R.: Sensitivity analysis for mean–variance portfolio problems. Manag. Sci. 37(8), 980–989 (1991)CrossRefMATHGoogle Scholar
  3. 3.
    Donoho, D.L., Elad, M.: Optimally sparse representation in general (nonorthogonal) dictionaries via \(\ell _1\) minimization. Proc. Natl. Acad. Sci. 100(5), 2197–2202 (2003)CrossRefMATHMathSciNetGoogle Scholar
  4. 4.
    Bruckstein, A.M., Donoho, D.L., Elad, M.: From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev. 51(1), 34–81 (2009)CrossRefMATHMathSciNetGoogle Scholar
  5. 5.
    Donoho, D.L., Huo, X.: Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inf. Theory 47(7), 2845–2862 (2001)CrossRefMATHMathSciNetGoogle Scholar
  6. 6.
    Elad, M., Bruckstein, A.M.: A generalized uncertainty principle and sparse representation in pairs of bases. IEEE Trans. Inf. Theory 48(9), 2558–2567 (2002)CrossRefMATHMathSciNetGoogle Scholar
  7. 7.
    Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)CrossRefMATHMathSciNetGoogle Scholar
  8. 8.
    Cohen, A., Dahmen, W., DeVore, R.: Compressed sensing and best \(k\)-term approximation. J. Am. Math. Soc. 22(1), 211–231 (2009)CrossRefMATHMathSciNetGoogle Scholar
  9. 9.
    Candes, E.J., Tao, T.: Decoding by linear programming. IEEE Trans. Inf. Theory 51(12), 4203–4215 (2005)CrossRefMATHMathSciNetGoogle Scholar
  10. 10.
    Zhang, Y.: Theory of compressive sensing via \(\ell _1\)-minimization: a non-RIP analysis and extensions. J. Oper. Res. Soc. China 1(1), 79–105 (2008)CrossRefGoogle Scholar
  11. 11.
    Candes, E.J., Plan, Y.: A probabilistic and RIPless theory of compressed sensing. IEEE Trans. Inf. Theory 57(11), 7235–7254 (2011)CrossRefMathSciNetGoogle Scholar
  12. 12.
    Chvatal, V.: Linear Programming. W.H. Freeman and Company, New York (1983)MATHGoogle Scholar
  13. 13.
    Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20(1), 33–61 (1998)CrossRefMathSciNetGoogle Scholar
  14. 14.
    Tibshirani, R.: Regression shrinkage and selection via the LASSO. J. R. Stat. Soc. Ser. B (Methodological) 58, 267–288 (1996)MATHMathSciNetGoogle Scholar
  15. 15.
    Fuchs, J.J.: On sparse representations in arbitrary redundant bases. IEEE Trans. Inf. Theory 50(6), 1341–1344 (2004)CrossRefMATHGoogle Scholar
  16. 16.
    Candès, E., Recht, B.: Simple bounds for recovering low-complexity models. Math. Program. 141, 577–589 (2013)CrossRefMATHMathSciNetGoogle Scholar
  17. 17.
    Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)CrossRefMATHGoogle Scholar
  18. 18.
    Tropp, J.A.: Recovery of short, complex linear combinations via \(\ell _1\) minimization. IEEE Trans. Inf. Theory 51(4), 1568–1570 (2005)CrossRefMATHMathSciNetGoogle Scholar
  19. 19.
    Grasmair, M., Scherzer, O., Haltmeier, M.: Necessary and sufficient conditions for linear convergence of \(\ell _1\)-regularization. Commun. Pure Appl. Math. 64(2), 161–182 (2011)CrossRefMATHMathSciNetGoogle Scholar
  20. 20.
    Dossal, C.: A necessary and sufficient condition for exact sparse recovery by \(\ell _1\) minimization. Comptes Rendus Mathematique 350(1), 117–120 (2012)CrossRefMATHMathSciNetGoogle Scholar
  21. 21.
    Fuchs, J.J.: Recovery of exact sparse representations in the presence of bounded noise. IEEE Trans. Inf. Theory 51(10), 3601–3608 (2005)CrossRefMATHGoogle Scholar
  22. 22.
    Tibshirani, R.J.: The LASSO problem and uniqueness. Electron. J.Stat. 7, 1456–1490 (2013)CrossRefMATHMathSciNetGoogle Scholar
  23. 23.
    Dantzig, G.B.: Linear Programming and Extensions. Princeton university press, Princeton, NJ (1998)MATHGoogle Scholar
  24. 24.
    Güler, O., Ye, Y.: Convergence behavior of interior-point algorithms. Math. Program. 60(1–3), 215–228 (1993)CrossRefMATHGoogle Scholar
  25. 25.
    Lorenz, D.A.: Constructing test instances for basis pursuit denoising. IEEE Trans. Signal Process. 61(5–8), 1210–1214 (2013)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Department of Mathematics and Systems Science, College of ScienceNational University of Defense TechnologyChangshaChina
  2. 2.Department of MathematicsUniversity of CaliforniaLos AngelesUSA

Personalised recommendations