Global optimality condition and fixed point continuation algorithm for non-Lipschitz ℓ p regularized matrix minimization
- 9 Downloads
Abstract
Regularized minimization problems with nonconvex, nonsmooth, even non-Lipschitz penalty functions have attracted much attention in recent years, owing to their wide applications in statistics, control, system identification and machine learning. In this paper, the non-Lipschitz ℓ p (0 < p < 1) regularized matrix minimization problem is studied. A global necessary optimality condition for this non-Lipschitz optimization problem is firstly obtained, specifically, the global optimal solutions for the problem are fixed points of the so-called p-thresholding operator which is matrix-valued and set-valued. Then a fixed point iterative scheme for the non-Lipschitz model is proposed, and the convergence analysis is also addressed in detail. Moreover, some acceleration techniques are adopted to improve the performance of this algorithm. The effectiveness of the proposed p-thresholding fixed point continuation (p-FPC) algorithm is demonstrated by numerical experiments on randomly generated and real matrix completion problems.
Keywords
ℓp regularized matrix minimization matrix completion problem p-thresholding operator global optimality condition fixed point continuation algorithmMSC(2010)
90C06 90C26 90C46 65F22 65F30Preview
Unable to display preview. Download preview PDF.
Notes
Acknowledgments
This work was supported by National Natural Science Foundation of China (Grant Nos. 11401124 and 71271021), the Scientific Research Projects for the Introduced Talents of Guizhou University (Grant No. 201343) and the Key Program of Natural Science Foundation of China (Grant No. 11431002). The authors are thankful to the two anonymous referees for their valuable suggestions and comments that helped us to revise the paper into the present form.
References
- 1.Attouch H, Bolte J. On the convergence of proximal algorithm for nonsmooth function involving analytic features. Math Program, 2009, 116: 5–16MathSciNetCrossRefMATHGoogle Scholar
- 2.Attouch H, Bolte J, Svaiter B F. Convergence of descent methods for semi-algebraic and tame problems: Proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Math Program, 2013, 137: 91–129MathSciNetCrossRefMATHGoogle Scholar
- 3.Cai J, Candès E, Shen Z. A singular value thresholding algorithm for matrix completion. SIAM J Optim, 2010, 20: 1956–1982MathSciNetCrossRefMATHGoogle Scholar
- 4.Candès E, Plan Y. Matrix completion with noise. In: Proceedings of the IEEE, vol. 98. New York: IEEE, 2010, 925–936CrossRefGoogle Scholar
- 5.Candès E, Recht B. Exact matrix completion via convex optimization. Found Comput Math, 2009, 9: 717–772MathSciNetCrossRefMATHGoogle Scholar
- 6.Candès E, Tao T. The power of convex relaxation: Near-optimal matrix completion. IEEE Trans Inform Theory, 2010, 56: 2053–2080MathSciNetCrossRefMATHGoogle Scholar
- 7.Cao W, Sun J, Xu Z. Fast image deconvolution using closed-form thresholding formulas of L q (q = 1/2, 2/3) regular-ization. J Vis Commun Image Represent, 2013, 24: 31–41CrossRefGoogle Scholar
- 8.Chartrand R. Exact reconstructions of sparse signals via nonconvex minimization. IEEE Signal Process Lett, 2007, 14: 707–710CrossRefGoogle Scholar
- 9.Chen X, Ge D, Wang Z, et al. Complexity of unconstrained l 2-l p minimization. Math Program, 2014, 143: 371–383MathSciNetCrossRefGoogle Scholar
- 10.Chen X, Niu L, Yuan Y. Optimality conditions and smoothing trust region Newton method for non-Lipschitz opti-mization. SIAM J Optim, 2013, 23: 1528–1552MathSciNetCrossRefMATHGoogle Scholar
- 11.Chen X, Xu F, Ye Y. Lower bound theory of nonzero entries in solutions of l 2-l p minimization. SIAM J Sci Comput, 2010, 32: 2832–2852MathSciNetCrossRefMATHGoogle Scholar
- 12.Chen Y, Xiu N, Peng D. Global solutions of non-Lipschitz S 2-S p minimization over the positive semidefinite cone. Optim Lett, 2014, 8: 2053–2064MathSciNetCrossRefMATHGoogle Scholar
- 13.Daubechies I, DeVore R, Fornasier M, et al. Iteratively reweighted least squares minimization for sparse recovery. Commun Pure Appl Math, 2010, 63: 1–38MathSciNetCrossRefMATHGoogle Scholar
- 14.Drineas P, Kannan R, MahoneyMW. Fast Monte Carlo algorithms for matrices II: Computing low-rank approximations to a matrix. SIAM J Comput, 2006, 36: 158–183MathSciNetCrossRefMATHGoogle Scholar
- 15.Efron B, Hastie T, Johnstone I M, et al. Least angle regression. Ann Statist, 2004, 32: 407–499MathSciNetCrossRefMATHGoogle Scholar
- 16.Fazel M, Hindi H, Boyd S. A rank minimization heuristic with application to minimum order system approximation. In: Proceedings of the American Control Conference. New York: IEEE, 2001, doi: 10.1109/ACC.2001.945730Google Scholar
- 17.Fazel M, Hindi H, Boyd S. Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices. In: Proceedings of the American Control Conference. New York: IEEE, 2003, doi: 10.1109/AC-C.2003.1243393Google Scholar
- 18.Foucart S, Lai M-J. Sparsest solutions of underdetermined linear systems via l q minimization for 0 < q 6 1. Appl Comput Harmon Anal, 2009, 26: 395–407MathSciNetCrossRefMATHGoogle Scholar
- 19.Hale E, Yin W, Zhang Y. A fixed-point continuation method for l 1-regularized minimization: Methodology and convergence. SIAM J Optim, 2008, 19: 1107–1130MathSciNetCrossRefMATHGoogle Scholar
- 20.Ji S, Sze K, Zhou Z, et al. Beyond convex relaxation: A polynomial-time non-convex optimization approach to network localization. In: IEEE Conference on Computer Communications. New York: IEEE, 2013, 2499–2507Google Scholar
- 21.Keshavan R, Montanari A, Oh S. Matrix completion from a few entries. IEEE Trans Inform Theory, 2010, 56: 2980–2998MathSciNetCrossRefMATHGoogle Scholar
- 22.Lai M-J, Xu Y, Yin W. Improved iteratively rewighted least squares for unconstrained smoothed l p minimization. SIAM J Numer Anal, 2013, 5: 927–957CrossRefMATHGoogle Scholar
- 23.Liu Z, Vandenberghe L. Interior-point method for nuclear norm approximation with application to system identification. SIAM J Matrix Anal Appl, 2009, 31: 1235–1256MathSciNetCrossRefMATHGoogle Scholar
- 24.Lu Y, Zhang L, Wu J. A smoothing majorization method for l 2 2 -l p p matrix minimization. Optim Method Softw, 2014, 30: 1–24Google Scholar
- 25.Lu Z. Iterative reweighted minimization methods for l p regularized unconstrained nonlinear programming. Math Program, 2014, 147: 277–307MathSciNetCrossRefMATHGoogle Scholar
- 26.Lu Z, Zhang Y, Li X. Penalty decomposition methods for rank minimization. Optim Method Softw, 2015, 30: 531–558MathSciNetCrossRefMATHGoogle Scholar
- 27.Lu Z, Zhang Y, Lu J. l p regularized low-rank approximation via iterative reweighted singular value minimization. Comput Optim Appl, 2017, 68: 619–642MathSciNetCrossRefMATHGoogle Scholar
- 28.Ma S, Goldfarb D, Chen L. Fixed point and Bregman iterative methods for matrix rank minimization. Math Program, 2011, 128: 321–353MathSciNetCrossRefMATHGoogle Scholar
- 29.Ma S, Li Q. Lower bound theory for Schatten-p regularized least squares problems. Technical report. Beijing: Beijing Institute of Technology, 2013Google Scholar
- 30.Mohan K, Fazel M. Iterative reweighted algorithms for matrix rank minimization. J Mach Learn Res, 2012, 13: 3253–3285MathSciNetMATHGoogle Scholar
- 31.Rakotomamonjy A, Flamary R, Gasso G, et al. l p-l q penalty for sparse linear and sparse multiple kernel multitask learning. IEEE Trans Neural Network, 2011, 22: 1307–1320CrossRefGoogle Scholar
- 32.Rohde A, Tsybakov A. Estimation of high-dimensional low-rank matrices. Ann Statist, 2011, 39: 887–930MathSciNetCrossRefMATHGoogle Scholar
- 33.Skelton R, Iwasaki T, Grigoriadis K. A Unified Algebraic Approach to Linear Control Design. Abingdon: Taylor and Francis, 1998Google Scholar
- 34.Sun Q. Recovery of sparsest signals via l q minimization. Appl Comput Harmon Anal, 2012, 32: 329–341MathSciNetCrossRefMATHGoogle Scholar
- 35.Toh K C, Yun S. An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems. Pac J Optim, 2010, 6: 615–640MathSciNetMATHGoogle Scholar
- 36.Wen Z, Yin W, Zhang Y. Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm. Math Program Comp, 2012, 4: 333–361MathSciNetCrossRefMATHGoogle Scholar
- 37.Xu Z, Chang X, Xu F, et al. L 1/2 regularization: A thresholding representation theory and a fast solver. IEEE Trans Neural Network Learn Syst, 2012, 23: 1013–1027CrossRefGoogle Scholar
- 38.Zeng J, Lin S, Wang Y, et al. L 1/2 regularization: Convergence of iterative half thresholding algorithm. IEEE Trans Signal Process, 2014, 62: 2317–2329MathSciNetCrossRefGoogle Scholar