Advertisement

Multidimensional Systems and Signal Processing

, Volume 30, Issue 1, pp 145–174 | Cite as

A new nonconvex approach to low-rank matrix completion with application to image inpainting

  • Yongchao Yu
  • Jigen PengEmail author
  • Shigang Yue
Article

Abstract

The problem of recovering a low-rank matrix from partial entries, known as low-rank matrix completion, has been extensively investigated in recent years. It can be viewed as a special case of the affine constrained rank minimization problem which is NP-hard in general and is computationally hard to solve in practice. One widely studied approach is to replace the matrix rank function by its nuclear-norm, which leads to the convex nuclear-norm minimization problem solved efficiently by many popular convex optimization algorithms. In this paper, we propose a new nonconvex approach to better approximate the rank function. The new approximation function is actually the Moreau envelope of the rank function (MER) which has an explicit expression. The new approximation problem of low-rank matrix completion based on MER can be converted to an optimization problem with two variables. We then adapt the proximal alternating minimization algorithm to solve it. The convergence (rate) of the proposed algorithm is proved and its accelerated version is also developed. Numerical experiments on completion of low-rank random matrices and standard image inpainting problems have shown that our algorithms have better performance than some state-of-art methods.

Keywords

Low-rank matrix completion Moreau envelope Proximal alternating minimization Image Inpainting 

Notes

Acknowledgements

The authors would like to thank the editor-in chief and the anonymous reviewers for their insightful and constructive comments, which help to enrich the content and improve the presentation of this paper. This research was supported by the National Natural Science Foundation of China under the grants 11771347, 91730306, 41390454, and 11271297, and supported by the Nanhu Scholars Program for Young Scholars of XYNU.

References

  1. Attouch, H., & Bolte, J. (2009). On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Mathematical Programming, 116(1), 5–16.MathSciNetCrossRefzbMATHGoogle Scholar
  2. Attouch, H., Bolte, J., Redont, P., & Soubeyran, A. (2010). Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the Kurdyka–Łojasiewicz inequality. Mathematics of Operations Research, 35(2), 438–457.MathSciNetCrossRefzbMATHGoogle Scholar
  3. Auslender, A. (1992). Asymptotic properties of the Fenchel dual functional and applications to decomposition problems. Journal of Optimization Theory and Applications, 73(3), 427–449.MathSciNetCrossRefzbMATHGoogle Scholar
  4. Bauschke, H. H., & Combettes, P. L. (2011). Convex analysis and monotone operator theory in Hilbert spaces. CMS Books in Mathematics. New York: Springer.CrossRefzbMATHGoogle Scholar
  5. Beck, A., & Teboulle, M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1), 183–202.MathSciNetCrossRefzbMATHGoogle Scholar
  6. Bertsekas, D. (1999). Nonlinear optimisation, 2nd ed., Athena, Belmont, Massachusetts.Google Scholar
  7. Bolte, J., Sabach, S., & Teboulle, M. (2013). Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Mathematical Programming, 146(1), 459–494.MathSciNetzbMATHGoogle Scholar
  8. Cai, J. F., Candès, E. J., & Shen, Z. W. (2010). A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 20(4), 1956–1982.MathSciNetCrossRefzbMATHGoogle Scholar
  9. Candès, E. J., & Recht, B. (2009). Exact matrix completion via convex optimization. Foundations of Computational Mathematics, 9(6), 717–772.MathSciNetCrossRefzbMATHGoogle Scholar
  10. Candès, E. J., Romberg, J., & Tao, T. (2006). Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics, 59(8), 1207–1223.MathSciNetCrossRefzbMATHGoogle Scholar
  11. Candès, E. J., & Tao, T. (2010). The power of convex relaxation: Near-optimal matrix completion. IEEE Transactions on Information Theory, 56(5), 2053–2080.MathSciNetCrossRefzbMATHGoogle Scholar
  12. Candès, E. J., & Wakin, M. (2008). An introduction to compressive sampling. IEEE Signal Processing Magazine, 25(2), 21–30.CrossRefGoogle Scholar
  13. Cao, W. F., Sun, J., & Xu, Z. B. (2013). Fast image deconvolution using closed-form thresholding formulas of \(L_q\) \((q = 1/2, 2/3)\) regularization. Journal of Visual Communication and Image Representation, 24(1), 31–41.CrossRefGoogle Scholar
  14. Chen, P., & Suter, D. (2004). Recovering the missing components in a large noisy low-rank matrix: Application to SFM source. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(8), 1051–1063.CrossRefGoogle Scholar
  15. Fan, J. Q., & Li, R. Z. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456), 1348–1360.MathSciNetCrossRefzbMATHGoogle Scholar
  16. Fazel, M. (2002). Matrix rank minimization with applications. Ph.D. thesis, Stanford University.Google Scholar
  17. Geng, J., Wang, L. S., & Wang, Y. F. (2015). A non-convex algorithm framework based on DC programming and DCA for matrix completion. Numerical Algorithms, 68(4), 903–921.MathSciNetCrossRefzbMATHGoogle Scholar
  18. Goldberg, D., Nichols, D., Oki, B. M., & Terry, D. (1992). Using collaborative filtering to weave an information tapestry. Communications of the ACM, 35(12), 61–70.CrossRefGoogle Scholar
  19. Hu, Y., Zhang, D. B., Ye, J. P., Li, X. L., & He, X. F. (2013). Fast and accurate matrix completion via truncated nuclear norm regularization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(9), 2117–2130.CrossRefGoogle Scholar
  20. Jin, Z. F., Wan, Z. P., Jiao, Y. L., & Lu, J. X. (2016). An alternating direction method with continuation for nonconvex low rank minimization. Journal of Scientific Computing, 66(2), 849–869.MathSciNetCrossRefzbMATHGoogle Scholar
  21. Lai, M. J., & Wang, J. Y. (2011). An unconstrained \(\ell _q\) minimization with \(0<q\le 1\) for sparse solution of underdetermined linear systems. SIAM Journal on Optimization, 21(1), 82–101.MathSciNetCrossRefGoogle Scholar
  22. Li, G., & Pong, T. (2016). Calculus of the exponent of Kurdyka–Łojasiewicz inequality and its applications to linear convergence of first-order methods. arXiv:1602.02915v1.
  23. Li, G., Mordukhovich, B. S., & Pham, T. S. (2015). New fractional error bounds for polynomial systems with applications to Hölderian stability in optimization and spectral theory of tensors. Mathematical Programming, 153(2), 333–362.MathSciNetCrossRefzbMATHGoogle Scholar
  24. Liu, Z., & Vandenberghe, L. (2009). Interior-point method for nuclear norm approximation with application to system identification. SIAM Journal on Matrix Analysis and Applications, 31(3), 1235–1256.MathSciNetCrossRefzbMATHGoogle Scholar
  25. Lu, Z. S., & Zhang, Y. (2015). Schatten-\(p\) quasi-norm regularized matrix optimization via iterative reweighted singular value minimization. http://www.optimization-online.org/DB_HTML/2015/11/5215.html.
  26. Lu, C. Y., Tang, J. H., Yan, S. C., & Lin, Z. C. (2015). Nonconvex nonsmooth low-rank minimization via iteratively reweighted nuclear norm. arXiv:1510.06895.
  27. Ma, S. Q., Goldfarb, D., & Chen, L. F. (2011). Fixed point and Bregman iterative methods for matrix rank minimization. Mathematical Programming, 128(1), 321–353.MathSciNetCrossRefzbMATHGoogle Scholar
  28. Mohammadi, M. M., Zadeh, M. B., Amini, A., & Jutten, C. (2013). Recovery of low-rank matrices under affine constraints via a smoothed rank function. arXiv:1308.2293.
  29. Mohammadi, M. M., Zadeh, M. B., & Jutten, C. (2009). A fast approach for overcomplete sparse decomposition based on smoothed \(\ell _0\)-norm. IEEE Transactions on Signal Processing, 57(1), 289–301.MathSciNetCrossRefzbMATHGoogle Scholar
  30. Recht, B., Fazel, M., & Parrilo, P. A. (2010). Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Review, 52(3), 471–501.MathSciNetCrossRefzbMATHGoogle Scholar
  31. Rockafellar, R. (1970). Convex analysis. Princeton: Princeton University Press.CrossRefzbMATHGoogle Scholar
  32. Singer, A. (2008). A remark on global positioning from local distances. Proceedings of the National Academy of Sciences of the United States of America, 105(28), 9507–9511.MathSciNetCrossRefzbMATHGoogle Scholar
  33. Srebro, N. (2004). Learning with matrix factorizations. Ph.D. thesis, Massachusetts Institute of Technology.Google Scholar
  34. Toh, K. C., & Yun, S. W. (2012). An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems. Pacific Journal of Optimization, 6(3), 615–640.MathSciNetzbMATHGoogle Scholar
  35. Tomasi, C., & Kanade, T. (1992). Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision, 9(2), 137C154.CrossRefGoogle Scholar
  36. Tseng, P. (2001). Convergence of a block coordinate descent method for nondifferentiable minimization. Journal of Optimization Theory and Applications, 109(3), 475–494.MathSciNetCrossRefzbMATHGoogle Scholar
  37. Tütüncü, R. H., Toh, K. C., & Todd, M. J. (2003). Solving semidefinite-quadratic-linear programs using SDPT3. Mathematical Programming, 95(2), 189–217.MathSciNetCrossRefzbMATHGoogle Scholar
  38. Yang, J. F., & Yuan, X. M. (2013). Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Mathematics of Computation, 82(281), 301–329.MathSciNetCrossRefzbMATHGoogle Scholar
  39. Zhang, S., & Xin, J. (2014). Minimization of transformed \(\ell _1\) penalty: Closed form representation and iterative thresholding algorithms. arXiv preprint arXiv:1412.5240.
  40. Zhang, S., Yin, P. H., & Xin, J. (2015). Transformed Schatten-1 iterative thresholding algorithms for matrix rank minimization and applications. arXiv:1506.04444.
  41. Zhang, C. H. (2010). Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 38(2), 894–942.MathSciNetCrossRefzbMATHGoogle Scholar
  42. Zhang, T. (2010). Analysis of multi-stage convex relaxation for sparse regularization. Journal of Machine Learning Research, 11(Mar), 1081–1107.MathSciNetzbMATHGoogle Scholar
  43. Zhao, Y. B. (2012). An approximation theory of matrix rank minimization and its application to quadratic equations. Linear Algebra and Its Applications, 437(1), 77–93.MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Mathematics and StatisticsXinyang Normal UniversityXinyangChina
  2. 2.School of Mathematics and StatisticsXi’an Jiaotong UniversityXi’anChina
  3. 3.School of Mathematics and Information Sciences, Guangzhou UniversityGuangZhouChina
  4. 4.School of Computer ScienceUniversity of LincolnLincolnUK

Personalised recommendations