Computational Optimization and Applications

, Volume 71, Issue 1, pp 5–52 | Cite as

A block coordinate variable metric linesearch based proximal gradient method

  • S. Bonettini
  • M. Prato
  • S. Rebegoldi


In this paper we propose an alternating block version of a variable metric linesearch proximal gradient method. This algorithm addresses problems where the objective function is the sum of a smooth term, whose variables may be coupled, plus a separable part given by the sum of two or more convex, possibly nonsmooth functions, each depending on a single block of variables. Our approach is characterized by the possibility of performing several proximal gradient steps for updating every block of variables and by the Armijo backtracking linesearch for adaptively computing the steplength parameter. Under the assumption that the objective function satisfies the Kurdyka-Łojasiewicz property at each point of its domain and the gradient of the smooth part is locally Lipschitz continuous, we prove the convergence of the iterates sequence generated by the method. Numerical experience on an image blind deconvolution problem show the improvements obtained by adopting a variable number of inner block iterations combined with a variable metric in the computation of the proximal operator.


Alternating optimization Proximal gradient methods Kurdyka-Łojasiewicz inequality 


  1. 1.
    Abboud, F., Chouzenoux, E., Pesquet, J.C., Chenot, J.H., Laborelli, L.: Dual block coordinate forward–backward algorithm with application to deconvolution and deinterlacing of video sequences. J. Math. Imaging Vis. 59(3), 415–431 (2017)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Almeida, M.S.C., Almeida, L.B.: Blind and semi-blind deblurring of natural images. IEEE Trans. Image Process. 19(1), 36–52 (2010)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Attouch, H., Bolte, J.: On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Math. Program. 116(1–2), 5–16 (2009)MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka-Łojasiewicz inequality. Math. Oper. Res. 35(2), 438–457 (2010)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods. Math. Program. 137(1–2), 91–129 (2013)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Ayers, G.R., Dainty, J.C.: Iterative blind deconvolution method and its applications. Opt. Lett. 13(7), 547–549 (1988)CrossRefGoogle Scholar
  7. 7.
    Bauschke, H.H., Bolte, J., Teboulle, M.: A descent lemma beyond Lipschitz gradient continuity: first-order methods revisited and applications. Math. Oper. Res. 4(1), 330–348 (2016)MathSciNetMATHGoogle Scholar
  8. 8.
    Bertero, M., Bindi, D., Boccacci, P., Cattaneo, M., Eva, C., Lanza, V.: A novel blind-deconvolution method with an application to seismology. Inverse Probl. 14(4), 815–833 (1998)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Bertero, M., Boccacci, P., Desiderà, G., Vicidomini, G.: Image deblurring with Poisson data: from cells to galaxies. Inverse Probl. 25(12), 123006 (2009)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Bertero, M., Lantéri, H., Zanni, L.: Iterative image reconstruction: a point of view. In: Censor, Y., Jiang, M., Louis, A.K. (eds.) Mathematical Methods in Biomedical Imaging and Intensity-Modulated Radiation Therapy (IMRT), pp. 37–63. Birkhauser, Pisa (2008)Google Scholar
  11. 11.
    Bertsekas, D.: Nonlinear Programming. Athena Scientific, Belmont (1999)MATHGoogle Scholar
  12. 12.
    Bolte, J., Combettes, P.L., Pesquet, J.C.: Alternating proximal algorithm for blind image recovery. In: Proceedings of the 17th International Conference on Image Processing, pp. 1673–1676 (2010)Google Scholar
  13. 13.
    Bolte, J., Daniilidis, A., Ley, O., Mazet, L.: Characterizations of Łojasiewicz inequalities: subgradient flows, talweg, convexity. Trans. Am. Math. Soc. 362(6), 3319–3363 (2010)CrossRefMATHGoogle Scholar
  14. 14.
    Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146(1–2), 459–494 (2014)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Bonettini, S.: Inexact block coordinate descent methods with application to the nonnegative matrix factorization. IMA J. Numer. Anal. 31(4), 1431–1452 (2011)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Bonettini, S., Cornelio, A., Prato, M.: A new semiblind deconvolution approach for Fourier-based image restoration: an application in astronomy. SIAM J. Imaging Sci. 6(3), 1736–1757 (2013)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Bonettini, S., Loris, I., Porta, F., Prato, M.: Variable metric inexact line-search based methods for nonsmooth optimization. SIAM J. Optim. 26(2), 891–921 (2016)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Bonettini, S., Loris, I., Porta, F., Prato, M., Rebegoldi, S.: On the convergence of a linesearch based proximal-gradient method for nonconvex optimization. Inverse Probl. 33(5), 055005 (2017)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Bonettini, S., Prato, M.: New convergence results for the scaled gradient projection method. Inverse Probl. 31(9), 095008 (2015)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    Bonettini, S., Prato, M., Rebegoldi, S.: A cyclic block coordinate descent method with generalized gradient projections. Appl. Math. Comput. 286, 288–300 (2016)MathSciNetGoogle Scholar
  21. 21.
    Bonettini, S., Zanella, R., Zanni, L.: A scaled gradient projection method for constrained image deblurring. Inverse Probl. 25(1), 015002 (2009)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Cassioli, A., Di Lorenzo, D., Sciandrone, M.: On the convergence of inexact block coordinate descent methods for constrained optimization. Eur. J. Oper. Res. 231(2), 274–281 (2013)MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Chambolle, A., Dossal, C.: On the convergence of the iterates of the “Fast Iterative Shrinkage/Thresholding Algorithm”. J. Optim. Theory Appl. 166(3), 968–982 (2015)MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Chouzenoux, E., Pesquet, J.C.: A stochastic majorize-minimize subspace algorithm for online penalized least squares estimation. IEEE Trans. Signal Process. 65(18), 4770–4783 (2017)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Chouzenoux, E., Pesquet, J.C., Repetti, A.: Variable metric forward–backward algorithm for minimizing the sum of a differentiable function and a convex function. J. Optim. Theory Appl. 162(1), 107–132 (2014)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Chouzenoux, E., Pesquet, J.C., Repetti, A.: A block coordinate variable metric forward–backward algorithm. J. Glob. Optim. 66(3), 457–485 (2016)MathSciNetCrossRefMATHGoogle Scholar
  27. 27.
    Combettes, P.L., Pesquet, J.C.: Proximal splitting methods in signal processing. In: Bauschke, H.H., Burachik, R.S., Combettes, P.L., Elser, V., Luke, D.R., Wolkowicz, H. (eds.) Fixed-Point Algorithms for Inverse Problems in Science and Engineering. Springer Optimization and Its Applications, pp. 185–212. Springer, New York (2011)CrossRefGoogle Scholar
  28. 28.
    Cornelio, A., Porta, F., Prato, M.: A convergent least-squares regularized blind deconvolution approach. Appl. Math. Comput. 259, 173–186 (2015)MathSciNetMATHGoogle Scholar
  29. 29.
    Fessler, J.A., Erdogan, H.: A paraboloidal surrogates algorithm for convergent penalized–likelihood emission image reconstruction. In: 1998 IEEE Nuclear Science Symposium Conference Record, pp. 1132–1135 (1998)Google Scholar
  30. 30.
    Frankel, P., Garrigos, G., Peypouquet, J.: Splitting methods with variable metric for Kurdyka-Łojasiewicz functions and general convergence rates. J. Opt. Theory Appl. 165(3), 874–900 (2015)CrossRefMATHGoogle Scholar
  31. 31.
    Grippo, L., Sciandrone, M.: Globally convergent block-coordinate techniques for unconstrained optimization. Optim. Method Softw. 10(4), 587–637 (1999)MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Grippo, L., Sciandrone, M.: On the convergence of the block nonlinear Gauss–Seidel method under convex constraints. Oper. Res. Lett. 26(3), 127–136 (2000)MathSciNetCrossRefMATHGoogle Scholar
  33. 33.
    Harmany, Z., Marcia, R., Willett, R.: This is SPIRAL-TAP: sparse Poisson intensity reconstruction algorithms—theory and practice. IEEE Trans. Image Process. 21(3), 1084–1096 (2012)MathSciNetCrossRefMATHGoogle Scholar
  34. 34.
    Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms. Springer, New York (1993)MATHGoogle Scholar
  35. 35.
    Kurdyka, K.: On gradients of functions definable in o-minimal structures. Ann. Inst. Fourier 48(3), 769–783 (1998)MathSciNetCrossRefMATHGoogle Scholar
  36. 36.
    Lantéri, H., Roche, M., Cuevas, O., Aime, C.: A general method to devise maximum likelihood signal restoration multiplicative algorithms with non-negativity constraints. Signal Process. 81(5), 945–974 (2001)CrossRefMATHGoogle Scholar
  37. 37.
    Lee, D., Seung, H.: Algorithms for non-negative matrix factorization. In: NIPS, pp. 556–562. MIT Press (2000)Google Scholar
  38. 38.
    Lin, C.J.: Projected gradient methods for nonnegative matrix factorization. Neural Comput. 19(10), 2756–2779 (2007)MathSciNetCrossRefMATHGoogle Scholar
  39. 39.
    Łojasiewicz, S.: Une propriété topologique des sous-ensembles analytiques réels. In: Les Équations aux Dérivées Partielles, pp. 87–89. Éditions du Centre National de la Recherche Scientifique, Paris (1963)Google Scholar
  40. 40.
    Lucy, L.B.: An iterative technique for the rectification of observed distributions. Astronom. J. 79(6), 745–754 (1974)CrossRefGoogle Scholar
  41. 41.
    Noll, D.: Convergence of non-smooth descent methods using the Kurdyka-Łojasiewicz inequality. J. Opt. Theory Appl. 160(2), 553–572 (2014)CrossRefMATHGoogle Scholar
  42. 42.
    Ochs, P.: Unifying abstract inexact convergence theorems for descent methods and block coordinate variable metric iPiano (2016). arXiv:1602.07283
  43. 43.
    Ochs, P., Chen, Y., Brox, T., Pock, T.: iPiano: inertial proximal algorithm for non-convex optimization. SIAM J. Imaging Sci. 7(2), 1388–1419 (2014)MathSciNetCrossRefMATHGoogle Scholar
  44. 44.
    Porta, F., Loris, I.: On some steplength approaches for proximal algorithms. Appl. Math. Comput. 253, 345–362 (2015)MathSciNetMATHGoogle Scholar
  45. 45.
    Prato, M., Camera, A.L., Bonettini, S., Bertero, M.: A convergent blind deconvolution method for post-adaptive-optics astronomical imaging. Inverse Probl. 29(6), 065017 (2013)MathSciNetCrossRefMATHGoogle Scholar
  46. 46.
    Prato, M., Cavicchioli, R., Zanni, L., Boccacci, P., Bertero, M.: Efficient deconvolution methods for astronomical imaging: algorithms and IDL-GPU codes. Astron. Astrophys. 539, A133 (2012)CrossRefGoogle Scholar
  47. 47.
    Prato, M., La Camera, A., Bonettini, S., Rebegoldi, S., Bertero, M., Boccacci, P.: A blind deconvolution method for ground based telescopes and Fizeau interferometers. New Astron. 40, 1–13 (2015)CrossRefGoogle Scholar
  48. 48.
    Razaviyayn, M., Hong, M., Luo, Z.Q.: A unified convergence analysis of block successive minimization methods for nonsmooth optimization. SIAM J. Optim. 23(2), 1126–1153 (2013)MathSciNetCrossRefMATHGoogle Scholar
  49. 49.
    Rockafellar, R.T., Wets, R.J.B., Wets, M.: Variational Analysis. Springer, Berlin (1998)CrossRefMATHGoogle Scholar
  50. 50.
    Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. J. Phys. D. 60(1–4), 259–268 (1992)MathSciNetCrossRefMATHGoogle Scholar
  51. 51.
    Salzo, S.: The variable metric forward–backward splitting algorithm under mild differentiability assumptions. SIAM J. Optim. 27(4), 2153–2181 (2017)MathSciNetCrossRefMATHGoogle Scholar
  52. 52.
    Salzo, S., Villa, S.: Inexact and accelerated proximal point algorithms. J. Convex Anal. 19(4), 1167–1192 (2012)MathSciNetMATHGoogle Scholar
  53. 53.
    Shen, X., Diamond, S., Udell, M., Gu, Y., Boyd, S.: Disciplined multi-convex programming (2016). arXiv:1609.03285
  54. 54.
    Staglianò, A., Boccacci, P., Bertero, M.: Analysis of an approximate model for Poisson data reconstruction and a related discrepancy principle. Inverse Probl. 27(12), 125003 (2011)MathSciNetCrossRefMATHGoogle Scholar
  55. 55.
    Tikhonov, N.A., Arsenin, V.Y.: Solution of Ill Posed Problems. Wiley, New York (1977)MATHGoogle Scholar
  56. 56.
    Tseng, P., Yun, S.: A coordinate gradient descent method for nonsmooth separable minimization. Math. Program. 117(1–2), 387–423 (2009)MathSciNetCrossRefMATHGoogle Scholar
  57. 57.
    Villa, S., Salzo, S., Baldassarre, L., Verri, A.: Accelerated and inexact forward–backward algorithms. SIAM J. Optim. 23(3), 1607–1633 (2013)MathSciNetCrossRefMATHGoogle Scholar
  58. 58.
    Xu, Y., Yin, W.: A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion. SIAM J. Imaging Sci. 6(3), 1758–1789 (2013)MathSciNetCrossRefMATHGoogle Scholar
  59. 59.
    Zalinescu, A.: Convex Analysis in General Vector Spaces. World Scientific Publishing Co., Inc., River Edge (2002)CrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Dipartimento di Scienze Fisiche, Informatiche e MatematicheUniversità di Modena e Reggio EmiliaModenaItaly
  2. 2.Dipartimento di Matematica e InformaticaUniversità di FerraraFerraraItaly

Personalised recommendations