Advertisement

On Inexact Relative-Error Hybrid Proximal Extragradient, Forward-Backward and Tseng’s Modified Forward-Backward Methods with Inertial Effects

  • M. Marques AlvesEmail author
  • Raul T. Marcavillaca
Article
  • 10 Downloads

Abstract

For solving monotone inclusion problems, we propose an inertial under-relaxed version of the relative-error hybrid proximal extragradient method. We study the asymptotic convergence of the method, as well as its nonasymptotic global convergence rates in terms of iteration complexity. We analyze the new method under more flexible assumptions than existing ones, both on the extrapolation and on the relative-error parameters. The approach is applied to two types of forward-backward type methods for solving structured monotone inclusions.

Keywords

Inertial Relaxed Proximal point method HPE method Pointwise Ergodic Iteration-complexity Forward-backward algorithm Tseng’s modified forward-backward algorithm 

Mathematics Subject Classification (2010)

90C25 90C30 47H05 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgments

We are thankful to the two anonymous referees for their constructive suggestions which have improved the first version of this paper.

References

  1. 1.
    Alvarez, F.: Weak convergence of a relaxed and inertial hybrid projection-proximal point algorithm for maximal monotone operators in Hilbert space. SIAM J. Optim. 14 (3), 773–782 (2003)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Alvarez, F., Attouch, H.: An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping. Set-Valued Anal. 9 (1–2), 3–11 (2001). Wellposedness in optimization and related topics (Gargnano, 1999)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Attouch, H., Cabot, A.: Convergence of a relaxed inertial forward-backward algorithm for structured monotone inclusions. Preprint hal-01708216 (2018)Google Scholar
  4. 4.
    Attouch, H., Cabot, A.: Convergence of a relaxed inertial proximal algorithm for maximally monotone operators. Preprint hal-01708905 (2018)Google Scholar
  5. 5.
    Attouch, H., Chbani, Z., Peypouquet, J., Redont, P.: Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity. Math. Program. 168(1–2, Ser. B), 123–175 (2018)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Attouch, H., Peypouquet, J.: The rate of convergence of Nesterov’s accelerated forward-backward method is actually faster than 1/k 2. SIAM J. Optim. 26(3), 1824–1834 (2016)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Attouch, H., Peypouquet, J., Redont, P.: Fast convex optimization via inertial dynamics with Hessian driven damping. J. Differential Equations 261(10), 5734–5783 (2016)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC. Springer, New York (2011). With a foreword by Hédy AttouchGoogle Scholar
  9. 9.
    Boṫ, R.I., Csetnek, E.R.: A hybrid proximal-extragradient algorithm with inertial effects. Numer. Funct. Anal. Optim. 36(8), 951–963 (2015)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Boṫ, R. I., Csetnek, E.R., Hendrich, C.: Inertial Douglas-Rachford splitting for monotone inclusion problems. Appl. Math. Comput. 256, 472–487 (2015)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Burachik, R.S., Iusem, A.N., Svaiter, B.F.: Enlargement of monotone operators with applications to variational inequalities. Set-Valued Anal. 5(2), 159–180 (1997)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Burachik, R.S., Sagastizábal, C. A., Svaiter, B.F.: 𝜖-enlargements of maximal monotone operators: theory and applications. In: Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods (Lausanne, 1997), Volume 22 of Appl. Optim., pp 25–43. Kluwer Acad. Publ., Dordrecht (1999)Google Scholar
  13. 13.
    Burachik, R.S., Svaiter, B.F.: 𝜖-enlargements of maximal monotone operators in Banach spaces. Set-Valued Anal. 7(2), 117–132 (1999)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Ceng, L.C., Mordukhovich, B.S., Yao, J.C.: Hybrid approximate proximal method with auxiliary variational inequality for vector optimization. J. Optim Theory Appl. 146(2), 267–303 (2010)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Chen, C., Chan, R.H., Ma, S., Yang, J.: Inertial proximal ADMM for linearly constrained separable convex optimization. SIAM J. Imaging Sci. 8(4), 2239–2267 (2015)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Chen, C., Ma, S., Yang, J.: A general inertial proximal point algorithm for mixed variational inequality problem. SIAM J. Optim. 25(4), 2120–2142 (2015)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Eckstein, J., Silva, P.J.S.: A practical relative error criterion for augmented Lagrangians. Math Program. 141(1-2, Ser. A), 319–348 (2013)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Eckstein, J., Yao, W.: Relative-error approximate versions of Douglas-Rachford splitting and special cases of the ADMM. Mathematical Programming (2017)Google Scholar
  19. 19.
    Glowinski, R., Osher, S.J., Yin, W. (eds.): Splitting Methods in Communication, Imaging, Science, and Engineering. Scientific Computation. Springer, Cham (2016)Google Scholar
  20. 20.
    He, Y., Monteiro, R.D.C.: An accelerated HPE-type algorithm for a class of composite convex-concave saddle-point problems. SIAM J. Optim. 26(1), 29–56 (2016)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Iusem, A.N., Sosa, W.: On the proximal point method for equilibrium problems in Hilbert spaces. Optimization 59(8), 1259–1274 (2010)MathSciNetzbMATHGoogle Scholar
  22. 22.
    Lions, P.-L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer Anal. 16(6), 964–979 (1979)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Lorenz, D.A., Pock, T.: An inertial forward-backward algorithm for monotone inclusions. J. Math Imaging Vis. 51(2), 311–325 (2015)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Lotito, P.A., Parente, L.A., Solodov, M.V.: A class of variable metric decomposition methods for monotone variational inclusions. J Convex Anal. 16(3-4), 857–880 (2009)MathSciNetzbMATHGoogle Scholar
  25. 25.
    Martinet, B.: Régularisation d’inéquations variationnelles par approximations successives. Rev. Française Informat, journal=Recherche Opé,rationnelle 4(Ser. R-3), 154–158 (1970)Google Scholar
  26. 26.
    Monteiro, R.D.C., Ortiz, C., Svaiter, B.F.: A first-order block-decomposition method for solving two-easy-block structured semidefinite programs. Math Program. Comput. 6(2), 103–150 (2014)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Monteiro, R.D.C., Ortiz, C., Svaiter, B.F.F.: Implementation of a block-decomposition algorithm for solving large-scale conic semidefinite programming problems. Comput. Optim Appl. 57(1), 45–69 (2014)MathSciNetzbMATHGoogle Scholar
  28. 28.
    Monteiro, R.D.C., Ortiz, C., Svaiter, B.F.F.: An adaptive accelerated first-order method for convex optimization. Comput. Optim Appl. 64(1), 31–73 (2016)MathSciNetzbMATHGoogle Scholar
  29. 29.
    Monteiro, R.D.C., Svaiter, B.F.: Complexity of variants of Tseng’s modified F-B splitting and Korpelevich’s methods for hemivariational inequalities with applications to saddle point and convex optimization problems. SIAM J. Optim. 21, 1688–1720 (2010)MathSciNetzbMATHGoogle Scholar
  30. 30.
    Monteiro, R.D.C., Svaiter, B.F.: On the complexity of the hybrid proximal extragradient method for the iterates and the ergodic mean. SIAM J. Optim. 20, 2755–2787 (2010)MathSciNetzbMATHGoogle Scholar
  31. 31.
    Monteiro, R.D.C., Svaiter, B.F.: Iteration-complexity of block-decomposition algorithms and the alternating direction method of multipliers. SIAM J. Optim. 23(1), 475–507 (2013)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Moudafi, A., Oliny, M.: Convergence of a splitting inertial proximal method for monotone operators. J. Comput. Appl Math. 155(2), 447–454 (2003)MathSciNetzbMATHGoogle Scholar
  33. 33.
    Passty, G.B.: Ergodic convergence to a zero of the sum of monotone operators in Hilbert space. J. Math. Anal. Appl. 72(2), 383–390 (1979)MathSciNetzbMATHGoogle Scholar
  34. 34.
    Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976)MathSciNetzbMATHGoogle Scholar
  35. 35.
    Solodov, M.V., Svaiter, B.F.: A hybrid approximate extragradient-proximal point algorithm using the enlargement of a maximal monotone operator. Set-Valued Anal. 7 (4), 323–345 (1999)MathSciNetzbMATHGoogle Scholar
  36. 36.
    Solodov, M.V., Svaiter, B.F.: A hybrid projection-proximal point algorithm. J. Convex Anal. 6(1), 59–70 (1999)MathSciNetzbMATHGoogle Scholar
  37. 37.
    Solodov, M.V., Svaiter, B.F.: An inexact hybrid generalized proximal point algorithm and some new results on the theory of Bregman functions. Math. Oper. Res. 25(2), 214–230 (2000)MathSciNetzbMATHGoogle Scholar
  38. 38.
    Solodov, M.V., Svaiter, B.F.: A unified framework for some inexact proximal point algorithms. Numer. Funct. Anal Optim. 22(7–8), 1013–1035 (2001)MathSciNetzbMATHGoogle Scholar
  39. 39.
    Svaiter, B.F.: A class of Fejér convergent algorithms, approximate resolvents and the hybrid proximal-extragradient method. J. Optim. Theory Appl. 162(1), 133–153 (2014)MathSciNetzbMATHGoogle Scholar
  40. 40.
    Svaiter, B.F.: Complexity of the relaxed hybrid proximal-extragradient method under the large-step condition. Preprint A766/2015, IMPA - Instituto Nacional de Matemática Pura e Aplicada, Estrada Dona Castorina 110, Rio de Janeiro, RJ Brasil, pp. 22460–320 (2015)Google Scholar
  41. 41.
    Tseng, P.: A modified forward-backward splitting method for maximal monotone mappings. SIAM J. Control Optim. 38(2), 431–446 (2000). (electronic)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Departamento de MatemáticaUniversidade Federal de Santa CatarinaFlorianópolisBrazil

Personalised recommendations