Advertisement

A Note on the Equivalence of Operator Splitting Methods

  • Walaa M. MoursiEmail author
  • Yuriy Zinchenko
Chapter

Abstract

This note provides a comprehensive discussion of the equivalences between some splitting methods. We survey known results concerning these equivalences which have been studied over the past few decades. In particular, we provide simplified proofs of the equivalence of the ADMM and the Douglas–Rachford method and the equivalence of the ADMM with intermediate update of multipliers and the Peaceman–Rachford method.

Keywords

Alternating Direction Method of Multipliers (ADMM) Chambolle–Pock method Douglas–Rachford algorithm Dykstra method Equivalence of splitting methods Fenchel–Rockafellar duality Peaceman–Rachford algorithm 

AMS 2010 Subject Classification

47H05 47H09 49M27 49M29 49N15 90C25 

Notes

Acknowledgements

WMM was supported by the Pacific Institute of Mathematics Postdoctoral Fellowship and the DIMACS/Simons Collaboration on Bridging Continuous and Discrete Optimization through NSF grant # CCF-1740425.

References

  1. 1.
    Attouch, H., Brézis, H.: Duality for the sum of convex functions in general Banach spaces. In: Aspects of Mathematics and Its Applications 34, pp. 125–133. North-Holland, Amsterdam (1986)Google Scholar
  2. 2.
    Attouch, H., Théra, M.: A general duality principle for the sum of two operators. J. Convex Anal. 3, 1–24 (1996)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Bauschke, H.H., Combettes, P.L.: A Dykstra-like algorithm for two monotone operators. Pacific J. Optim. 4, 383–391 (2008)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Bauschke, H.H., Boţ, R.I., Hare, W.L., Moursi, W.M.: Attouch–Théra duality revisited: paramonotonicity and operator splitting. J. Approx. Th. 164, 1065–1084 (2012)CrossRefGoogle Scholar
  5. 5.
    Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Second Edition. Springer (2017)CrossRefGoogle Scholar
  6. 6.
    Bauschke, H.H., Koch, V.R.: Projection methods: Swiss Army knives for solving feasibility and best approximation problems with halfspaces. In: Infinite Products of Operators and Their Applications, Contemp. Math. vol. 636, pp. 1–40 (2012)Google Scholar
  7. 7.
    Beck, A.: First-Order Methods in Optimization, SIAM (2017)Google Scholar
  8. 8.
    Boţ, R.I, Csetnek, E.R.: ADMM for monotone operators: convergence analysis and rates. Advances Comp. Math. 45, 327–359 (2019)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning 3, 1–122 (2011)CrossRefGoogle Scholar
  10. 10.
    Boyle, J.P., Dykstra, R.L.: A method for finding projections onto the intersection of convex sets in Hilbert spaces. Lecture Notes in Statistics 37, 28–47 (1986)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vis. 40, 120–145 (2011)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Combettes, P.L.: Solving monotone inclusions via compositions of nonexpansive averaged operators. Optimization 53, 475–504 (2004)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Briceño-Arias, L.M., Combettes, P.L.: A monotone + skew splitting model for composite monotone inclusions in duality. SIAM J. Optim. 21, 1230–1250 (2011)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Combettes, P.L., Pesquet, J.-C.: Primal-dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum type monotone operators. Set-Valued Var. Anal. 20, 307–330 (2012)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Combettes, P.L., Pesquet, J.-C.: A proximal decomposition method for solving convex variational inverse problems. Inverse Problems 24, article 065014 (2008)Google Scholar
  16. 16.
    Combettes, P.L., Pesquet, J.-C.: Proximal splitting methods in signal processing. In: Fixed-Point Algorithms for Inverse Problems in Science and Engineering, vol. 49, pp. 185–212. Springer, New York (2011)Google Scholar
  17. 17.
    Combettes, P.L., Vũ, B.-C.: Variable metric forward–backward splitting with applications to monotone inclusions in duality. Optimization 63, 1289–1318 (2014)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Condat, L.: A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. J. Optim. Th. Appl. 158, 460–479 (2013)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Deutsch, F.: Best Approximation in Inner Product Spaces, Springer (2001)Google Scholar
  20. 20.
    Eckstein, J.: Splitting Methods for Monotone Operators with Applications to Parallel Optimization, Ph.D. thesis, MIT (1989)Google Scholar
  21. 21.
    Eckstein, J., Bertsekas, D.P.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Prog. 55, 293–318 (1992)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Gabay, D.: Applications of the method of multipliers to variational inequalities. In: Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, vol. 15, pp. 299–331. North-Holland, Amsterdam (1983)Google Scholar
  23. 23.
    Gossez, J.-P.: Opérateurs monotones non linéaires dans les espaces de Banach non réflexifs. J. Math. Anal. Appl. 34, 371–395 (1971)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16, 964–979 (1979)MathSciNetCrossRefGoogle Scholar
  25. 25.
    O’Connor, D., Vandenberghe, L.: On the equivalence of the primal-dual hybrid gradient method and Douglas–Rachford splitting. Math. Prog. (Ser. A) (2018), https://doi.org/10.1007/s10107-018-1321-1.
  26. 26.
    Riesz, F., Sz.-Nagy, B.: Functional Analysis. Dover paperback (1990)Google Scholar
  27. 27.
    Rockafellar, R.T.: On the maximal monotonicity of subdifferential mappings. Pacific J. Math. 33, 209–216 (1970)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Stoer, J., Bulirsch, R.: Introduction to Numerical Analysis. Third Edition. Springer-Verlag (2002)CrossRefGoogle Scholar
  29. 29.
    Tseng, P.: Applications of a splitting algorithm to decomposition in convex programming and variational inequalities. SIAM J. Con. Optim. 29, 119–138 (1991)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Vũ, B.C.: A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv. Comput. Math. 38, 667–681 (2013)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Electrical EngineeringStanford UniversityStanfordUSA
  2. 2.Faculty of Science, Mathematics DepartmentMansoura UniversityMansouraEgypt
  3. 3.University of Calgary, Department of Mathematics and StatisticsCalgaryCanada

Personalised recommendations