Advertisement

Journal of Global Optimization

, Volume 73, Issue 1, pp 83–112 | Cite as

A Lyapunov-type approach to convergence of the Douglas–Rachford algorithm for a nonconvex setting

  • Minh N. DaoEmail author
  • Matthew K. Tam
Article

Abstract

The Douglas–Rachford projection algorithm is an iterative method used to find a point in the intersection of closed constraint sets. The algorithm has been experimentally observed to solve various nonconvex feasibility problems; an observation which current theory cannot sufficiently explain. In this paper, we prove convergence of the Douglas–Rachford algorithm in a potentially nonconvex setting. Our analysis relies on the existence of a Lyapunov-type functional whose convexity properties are not tantamount to convexity of the original constraint sets. Moreover, we provide various nonconvex examples in which our framework proves global convergence of the algorithm.

Keywords

Douglas–Rachford algorithm Feasibility problem Global convergence Graph of a function Linear convergence Lyapunov function Method of alternating projections Newton’s method Nonconvex set Projection Stability Zero of a function 

Mathematics Subject Classification

90C26 47H10 37B25 

Notes

Acknowledgements

This paper is dedicated to the memory of Jonathan M. Borwein and his enthusiasm for the Douglas–Rachford algorithm. MND was partially supported by the Australian Research Council Discovery Project DP160101537 and a Startup Research Grant from the University of Newcastle. He wishes to acknowledge the hospitality and the support of D. Russell Luke during his visit to Universität Göttingen. MKT was partially supported by the Deutsche Forschungsgemeinschaft RTG 2088 and a Postdoctoral Fellowship from the Alexander von Humboldt Foundation. The authors wish to thank the three anonymous referees for their comments and suggestions.

References

  1. 1.
    Aragón Artacho, F.J., Borwein, J.M.: Global convergence of a nonconvex Douglas–Rachford iteration. J. Glob. Optim. 57(3), 753–769 (2013)zbMATHGoogle Scholar
  2. 2.
    Aragón Artacho, F.J., Borwein, J.M., Tam, M.K.: Global behavior of the Douglas–Rachford method for a nonconvex feasibility problem. J. Glob. Optim. 65(2), 309–327 (2016)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Barnsley, M.F.: Fractals Everywhere, 2nd edn. Morgen Kaufman, Burlington (1993)zbMATHGoogle Scholar
  4. 4.
    Bauschke, H.H., Bello Cruz, J.Y., Nghia, T.T.A., Phan, H.M., Wang, X.: The rate of linear convergence of the Douglas–Rachford algorithm for subspaces is the cosine of the Friedrichs angle. J. Approx. Theory 185, 63–79 (2014)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York (2011)zbMATHGoogle Scholar
  6. 6.
    Bauschke, H.H., Combettes, P.L., Luke, D.R.: Phase retrieval, error reduction algorithm, and Fienup variants: a view from convex optimization. JOSA A 19(7), 1334–1345 (2002)MathSciNetGoogle Scholar
  7. 7.
    Bauschke, H.H., Combettes, P.L., Luke, D.R.: Finding best approximation pairs relative to two closed convex sets in Hilbert spaces. J. Approx. Theory 127(2), 178–192 (2004)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Bauschke, H.H., Combettes, P.L., Noll, D.: Joint minimization with alternating Bregman proximity operators. Pac. J. Optim. 2(3), 401–424 (2006)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Bauschke, H.H., Dao, M.N.: On the finite convergence of the Douglas–Rachford algorithm for solving (not necessarily convex) feasibility problems in Euclidean spaces. SIAM J. Optim. 27(1), 507–537 (2017)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Bauschke, H.H., Dao, M.N., Moursi, W.M.: On Fejér monotone sequences and nonexpansive mappings. Linear Nonlinear Anal. 1(2), 287–295 (2015)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Bauschke, H.H., Dao, M.N., Moursi, W.M.: The Douglas–Rachford algorithm in the affine-convex case. Oper. Res. Lett. 44(3), 379–382 (2016)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Bauschke, H.H., Dao, M.N., Noll, D., Phan, H.M.: Proximal point algorithm, Douglas–Rachford algorithm and alternating projections: a case study. J. Convex Anal. 23(1), 237–261 (2016)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Bauschke, H.H., Dao, M.N., Noll, D., Phan, H.M.: On Slater’s condition and finite convergence of the Douglas–Rachford algorithm for solving convex feasibility problems in Euclidean spaces. J. Glob. Optim. 65(2), 329–349 (2016)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Bauschke, H.H., Moursi, W.M.: On the order of the operators in the Douglas–Rachford algorithm. Optim. Lett. 10(3), 447–455 (2016)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Bauschke, H.H., Moursi, W.M.: On the Douglas–Rachford algorithm. Math. Program. A 164(1), 263–284 (2017)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Bauschke, H.H., Wang, C., Wang, X., Xu, J.: On subgradient projections. SIAM J. Optim. 25, 1064–1082 (2015)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Bauschke, H.H., Wang, C., Wang, X., Xu, J.: Subgradient projectors: extensions, theory, and characterizations. Set-Valued Var. Anal. (2017).  https://doi.org/10.1007/s11228-017-0415-x Google Scholar
  18. 18.
    Benoist, J.: The Douglas–Rachford algorithm for the case of the sphere and the line. J. Glob. Optim. 63(2), 363–380 (2015)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Borwein, J.M., Sims, B.: The Douglas–Rachford algorithm in the absence of convexity. In: Bauschke, H.H., Burachik, R., Combettes, P.L., Elser, V., Luke, D.R., Wolkowicz, H. (eds.) Fixed-Point Algorithms for Inverse Problems in Science and Engineering, pp. 93–109. Springer, New York (2011)Google Scholar
  20. 20.
    Dao, M.N., Phan, H.M.: Linear convergence of projection algorithms. Math. Oper. Res. (to appear). arXiv:1609.00341
  21. 21.
    Dao, M.N., Phan, H.M.: Linear convergence of the generalized Douglas–Rachford algorithm for feasibility problems. J. Glob. Optim. (2018).  https://doi.org/10.1007/s10898-018-0654-x MathSciNetzbMATHGoogle Scholar
  22. 22.
    Douglas, J., Rachford, H.H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82, 421–439 (1956)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Drusvyatskiy, D., Ioffe, A.D., Lewis, A.S.: Transversality and alternating projections for nonconvex sets. Found. Comput. Math. 15(6), 1637–1651 (2015)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Giladi, O.: A remark on the convergence of the Douglas–Rachford iteration in a non-convex setting. Set-Valued Var. Anal. 26(2), 207–225 (2018)MathSciNetzbMATHGoogle Scholar
  25. 25.
    Hesse, R., Luke, D.R.: Nonconvex notions of regularity and convergence of fundamental algorithms for feasibility problems. SIAM J. Optim. 23(4), 2397–2419 (2013)MathSciNetzbMATHGoogle Scholar
  26. 26.
    Lewis, A.S., Luke, D.R., Malick, J.: Local linear convergence for alternating and averaged nonconvex projections. Found. Comput. Math. 9(4), 485–513 (2009)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Lindstrom, S.B., Sims, B., Skerritt, M.: Computing intersections of implicitly specified plane curves. J. Nonlinear Convex Anal. 18(3), 347–359 (2017)MathSciNetzbMATHGoogle Scholar
  28. 28.
    Lions, P.-L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979)MathSciNetzbMATHGoogle Scholar
  29. 29.
    Mordukhovich, B.: Variational Analysis and Generalized Differentiation I. Basic Theory. Springer, Berlin (2006)Google Scholar
  30. 30.
    Phan, H.M.: Linear convergence of the Douglas–Rachford method for two closed sets. Optimization 65(2), 369–385 (2016)MathSciNetzbMATHGoogle Scholar
  31. 31.
    Noll, D., Rondepierre, A.: On local convergence of the method of alternating projections. Found. Comput. Math. 16(2), 425–455 (2016)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer, Berlin (1998)zbMATHGoogle Scholar
  33. 33.
    Sherman, J., Morrison, W.J.: Adjustment of an inverse matrix corresponding to a change in one element of a given matrix. Ann. Math. Stat. 21(1), 124–127 (1950)MathSciNetzbMATHGoogle Scholar
  34. 34.
    Svaiter, B.F.: On weak convergence of the Douglas–Rachford method. SIAM J. Control Optim. 49(1), 280–287 (2011)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.CARMAUniversity of NewcastleCallaghanAustralia
  2. 2.Institut für Numerische und Angewandte MathematikUniversität GöttingenGöttingenGermany

Personalised recommendations