Skip to main content
SpringerLink
Log in
Book cover

Jonathan M. Borwein Commemorative Conference

JBCC 2017: From Analysis to Visualization pp 75–98Cite as

Comparing Averaged Relaxed Cutters and Projection Methods: Theory and Examples

  • Conference paper
  • First Online:

Part of the book series: Springer Proceedings in Mathematics & Statistics ((PROMS,volume 313))

Abstract

We focus on the convergence analysis of averaged relaxations of cutters, specifically for variants that—depending upon how parameters are chosen—resemble alternating projections, the Douglas–Rachford method, relaxed reflect-reflect, or the Peaceman–Rachford method. Such methods are frequently used to solve convex feasibility problems. The standard convergence analysis of projection algorithms is based on the firm nonexpansivity property of the relevant operators. However, if the projections onto the constraint sets are replaced by cutters (which may be thought of as maps that project onto separating hyperplanes), the firm nonexpansivity is lost. We provide a proof of convergence for a family of related averaged relaxed cutter methods under reasonable assumptions, relying on a simple geometric argument. This allows us to clarify fine details related to the allowable choice of the relaxation parameters, highlighting the distinction between the exact (firmly nonexpansive) and approximate (strongly quasinonexpansive) settings. We provide illustrative examples and discuss practical implementations of the method.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Aleyner, A., Reich, S.: Block-iterative algorithms for solving convex feasibility problems in Hilbert and in Banach spaces. J. Math. Anal. Appl. 343(1), 427–435 (2008). https://doi.org/10.1016/j.jmaa.2008.01.087

  2. Aragón Artacho, F.J., Borwein, J.M.: Global convergence of a non-convex Douglas-Rachford iteration. J. Glob. Optim. 57(3), 753–769 (2012)

    Article  MathSciNet  Google Scholar 

  3. Aragón Artacho, F.J., Borwein, J.M., Tam, M.K.: Douglas–Rachford feasibility methods for matrix completion problems. ANZIAM J. 55(4), 299–326 (2014). https://doi.org/10.1017/S1446181114000145

  4. Aragón Artacho, F.J., Borwein, J.M., Tam, M.K.: Recent results on Douglas–Rachford methods for combinatorial optimization problems. J. Optim. Theory Appl. 163(1), 1–30 (2014). https://doi.org/10.1007/s10957-013-0488-0

  5. Aragón Artacho, F.J., Borwein, J.M., Tam, M.K.: Global behavior of the Douglas–Rachford method for a nonconvex feasibility problem. J. Global Optim. 65(2), 309–327 (2016). https://doi.org/10.1007/s10898-015-0380-6

  6. Aragón Artacho, F.J., Campoy, R.: Solving graph coloring problems with the Douglas–Rachford algorithm. Set-Valued and Variational Analysis pp. 1–28 (2017)

    Google Scholar 

  7. Aragón Artacho, F.J., Campoy, R.: A new projection method for finding the closest point in the intersection of convex sets. Comput. Optim. Appl. 69(1), 99–132 (2018). https://doi.org/10.1007/s10589-017-9942-5

  8. Artacho, F.J.A., Campoy, R.: Optimal rates of linear convergence of the averaged alternating modified reflections method for two subspaces (2017). arXiv:1711.06521

  9. Baillon, J.B., Combettes, P.L., Cominetti, R.: There is no variational characterization of the cycles in the method of periodic projections. J. Funct. Anal. 262(1), 400–408 (2012). https://doi.org/10.1016/j.jfa.2011.09.002

  10. Bauschke, H.H., Combettes, P.L.: A weak-to-strong convergence principle for fejér-monotone methods in hilbert spaces. Mathematics of Operations Research 26(2), 248–264. http://www.jstor.org/stable/3690618 (2001)

  11. Bauschke, H.H., Combettes, P.L.: Convex analysis and monotone operator theory in Hilbert spaces, 2nd edn. CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-48311-5. With a foreword by Hédy Attouch

  12. Bauschke, H.H., Combettes, P.L., Luke, D.R.: Phase retrieval, error reduction algorithm, and Fienup variants: a view from convex optimization. J. Opt. Soc. Am. A 19(7), 1334–1345 (2002). https://doi.org/10.1364/JOSAA.19.001334

  13. Bauschke, H.H., Dao, M.N., Noll, D., Phan, H.M.: On Slater’s condition and finite convergence of the Douglas–Rachford algorithm for solving convex feasibility problems in Euclidean spaces. J. Global Optim. 65(2), 329–349 (2016). https://doi.org/10.1007/s10898-015-0373-5

  14. Bauschke, H.H., Wang, C., Wang, X., Xu, J.: On subgradient projectors. SIAM J. Optim. 25(2), 1064–1082 (2015)

    Article  MathSciNet  Google Scholar 

  15. Bauschke, H.H., Wang, C., Wang, X., Xu, J.: On the finite convergence of a projected cutter method. J. Optim. Theory Appl. 165(3), 901–916 (2015)

    Article  MathSciNet  Google Scholar 

  16. Bauschke, H.H., Wang, C., Wang, X., Xu, J.: Subgradient projectors: extensions, theory, and characterizations. Set-Valued Var. Anal. 1–70 (2017)

    Google Scholar 

  17. Bello Cruz, J.Y., Díaz Millán, R.: A relaxed-projection splitting algorithm for variational inequalities in Hilbert spaces. J. Global Optim. 65(3), 597–614 (2016). https://doi.org/10.1007/s10898-015-0397-x

  18. Bello Cruz, J.Y., Iusem, A.N.: An explicit algorithm for monotone variational inequalities. Optimization 61(7), 855–871 (2012). https://doi.org/10.1080/02331934.2010.536232

  19. Benoist, J.: The Douglas-Rachford algorithm for the case of the sphere and the line. J. Glob. Optim. 63, 363–380 (2015)

    Article  MathSciNet  Google Scholar 

  20. Borwein, J.M.: The life of modern homo habilis mathematicus: experimental computation and visual theorems. Tools and Mathematics. Mathematics Education Library, vol. 347, pp. 23–90. Springer, Berlin (2016)

    Chapter  Google Scholar 

  21. Borwein, J.M., Li, G., Tam, M.K.: Convergence rate analysis for averaged fixed point iterations in common fixed point problems. SIAM J. Optim. 27(1), 1–33 (2017)

    Article  MathSciNet  Google Scholar 

  22. Borwein, J.M., Li, G., Yao, L.: Analysis of the convergence rate for the cyclic projection algorithm applied to basic semialgebraic convex sets. SIAM J. Optim. 24(1), 498–527 (2014). https://doi.org/10.1137/130919052

  23. Borwein, J.M., Lindstrom, S.B., Sims, B., Schneider, A., Skerritt, M.P.: Dynamics of the Douglas-Rachford method for ellipses and p-spheres. Set-Valued Var. Anal. 26(2), 385–403 (2018)

    Article  MathSciNet  Google Scholar 

  24. Borwein, J.M., Sims, B.: The Douglas–Rachford algorithm in the absence of convexity. Fixed-Point Algorithms for Inverse Problems in Science and Engineering. Springer Optimization and Its Applications, vol. 49, pp. 93–109. Springer, New York (2011). https://doi.org/10.1007/978-1-4419-9569-8_6

  25. Cegielski, A.: Iterative Methods for Fixed Point Problems in Hilbert Spaces. Lecture Notes in Mathematics, vol. 2057. Springer, Heidelberg (2012)

    MATH  Google Scholar 

  26. Cegielski, A., Reich, S., Zalas, R.: Regular sequences of quasi-nonexpansive operators and their applications (2017)

    Google Scholar 

  27. Censor, Y., Lent, A.: Cyclic subgradient projections. Math. Program. 24(1), 233–235 (1982)

    Article  MathSciNet  Google Scholar 

  28. Censor, Y., Zenios, S.A.: Parallel Optimization: Theory, Algorithms, and Applications. Oxford University Press on Demand (1997)

    Google Scholar 

  29. Combettes, P.L.: Convex set theoretic image recovery by extrapolated iterations of parallel subgradient projections. IEEE Trans. Image Process. 6(4), 493–506 (1997)

    Article  Google Scholar 

  30. Combettes, P.L.: Quasi-Fejérian analysis of some optimization algorithms 8, 115–152 http://www.sciencedirect.com/science/article/pii/S1570579X01800100 (2001)

  31. Cominetti, R., Roshchina, V., Williamson, A.: A counterexample to De Pierro’s conjecture on the convergence of under-relaxed cyclic projections (2018)

    Google Scholar 

  32. Dao, M.N., Phan, H.M.: Linear convergence of projection algorithms (2016). arXiv:1609.00341

  33. Dao, M.N., Phan, H.M.: Linear convergence of the generalized Douglas–Rachford algorithm for feasibility problems (2017). arXiv:1710.09814

  34. De Pierro, A.R.: From parallel to sequential projection methods and vice versa in convex feasibility: results and conjectures. Inherently Parallel Algorithms in Feasibility and Optimization and Their Applications (Haifa, 2000). Studies in Computational Mathematics, vol. 8, pp. 187–201. North-Holland, Amsterdam (2001). https://doi.org/10.1016/S1570-579X(01)80012-4

  35. Douglas Jr., J., Rachford Jr., H.H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82, 421–439 (1956). https://doi.org/10.2307/1993056

  36. Drusvyatskiy, D., Li, G., Wolkowicz, H.: A note on alternating projections for ill-posed semidefinite feasibility problems. Math. Program. 162(1-2, Ser. A), 537–548 (2017). https://doi.org/10.1007/s10107-016-1048-9

  37. Elser, V.: Matrix product constraints by projection methods. J. Global Optim. 68(2), 329–355 (2017)

    Article  MathSciNet  Google Scholar 

  38. Fukushima, M.: A relaxed projection method for variational inequalities. Math. Program. 35(1), 58–70 (1986). https://doi.org/10.1007/BF01589441

  39. Hundal, H.S.: An alternating projection that does not converge in norm. Nonlinear Analysis: Theory, Methods & Applications 57(1), 35–61. http://www.sciencedirect.com/science/article/pii/S0362546X03004218 (2004). https://doi.org/10.1016/j.na.2003.11.004

  40. Kruger, A.Y., Luke, D.R., Thao, N.H.: About subtransversality of collections of sets. Set-Valued Var. Anal. 25(4), 701–729 (2017). https://doi.org/10.1007/s11228-017-0436-5

  41. Lamichhane, B.P., Lindstrom, S.B., Sims, B.: Application of projection algorithms to differential equations: boundary value problems (2017). arXiv:1705.11032

  42. Li, G., Pong, T.K.: Douglas-Rachford splitting for nonconvex optimization with application to nonconvex feasibility problems. Math. Program. 159(1-2, Ser. A), 371–401 (2016). https://doi.org/10.1007/s10107-015-0963-5

  43. Lindstrom, S.B., Sims, B.: Survey: sixty years of Douglas–Rachford (2018). arXiv:1809.07181

  44. Lindstrom, S.B., Sims, B., Skerritt, M.P.: Computing intersections of implicitly specified plane curves. Nonlinear Conv. Anal. 18(3), 347–359 (2017)

    MathSciNet  MATH  Google Scholar 

  45. Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979). https://doi.org/10.1137/0716071

  46. Littlewood, J.E.: A Mathematician’s Miscellany. Methuen London (1953)

    Google Scholar 

  47. Pierra, G.: Decomposition through formalization in a product space. Math. Program. 28(1), 96–115 (1984)

    Article  MathSciNet  Google Scholar 

  48. Polyak, B.T.: Introduction to optimization. Translations Series in Mathematics and Engineering. Optimization Software (1987)

    Google Scholar 

  49. Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)

    Google Scholar 

  50. Spingarn, J.E.: Partial inverse of a monotone operator. Appl. Math. Optim. 10(1), 247–265 (1983)

    Article  MathSciNet  Google Scholar 

  51. Svaiter, B.F.: On weak convergence of the Douglas-Rachford method. SIAM J. Control Optim. 49(1), 280–287 (2011)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

We are grateful to Heinz Bauschke for pointing out useful references. We also extend our gratitude to the organisers and participants of the workshop Splitting Algorithms, Modern Operator Theory and Applications held at Casa Matematica Oaxaca, Mexico, from 17–22 September 2017 and to the Australian Research Council for supporting our travel expenses via project DE150100240. We are also grateful for the helpful feedback of two anonymous referees, whose suggestions greatly improved this manuscript.

We dedicate this work to the memory of Jonathan M. Borwein, whose influence on the present authors and on the Australian mathematical community cannot be overstated.

Author information

Authors and Affiliations

  1. Federal Institute of Goiás, Goiás, Brazil

    Reinier Díaz Millán

  2. School of Mathematical and Physical Sciences, Centre for Computer-assisted Research Mathematics and its Applications (CARMA), The University of Newcastle, Callaghan, NSW, Australia

    Scott B. Lindstrom

  3. UNSW Sydney, Sydney, NSW, Australia

    Vera Roshchina

Authors
  1. Reinier Díaz Millán
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Scott B. Lindstrom
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vera Roshchina
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Scott B. Lindstrom .

Editor information

Editors and Affiliations

  1. Lawrence Berkeley National Laboratory, Berkeley, CA, USA

    David H. Bailey

  2. Western University, London, ON, Canada

    Naomi Simone Borwein

  3. Mathematical Sciences Institute, Australian National University, Canberra, ACT, Australia

    Richard P. Brent

  4. School of IT and Mathematical Sciences, University of South Australia, Mawson Lakes, SA, Australia

    Regina S. Burachik

  5. School of Mathematical and Physical Sciences, University of Newcastle, Callaghan, NSW, Australia

    Judy-anne Heather Osborn

  6. School of Mathematical and Physical Sciences, University of Newcastle, Callaghan, NSW, Australia

    Brailey Sims

  7. Department of Mathematics, Western Michigan University, Kalamazoo, MI, USA

    Qiji J. Zhu

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Millán, R.D., Lindstrom, S.B., Roshchina, V. (2020). Comparing Averaged Relaxed Cutters and Projection Methods: Theory and Examples. In: Bailey, D., et al. From Analysis to Visualization. JBCC 2017. Springer Proceedings in Mathematics & Statistics, vol 313. Springer, Cham. https://doi.org/10.1007/978-3-030-36568-4_5

Download citation

Publish with us

Policies and ethics

Search

Navigation