Advertisement

Set-Valued and Variational Analysis

, Volume 26, Issue 3, pp 469–492 | Cite as

Computing Proximal Points of Convex Functions with Inexact Subgradients

  • W. Hare
  • C. Planiden
Article
  • 85 Downloads

Abstract

Locating proximal points is a component of numerous minimization algorithms. This work focuses on developing a method to find the proximal point of a convex function at a point, given an inexact oracle. Our method assumes that exact function values are at hand, but exact subgradients are either not available or not useful. We use approximate subgradients to build a model of the objective function, and prove that the method converges to the true prox-point within acceptable tolerance. The subgradient g k used at each step k is such that the distance from g k to the true subdifferential of the objective function at the current iteration point is bounded by some fixed ε > 0. The algorithm includes a novel tilt-correct step applied to the approximate subgradient.

Keywords

Bundle methods Convex optimization Cutting-plane methods Inexact subgradient Proximal point 

Mathematics Subject Classifications

Primary 49M30 65K10 Secondary 90C20 90C56 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Al-Khayyal, F., Kyparisis, J.: Finite convergence of algorithms for nonlinear programs and variational inequalities. J. Optim. Theory Appl. 70(2), 319–332 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Bagirov, A.M., Karasözen, B., Sezer, M.: Discrete gradient method: derivative-free method for nonsmooth optimization. J. Optim. Theory Appl. 137(2), 317–334 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York (2011)CrossRefzbMATHGoogle Scholar
  4. 4.
    Bonnans, J.F., Gilbert, J.C., Lemaréchal, C., Sagastizábal, C.: Numerical optimization: Theoretical and Practical Aspects, 2nd edn. Springer, Berlin (2006)zbMATHGoogle Scholar
  5. 5.
    Coleman, T.F., Hulbert, L.: A Globally and Superlinearly Convergent Algorithm for Convex Quadratic Programs with Simple Bounds. Technical report, Cornell University (1990)Google Scholar
  6. 6.
    Combettes, P.L., Pesquet, J.-C.: Proximal splitting methods in signal processing. In: Fixed-point algorithms for inverse problems in science and engineering, pp 185–212. Springer (2011)Google Scholar
  7. 7.
    Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to derivative-free optimization, volume 8 Siam (2009)Google Scholar
  8. 8.
    Correa, R., Lemaréchal, C.: Convergence of some algorithms for convex minimization. Math. Program. 62(1-3), 261–275 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    de Oliveira, W., Sagastizábal, C., Lemaréchal, C.: Convex proximal bundle methods in depth: a unified analysis for inexact oracles. Math. Program. 148(1-2), 241–277 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Douglas, J., Rachford, H.H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82(2), 421–439 (1956)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Rey, P.A., Sagastizábal, C.: Dynamical adjustment of the prox-parameter in bundle methods. JOURNAL = Optimization. Optimization A Journal of Mathematical Programming and Operations Research 51(2), 423–447 (2002)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Ferris, M.C.: Finite termination of the proximal point algorithm. Math. Program. 50(1-3), 359–366 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Fisher, M.L.: An applications oriented guide to Lagrangian relaxation. Interfaces 15(2), 10–21 (1985)CrossRefGoogle Scholar
  14. 14.
    Gupal, A.M.: A method for the minimization of almost-differentiable functions. Cybern. Syst. Anal. 13(1), 115–117 (1977)MathSciNetGoogle Scholar
  15. 15.
    Hare, W.: Numerical analysis of 𝓥𝓤-decomposition, 𝓤-gradient, and 𝓤-Hessian approximations. SIAM J. Optim. 24(4), 1890–1913 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Hare, W., Nutini, J.: A derivative-free approximate gradient sampling algorithm for finite minimax problems. Comput. Optim. Appl. 56(1), 1–38 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Hare, W., Nutini, J., Tesfamariam, S.: A survey of non-gradient optimization methods in structural engineering. Adv. Eng. Softw. 59, 19–28 (2013)CrossRefGoogle Scholar
  18. 18.
    Hare, W., Sagastizábal, C.: Computing proximal points of nonconvex functions. Math Program 116(1-2, Ser. B), 221–258 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Hare, W., Sagastizábal, C.: A redistributed proximal bundle method for nonconvex optimization. SIAM J. Optim. 20(5), 2442–2473 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Hare, W., Sagastizábal, C., Solodov, M.: A proximal bundle method for nonsmooth nonconvex functions with inexact information. Comput. Optim. Appl. 63 (1), 1–28 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Hintermüller, M.: A proximal bundle method based on approximate subgradients. Comput. Optim. Appl. 20(3), 245–266 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Hiriart-Urruty, J.B., Lemarechal, C.: Convex analysis and minimization algorithms. Grundlehren der mathematischen Wissenschaften, 305 (1993)Google Scholar
  23. 23.
    Kalvelagen, E.: Benders decomposition with gams (2002)Google Scholar
  24. 24.
    Karas, E., Ribeiro, A., Sagastizábal, C., Solodov, M.: A bundle-filter method for nonsmooth convex constrained optimization. Math. Program. 116(1-2), 297–320 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Kelley, C.T.: Iterative methods for optimization, volume 18 Siam (1999)Google Scholar
  26. 26.
    Kiwiel, K.C.: Proximity control in bundle methods for convex nondifferentiable minimization. Math. Program. 46(1-3), 105–122 (1990)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Kiwiel, K.C.: Approximations in proximal bundle methods and decomposition of convex programs. J. Optim. Theory Appl. 84(3), 529–548 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Kiwiel, K.C.: A nonderivative version of the gradient sampling algorithm for nonsmooth nonconvex optimization. SIAM J. Optim. 20(4), 1983–1994 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Lemaréchal, C., Sagastizábal, C.: Variable metric bundle methods: from conceptual to implementable forms. Math. Program. 76(3), 393–410 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Lewis, A.S., Wright, S.J.: A proximal method for composite minimization. Math. Program., 1–46 (2008)Google Scholar
  31. 31.
    Lukšan, L., Vlcek, J.: Test problems for nonsmooth unconstrained and linearly constrained optimization. Technická zpráva 798 (2000)Google Scholar
  32. 32.
    Martinet, B.: Régularisation d’inéquations variationnelles par approximations successives, Rev. Française Informat. Recherche Opé,rationnelle 4(Ser. R-3), 154–158 (1970)zbMATHGoogle Scholar
  33. 33.
    Mayer, J.: Stochastic linear programming algorithms: A comparison based on a model management system, volume 1. CRC Press (1998)Google Scholar
  34. 34.
    Mifflin, R., Sagastizábal, C.: A VU-algorithm for convex minimization. Math Program. 104(2-3, Ser. B), 583–608 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Moreau, J.-J.: Propriétés des applications “prox”. C. R. Acad. Sci. Paris 256, 1069–1071 (1963)MathSciNetzbMATHGoogle Scholar
  36. 36.
    Moreau, J.-J.: Proximité et dualité dans un espace hilbertien. Bull. Soc. Math. France 93, 273–299 (1965)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optimization 14(5), 877–898 (1976)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Rockafellar, R.T., Wets, R.J.: Variational analysis. Springer, Berlin (1998)CrossRefzbMATHGoogle Scholar
  39. 39.
    Sagastizábal, C.: Composite proximal bundle method. Math. Program. 140(1), 189–233 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Shen, J., Liu, X.-Q., Guo, F.-F., Wang, S.-X.: An approximate redistributed proximal bundle method with inexact data for minimizing nonsmooth nonconvex functions. Math. Probl. Eng., 2015 (2015)Google Scholar
  41. 41.
    Shen, J., Xia, Z.-Q., Pang, L.-P.: A proximal bundle method with inexact data for convex nondifferentiable minimization. Nonlinear Analysis Theory, Methods & Applications 66(9), 2016–2027 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  42. 42.
    Solodov, M.V.: A bundle method for a class of bilevel nonsmooth convex minimization problems. SIAM J. Optim. 18(1), 242–259 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Sun, W.-Y., Sampaio, R.J.B., Candido, M.A.B.: Proximal point algorithm for minimization of dc function. Journal of Computational Mathematics-International Edition 21(4), 451–462 (2003)MathSciNetzbMATHGoogle Scholar
  44. 44.
    Tseng, P.: Approximation accuracy, gradient methods, and error bound for structured convex optimization. Math. Program. 125(2), 263–295 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Wang, C., Xu, A.: An inexact accelerated proximal gradient method and a dual newton-cg method for the maximal entropy problem. J. Optim. Theory Appl. 157(2), 436–450 (2013)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  1. 1.MathematicsUniversity of British ColumbiaKelownaCanada

Personalised recommendations