Advertisement

Set-Valued and Variational Analysis

, Volume 26, Issue 3, pp 469–492 | Cite as

Computing Proximal Points of Convex Functions with Inexact Subgradients

  • W. Hare
  • C. Planiden
Article
  • 75 Downloads

Abstract

Locating proximal points is a component of numerous minimization algorithms. This work focuses on developing a method to find the proximal point of a convex function at a point, given an inexact oracle. Our method assumes that exact function values are at hand, but exact subgradients are either not available or not useful. We use approximate subgradients to build a model of the objective function, and prove that the method converges to the true prox-point within acceptable tolerance. The subgradient g k used at each step k is such that the distance from g k to the true subdifferential of the objective function at the current iteration point is bounded by some fixed ε > 0. The algorithm includes a novel tilt-correct step applied to the approximate subgradient.

Keywords

Bundle methods Convex optimization Cutting-plane methods Inexact subgradient Proximal point 

Mathematics Subject Classifications

Primary 49M30 65K10 Secondary 90C20 90C56 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Al-Khayyal, F., Kyparisis, J.: Finite convergence of algorithms for nonlinear programs and variational inequalities. J. Optim. Theory Appl. 70(2), 319–332 (1991)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Bagirov, A.M., Karasözen, B., Sezer, M.: Discrete gradient method: derivative-free method for nonsmooth optimization. J. Optim. Theory Appl. 137(2), 317–334 (2008)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York (2011)CrossRefMATHGoogle Scholar
  4. 4.
    Bonnans, J.F., Gilbert, J.C., Lemaréchal, C., Sagastizábal, C.: Numerical optimization: Theoretical and Practical Aspects, 2nd edn. Springer, Berlin (2006)MATHGoogle Scholar
  5. 5.
    Coleman, T.F., Hulbert, L.: A Globally and Superlinearly Convergent Algorithm for Convex Quadratic Programs with Simple Bounds. Technical report, Cornell University (1990)Google Scholar
  6. 6.
    Combettes, P.L., Pesquet, J.-C.: Proximal splitting methods in signal processing. In: Fixed-point algorithms for inverse problems in science and engineering, pp 185–212. Springer (2011)Google Scholar
  7. 7.
    Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to derivative-free optimization, volume 8 Siam (2009)Google Scholar
  8. 8.
    Correa, R., Lemaréchal, C.: Convergence of some algorithms for convex minimization. Math. Program. 62(1-3), 261–275 (1993)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    de Oliveira, W., Sagastizábal, C., Lemaréchal, C.: Convex proximal bundle methods in depth: a unified analysis for inexact oracles. Math. Program. 148(1-2), 241–277 (2014)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Douglas, J., Rachford, H.H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82(2), 421–439 (1956)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Rey, P.A., Sagastizábal, C.: Dynamical adjustment of the prox-parameter in bundle methods. JOURNAL = Optimization. Optimization A Journal of Mathematical Programming and Operations Research 51(2), 423–447 (2002)MathSciNetMATHGoogle Scholar
  12. 12.
    Ferris, M.C.: Finite termination of the proximal point algorithm. Math. Program. 50(1-3), 359–366 (1991)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Fisher, M.L.: An applications oriented guide to Lagrangian relaxation. Interfaces 15(2), 10–21 (1985)CrossRefGoogle Scholar
  14. 14.
    Gupal, A.M.: A method for the minimization of almost-differentiable functions. Cybern. Syst. Anal. 13(1), 115–117 (1977)MathSciNetGoogle Scholar
  15. 15.
    Hare, W.: Numerical analysis of 𝓥𝓤-decomposition, 𝓤-gradient, and 𝓤-Hessian approximations. SIAM J. Optim. 24(4), 1890–1913 (2014)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Hare, W., Nutini, J.: A derivative-free approximate gradient sampling algorithm for finite minimax problems. Comput. Optim. Appl. 56(1), 1–38 (2013)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Hare, W., Nutini, J., Tesfamariam, S.: A survey of non-gradient optimization methods in structural engineering. Adv. Eng. Softw. 59, 19–28 (2013)CrossRefGoogle Scholar
  18. 18.
    Hare, W., Sagastizábal, C.: Computing proximal points of nonconvex functions. Math Program 116(1-2, Ser. B), 221–258 (2009)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Hare, W., Sagastizábal, C.: A redistributed proximal bundle method for nonconvex optimization. SIAM J. Optim. 20(5), 2442–2473 (2010)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    Hare, W., Sagastizábal, C., Solodov, M.: A proximal bundle method for nonsmooth nonconvex functions with inexact information. Comput. Optim. Appl. 63 (1), 1–28 (2016)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Hintermüller, M.: A proximal bundle method based on approximate subgradients. Comput. Optim. Appl. 20(3), 245–266 (2001)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Hiriart-Urruty, J.B., Lemarechal, C.: Convex analysis and minimization algorithms. Grundlehren der mathematischen Wissenschaften, 305 (1993)Google Scholar
  23. 23.
    Kalvelagen, E.: Benders decomposition with gams (2002)Google Scholar
  24. 24.
    Karas, E., Ribeiro, A., Sagastizábal, C., Solodov, M.: A bundle-filter method for nonsmooth convex constrained optimization. Math. Program. 116(1-2), 297–320 (2009)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Kelley, C.T.: Iterative methods for optimization, volume 18 Siam (1999)Google Scholar
  26. 26.
    Kiwiel, K.C.: Proximity control in bundle methods for convex nondifferentiable minimization. Math. Program. 46(1-3), 105–122 (1990)MathSciNetCrossRefMATHGoogle Scholar
  27. 27.
    Kiwiel, K.C.: Approximations in proximal bundle methods and decomposition of convex programs. J. Optim. Theory Appl. 84(3), 529–548 (1995)MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Kiwiel, K.C.: A nonderivative version of the gradient sampling algorithm for nonsmooth nonconvex optimization. SIAM J. Optim. 20(4), 1983–1994 (2010)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Lemaréchal, C., Sagastizábal, C.: Variable metric bundle methods: from conceptual to implementable forms. Math. Program. 76(3), 393–410 (1997)MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Lewis, A.S., Wright, S.J.: A proximal method for composite minimization. Math. Program., 1–46 (2008)Google Scholar
  31. 31.
    Lukšan, L., Vlcek, J.: Test problems for nonsmooth unconstrained and linearly constrained optimization. Technická zpráva 798 (2000)Google Scholar
  32. 32.
    Martinet, B.: Régularisation d’inéquations variationnelles par approximations successives, Rev. Française Informat. Recherche Opé,rationnelle 4(Ser. R-3), 154–158 (1970)MATHGoogle Scholar
  33. 33.
    Mayer, J.: Stochastic linear programming algorithms: A comparison based on a model management system, volume 1. CRC Press (1998)Google Scholar
  34. 34.
    Mifflin, R., Sagastizábal, C.: A VU-algorithm for convex minimization. Math Program. 104(2-3, Ser. B), 583–608 (2005)MathSciNetCrossRefMATHGoogle Scholar
  35. 35.
    Moreau, J.-J.: Propriétés des applications “prox”. C. R. Acad. Sci. Paris 256, 1069–1071 (1963)MathSciNetMATHGoogle Scholar
  36. 36.
    Moreau, J.-J.: Proximité et dualité dans un espace hilbertien. Bull. Soc. Math. France 93, 273–299 (1965)MathSciNetCrossRefMATHGoogle Scholar
  37. 37.
    Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optimization 14(5), 877–898 (1976)MathSciNetCrossRefMATHGoogle Scholar
  38. 38.
    Rockafellar, R.T., Wets, R.J.: Variational analysis. Springer, Berlin (1998)CrossRefMATHGoogle Scholar
  39. 39.
    Sagastizábal, C.: Composite proximal bundle method. Math. Program. 140(1), 189–233 (2013)MathSciNetCrossRefMATHGoogle Scholar
  40. 40.
    Shen, J., Liu, X.-Q., Guo, F.-F., Wang, S.-X.: An approximate redistributed proximal bundle method with inexact data for minimizing nonsmooth nonconvex functions. Math. Probl. Eng., 2015 (2015)Google Scholar
  41. 41.
    Shen, J., Xia, Z.-Q., Pang, L.-P.: A proximal bundle method with inexact data for convex nondifferentiable minimization. Nonlinear Analysis Theory, Methods & Applications 66(9), 2016–2027 (2007)MathSciNetCrossRefMATHGoogle Scholar
  42. 42.
    Solodov, M.V.: A bundle method for a class of bilevel nonsmooth convex minimization problems. SIAM J. Optim. 18(1), 242–259 (2007)MathSciNetCrossRefMATHGoogle Scholar
  43. 43.
    Sun, W.-Y., Sampaio, R.J.B., Candido, M.A.B.: Proximal point algorithm for minimization of dc function. Journal of Computational Mathematics-International Edition 21(4), 451–462 (2003)MathSciNetMATHGoogle Scholar
  44. 44.
    Tseng, P.: Approximation accuracy, gradient methods, and error bound for structured convex optimization. Math. Program. 125(2), 263–295 (2010)MathSciNetCrossRefMATHGoogle Scholar
  45. 45.
    Wang, C., Xu, A.: An inexact accelerated proximal gradient method and a dual newton-cg method for the maximal entropy problem. J. Optim. Theory Appl. 157(2), 436–450 (2013)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  1. 1.MathematicsUniversity of British ColumbiaKelownaCanada

Personalised recommendations