Advertisement

Nonsmooth and Nonconvex Optimization via Approximate Difference-of-Convex Decompositions

  • Wim van Ackooij
  • Welington de OliveiraEmail author
Article
  • 134 Downloads

Abstract

We propose an optimization technique for computing stationary points of a broad class of nonsmooth and nonconvex programming problems. The proposed approach (approximately) decomposes the objective function as the difference of two convex functions and performs inexact optimization of the resulting (convex) subproblems. We prove global convergence of our method in the sense that, for an arbitrary starting point, every accumulation point of the sequence of iterates is a Clarke-stationary solution. The given approach is validated by encouraging numerical results on several nonsmooth and nonconvex distributionally robust optimization problems.

Keywords

Nonconvex programming Nonsmooth optimization Lower-\(C^2\) functions DC decomposition 

Mathematics Subject Classification

49J52 49J53 49K99 90C26 

Notes

Acknowledgements

The authors would like to acknowledge financial support from the Gaspard-Monge program for Optimization and Operations Research (PGMO) project “Optimization & stability of stochastic unit-commitment problems.”

References

  1. 1.
    Hartman, P.: On functions representable as a difference of convex functions. Pac. J. Math. 9(3), 167–198 (1959)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Rockafellar, R., Wets, R.J.B.: Variational Analysis, Grundlehren der mathematischen Wissenschaften, vol. 317, 3rd edn. Springer, Berlin (2009)Google Scholar
  3. 3.
    Pflug, G.C., Pohl, M.: A review on ambiguity in stochastic portfolio optimization. Set Valued Var. Anal. (2017).  https://doi.org/10.1007/s11228-017-0458-z zbMATHGoogle Scholar
  4. 4.
    Esfahani, P.M., Kuhn, D.: Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations. Math. Program. 171(1–2), 115–166 (2018)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Curtis, F.E., Mitchell, T., Overton, M.L.: A BFGS-SQP method for nonsmooth, nonconvex, constrained optimization and its evaluation using relative minimization profiles. Optim. Methods Softw. 32(1), 148–181 (2017)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Ginchev, I., Gintcheva, D.: Characterization and recognition of D.C. functions. J. Glob. Optim. 57(3), 633–647 (2012).  https://doi.org/10.1007/s10898-012-9964-6 MathSciNetzbMATHGoogle Scholar
  7. 7.
    Toa, P.D., Souad, E.B.: Duality in D.C. (difference of convex functions) optimization. Subgradient methods. In: Hoffmann, K.H., Zowe, J., Hiriart-Urruty, J., Lemarechal, C. (eds.) Trends in Mathematical Optimization. International Series of Numerical Mathematics, vol. 84, pp. 277–293. Birkhauser, Basel (1988)Google Scholar
  8. 8.
    de Oliveira, W., Tcheou, M.: An inertial algorithm for DC programming. Set Valued Var. Anal. (2018).  https://doi.org/10.1007/s11228-018-0497-0
  9. 9.
    Toa, P.D.: Exact penalty in D.C. programming. Vietnam J. Math. 27(2), 169–178 (1999)MathSciNetGoogle Scholar
  10. 10.
    Tao, P.D., Le Thi, H.A.: Convex analysis approach to DC programming: theory, algorithms and applications. Acta Math. Vietnam. 22(1), 289–355 (1997)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Pang, J.S., Razaviyayn, M., Alvarado, A.: Computing B-stationary points of nonsmooth DC programs. Math. Oper. Res. 42(1), 95–118 (2017).  https://doi.org/10.1287/moor.2016.0795 MathSciNetzbMATHGoogle Scholar
  12. 12.
    Strekalovsky, A.S.: On local search in D.C. optimization problems. Appl. Math. Comput. 255(1), 73–83 (2015)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Strekalovsky, A.S., Minarchenko, I.M.: A local search method for optimisation problem with D.C. inequality constraints. Appl. Math. Model. 1(58), 229–244 (2018)Google Scholar
  14. 14.
    Joki, K., Bagirov, A.M., Karmitsa, N., Mäkelä, M.M.: A proximal bundle method for nonsmooth DC optimization utilizing nonconvex cutting planes. J. Glob. Optim. 68(3), 501–535 (2017)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Gaudioso, M., Giallombardo, G., Miglionico, G., Bagirov, A.M.: Minimizing nonsmooth DC functions via successive DC piecewise-affine approximations. J. Glob. Optim. 71, 37–55 (2018)MathSciNetzbMATHGoogle Scholar
  16. 16.
    de Oliveira, W.: Proximal bundle methods for nonsmooth DC programming. J. Glob. Optim. (2019).  https://doi.org/10.1007/s10898-019-00755-4 Google Scholar
  17. 17.
    Le Thi, H.A., Tao, P.D.: DC programming in communication systems: challenging problems and methods. Vietnam J. Comput. Sci. 1(1), 15–28 (2014).  https://doi.org/10.1007/s40595-013-0010-5 Google Scholar
  18. 18.
    Hare, W., Sagastizábal, C.: Computing proximal points of nonconvex functions. Math. Program. 116(1–2), 221–258 (2009)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Hare, W., Sagastizábal, C.: A redistributed proximal bundle method for nonconvex optimization. SIAM J. Optim. 20(5), 2442–2473 (2010)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Hare, W., Sagastizábal, C., Solodov, M.: A proximal bundle method for nonconvex functions with inexact oracles. Comput. Optim. Appl. 63(1), 1–28 (2016)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Karmitsa, N., Gaudioso, M., Joki, K.: Diagonal bundle method with convex and concave updates for large-scale nonconvex and nonsmooth optimization. Optim. Methods Softw. (2017).  https://doi.org/10.1080/10556788.2017.1389941 zbMATHGoogle Scholar
  22. 22.
    Dao, M.N., Gwinner, J., Noll, D., Ovcharova, N.: Nonconvex bundle method with application to a delamination problem. Comput. Optim. Appl. 65(1), 173–203 (2016)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Fuduli, A., Gaudioso, M., Giallombardo, G.: Minimizing nonconvex nonsmooth functions via cutting planes and proximity control. SIAM J. Optim. 14(3), 743–756 (2004)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Fuduli, A., Gaudioso, M., Giallombardo, G.: A DC piecewise affine model and a bundling technique in nonconvex nonsmooth minimization. Optim. Methods Softw. 19, 89–102 (2004)MathSciNetzbMATHGoogle Scholar
  25. 25.
    Haarala, N., Miettinen, K., Mäkelä, M.M.: Globally convergent limited memory bundle method for large-scale nonsmooth optimization. Math. Program. 109(1), 181–205 (2007)MathSciNetzbMATHGoogle Scholar
  26. 26.
    Kiwiel, K.C.: A linearization algorithm for nonsmooth minimization. Math. Oper. Res. 10, 185–194 (1985)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Kiwiel, K.: Restricted step and Levenberg–Marquardt techniques in proximal bundle methods for nonconvex nondifferentiable optimization. SIAM J. Optim. 6(1), 227–249 (1996)MathSciNetzbMATHGoogle Scholar
  28. 28.
    Lukšan, L., Vlček, J.: A bundle-newton method for nonsmooth unconstrained minimization. Math. Program. 83(1–3), 373–391 (1998)MathSciNetzbMATHGoogle Scholar
  29. 29.
    Mäkelä, M.M., Neittaanmäki, P.: Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control. World Scientific Publishing Co., River Edge (1992)zbMATHGoogle Scholar
  30. 30.
    Schramm, H., Zowe, J.: A version of the bundle idea for minimizing a nonsmooth function: conceptual idea, convergence analysis, numerical results. SIAM J. Optim. 2, 121–152 (1992)MathSciNetzbMATHGoogle Scholar
  31. 31.
    Vlček, J., Lukšan, L.: Globally convergent variable metric method for nonconvex nondifferentiable unconstrained optimization. J. Optim. Theory Appl. 111, 407–430 (2001)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Mifflin, R.: A modification and extension of Lemaréchal’s algorithm for nonsmooth optimization. Math. Program. Study 17(1), 77–90 (1982)MathSciNetzbMATHGoogle Scholar
  33. 33.
    Lemaréchal, C., Nemirovskii, A., Nesterov, Y.: New variants of bundle methods. Math. Program. 69(1), 111–147 (1995)MathSciNetzbMATHGoogle Scholar
  34. 34.
    van Ackooij, W., de Oliveira, W.: Level bundle methods for constrained convex optimization with various oracles. Comput. Optim. Appl. 57(3), 555–597 (2014)MathSciNetzbMATHGoogle Scholar
  35. 35.
    Noll, D.: Bundle method for non-convex minimization with inexact subgradients and function values. In: Bailey, D.H., Bauschke, H.H., Borwein, P., Garvan, F., Théra, M., Vanderwerff, J., Wolkowicz, H. (eds.) Computational and Analytical Mathematics. Springer Proceedings in Mathematics and Statistics, vol. 50, pp. 555–592. Springer, Berlin (2013)Google Scholar
  36. 36.
    Luc, D.T., Van Ngai, H., Théra, M.: On \(\epsilon \)-monotonicity and \(\epsilon \)-convexity. In: Ioffe, A., Reich, S., Shafrir, I. (eds.) Calculus of Variations and Differential Equations. Research Notes in Mathematics, vol. 410, pp. 82–100. Chapman and Hall/CRC, London (2000)Google Scholar
  37. 37.
    Daniildis, A., Georgiev, P.: Approximate convexity and submonotonicity. J. Math. Anal. Appl. 291, 117–144 (2004)MathSciNetGoogle Scholar
  38. 38.
    Apkarian, P., Noll, D., Prot, O.: A proximity control algorithm to minimize nonsmooth and nonconvex semi-infinite maximum eigenvalue functions. J. Convex Anal. 16(3–4), 641–666 (2009)MathSciNetzbMATHGoogle Scholar
  39. 39.
    Clarke, F.: Optimisation and nonsmooth analysis. Classics in applied mathematics. Soc. Ind. Appl. Math. (1987).  https://doi.org/10.1137/1.9781611971309 Google Scholar
  40. 40.
    Mordukhovich, B.S.: Variational Analysis and Generalized Differentiation I. Basic Theory. Grundlehren der mathematischen Wissenschaften, vol. 330. Springer, Heidelberg (2006)Google Scholar
  41. 41.
    Borwein, J.M., Preiss, D.: A smooth variational principle with applications to subdifferentiability and differentiability of convex functions. Trans. Am. Math. Soc. 303, 517–527 (1987)MathSciNetzbMATHGoogle Scholar
  42. 42.
    Borwein, J.M., Zhu, Q.J.: Techniques of Variational Analysis, CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC, vol. 20. Springer, New York (2005)Google Scholar
  43. 43.
    Kruger, A.Y.: On Fréchet subdifferentials. J. Math. Sci. 116(3), 3325–3358 (2003)MathSciNetzbMATHGoogle Scholar
  44. 44.
    Kelley, J.E.: The cutting-plane method for solving convex programs. J. Soc. Ind. Appl. Math. 8(4), 703–712 (1960)MathSciNetzbMATHGoogle Scholar
  45. 45.
    Frangioni, A.: Standard Bundle Methods: Untrusted Models and Duality. Technical Report del Dipartimento di Informatica, TR. University of Pisa, Pisa, IT (submitted)Google Scholar
  46. 46.
    de Oliveira, W.: Target radius methods for nonsmooth convex optimization. Oper. Res. Lett. 45(6), 659–664 (2017)MathSciNetzbMATHGoogle Scholar
  47. 47.
    Bello-Cruz, J.Y., de Oliveira, W.: Level bundle-like algorithms for convex optimization. J. Glob. Optim. 59(4), 787–809 (2014).  https://doi.org/10.1007/s10898-013-0096-4 MathSciNetzbMATHGoogle Scholar
  48. 48.
    Ferrier, C.: Bornes duales de problèmes d’optimisation polynomiaux. Ph.D. thesis, Université Paul Sabatier, Toulouse (1997)Google Scholar
  49. 49.
    Ferrier, C.: Computation of the distance to semi-algebraic sets. ESAIM Control Optim. Calc. Var. 5(1), 139–156 (2000)MathSciNetzbMATHGoogle Scholar
  50. 50.
    Beltran, F., de Oliveira, W., Finardi, E.C.: Application of scenario tree reduction via quadratic process to medium-term hydrothermal scheduling problem. IEEE Trans. Power Syst. 32(6), 4351–4361 (2017)Google Scholar
  51. 51.
    Trivedi, P.K., Zimmer, D.M.: Copula modeling: an introduction for practitioners. Found. Trends Econ. 1(1), 1–111 (2007).  https://doi.org/10.1561/0800000005 zbMATHGoogle Scholar
  52. 52.
    Nelsen, R.B.: An Introduction to Copulas. Springer Series in Statistics, 2nd edn. Springer, New York (2006)zbMATHGoogle Scholar
  53. 53.
    McNeil, A., Nešlehová, J.: Multivariate archimedian copulas, d-monotone functions and \(l_1\) norm symmetric distributions. Ann. Stat. 37, 3059–3097 (2009)zbMATHGoogle Scholar
  54. 54.
    van Ackooij, W., de Oliveira, W.: Convexity and optimization with copulæ structured probabilistic constraints. Optim. J. Math. Program. Oper. Res. 65(7), 1349–1376 (2016).  https://doi.org/10.1080/02331934.2016.1179302 zbMATHGoogle Scholar
  55. 55.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002).  https://doi.org/10.1007/s101070100263 MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.EDF R&DÉlectricité de FranceParisFrance
  2. 2.MINES ParisTech, PSL – Research University, CMA – Centre de Mathématiques AppliquéesSophia AntipolisFrance

Personalised recommendations