Computational Optimization and Applications

, Volume 57, Issue 3, pp 555–597 | Cite as

Level bundle methods for constrained convex optimization with various oracles

  • Wim van Ackooij
  • Welington de Oliveira


We propose restricted memory level bundle methods for minimizing constrained convex nonsmooth optimization problems whose objective and constraint functions are known through oracles (black-boxes) that might provide inexact information. Our approach is general and covers many instances of inexact oracles, such as upper, lower and on-demand accuracy oracles. We show that the proposed level bundle methods are convergent as long as the memory is restricted to at least four well chosen linearizations: two linearizations for the objective function, and two linearizations for the constraints. The proposed methods are particularly suitable for both joint chance-constrained problems and two-stage stochastic programs with risk measure constraints. The approach is assessed on realistic joint constrained energy problems, arising when dealing with robust cascaded-reservoir management.


Nonsmooth optimization Stochastic optimization Level bundle method Chance constrained programming 



The authors are grateful to the reviewer for his suggestions of improvement.


  1. 1.
    Apkarian, P., Noll, D., Rondepierre, A.: Mixed H 2/H control via nonsmooth optimization. SIAM J. Control Optim. 47(3), 1516–1546 (2008) CrossRefMATHMathSciNetGoogle Scholar
  2. 2.
    de Oliveira, W., Sagastizábal, C.: Level bundle methods for oracles with on demand accuracy (2012). Available at
  3. 3.
    de Oliveira, W., Sagastizábal, C., Scheimberg, S.: Inexact bundle methods for two-stage stochastic programming. SIAM J. Optim. 21(2), 517–544 (2011). doi: 10.1137/100808289 CrossRefMATHMathSciNetGoogle Scholar
  4. 4.
    de Oliveira, W., Sagastizábal, C., Lemaréchal, C.: Bundle methods in depth: a unified analysis for inexact oracles (2013). Available at
  5. 5.
    Dentcheva, D., Martinez, G.: Two-stage stochastic optimization problems with stochastic ordering constraints on the recourse. Eur. J. Oper. Res. 219(1), 1–8 (2012) CrossRefMATHMathSciNetGoogle Scholar
  6. 6.
    Fábián, C.: Bundle-type methods for inexact data. In: Csendes, T., Rapcsák, T. (eds.) Proceedings of the XXIV Hungarian Operations Researc Conference, Veszprém, 1999, vol. 8, pp. 35–55 (2000) Google Scholar
  7. 7.
    Fábián, C.: Computational aspects of risk-averse optimisation in two-stage stochastic models. Tech. rep, Institute of Informatics, Kecskemét College, Hungary (2013). Optimization Online report
  8. 8.
    Bello-Cruz, J., de Oliveria, W.: Level bundle-like algorithms for convex optimization. J. Glob. Optim. (2013, to appear). doi: 10.1007/s10898-013-0096-4
  9. 9.
    Fábián, C., Szőke, Z.: Solving two-stage stochastic programming problems with level decomposition. Comput. Manag. Sci. 4, 313–353 (2007) CrossRefMATHMathSciNetGoogle Scholar
  10. 10.
    Genz, A.: Numerical computation of multivariate normal probabilities. J. Comput. Graph. Stat. 1, 141–149 (1992) Google Scholar
  11. 11.
    Genz, A., Bretz, F.: Computation of Multivariate Normal and t Probabilities. Lecture Notes in Statistics, vol. 195. Springer, Dordrecht (2009) CrossRefMATHGoogle Scholar
  12. 12.
    Henrion, R., Römisch, W.: Lipschitz and differentiability properties of quasi-concave and singular normal distribution functions. Ann. Oper. Res. 177, 115–125 (2010) CrossRefMATHMathSciNetGoogle Scholar
  13. 13.
    Hintermüller, M.: A proximal bundle method based on approximate subgradients. Comput. Optim. Appl. 20, 245–266 (2001). doi: 10.1023/A:1011259017643 CrossRefMATHMathSciNetGoogle Scholar
  14. 14.
    Hiriart-Urruty, J., Lemaréchal, C.: Convex Analysis and Minimization Algorithms II, 2nd edn. Grundlehren der mathematischen Wissenschaften, vol. 306. Springer, Berlin (1996) Google Scholar
  15. 15.
    Karas, E., Ribeiro, A., Sagastizábal, C., Solodov, M.: A bundle-filter method for nonsmooth convex constrained optimization. Math. Program. 116(1), 297–320 (2008). doi: 10.1007/s10107-007-0123-7 Google Scholar
  16. 16.
    Kiwiel, K.: Proximal level bundle methods for convex nondifferentiable optimization, saddle-point problems and variational inequalities. Math. Program. 69(1), 89–109 (1995) MATHMathSciNetGoogle Scholar
  17. 17.
    Kiwiel, K.: A proximal bundle method with approximate subgradient linearizations. SIAM J. Optim. 16(4), 1007–1023 (2006) CrossRefMATHMathSciNetGoogle Scholar
  18. 18.
    Kiwiel, K.: A method of centers with approximate subgradient linearizations for nonsmooth convex optimization. SIAM J. Optim. 18(4), 1467–1489 (2008) CrossRefMathSciNetGoogle Scholar
  19. 19.
    Kiwiel, K.: An inexact bundle approach to cutting stock problems. INFORMS J. Comput. 22, 131–143 (2010) CrossRefMATHMathSciNetGoogle Scholar
  20. 20.
    Kiwiel, K., Lemaréchal, C.: An inexact bundle variant suited to column generation. Math. Program. 118(1), 177–206 (2009) CrossRefMATHMathSciNetGoogle Scholar
  21. 21.
    Lan, G., Nemirovski, A., Shapiro, A.: Validation analysis of mirror descent stochastic approximation method. Math. Program. 134(2), 425–458 (2012). doi: 10.1007/s10107-011-0442-6 CrossRefMATHMathSciNetGoogle Scholar
  22. 22.
    Lemaréchal, C., Nemirovskii, A., Nesterov, Y.: New variants of bundle methods. Math. Program. 69(1), 111–147 (1995) CrossRefMATHGoogle Scholar
  23. 23.
    Mayer, J.: On the numerical solution of jointly chance constrained problems. Chapter 12 in [38], 1st edn. Springer (2000) Google Scholar
  24. 24.
    Prékopa, A.: On probabilistic constrained programming. Proc. Princet. Symp. Math. Prog. 28, 113–138 (1970) Google Scholar
  25. 25.
    Prékopa, A.: Stochastic Programming. Kluwer, Dordrecht (1995) CrossRefGoogle Scholar
  26. 26.
    Prékopa, A.: Probabilistic programming. In [31] (Chap. 5). Elsevier, Amsterdam (2003) Google Scholar
  27. 27.
    Prékopa, A., Szántai, T.: Flood control reservoir system design using stochastic programming. Math. Program. Stud. 9, 138–151 (1978) CrossRefGoogle Scholar
  28. 28.
    Rockafellar, R.: Convex Analysis, 1st edn. Princeton University Press, Princeton (1970) MATHGoogle Scholar
  29. 29.
    Rockafellar, R., Uryas’ev, S.: Optimization of conditional value-at-risk. J. Risk 2, 21–42 (2000) Google Scholar
  30. 30.
    Rockafellar, R., Uryas’ev, S.: Conditional value at risk: optimization approach. In: Stochastic optimization: algorithms and applications, pp. 411–435 (2001) Google Scholar
  31. 31.
    Ruszczyński, A., Shapiro, A.: Stochastic Programming. Handbooks in Operations Research and Management Science, vol. 10. Elsevier, Amsterdam (2003) MATHGoogle Scholar
  32. 32.
    Sagastizábal, C., Solodov, M.: An infeasible bundle method for nonsmooth convex constrained optimization without a penalty function or a filter. SIAM J. Optim. 16(1), 146–169 (2005). CrossRefMATHMathSciNetGoogle Scholar
  33. 33.
    Shapiro, A., Dentcheva, D., Ruszczyński, A.: Lectures on Stochastic Programming. Modeling and Theory. MPS-SIAM Series on Optimization, vol. 9. SIAM and MPS, Philadelphia (2009) CrossRefMATHGoogle Scholar
  34. 34.
    Szántai, T.: A computer code for solution of probabilistic-constrained stochastic programming problems. In: Ermoliev, Y., Wets, R.J.-B. (eds.) Numerical Techniques for Stochastic Optimization, pp. 229–235 (1988) CrossRefGoogle Scholar
  35. 35.
    Uryas’ev, S.: A differentation formula for integrals over sets given by inclusion. Numer. Funct. Anal. Optim. 10(7&8), 827–841 (1989) CrossRefMATHMathSciNetGoogle Scholar
  36. 36.
    Uryas’ev, S.: Derivatives of probability functions and some applications. Ann. Oper. Res. 56, 287–311 (1995) CrossRefMathSciNetGoogle Scholar
  37. 37.
    Uryas’ev, S.: Introduction to the theory of probabilistic functions and percentiles (value-at-risk). Chap. 1 in [38]. Kluwer Academic Publishers, Dordrecht (2000) Google Scholar
  38. 38.
    Uryas’ev, S. (ed.): Probabilistic Constrained Optimization: Methodology and Applications. Kluwer Academic Publishers, Dordrecht (2000) Google Scholar
  39. 39.
    van Ackooij, W.: Decomposition approaches for block-structured chance-constrained programs with application to hydro-thermal unit-commitment. Submitted; Preprint CR-2012-08 (2012, submitted).
  40. 40.
    van Ackooij, W., Sagastizábal, C.: Constrained bundle methods for upper inexact oracles with application to joint chance constrained energy problems pp. 1–25 (2012, submitted). Preprint available at.
  41. 41.
    van Ackooij, W., Zorgati, R.: Estimating the probabilistic contents of Gaussian rectangles faster in joint chance constrained programming for hydro reservoir management. In: EngOpt2012, 3rd International Conference on Engineering Optimization (2012) Google Scholar
  42. 42.
    van Ackooij, W., Henrion, R., Möller, A., Zorgati, R.: On probabilistic constraints induced by rectangular sets and multivariate normal distributions. Math. Methods Oper. Res. 71(3), 535–549 (2010) CrossRefMATHMathSciNetGoogle Scholar
  43. 43.
    van Ackooij, W., Henrion, R., Möller, A., Zorgati, R.: Joint chance constrained programming for hydro reservoir management. Optim. Eng. (2011, to appear).
  44. 44.
    Veinott, A.: The supporting hyperplane method for unimodal programming. Oper. Res. 15, 147–152 (1967) CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.OSIRISEDF R&DClamartFrance
  2. 2.Ecole Centrale ParisChâtenay-MalabryFrance
  3. 3.Instituto Nacional de Matemática Pura e Aplicada—IMPARio de JaneiroBrazil

Personalised recommendations