Advertisement

Risk and complexity in scenario optimization

  • S. Garatti
  • M. C. CampiEmail author
Full Length Paper Series B
First Online:
  • 146 Downloads

Abstract

Scenario optimization is a broad methodology to perform optimization based on empirical knowledge. One collects previous cases, called “scenarios”, for the set-up in which optimization is being performed, and makes a decision that is optimal for the cases that have been collected. For convex optimization, a solid theory has been developed that provides guarantees of performance, and constraint satisfaction, of the scenario solution. In this paper, we open a new direction of investigation: the risk that a performance is not achieved, or that constraints are violated, is studied jointly with the complexity (as precisely defined in the paper) of the solution. It is shown that the joint probability distribution of risk and complexity is concentrated in such a way that the complexity carries fundamental information to tightly judge the risk. This result is obtained without requiring extra knowledge on the underlying optimization problem than that carried by the scenarios; in particular, no extra knowledge on the distribution by which scenarios are generated is assumed, so that the result is broadly applicable. This deep-seated result unveils a fundamental and general structure of data-driven optimization and suggests practical approaches for risk assessment.

Keywords

Data-driven optimization Scenario approach Stochastic optimization Probabilistic constraints 

Mathematics Subject Classification

90C15 90C25 62C12 
This is a preview of subscription content, log in to check access.

Notes

Supplementary material

Open image in new window

References

  1. 1.
    Alamo, T., Tempo, R., Camacho, E.: A randomized strategy for probabilistic solutions of uncertain feasibility and optimization problems. IEEE Trans. Autom. Control 54(11), 2545–2559 (2009)zbMATHCrossRefGoogle Scholar
  2. 2.
    Baronio, F., Baronio, M., Campi, M., Caré, A., Garatti, S.: Ventricular defribillation: classification with GEM and a roadmap for future investigations. In: Proceedings of the 56th IEEE Conference on Decision and Control. Melbourne, Australia (2017)Google Scholar
  3. 3.
    Bayraksan, G., Morton, D.: Assessing solution quality in stochastic programs. Math. Program. 108, 495–514 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Bayraksan, G., Morton, D.: Assessing solution quality in stochastic programs via sampling. In: Oskoorouchi, M. (ed.) Tutorials in Operations Research, pp. 102–122. Informs (2009)Google Scholar
  5. 5.
    Ben-Tal, A., Nemirovski, A.: On safe tractable approximations of chance-constrained linear matrix inequalities. Math. Oper. Res. 34(1), 1–25 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Bertsimas, D., Gupta, V., Kallus, N.: Data-driven robust optimization. Math. Program. 167, 235–292 (2018)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Bertsimas, D., Gupta, V., Kallus, N.: Robust sample average approximation. Math. Program. 171, 217–282 (2018)MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Bertsimas, D., Thiele, A.: Robust and data-driven optimization: modern decision-making under uncertainty. In: Tutorials on Operations Research. INFORMS (2006)Google Scholar
  9. 9.
    Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)zbMATHCrossRefGoogle Scholar
  10. 10.
    Calafiore, G., Campi, M.: Uncertain convex programs: randomized solutions and confidence levels. Math. Program. 102(1), 25–46 (2005).  https://doi.org/10.1007/s10107-003-0499-y MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Calafiore, G., Campi, M.: The scenario approach to robust control design. IEEE Trans. Autom. Control 51(5), 742–753 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Calafiore, G., El Ghaoui, L.: On distributionally robust chance-constrained linear programs. Math. Program. 130(1), 1–22 (2006)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Carè, A., Ramponi, F.A., Campi, M.C.: A new classification algorithm with guaranteed sensitivity and specificity for medical applications. IEEE Control Syst. Lett. 2, 393–398 (2018)CrossRefGoogle Scholar
  14. 14.
    Campi, M.: Classification with guaranteed probability of error. Mach. Learn. 80, 63–84 (2010)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Campi, M., Calafiore, G., Garatti, S.: Interval predictor models: identification and reliability. Automatica 45(2), 382–392 (2009).  https://doi.org/10.1016/j.automatica.2008.09.004 MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Campi, M., Carè, A.: Random convex programs with \(l_1\)-regularization: sparsity and generalization. SIAM J. Control Optim. 51(5), 3532–3557 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    Campi, M., Garatti, S.: The exact feasibility of randomized solutions of uncertain convex programs. SIAM J. Optim. 19(3), 1211–1230 (2008).  https://doi.org/10.1137/07069821X MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Campi, M., Garatti, S.: A sampling-and-discarding approach to chance-constrained optimization: feasibility and optimality. J. Optim. Theory Appl. 148(2), 257–280 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Campi, M., Garatti, S.: Wait-and-judge scenario optimization. Math. Program. 167(1), 155–189 (2018)MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Campi, M., Garatti, S., Prandini, M.: The scenario approach for systems and control design. Annu. Rev. Control 33(2), 149–157 (2009).  https://doi.org/10.1016/j.arcontrol.2009.07.001 CrossRefGoogle Scholar
  21. 21.
    Carè, A., Garatti, S., Campi, M.: FAST—fast algorithm for the scenario technique. Oper. Res. 62(3), 662–671 (2014)MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Carè, A., Garatti, S., Campi, M.: Scenario min-max optimization and the risk of empirical costs. SIAM J. Optim. 25(4), 2061–2080 (2015)MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Crespo, L., Giesy, D., Kenny, S.: Interval predictor models with a formal characterization of uncertainty and reliability. In: Proceedings of the 53rd IEEE Conference on Decision and Control (CDC), pp. 5991–5996. Los Angeles, CA, USA (2014)Google Scholar
  24. 24.
    Crespo, L., Kenny, S., Giesy, D.: Random predictor models for rigorous uncertainty quantification. Int. J. Uncertain. Quantif. 5(5), 469–489 (2015)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Crespo, L., Kenny, S., Giesy, D., Norman, R., Blattnig, S.: Application of interval predictor models to space radiation shielding. In: Proceedings of the 18th AIAA Non-Deterministic Approaches Conference. San Diego, CA, USA (2016)Google Scholar
  26. 26.
    de Mello, T.H.: Variable-sample methods for stochastic optimization. ACM Trans. Model. Comput. Simul. 13, 108–133 (2003)zbMATHCrossRefGoogle Scholar
  27. 27.
    de Mello, T.H., Bayraksan, G.: Monte Carlo sampling-based methods for stochastic optimization. Surv. Oper. Res. Manag. Sci. 19(1), 56–85 (2014)MathSciNetGoogle Scholar
  28. 28.
    Delage, E., Ye, Y.: Distributionally robust optimization under moment uncertainty with application to data-driven problems. Oper. Res. 58(3), 596–612 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Dentcheva, D.: Optimization models with probabilistic constraints. In: Calafiore, G., Dabbene, F. (eds.) Probabilistic and Randomized Methods for Design Under Uncertainty. Springer, London (2006)Google Scholar
  30. 30.
    Erdogan, E., Iyengar, G.: Ambiguous chance constrained problems and robust optimization. Math. Program. 107, 37–61 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Esfahani, P., Sutter, T., Lygeros, J.: Performance bounds for the scenario approach and an extension to a class of non-convex programs. IEEE Trans. Autom. Control 60(1), 46–58 (2015)MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Esfahani, P.M., Kuhn, D.: Data-driven distributionally robust optimization using the wasserstein metric: performance guarantees and tractable reformulations. Math. Program. (2017).  https://doi.org/10.1007/s10107-017-1172-1 CrossRefzbMATHGoogle Scholar
  33. 33.
    Fabozzi, F., Kolm, P., Pachamanova, D., Focardi, S.: Robust Portfolio Optimization and Management. Wiley, Hoboken (2010)Google Scholar
  34. 34.
    Garatti, S., Campi, M.: Modulating robustness in control design: principles and algorithms. IEEE Control Syst. Mag. 33(2), 36–51 (2013).  https://doi.org/10.1109/MCS.2012.2234964 MathSciNetCrossRefGoogle Scholar
  35. 35.
    Goh, J., Sim, M.: Distributionally robust optimization and its tractable approximations. Oper. Res. 58, 902–917 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  36. 36.
    Grammatico, S., Zhang, X., Margellos, K., Goulart, P., Lygeros, J.: A scenario approach for non-convex control design. IEEE Trans. Autom. Control 61(2), 334–345 (2016)MathSciNetzbMATHGoogle Scholar
  37. 37.
    Gupta, V.: Near-optimal ambiguity sets for distributionally robust optimization. Manag. Sci. 65(9), 4242–4260 (2019)CrossRefGoogle Scholar
  38. 38.
    Hanasusanto, G., Roitch, V., Kuhn, D., Wiesemann, W.: A distributionally robust perspective on uncertainty quantification and chance constrained programming. Math. Program. 151(1), 35–62 (2015)MathSciNetzbMATHCrossRefGoogle Scholar
  39. 39.
    Hong, L., Hu, Z., Liu, G.: Monte Carlo methods for value-at-risk and conditional value-at-risk: a review. ACM Trans. Model. Comput. Simul. 24(4), 22:1–22:37 (2014)MathSciNetzbMATHGoogle Scholar
  40. 40.
    Hu, Z., Hong, L.: Kullback–Leiber divergence constrained distributionally robust optimization (2013). http://www.optimization-online.org/DB_HTML/2012/11/3677.html
  41. 41.
    Jiang, R., Guan, Y.: Data-driven chance constrained stochastic program. Math. Program. 158(1–2), 291–327 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  42. 42.
    Lam, H.: Recovering best statistical guarantees via the empirical divergence-based distributionally robust optimization. Oper. Res. 67(4), 1090–1105 (2019)MathSciNetGoogle Scholar
  43. 43.
    Linderoth, J., Shapiro, A., Wright, S.: The empirical behavior of sampling methods for stochastic programming. Ann. Oper. Res. 142, 215–241 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  44. 44.
    Luedtke, J., Ahmed, S.: A sample approximation approach for optimization with probabilistic constraints. SIAM J. Optim. 19, 674–699 (2008).  https://doi.org/10.1137/070702928 MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Luedtke, J., Ahmed, S., Nemhauser, G.: An integer programming approach for linear programs with probabilistic constraints. Math. Program. 122(2), 247–272 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  46. 46.
    Mak, W., Morton, D., Wood, R.: Monte Carlo bounding techniques for determing solution quality in stochastic programs. Oper. Res. Lett. 24, 47–56 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  47. 47.
    Margellos, K., Prandini, M., Lygeros, J.: On the connection between compression learning and scenario based single-stage and cascading optimization problems. IEEE Trans. Autom. Control 60(10), 2716–2721 (2015)MathSciNetzbMATHCrossRefGoogle Scholar
  48. 48.
    Nemirovski, A.: On safe tractable approximations of chance constraints. Eur. J. Oper. Res. 219, 707–718 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  49. 49.
    Nemirovski, A., Shapiro, A.: Convex approximations of chance constrained programs. SIAM J. Optim. 17(4), 969–996 (2006).  https://doi.org/10.1137/050622328 MathSciNetCrossRefzbMATHGoogle Scholar
  50. 50.
    Nemirovski, A., Shapiro, A.: Scenario approximations of chance constraints. In: Calafiore, G., Dabbene, F. (eds.) Probabilistic and Randomized Methods for Design Under Uncertainty. Springer, London (2006)Google Scholar
  51. 51.
    Pagnoncelli, B., Ahmed, S., Shapiro, A.: Sample average approximation method for chance constrained programming: theory and applications. J. Optim. Theory Appl. 142(2), 399–416 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  52. 52.
    Pagnoncelli, B., Reich, D., Campi, M.: Risk-return trade-off with the scenario approach in practice: a case study in portfolio selection. J. Optim. Theory Appl. 155(2), 707–722 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  53. 53.
    Pagnoncelli, B., Vanduffel, S.: A provisioning problem with stochastic payments. Eur. J. Oper. Res. 221(2), 445–453 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  54. 54.
    Pflug, G., Wozabal, D.: Ambiguity in portfolio selection. Quant. Finance 7, 435–442 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  55. 55.
    Schildbach, G., Fagiano, L., Frei, C., Morari, M.: The scenario approach for stochastic model predictive control with bounds on closed-loop constraint violations. Automatica 50(12), 3009–3018 (2014)MathSciNetzbMATHCrossRefGoogle Scholar
  56. 56.
    Schildbach, G., Fagiano, L., Morari, M.: Randomized solutions to convex programs with multiple chance constraints. SIAM J. Optim. 23(4), 2479–2501 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  57. 57.
    Shapiro, A.: Monte–Carlo sampling methods. In: Ruszczyński, A., Shapiro, A. (eds.) Stochastic Programming, volume 10 of Handbooks in Operations Research and Management Science. Elsevier, London (2003)Google Scholar
  58. 58.
    Shapiro, A., Dentcheva, D., Ruszczyński, A.: Lectures on Stochastic Programming: Modeling and Theory. MPS-SIAM, Philadelphia (2009)zbMATHCrossRefGoogle Scholar
  59. 59.
    Shiryaev, A.: Probability. Springer, New York (1996)zbMATHCrossRefGoogle Scholar
  60. 60.
    Thiele, A.: Robust stochastic programming with uncertain probabilities. IMA J. Manag. Math. 19(3), 289–321 (2008).  https://doi.org/10.1093/imaman/dpm011 MathSciNetCrossRefzbMATHGoogle Scholar
  61. 61.
    Van Parys, B., Esfahani, P., Kuhn, D.: From data to decisions: distributionally robust optimization is optimal. (2017). arxiv:1704.04118
  62. 62.
    Vayanos, P., Kuhn, D., Rustem, B.: A constraint sampling approach for multistage robust optimization. Automatica 48(3), 459–471 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  63. 63.
    Welsh, J., Kong, H.: Robust experiment design through randomisation with chance constraints. In: Proceedings of the 18th IFAC World Congress, Milan, Italy (2011)Google Scholar
  64. 64.
    Welsh, J., Rojas, C.: A scenario based approach to robust experiment design. In: Proceedings of the 15th IFAC Symposium on System Identification. Saint-Malo, France (2009)CrossRefGoogle Scholar
  65. 65.
    Wieseman, W., Kuhn, D., Sim, M.: Distributionally robust convex optimization. Oper. Res. 62, 1358–1376 (2014)MathSciNetzbMATHCrossRefGoogle Scholar
  66. 66.
    Wozabal, D.: A framework for optimization under ambiguity. Ann. Oper. Res. 193, 21–47 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  67. 67.
    Zhang, X., Grammatico, S., Schildbach, G., Goulart, P., Lygeros, J.: On the sample size of random convex programs with structured dependence on the uncertainty. Automatica 60, 182–188 (2015)MathSciNetzbMATHCrossRefGoogle Scholar
  68. 68.
    Zhou, Z., Cogill, R.: Reliable approximations of probability-constrained stochastic linear-quadratic control. Automatica 49(8), 2435–2439 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  69. 69.
    Zymler, S., Kuhn, D., Rustem, B.: Distributionally robust joint chance constraints with second-order moment information. Math. Program. 137(1–2), 167–198 (2013)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society 2019

Authors and Affiliations

  1. 1.Dipartimento di Elettronica, Informazione e BioingegneriaPolitecnico di MilanoMilanItaly
  2. 2.Dipartimento di Ingegneria dell’InformazioneUniversità di BresciaBresciaItaly

Personalised recommendations

Cite article

Buy options