Minimizing buffered probability of exceedance by progressive hedging

  • R. Tyrrell RockafellarEmail author
  • Stan Uryasev
Full Length Paper Series B


Stochastic programming problems have for a long time been posed in terms of minimizing the expected value of a random variable influenced by decision variables, but alternative objectives can also be considered, such as minimizing a measure of risk. Here something different is introduced: minimizing the buffered probability of exceedance for a specified loss threshold. The buffered version of the traditional concept of probability of exceedance has recently been developed with many attractive properties that are conducive to successful optimization, in contrast to the usual concept, which is often posed simply as the probability of failure. The main contribution here is to demonstrate that in minimizing buffered probability of exceedance the underlying convexities in a stochastic programming problem can be maintained and the progressive hedging algorithm can be employed to compute a solution.


Convex stochastic programming problems Probability of failure Probability of exceedance Buffered probability of failure Buffered probability of exceedance Quantiles Superquantiles Conditional value-at-risk Progressive hedging algorithm 

Mathematics Subject Classification




This research was sponsored for both authors by DARPA EQUiPS Grant SNL 014150709. The authors are grateful also to Dr. Viktor Kuzmeno for help with conducting the numerical case study.


  1. 1.
    Davis, R.A., Uryasev, S.: Analysis of tropical storm damage using buffered probability of exceedance. Nat. Hazards (2016). CrossRefGoogle Scholar
  2. 2.
    Kouri, D.P., Shapiro, A.: Optimization of PDEs with uncertain inputs. In: Antil, H., Kouri, D., Lacasse, M.D., Ridzal, D. (eds.) Frontiers in PDE-Constrained Optimization. The IMA Volumes in Mathematics and Its Applications, vol. 163. Springer, New York, NY (2018)Google Scholar
  3. 3.
    Kouri, D.P.: Higher-moment buffered probability. Optm. Lett. 13, 657–672 (2019) MathSciNetCrossRefGoogle Scholar
  4. 4.
    Mafusalov, A., Uryasev, S.: Buffered probability of exceedance: mathematical properties and optimization. SIAM. J. Optim. 28, 1077–1103 (2018) MathSciNetCrossRefGoogle Scholar
  5. 5.
    Mafusalov, A., Shapiro, A., Uryasev, S.: Estimation and asymptotics for buffered probability of exceedance. Eur. J. Oper. Res. 270, 826–836 (2018)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Norton, M., Uryasev, S.: Maximization of AUC and Buffered AUC in binary classification. Math. Program. (2018). CrossRefzbMATHGoogle Scholar
  7. 7.
    Norton, M., Mafusalov, A., Uryasev, S.: Soft margin support vector classification as buffered probability minimization. J. Mach. Learn. Res. 18, 1–43 (2017)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Norton, M., Mafusalov, A., Uryasev, S.: Cardinality of upper average and application to network optimization. SIAM J. Optim. 28(2), 1726–1750 (2018) MathSciNetCrossRefGoogle Scholar
  9. 9.
    Norton, M., Khokhlov, V., Uryasev, S.: Calculating CVaR and bPOE for common probability distributions with application to portfolio optimization and density estimation. Ann. Oper. Res. arXiv:1811.11301 (submitted for publication)
  10. 10.
    Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)CrossRefGoogle Scholar
  11. 11.
    Rockafellar, R.T.: Solving stochastic programming problems with risk measures by progessive hedging. Set-valued Variational Anal. 26, 759–768 (2017)CrossRefGoogle Scholar
  12. 12.
    Rockafellar, R.T., Royset, J.O.: On buffered failure probability in design and optimization of structures. J. Reliab. Eng. Syst. Saf. 95, 499–510 (2010)CrossRefGoogle Scholar
  13. 13.
    Rockafellar, R.T., Royset, J.O.: Superquantiles and their applications to risk, random variables, and regression. In: Tutorials in Operations Research, pp. 151–167. INFORMS (2013) Google Scholar
  14. 14.
    Rockafellar, R.T., Royset, J.O., Miranda, S.J.: Superquantile regression with applications to buffered reliability, uncertainty quantification and conditional value-at-risk. Eur. J. Oper. Res. 234, 140–154 (2014)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Rockafellar, R.T., Royset, J.O.: Engineering decisions under risk averseness. J. Risk Uncertain. Eng. Syst. Part A Civ. Eng. 1(2), 04015005 (2015). CrossRefGoogle Scholar
  16. 16.
    Rockafellar, R.T., Royset, J.O.: Risk measures in engineering design under uncertainty. In: Proceedings of the 12th International Conference on Applications of Statistics and Probability in Civil Engineering (ICASP), Vancouver (2015).
  17. 17.
    Rockafellar, R.T., Royset, J.O., Harajli, M.M.: Importance sampling in the evaluation and optimization of buffered probability of failure. In: Proceedings of the 12th International Conference on Applications of Statistics and Probability in Civil Engineering (ICASP), Vancouver (2015).
  18. 18.
    Rockafellar, R.T., Royset, J.O.: Measures of residual risk with connections to regression, risk tracking, surrogate models and ambiguity. SIAM J. Optim. 25, 1179–1208 (2015)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Rockafellar, R.T., Royset, J.O.: Superquantile/CVaR risk measures: second-order theory. Ann. Oper. Res. 262, 3–29 (2018)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Rockafellar, R.T., Royset, J.O.: Random variables, monotone relations and convex analysis. Math. Program. B 128, 297–331 (2014)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Rockafellar, R.T., Sun, J.: Soving monotone stochastic variational inequalities and complementarity problems by progressive hedging. Math. Program. B 174, 453–471 (2018)CrossRefGoogle Scholar
  22. 22.
    Rockafellar, R.T., Sun, J.: Solving Lagrangian variational inequalities with applications to stochastic programming. Math. Program. (2020).
  23. 23.
    Rockafellar, R.T., Uryasev, S.: Optimization of conditional value-at-risk. J. Risk 2, 21–43 (2000)CrossRefGoogle Scholar
  24. 24.
    Rockafellar, R.T., Uryasev, S.: Conditional value-at-risk for general loss distributions. J. Bank. Finance 26, 1443–1471 (2002)CrossRefGoogle Scholar
  25. 25.
    Rockafellar, R.T., Uryasev, S.: The fundamental risk quadrangle in risk management, optimization and statistical esimation. Surv. Oper. Res. Manag. Sci. 18, 33–53 (2013)MathSciNetGoogle Scholar
  26. 26.
    Rockafellar, R.T., Uryasev, S., Zabarankin, M.: Generalized deviations in risk analysis. Finance Stoch. 10, 51–74 (2006)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Rockafellar, R.T., Wets, R.J.-B.: Nonanticipativity and \({\cal{L}}^1\)-martingales in stochastic optimization problems. Stoch. Syst. Model. Identif. Optim. Math. Programm. Study 6, 170–187 (1976)CrossRefGoogle Scholar
  28. 28.
    Rockafellar, R.T., Wets, R.J.-B.: Scenarios and policy aggregation in optimization under uncertainty. Math. Oper. Res. 16, 119–147 (1991)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Rockafellar, R.T., Wets, R.J.-B.: Stochastic variational inequalities: single-stage to multistage. Math. Program. B 165, 291–330 (2017)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Shang, D., Kuzmenko, V., Uryasev, S.: Cash flow matching with risks controlled by buffered probability of exceedance and conditional value-at-risk. Ann. Oper. Res. 260, 501–514 (2016)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Uryasev, S.: Buffered probability of exceedance and buffered service level: definitions and properties, Research Report 2014-3. ISE Department, University of Florida (2014)Google Scholar
  32. 32.
    Watson, J.-P., Woodruff, D.L.: Progressive hedging innovations for a class of stochastic mixed-integer resource allocation problems. Comput. Manag. Sci. 8, 355–370 (2010)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Zhitlukhin, M.: Monotone Sharpe ratios and related measures of investment performance (2018). arXiv:1809.10193

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society 2020

Authors and Affiliations

  1. 1.Department of MathematicsUniversity of WashingtonSeattleUSA
  2. 2.Department of Systems and Industrial EngineeringUniversity of FloridaGainesvilleUSA

Personalised recommendations