The subdifferential of measurable composite max integrands and smoothing approximation

  • James V. Burke
  • Xiaojun ChenEmail author
  • Hailin Sun
Full Length Paper Series B


The subdifferential calculus for the expectation of nonsmooth random integrands involves many fundamental and challenging problems in stochastic optimization. It is known that for Clarke regular integrands, the Clarke subdifferential of the expectation equals the expectation of their Clarke subdifferential. In particular, this holds for convex integrands. However, little is known about the calculation of Clarke subgradients for the expectation of non-regular integrands. The focus of this contribution is to approximate Clarke subgradients for the expectation of random integrands by smoothing methods applied to the integrand. A framework for how to proceed along this path is developed and then applied to a class of measurable composite max integrands. This class contains non-regular integrands from stochastic complementarity problems as well as stochastic optimization problems arising in statistical learning.


Stochastic optimization Clarke subgradient Smoothing Non-regular integrands 

Mathematics Subject Classification




We would like to thank Associate Editor and two referees for their helpful comments.


  1. 1.
    Ahn, M., Pang, J.S., Xin, J.: Difference-of-convex learning I: directional stationarity, optimality, and sparsity. SIAM J. Optim. 27, 1637–1665 (2017)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Aumann, R.J.: Integrals of set-valued functions. J. Math. Anal. Appl. 12, 1–12 (1965)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Blundell, R., Powell, J.L.: Censored regression quantiles with endogenous regression. J. Econom. 141, 65–83 (2007)CrossRefGoogle Scholar
  4. 4.
    Burke, J.V.: Second order necessary and sufficient conditions for convex composite NDO. Math. Program. 38, 287–302 (1987)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Burke, J.V., Hoheisel, T.: Epi-convergent smoothing with applications to convex composite functions. SIAM J. Optim. 23, 1457–1479 (2013) MathSciNetCrossRefGoogle Scholar
  6. 6.
    Burke, J.V., Hoheisel, T., Kanzow, C.: Gradient consistency for integral-convolution smoothing functions. Set-Valued Var. Anal. 21, 359–376 (2013)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Burke, J.V., Luke, D.R.: Variational analysis applied to the problem of optimal phase retieval. SIAM J. Control Optim. 42, 576–595 (2003)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Chen, C., Mangasarian, O.L.: A class of smoothing functions for nonlinear and mixed complementarity problems. Comput. Optim. Appl. 5, 97–138 (1996)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Chen, X.: Smoothing methods for nonsmooth, nonconvex minimization. Math. Program. 134, 71–99 (2012)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Chen, X., Fukushima, M.: A smoothing method for a mathematical program with P-matrix linear complementarity constraints. Comput. Optim. Appl. 27, 223–246 (2004)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Chen, X., Fukushima, M.: Expected residual minimization method for stochastic linear complementarity problems. Math. Oper. Res. 30, 1022–1038 (2005)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Chen, X., Qi, L., Sun, D.: Global and superlinear convergence of the smoothing Newton method and its application to general box constrained variational inequalities. Math. Comput. 67, 519–540 (1998)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Chen, X., Wets, R., Zhang, Y.: Stochastic variational inequalities: residual minimization smoothing sample average approximations. SIAM J. Optim. 22, 649–673 (2012)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Clarke, F.H.: Optimization and Nonsmooth Analysis, Volume 5 of Classics in Applied Mathematics. SIAM, Philadelphia (1990)CrossRefGoogle Scholar
  15. 15.
    Clarke, F.H., Ledyaev, Y.S., Stern, R.J., Wolenski, P.R.: Nonsmooth Analysis and Control Theory. Springer, New York (1998)zbMATHGoogle Scholar
  16. 16.
    Dunford, N., Schwartz, J.T.: Linear Operators, Part1: General Theory. Wiley, Hoboken (1988)Google Scholar
  17. 17.
    Folland, G.B.: Real Analysis, 2nd edn. Wiley, New York (1999)zbMATHGoogle Scholar
  18. 18.
    Hildenbrand, W.: Core and Equilibria of a Large Economy. Princeton University Press, Princeton (1974)zbMATHGoogle Scholar
  19. 19.
    Lyapunov, A.: Sur les fonctions-vecteur complétement additives. Bull. Acad. Sci. USSR Ser. Math. 4, 465–478 (1940)Google Scholar
  20. 20.
    Mordukhvich, B.: Variational Analysis and Generalized Differentiation II. Springer, Berlin (2006)CrossRefGoogle Scholar
  21. 21.
    Nemirovski, A., Juditsky, A., Lan, G., Shapiro, A.: Robust stochastic approximation approach to stochastic programming. SIAM J. Optim. 19, 1574–1906 (2009)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Ralph, D., Xu, H.: Convergence of stationary points of sample average two stage stochastic programs: a generalized equation approach. Math. Oper. Res. 36, 568–592 (2011)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Richter, H.: Verallgemeinerung eines in der statistik benötigten satzes der masstheorie. Math. Ann. 150, 85–90 (1963)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Rockafellar, R.T., Uryasev, S.: Optimization of conditional value-at-risk. J. Risk 2, 21–41 (2000)CrossRefGoogle Scholar
  25. 25.
    Rockafellar, R.T.: Integral functionals, normal integrands and measurable selections. In: Nonlinear Operators in the Calculus of Variations, Volume 543 in Lecture Notes in Mathematics, pp. 157–207. Springer, New York (1976)Google Scholar
  26. 26.
    Rockafellar, R.T., Wets, R.: On the interchange of subdifferentiation and conditional expectation for convex functions. Stochastics 7, 173–182 (1982)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Rockafellar, R.T., Wets, R.: Variational Analysis. Springer, New York (1998)CrossRefGoogle Scholar
  28. 28.
    Rudin, W.: Principles of Mathematical Analysis, 3rd edn. McGraw-Hill, San Francisco (1976)zbMATHGoogle Scholar
  29. 29.
    Shapiro, A., Dentcheva, D., Ruszczynski, A.: Lectures on Stochastic Programming: Modeling and Theory. SIAM, Philadelphia (2009)CrossRefGoogle Scholar
  30. 30.
    Tardella, F.: A new proof of the Lyapunov convexity theorem. SIAM J. Control Optim. 28, 478–481 (1990)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Wets, R.: Stochastic programming. In: Nemhauser, G.L., et al. (eds.) Handbooks in OR & MS, vol. 1, pp. 573–629 (1989)Google Scholar
  32. 32.
    Xu, H.: An implicit programming approach for a class of stochastic mathematical programs with complementarity constraints. SIAM J. Optim. 16, 670–696 (2006)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Xu, H., Ye, J.J.: Necessary optimality conditions for two-stage stochastic programs with equilibaium constraints. SIAM J. Optim. 20, 1685–1715 (2010)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Xu, H., Zhang, D.: Smooth sample average approximation of stationary points in nonsmooth stochastic optimization and applications. Math. Program. 119, 371–401 (2009)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society 2019

Authors and Affiliations

  1. 1.Department of Applied MathematicsUniversity of WashingtonSeattleUSA
  2. 2.Department of Applied MathematicsHong Kong Polytechnic UniversityHong KongChina
  3. 3.Jiangsu Key Lab for NSLSCS, School of Mathematical SciencesNanjing Normal UniversityNanjingChina

Personalised recommendations