# The subdifferential of measurable composite max integrands and smoothing approximation

- 63 Downloads

## Abstract

The subdifferential calculus for the expectation of nonsmooth random integrands involves many fundamental and challenging problems in stochastic optimization. It is known that for Clarke regular integrands, the Clarke subdifferential of the expectation equals the expectation of their Clarke subdifferential. In particular, this holds for convex integrands. However, little is known about the calculation of Clarke subgradients for the expectation of non-regular integrands. The focus of this contribution is to approximate Clarke subgradients for the expectation of random integrands by smoothing methods applied to the integrand. A framework for how to proceed along this path is developed and then applied to a class of *measurable composite max integrands*. This class contains non-regular integrands from stochastic complementarity problems as well as stochastic optimization problems arising in statistical learning.

## Keywords

Stochastic optimization Clarke subgradient Smoothing Non-regular integrands## Mathematics Subject Classification

90C15## Notes

### Acknowledgements

We would like to thank Associate Editor and two referees for their helpful comments.

## References

- 1.Ahn, M., Pang, J.S., Xin, J.: Difference-of-convex learning I: directional stationarity, optimality, and sparsity. SIAM J. Optim.
**27**, 1637–1665 (2017)MathSciNetCrossRefGoogle Scholar - 2.Aumann, R.J.: Integrals of set-valued functions. J. Math. Anal. Appl.
**12**, 1–12 (1965)MathSciNetCrossRefGoogle Scholar - 3.Blundell, R., Powell, J.L.: Censored regression quantiles with endogenous regression. J. Econom.
**141**, 65–83 (2007)CrossRefGoogle Scholar - 4.Burke, J.V.: Second order necessary and sufficient conditions for convex composite NDO. Math. Program.
**38**, 287–302 (1987)MathSciNetCrossRefGoogle Scholar - 5.Burke, J.V., Hoheisel, T.: Epi-convergent smoothing with applications to convex composite functions. SIAM J. Optim.
**23**, 1457–1479 (2013) MathSciNetCrossRefGoogle Scholar - 6.Burke, J.V., Hoheisel, T., Kanzow, C.: Gradient consistency for integral-convolution smoothing functions. Set-Valued Var. Anal.
**21**, 359–376 (2013)MathSciNetCrossRefGoogle Scholar - 7.Burke, J.V., Luke, D.R.: Variational analysis applied to the problem of optimal phase retieval. SIAM J. Control Optim.
**42**, 576–595 (2003)MathSciNetCrossRefGoogle Scholar - 8.Chen, C., Mangasarian, O.L.: A class of smoothing functions for nonlinear and mixed complementarity problems. Comput. Optim. Appl.
**5**, 97–138 (1996)MathSciNetCrossRefGoogle Scholar - 9.Chen, X.: Smoothing methods for nonsmooth, nonconvex minimization. Math. Program.
**134**, 71–99 (2012)MathSciNetCrossRefGoogle Scholar - 10.Chen, X., Fukushima, M.: A smoothing method for a mathematical program with P-matrix linear complementarity constraints. Comput. Optim. Appl.
**27**, 223–246 (2004)MathSciNetCrossRefGoogle Scholar - 11.Chen, X., Fukushima, M.: Expected residual minimization method for stochastic linear complementarity problems. Math. Oper. Res.
**30**, 1022–1038 (2005)MathSciNetCrossRefGoogle Scholar - 12.Chen, X., Qi, L., Sun, D.: Global and superlinear convergence of the smoothing Newton method and its application to general box constrained variational inequalities. Math. Comput.
**67**, 519–540 (1998)MathSciNetCrossRefGoogle Scholar - 13.Chen, X., Wets, R., Zhang, Y.: Stochastic variational inequalities: residual minimization smoothing sample average approximations. SIAM J. Optim.
**22**, 649–673 (2012)MathSciNetCrossRefGoogle Scholar - 14.Clarke, F.H.: Optimization and Nonsmooth Analysis, Volume 5 of Classics in Applied Mathematics. SIAM, Philadelphia (1990)CrossRefGoogle Scholar
- 15.Clarke, F.H., Ledyaev, Y.S., Stern, R.J., Wolenski, P.R.: Nonsmooth Analysis and Control Theory. Springer, New York (1998)zbMATHGoogle Scholar
- 16.Dunford, N., Schwartz, J.T.: Linear Operators, Part1: General Theory. Wiley, Hoboken (1988)Google Scholar
- 17.Folland, G.B.: Real Analysis, 2nd edn. Wiley, New York (1999)zbMATHGoogle Scholar
- 18.Hildenbrand, W.: Core and Equilibria of a Large Economy. Princeton University Press, Princeton (1974)zbMATHGoogle Scholar
- 19.Lyapunov, A.: Sur les fonctions-vecteur complétement additives. Bull. Acad. Sci. USSR Ser. Math.
**4**, 465–478 (1940)Google Scholar - 20.Mordukhvich, B.: Variational Analysis and Generalized Differentiation II. Springer, Berlin (2006)CrossRefGoogle Scholar
- 21.Nemirovski, A., Juditsky, A., Lan, G., Shapiro, A.: Robust stochastic approximation approach to stochastic programming. SIAM J. Optim.
**19**, 1574–1906 (2009)MathSciNetCrossRefGoogle Scholar - 22.Ralph, D., Xu, H.: Convergence of stationary points of sample average two stage stochastic programs: a generalized equation approach. Math. Oper. Res.
**36**, 568–592 (2011)MathSciNetCrossRefGoogle Scholar - 23.Richter, H.: Verallgemeinerung eines in der statistik benötigten satzes der masstheorie. Math. Ann.
**150**, 85–90 (1963)MathSciNetCrossRefGoogle Scholar - 24.Rockafellar, R.T., Uryasev, S.: Optimization of conditional value-at-risk. J. Risk
**2**, 21–41 (2000)CrossRefGoogle Scholar - 25.Rockafellar, R.T.: Integral functionals, normal integrands and measurable selections. In: Nonlinear Operators in the Calculus of Variations, Volume 543 in Lecture Notes in Mathematics, pp. 157–207. Springer, New York (1976)Google Scholar
- 26.Rockafellar, R.T., Wets, R.: On the interchange of subdifferentiation and conditional expectation for convex functions. Stochastics
**7**, 173–182 (1982)MathSciNetCrossRefGoogle Scholar - 27.Rockafellar, R.T., Wets, R.: Variational Analysis. Springer, New York (1998)CrossRefGoogle Scholar
- 28.Rudin, W.: Principles of Mathematical Analysis, 3rd edn. McGraw-Hill, San Francisco (1976)zbMATHGoogle Scholar
- 29.Shapiro, A., Dentcheva, D., Ruszczynski, A.: Lectures on Stochastic Programming: Modeling and Theory. SIAM, Philadelphia (2009)CrossRefGoogle Scholar
- 30.Tardella, F.: A new proof of the Lyapunov convexity theorem. SIAM J. Control Optim.
**28**, 478–481 (1990)MathSciNetCrossRefGoogle Scholar - 31.Wets, R.: Stochastic programming. In: Nemhauser, G.L., et al. (eds.) Handbooks in OR & MS, vol. 1, pp. 573–629 (1989)Google Scholar
- 32.Xu, H.: An implicit programming approach for a class of stochastic mathematical programs with complementarity constraints. SIAM J. Optim.
**16**, 670–696 (2006)MathSciNetCrossRefGoogle Scholar - 33.Xu, H., Ye, J.J.: Necessary optimality conditions for two-stage stochastic programs with equilibaium constraints. SIAM J. Optim.
**20**, 1685–1715 (2010)MathSciNetCrossRefGoogle Scholar - 34.Xu, H., Zhang, D.: Smooth sample average approximation of stationary points in nonsmooth stochastic optimization and applications. Math. Program.
**119**, 371–401 (2009)MathSciNetCrossRefGoogle Scholar