Advertisement

Mathematical Programming

, Volume 174, Issue 1–2, pp 195–222 | Cite as

On the pervasiveness of difference-convexity in optimization and statistics

  • Maher Nouiehed
  • Jong-Shi Pang
  • Meisam RazaviyaynEmail author
Full Length Paper Series B

Abstract

With the increasing interest in applying the methodology of difference-of-convex (dc) optimization to diverse problems in engineering and statistics, this paper establishes the dc property of many functions in various areas of applications not previously known to be of this class. Motivated by a quadratic programming based recourse function in two-stage stochastic programming, we show that the (optimal) value function of a copositive (thus not necessarily convex) quadratic program is dc on the domain of finiteness of the program when the matrix in the objective function’s quadratic term and the constraint matrix are fixed. The proof of this result is based on a dc decomposition of a piecewise \(\hbox {LC}^1\) function (i.e., functions with Lipschitz gradients). Armed with these new results and known properties of dc functions existed in the literature, we show that many composite statistical functions in risk analysis, including the value-at-risk (VaR), conditional value-at-risk (CVaR), optimized certainty equivalent, and the expectation-based, VaR-based, and CVaR-based random deviation functionals are all dc. Adding the known class of dc surrogate sparsity functions that are employed as approximations of the \(\ell _0\) function in statistical learning, our work significantly expands the classes of dc functions and positions them for fruitful applications.

Mathematics Subject Classification

90C26 

Notes

Acknowledgements

The second author gratefully acknowledges the discussion with Professors Le Thi Hoi An and Pham Dinh Tao in the early stage of this work during his visit to the Université de Lorraine, Metz in June 2016. The three authors acknowledge their fruitful discussion with Professor Defeng Sun at the National University of Singapore during his visit to the University of Southern California. They are also grateful to Professor Marc Teboulle for drawing their attention to the references [5, 6, 7] that introduce and revisit the OCE. The constructive comments of two referees are also gratefully acknowledged. In particular, the authors are particularly grateful to a referee who has been very patient with our repeated misunderstanding of the work [57] that is now correctly summarized at the end of Sect. 3.1.

References

  1. 1.
    Ahn, M., Pang, J.S., Xin, J.: Difference-of-convex learning: directional stationarity, optimality, and sparsity. SIAM J. Optim. 27, 1637–1665 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Alexandroff, A.D.: Surfaces represented by the difference of convex functions. Doklady Akademii Nauk SSSR (N.S.) 72, 613–616 (1950). [English translation: Siberian Èlektron. Mathetical. Izv. 9, 360–376 (2012)]Google Scholar
  3. 3.
    Alvarado, A., Scutari, G., Pang, J.S.: A new decomposition method for multiuser DC-programming and its application to physical layer security. IEEE Trans. Signal Process. 62, 2984–2998 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Bačák, M., Borwein, J.M.: On difference convexity of locally Lipschitz functions. Optimization 60, 961–978 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Ben-Tal, A., Teboulle, M.: Expected utility, penalty functions and duality in stochastic nonlinear programmi. Manage. Sci. 32, 14451466 (1986)CrossRefzbMATHGoogle Scholar
  6. 6.
    Ben-Tal, A., Teboulle, M.: Penalty functions and duality in stochastic programming via \(\varphi \)-divergence functionals. Math. Oper. Res. 12, 224–240 (1987)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Ben-Tal, A., Teboulle, M.: An old-new concept of convex risk measures: the optimized certainty equivalent. Math. Finance 17, 449–476 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Birge, J.R., Louveaux, F.: Introduction to Stochastic Programming. Springer Series in Operations Research. Springer, New York (1997)zbMATHGoogle Scholar
  9. 9.
    Chang, T.-H., Hong, M., Pang, J.S.: Local minimizers and second-order conditions in composite piecewise programming via directional derivatives. Preprint arXiv:1709.05758 (2017)
  10. 10.
    Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)zbMATHGoogle Scholar
  11. 11.
    Cui, Y., Pang, J.-S., Sen, B.: Composite difference-max programs for modern statistical estimation problems. Preprint arXiv:1803.00205 (2018)
  12. 12.
    Eaves, B.C.: On quadratic programming. Manage. Sci. 17, 698–711 (1971)CrossRefzbMATHGoogle Scholar
  13. 13.
    Facchinei, F., Pang, J.S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, Volumes I and II. Springer, New York (2003)zbMATHGoogle Scholar
  14. 14.
    Fan, J., Xue, L., Zou, H.: Strong oracle optimality of folded concave penalized estimation. Ann. Stat. 42(2014), 819–849 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Giannessi, F., Tomasin, E.: Nonconvex quadratic programs, linear complementarity problems, and integer linear programs. In: Conti, R., Ruberti, A. (eds) Fifth Conference on Optimization Techniques (Rome 1973), Part I, Lecture Notes in Computer Science, Vol. 3, pp. 437–449. Springer, Berlin (1973)Google Scholar
  16. 16.
    Gotoh, J., Takeda, A., Tono, K.: DC formulations and algorithms for sparse optimization problems. Math. Program. Ser. B. (2017).  https://doi.org/10.1007/s10107-017-1181-0
  17. 17.
    Hahn, G., Banergjee, M., Sen, B.: Parameter Estimation and Inference in a Continuous Piecewise Linear Regression Model. Manuscript, Department of Statistics. Columbia University, New York (2016)Google Scholar
  18. 18.
    Hartman, P.: On functions representable as a difference of convex functions. Pac. J. Math. 9, 707–713 (1959)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Hiriart-Urruty, J.B.: Generalized differentiability, duality and optimization for problems dealing with differences of convex functions. In: Ponstein, J. (eds) Convexity and Duality in Optimization. Proceedings of the Symposium on Convexity and Duality in Optimization Held at the University of Groningen, The Netherlands June 22, 1984, pp. 37–70 (1985)Google Scholar
  20. 20.
    Horst, R., Thoai, N.V.: D.C. programming: overview. J. Optim. Theory Appl. 103, 1–43 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Hu, J., Mitchell, J.E., Pang, J.S.: An LPCC approach to nonconvex quadratic programs. Math. Program. 133, 243–277 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Jara-Moroni, F., Pang, J.-S., Wächter, A.: A study of the difference-of-convex approach for solving linear programs with complementarity constraints. Math. Program. 169(1), 221–254 (2018)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Le Thi, H.A., Pham, D.T.: The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Ann. Oper. Res. 133, 25–46 (2005)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Le Thi, H.A., Pham, D.T.: Recent advances in DC programming and DCA. Trans. Comput. Collect. Intell. 8342, 1–37 (2014)Google Scholar
  25. 25.
    Le Thi, H.A., Pham, D.T.: DC Programming and DCA: Thirty years of Developments Manuscript. University of Lorraine, Lorraine (2016)zbMATHGoogle Scholar
  26. 26.
    Le Thi, H.A., Pham, D.T., Vo, X.T.: DC approximation approaches for sparse optimization. Eur. J. Oper. Res. 244, 26–46 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Lee, G.M., Tam, N.N., Yen, N.D.: Quadratic Programming and Affine Variational Inequalities A Qualitative Study. Springer, New York (2005)zbMATHGoogle Scholar
  28. 28.
    Luo, Z.Q., Tseng, P.: Error bound and convergence analysis of matrix splitting algorithms for the affine variational inequality problem. SIAM J. Optim. 2, 43–54 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Luo, Z.Q., Pang, J.S., Ralph, D.: Mathematical Programs With Equilibrium Constraints. Cambridge University Press, Cambridge (1996)CrossRefzbMATHGoogle Scholar
  30. 30.
    Nash, J.: Non-cooperative games. Ann. Math. 54(2), 286–295 (1951)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Orgeta, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Classics in Applied Mathematics. SIAM Publications, Philadelphia (2000)Google Scholar
  32. 32.
    Ovchinnikov, S.: Max-min representation of piecewise linear functions. Contrib. Algebra Geom. 43, 297–302 (2002)MathSciNetzbMATHGoogle Scholar
  33. 33.
    Pang, J.S., Tao, M.: Decomposition methods for computing directional stationary solutions of a class of non-smooth nonconvex optimization problems. SIAM J. Optim. Submitted January (2017)Google Scholar
  34. 34.
    Pang, J.S., Razaviyayn, M., Alvarado, A.: Computing B-stationary points of nonsmooth dc programs. Math. Oper. Res. 42, 95–118 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Pang, J.-S., Sen, S., Shanbhag, U.V.: Two-stage non-cooperative games with risk-averse players. Math. Program. 165(1), 235–290 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Pham, D.T., Le Thi, H.A.: Convex analysis approach to DC programming: theory, algorithm and applications. Acta Math. Vietnam. 22, 289–355 (1997)MathSciNetzbMATHGoogle Scholar
  37. 37.
    Razaviyayn, M., Hong, M., Luo, Z.Q., Pang, J.S.: Parallel successive convex approximation for nonsmooth nonconvex optimization. Adv. Neural Inf. Process. Syst. (NIPS), 1440–1448 (2014)Google Scholar
  38. 38.
    Razaviyayn, M.: Successive Convex Approximations: Analysis and Applications. Ph.D. Dissertation. Department of Electrical and Computer Engineering. University of Minnesota (Minneapolis 2014)Google Scholar
  39. 39.
    Razaviyayn, M., Hong, M., Luo, Z.Q.: A unified convergence analysis of block successive minimization methods for nonsmooth optimization. SIAM J. Optim. 23, 1126–1153 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Razaviyayn, M., Hong, M., Luo, Z.Q., Pang, J.S.: A unified algorithmic framework for block-structured optimization involving big data. IEEE Signal Process. Mag. 33, 57–77 (2016)CrossRefGoogle Scholar
  41. 41.
    Razaviyayn, M., Sanjabi, M., Luo, Z.Q.: A stochastic successive minimization method for nonsmooth nonconvex optimization with applications to transceiver design in wireless communication networks. Math. Program. Ser. B 157, 515–545 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  42. 42.
    Rockafellar, R.T., Uryasev, S.: Optimization of conditional value-at-risk. J. Risk 2, 21–41 (2000)CrossRefGoogle Scholar
  43. 43.
    Rockafellar, R.T., Uryasev, S.: Conditional value-at-risk for general loss distributions. J. Bank. Finance 7, 1143–1471 (2002)Google Scholar
  44. 44.
    Rockafellar, R.T., Uryasev, S.: The fundamental risk quadrangle in risk management, optimization, and statistical estimation. Surv. Oper. Res. Manag. Sci. 18, 33–53 (2013)MathSciNetGoogle Scholar
  45. 45.
    Rockafellar, R.T., Uryasev, S., Zabarankin, M.: Optimality conditions in portfolio analysis with general deviation measures. Math. Program. Ser. B 108, 515–540 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  46. 46.
    Rockafellar, R.T., Uryasev, S., Zabarankin, M.: Generalized deviations in risk analysis. Finance Stochast. 10, 51–74 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Sarykalin, S., Uryasev, S.: Value-at-risk versus conditional value-at-risk in risk management and optimization. Tutor. Oper. Res. 2008, 269–294 (2008)Google Scholar
  48. 48.
    Scholtes, S.: Intoduction to Piecewise Differentiable Functions. Springer, Berlin (2012)CrossRefzbMATHGoogle Scholar
  49. 49.
    Scutari, G., Facchinei, F., Palomar, D.P., Pang, J.S., Song, P.: Decomposition by partial linearization: parallel optimization of multiuser systems. IEEE Trans. Signal Process. 62, 641–656 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  50. 50.
    Scutari, G., Alvarado, A., Pang, J.S.: A new decomposition method for multiuser DC-programming and its application to physical layer security. IEEE Trans. Signal Process. 62, 2984–2998 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  51. 51.
    Shapiro, A., Dentcheva, D., Ruszczynski, A.: Lectures on Stochastic Programming: Modeling and Theory. SIAM Publications, Philadelphia (2009)CrossRefzbMATHGoogle Scholar
  52. 52.
    Sun, J.: On the structure of convex piecewise quadratic functions. J. Optim. Theory Appl. 72, 499–510 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  53. 53.
    Tuy, H.: Convex Analysis and Global Optimization, 2nd edn, vol. 110. Springer Optimization and its Applications (2016). [First edition. Kluwer Publishers (Dordrecht 1998)]Google Scholar
  54. 54.
    Tuy, H.: Global minimization of a difference of two convex functions. Math. Program. Study 30, 150–187 (1987)MathSciNetCrossRefzbMATHGoogle Scholar
  55. 55.
    Veselý, L., Zajíček, L.: Delta-convex mappings between Banach spaces and applications. Dissertationes Math. (Rozprawy Mat.) 289 (1989)Google Scholar
  56. 56.
    Veselý, L., Zajíček, L.: On composition of DC functions and mappings. J. Conv. Anal. 16(2), 423–439 (2009)zbMATHGoogle Scholar
  57. 57.
    Wozabal, D.: Value-at-risk optimization using the difference of convex algorithm. OR Spectr. 34, 861–883 (2012)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society 2018

Authors and Affiliations

  • Maher Nouiehed
    • 1
  • Jong-Shi Pang
    • 1
  • Meisam Razaviyayn
    • 1
    Email author
  1. 1.Daniel J. Epstein Department of Industrial and Systems EngineeringUniversity of Southern CaliforniaLos AngelesUSA

Personalised recommendations