Journal of Optimization Theory and Applications

, Volume 178, Issue 2, pp 383–410 | Cite as

Second-Order Optimality Conditions and Improved Convergence Results for Regularization Methods for Cardinality-Constrained Optimization Problems

  • Max Bucher
  • Alexandra SchwartzEmail author


We consider nonlinear optimization problems with cardinality constraints. Based on a continuous reformulation, we introduce second-order necessary and sufficient optimality conditions. Under such a second-order condition, we can guarantee local uniqueness of Mordukhovich stationary points. Finally, we use this observation to provide extended local convergence theory for a Scholtes-type regularization method, which guarantees the existence and convergence of iterates under suitable assumptions. This convergence theory can also be applied to other regularization schemes.


Cardinality constraints Strong stationarity Mordukhovich stationarity Second-order optimality conditions Regularization method Scholtes regularization 

Mathematics Subject Classification

90C27 90C30 90C33 90C46 65K05 



The work of Alexandra Schwartz and Max Bucher is supported by the ‘Excellence Initiative’ of the German Federal and State Governments and the Graduate School of Computational Engineering at Technische Universität Darmstadt. The authors would like to thank two anonymous referees and the editors for carefully reading the manuscript and asking the right questions, which helped to improve the paper.


  1. 1.
    Candes, E., Wakin, M.: An introduction to compressive sampling. IEEE Signal Process. Mag. 25(2), 21–30 (2008)CrossRefGoogle Scholar
  2. 2.
    Miller, A.: Subset Selection in Regression, 2nd edn. Chapman & Hall/CRC Press, Boca Raton (2002)CrossRefzbMATHGoogle Scholar
  3. 3.
    Weston, J., Elisseeff, A., Schölkopf, B., Kaelbling, P.: The use of zero-norm with linear models and kernel methods. J. Mach. Learn. Res. 1439–1461 (2003)Google Scholar
  4. 4.
    Galati, M.: Decomposition Methods for Integer Linear Programming. Ph.D. thesis (2010)Google Scholar
  5. 5.
    Gade, D., Küçükyavuz, S.: Formulations for dynamic lot sizing with service levels. Naval Res. Logist. (NRL) 60(2), 87–101 (2013)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Bienstock, D.: Computational study of a family of mixed-integer quadratic programming problems. Math. Program. 74(2), 121–140 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Bertsimas, D., Shioda, R.: Algorithm for cardinality-constrained quadratic optimization. Comput. Optim. Appl. 43(1), 1–22 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Lorenzo, D.D., Liuzzi, G., Rinaldi, F., Schoen, F., Sciandrone, M.: A concave optimization-based approach for sparse portfolio selection. Optim. Methods Softw. 27(6), 983–1000 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Murray, W., Shek, H.: A local relaxation method for the cardinality constrained portfolio optimization problem. Comput. Optim. Appl. 53(3), 681–709 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Loh, P.L., Wainwright, M.J.: Support Recovery Without Incoherence: A Case for Nonconvex Regularization. arXiv preprint arXiv:1412.5632 (2014)
  11. 11.
    Beck, A., Eldar, Y.C.: Sparsity constrained nonlinear optimization: optimality conditions and algorithms. SIAM J. Optim. 23(3), 1480–1509 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Pan, L., Xiu, N., Fan, J.: Optimality conditions for sparse nonlinear programming. Sci. China Math. 60(5), 759–776 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Burdakov, O.P., Kanzow, C., Schwartz, A.: Mathematical programs with cardinality constraints: reformulation by complementarity-type conditions and a regularization method. SIAM J. Optim. 26(1), 397–425 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Feng, M., Mitchell, J.E., Pang, J.S., Shen, X., Wächter, A.: Complementarity formulations of l0-norm optimization problems. Industrial Engineering and Management Sciences. Technical Report. Northwestern University, Evanston, IL, USA (2013)Google Scholar
  15. 15.
    Červinka, M., Kanzow, C., Schwartz, A.: Constraint qualifications and optimality conditions for optimization problems with cardinality constraints. Math. Program. 160(1–2), 353–377 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Branda, M., Bucher, M., Červinka, M., Schwartz, A.: Convergence of a scholtes-type regularization method for cardinality-constrained optimization problems with an application in sparse robust portfolio optimization. Comput. Optim. Appl. (2018).
  17. 17.
    Yuan, G., Ghanem, B.: Sparsity constrained minimization via mathematical programming with equilibrium constraints. arXiv preprint arXiv:1608.04430 (2016)
  18. 18.
    Adam, L., Branda, M.: Nonlinear chance constrained problems: optimality conditions, regularization and solvers. J. Optim. Theory Appl. 170(2), 419–436 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Curtis, F.E., Wächter, A., Zavala, V.M.: A Sequential Algorithm for Solving Nonlinear Optimization Problems with Chance Constraints. Tech. Rep. 16T-012, COR@L Laboratory, Department of ISE, Lehigh University (2016)Google Scholar
  20. 20.
    Luo, Z.Q., Pang, J.S., Ralph, D.: Mathematical Programs with Equilibrium Constraints. Cambridge University Press, Cambridge (1996)CrossRefzbMATHGoogle Scholar
  21. 21.
    Outrata, J., Kočvara, M., Zowe, J.: Nonsmooth Approach to Optimization Problems with Equilibrium Constraints. Kluwer Academic Publishers, Dordrecht (1998). (Nonconvex Optimization and its Applications)CrossRefzbMATHGoogle Scholar
  22. 22.
    Guo, L., Lin, G.H., Ye, J.J.: Second-order optimality conditions for mathematical programs with equilibrium constraints. J. Optim. Theory Appl. 158(1), 33–64 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Hoheisel, T., Kanzow, C.: First-and second-order optimality conditions for mathematical programs with vanishing constraints. Appl. Math. 52(6), 495–514 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Scheel, H., Scholtes, S.: Mathematical programs with complementarity constraints: stationarity, optimality, and sensitivity. Math. Oper. Res. 25(1), 1–22 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Nocedal, J., Wright, S.J.: Numerical Optimization, 2, ed edn. Springer series in operations research and financial engineering. Springer, New York [u.a.] (2006)Google Scholar
  26. 26.
    Steffensen, S., Ulbrich, M.: A new relaxation scheme for mathematical programs with equilibrium constraints. SIAM J. Optim. 20(5), 2504–2539 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Scholtes, S.: Convergence properties of a regularization scheme for mathematical programs with complementarity constraints. SIAM J. Optim. 11(4), 918–936 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Bertsekas, D.P., Ozdaglar, A.E.: Pseudonormality and a lagrange multiplier theory for constrained optimization. J. Optim. Theory Appl. 114(2), 287–343 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Kanzow, C., Schwartz, A.: Mathematical programs with equilibrium constraints: enhanced fritz john-conditions, new constraint qualifications, and improved exact penalty results. SIAM J. Optim. 20(5), 2730–2753 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Kanzow, C., Schwartz, A.: The price of inexactness: convergence properties of relaxation methods for mathematical programs with complementarity constraints revisited. Math. Oper. Res. 40(2), 253–275 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Kirst, P., Rigterink, F., Stein, O.: Global optimization of disjunctive programs. J. Glob. Optim. 69(2), 283–307 (2017)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Graduate School Computational EngineeringTechnische Universität DarmstadtDarmstadtGermany

Personalised recommendations