A Unified Approach to the Global Exactness of Penalty and Augmented Lagrangian Functions I: Parametric Exactness

Article
  • 6 Downloads

Abstract

In this two-part study, we develop a unified approach to the analysis of the global exactness of various penalty and augmented Lagrangian functions for constrained optimization problems in finite-dimensional spaces. This approach allows one to verify in a simple and straightforward manner whether a given penalty/augmented Lagrangian function is exact, i.e., whether the problem of unconstrained minimization of this function is equivalent (in some sense) to the original constrained problem, provided the penalty parameter is sufficiently large. Our approach is based on the so-called localization principle that reduces the study of global exactness to a local analysis of a chosen merit function near globally optimal solutions. In turn, such local analysis can be performed with the use of optimality conditions and constraint qualifications. In the first paper, we introduce the concept of global parametric exactness and derive the localization principle in the parametric form. With the use of this version of the localization principle, we recover existing simple, necessary, and sufficient conditions for the global exactness of linear penalty functions and for the existence of augmented Lagrange multipliers of Rockafellar–Wets’ augmented Lagrangian. We also present completely new necessary and sufficient conditions for the global exactness of general nonlinear penalty functions and for the global exactness of a continuously differentiable penalty function for nonlinear second-order cone programming problems. We briefly discuss how one can construct a continuously differentiable exact penalty function for nonlinear semidefinite programming problems as well.

Keywords

Penalty function Augmented Lagrangian function Exactness Localization principle Semidefinite programming 

Mathematics Subject Classification

65K05 90C30 

Notes

Acknowledgements

The author is grateful to Professor Franco Giannessi for pointing out to the author the importance of development of a general theory of the exactness of penalty and augmented Lagrangian functions several years ago. The author also wishes to express his thanks to Professor X. Yang for thoughtful and stimulating comments that helped to improve the quality of the article.

References

  1. 1.
    Fiacco, A.V., McCormick, G.P.: Nonlinear Programming: Sequential Unconstrained Minimization Techniques. SIAM, Philadelphia (1990)CrossRefMATHGoogle Scholar
  2. 2.
    Auslender, A.: Penalty and barrier methods: a unified framework. SIAM J. Optim. 10, 211–230 (1999)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Birgin, E.G., Martinez, J.M.: Practical Augmented Lagrangian Methods for Constrained Optimization. SIAM, Philadelphia (2014)CrossRefMATHGoogle Scholar
  4. 4.
    Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer, Berlin (1998)CrossRefMATHGoogle Scholar
  5. 5.
    Huang, X.X., Yang, X.Q.: A unified augmented Lagrangian approach to duality and exact penalization. Math. Oper. Res. 28, 533–552 (2003)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Zhou, Y.Y., Yang, X.Q.: Some results about duality and exact penalization. J. Glob. Optim. 29, 497–509 (2004)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Huang, X.X., Yang, X.Q.: Further study on augmented Lagrangian duality theory. J. Glob. Optim. 31, 193–210 (2005)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Zhou, Y.Y., Yang, X.Q.: Augmented Lagrangian functions for constrained optimization problems. J. Glob. Optim. 52, 95–108 (2012)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Wang, C.Y., Yang, X.Q., Yang, X.M.: Nonlinear augmented Lagrangian and duality theory. Math. Oper. Res. 38, 740–760 (2013)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Rubinov, A.M., Yang, X.: Lagrange-Type Functions in Constrained Non-convex Optimization. Kluwer Academic Publishers, Dordrecht (2003)CrossRefMATHGoogle Scholar
  11. 11.
    Wang, C.Y., Yang, X.Q., Yang, X.M.: Unified nonlinear Lagrangian approach to duality and optimal paths. J. Optim. Theory Appl. 135, 85–100 (2007)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Giannessi, F.: Constrained Optimization and Image Space Analysis. Separation of Sets and Optimality Conditions, vol. 1. Springer, New York (2005)MATHGoogle Scholar
  13. 13.
    Li, J., Feng, S., Zhang, Z.: A unified approach for constrained extremum problems: image space analysis. J. Optim. Theory Appl. 159, 69–92 (2013)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Zhu, S.K., Li, S.J.: Unified duality theory for constrained extremum problems. Part I: image space analysis. J. Optim. Theory Appl. 161, 738–762 (2014)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Zhu, S.K., Li, S.J.: Unified duality theory for constrained extremum problems. Part II: special duality schemes. J. Optim. Theory Appl. 161, 763–782 (2014)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Di Pillo, G., Grippo, L.: Exact penalty functions in constrained optimization. SIAM J. Control. Optim. 27, 1333–1360 (1989)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Yevtushenko, YuG, Zhadan, V.G.: Exact auxiliary functions in optimization problems. U.S.S.R. Comput. Math. Math. Phys. 30, 31–42 (1990)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Di Pillo, G.: Exact penalty methods. In: Spedicato, E. (ed.) Algorithms for Continuous Optimization: The State of the Art, pp. 1–45. Kluwer Academic Press, Boston (1994)Google Scholar
  19. 19.
    Eremin, I.I.: Penalty method in convex programming. Soviet Math. Dokl. 8, 459–462 (1966)MATHGoogle Scholar
  20. 20.
    Zangwill, W.I.: Nonlinear programming via penalty functions. Manag. Sci. 13, 344–358 (1967)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Han, S.P., Mangasarian, O.L.: Exact penalty functions in nonlinear programming. Math. Program. 17, 251–269 (1979)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Di Pillo, G., Grippo, L.: On the exactness of a class of nondifferentiable penalty functions. J. Optim. Theory Appl. 57, 399–410 (1988)MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Burke, J.V.: An exact penalization viewpoint on constrained optimization. SIAM J. Control. Optim. 29, 968–998 (1991)MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Demyanov, V.F.: Nonsmooth optimization. In: Di Pillo, G., Schoen, F. (eds.) Nonlinear Optimization. Lecture Notes in Mathematics, vol. 1989, pp. 55–164. Springer, Berlin (2010)CrossRefGoogle Scholar
  25. 25.
    Dolgopolik, M.V.: A unifying theory of exactness of linear penalty functions. Optimization 65, 1167–1202 (2016)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Fletcher, R.: A class of methods for nonlinear programming with termination and convergence properties. In: Abadie, J. (ed.) Integer and Nonlinear Programming, pp. 157–173. North-Holland, Amsterdam (1970)Google Scholar
  27. 27.
    Fletcher, R.: An exact penalty function for nonlinear programming with inequalities. Math. Program. 5, 129–150 (1973)MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Di Pillo, G., Grippo, L.: A continuously differentiable exact penalty function for nonlinear programming problems with inequality constraints. SIAM J. Control Optim. 23, 72–84 (1985)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Lucidi, S.: New results on a continuously differentiable exact penalty function. SIAM J. Optim. 2, 558–574 (1992)MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Fukuda, E.H., Silva, P.J.S., Fukushima, M.: Differentiable exact penalty functions for nonlinear second-order cone programs. SIAM J. Optim. 22, 1607–1633 (2012)MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    Di Pillo, G., Grippo, L.: A new class of augmented Lagrangians in nonlinear programming. SIAM J. Control Optim. 17, 618–628 (1979)MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Di Pillo, G., Grippo, L.: A new augmented Lagrangian function for inequality constraints in nonlinear programming problems. J. Optim. Theory Appl. 36, 495–519 (1982)MathSciNetCrossRefMATHGoogle Scholar
  33. 33.
    Di Pillo, G., Lucidi, S.: An augmented Lagrangian function with improved exactness properties. SIAM J. Optim. 12, 376–406 (2001)MathSciNetCrossRefMATHGoogle Scholar
  34. 34.
    Fukuda, E.H., Lourenco, B.F.: Exact augmented Lagrangian functions for nonlinear semidefinite programming. arXiv:1705.06551 (2017)
  35. 35.
    Dolgopolik, M.V.: Augmented Lagrangian functions for cone constrained optimization: the existence of global saddle points and exact penalty property. J. Glob. Optim. (2018).  https://doi.org/10.1007/s10898-017-0603-0 Google Scholar
  36. 36.
    Rubinov, A.M., Glover, B.M., Yang, X.Q.: Decreasing functions with applications to penalization. SIAM J. Optim. 10, 289–313 (1999)MathSciNetCrossRefMATHGoogle Scholar
  37. 37.
    Rubinov, A.M., Yang, X.Q., Bagirov, A.M.: Penalty functions with a small penalty parameter. Optim. Methods Softw. 17, 931–964 (2002)MathSciNetCrossRefMATHGoogle Scholar
  38. 38.
    Rubinov, A.M., Gasimov, R.N.: Strictly increasing positively homogeneous functions with applications to exact penalization. Optimization 52, 1–28 (2003)MathSciNetCrossRefMATHGoogle Scholar
  39. 39.
    Yang, X.Q., Huang, X.X.: Partially strictly monotone and nonlinear penalty functions for constrained mathematical programs. Comput. Optim. Appl. 25, 293–311 (2003)MathSciNetCrossRefMATHGoogle Scholar
  40. 40.
    Huyer, W., Neumaier, A.: A new exact penalty function. SIAM J. Optim. 13, 1141–1158 (2003)MathSciNetCrossRefMATHGoogle Scholar
  41. 41.
    Wang, C., Ma, C., Zhou, J.: A new class of exact penalty functions and penalty algorithms. J. Glob. Optim. 58, 51–73 (2014)MathSciNetCrossRefMATHGoogle Scholar
  42. 42.
    Dolgopolik, M.V.: A unifying theory of exactness of linear penalty functions II: parametric penalty functions. Optimization 66, 1577–1622 (2017)MathSciNetCrossRefMATHGoogle Scholar
  43. 43.
    Shapiro, A., Sun, J.: Some properties of the augmented Lagrangian in cone constrained optimization. Math. Oper. Res. 29, 479–491 (2004)MathSciNetCrossRefMATHGoogle Scholar
  44. 44.
    Zhou, Y.Y., Zhou, J.C., Yang, X.Q.: Existence of augmented Lagrange multipliers for cone constrained optimization problems. J. Glob. Optim. 58, 243–260 (2014)MathSciNetCrossRefMATHGoogle Scholar
  45. 45.
    Dolgopolik, M.V.: Existence of augmented Lagrange multipliers: reduction to exact penalty functions and localization principle. Math. Program. 166, 297–326 (2017)MathSciNetCrossRefMATHGoogle Scholar
  46. 46.
    Dolgopolik, M.V.: A unified approach to the global exactness of penalty and augmented Lagrangian functions I: parametric exactness. arXiv:1709.07073 (2017)
  47. 47.
    Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer, New York (2000)CrossRefMATHGoogle Scholar
  48. 48.
    Liu, Y.J., Zhang, L.W.: Convergence of the augmented Lagrangian method for nonlinear optimization problems over second-order cones. J. Optim. Theory Appl. 139, 557–575 (2008)MathSciNetCrossRefMATHGoogle Scholar
  49. 49.
    Sun, D., Sun, J., Zhang, L.: The rate of convergence of the augmented Lagrangian method for nonlinear semidefinite programming. Math. Program. 114, 349–391 (2008)MathSciNetCrossRefMATHGoogle Scholar
  50. 50.
    Wu, H., Luo, H., Ding, X., Chen, G.: Global convergence of modified augmented Lagrangian methods for nonlinear semidefinite programming. Comput. Optim. Appl. 56, 531–558 (2013)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Institute of Problems of Mechanical Engineering, Russian Academy of SciencesSaint Petersburg State UniversitySaint PetersburgRussia

Personalised recommendations