Implementation of Variable Metric Methods for Constrained Optimization Based on an Augmented Lagrangian Functional

  • N. H. Engersbach
  • W. A. Gruver
Part of the Lecture Notes in Computer Science book series (LNCIS)


Penalization and gradient projection are two of the simplest and most useful concepts from nonlinear programming. Both provide a means for extending unconstrained gradient descent techniques to accommodate equality and inequality constraints. It is known that penalty function methods may be used to solve a wide class of problems, even those involving nonconvex constraints. In practice, however, the Hessian matrix of the objective functional becomes ill-conditioned, causing convergence difficulties [1]. On the other hand, gradient projection is a concept that involves linear approximation of the constraints and, therefore, is not inherently suited for nonlinear constraints. In the gradient projection algorithm of Rosen [2], which was originally designed for linear constraints forming a bounded convex region, nonlinear constraints were to be accommodated by a restoration step. This procedure is known to be unsatisfactory if a minimum on the tangent plane approximation lies far from the constraint. The difficulty can be avoided by a simple modification of the gradient projection concept that penalizes violations of the constraints along the search direction [3,4].


Gradient Projection Nonlinear Constraint Velocity Increment Penalty Function Method Transfer Orbit 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lootsma, F. A.: A Survey of Methods for Solving Constrained Minimization Problems via Unconstrained Minimization, in F. Lootsma (ed), Numerical Methods for Nonlinear Optimization, Academic Press, New York, 313–347 (1972).Google Scholar
  2. 2.
    Rosen, J. B.: The Gradient Projection Method for Nonlinear Programming; Part I: Linear Constraints, J. SIAM, 8, 181–217 (1960); Part II: Nonlinear Constraints, J. SIAM, 9, 414–443 (1961).MATHGoogle Scholar
  3. 3.
    Miele, A., Huang, H., and Heideman, J.: Sequential Gradient-Restoration Algorithm for the Minimization of Constrained Functions — Ordinary and Conjugate Gradient Versions, J. Optimization Theory and Applications, 4, No 4, 213–242 (1969).MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Kelley, H. J. and Speyer, J. L.: Accelerated Gradient Projection, in Lectures in Mathematics, 132, Springer Verlag, Berlin-Heidelberg, 151–158 (1970).Google Scholar
  5. 5.
    Hestenes, M. R.: Multiplier and Gradient Methods, in L. Zadeh (ed), Computing Methods in Optimization Problems, 2, Academic Press, New York, 143–163 (1969).Google Scholar
  6. 6.
    Powell, M. J. D.: A Method for Nonlinear Constraints in Minimization Problems, in R. Fletcher (ed), Optimization, Academic Press, New York, 283–298 (1969).Google Scholar
  7. 7.
    Roode, J. D.: Generalized Lagrangian Functions in Mathematical Programming, Thesis, University of Leiden, Netherlands, (1968).Google Scholar
  8. 8.
    Rockafellar, R. T.: Augmented Lagrange Multiplier Functions and Duality in Nonconvex Programming, SIAM J. of Control, to appear.Google Scholar
  9. 9.
    Haarhoff, P. C. and Buys, J. D. . A New Method for the Optimization of a Nonlinear Function Subject to Nonlinear Constraints, Computer J., 13, 178–184 (1970).MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Miele, A., Cragg, E., Iyer, R., and Levy, A.: Use of the Augmented Penalty Function in Mathematical Programming Problems, Part 1, J. Optimization Theory and Applications, 8, 115–130 (1971).MathSciNetCrossRefGoogle Scholar
  11. 11.
    Martensson, K.: Methods for Constrained Function Minimization, Report 7107, Div. of Automatic Control, Lund Institute of Technology, Sweden, March 1971.Google Scholar
  12. 12.
    Glad, T.: Lagrange Multiplier Methods for Minimization Under Equality Constraints, Report 7323, Div. of Automatic Control, Lund Institute of Technology, Sweden, August 1973.Google Scholar
  13. 13.
    Tripathi, S. S. and Narendra, K. S.: Constrained Optimization Problems Using Multiplier Methods, J. Optimization Theory and Applications, 9, 59–70 (1972).MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Wierzbicki, A. P.: A Penalty Function Shifting Method in Constrained Static Optimization and its Convergence Properties, Archiwum Automatyki i Telemechaniki, 16, 395–416 (1971).MathSciNetMATHGoogle Scholar
  15. 15.
    Gruver, W. A. and Engersbach, N. H.: Nonlinear Programming by Projection-Restoration Applied to Optimal Geostationary Satellite Positioning, AIAA Journal, December 1974.Google Scholar
  16. 16.
    Engersbach, N. H. and Gruver, W. A.: Constrained Optimization Based on Generalized Exterior Point Methods, Report IB013–72/10, Deutsche Forschungs- und Versuchsanstalt für Luft- und Raumfahrt, December 1972.Google Scholar
  17. 17.
    Broyden, C. G.: Quasi-Newton Methods, in W. Murray (ed), Numerical Methods for Unconstrained Optimization, Academic Press, 87–106 (1972).Google Scholar
  18. 18.
    Goldfarb, D.: Extension of Davidon’s Variable Metric Method to Maximization Under Linear Inequality and Equality Constraints, SIAM J. Applied Math., 17, 739–764, July 1969.MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Kelley, H. J., Denham, W., Johnson, I., and Wheatley, P.: An Accelerated Gradient Method for Parameter Optimization with Nonlinear Constraints, J. Astro-nautical Sciences, 13, No 4, 166–169, July-August 1966.MathSciNetGoogle Scholar
  20. 20.
    Goldfarb, D.: A Family of Variable-Metric Methods Derived by Variational Means, Maths. Computation, 24, 23–26 (1970).MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Zangwill, W. I.: Nonlinear Programming, Prentice-Hall, Englewood Cliffs, N. J., Chapter 13 (1969).Google Scholar
  22. 22.
    Gruver, W. A. and Engersbach, N.: A Mathematical Programming Approach to the Optimization of Constrained, Impulsive, Minimum-Fuel Trajectories, Report IB013–72/3, Deutsche Forschungs- und Versuchsanstalt für Luft- und Raumfahrt, June 1972.Google Scholar
  23. 23.
    Eckstein, M. C. and Jochim, E. F.: Vorläufige Untersuchung zur Bahnoptimierung für die Helio-C Mission, Report IB522–73/1, Deutsche Forschungs- und Versuchsanstalt für Luft- und Raumfahrt, March 1973.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1975

Authors and Affiliations

  • N. H. Engersbach
    • 1
  • W. A. Gruver
    • 2
  1. 1.DFVLR Institut für Dynamik der FlugsystemeOberpfaffenhofenGermany
  2. 2.Institut für RegelungstechnikTechnische Hochschule DarmstadtDarmstadtGermany

Personalised recommendations