Spectral Gradient Methods for Linearly Constrained Optimization

  • J. M. Martínez
  • E. A. Pilotta
  • M. Raydan


Linearly constrained optimization problems with simple bounds are considered in the present work. First, a preconditioned spectral gradient method is defined for the case in which no simple bounds are present. This algorithm can be viewed as a quasi-Newton method in which the approximate Hessians satisfy a weak secant equation. The spectral choice of steplength is embedded into the Hessian approximation and the whole process is combined with a nonmonotone line search strategy. The simple bounds are then taken into account by placing them in an exponential penalty term that modifies the objective function. The exponential penalty scheme defines the outer iterations of the process. Each outer iteration involves the application of the previously defined preconditioned spectral gradient method for linear equality constrained problems. Therefore, an equality constrained convex quadratic programming problem needs to be solved at every inner iteration. The associated extended KKT matrix remains constant unless the process is reinitiated. In ordinary inner iterations, only the right-hand side of the KKT system changes. Therefore, suitable sparse factorization techniques can be applied and exploited effectively. Encouraging numerical experiments are presented.


Linearly constrained optimization quasi-Newton methods exponential penalty methods spectral gradient method nonmonotone line search 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barzilai, J., Borwein, J. M. 1988 Two-Point Stepsize Gradient Methods IMA Journal of Numerical Analysis8141148Google Scholar
  2. 2.
    Raydan, M. 1993 On the Barzilai and Borwein Choice of Steplength for the Gradient Method IMA Journal of Numerical Analysis13321326Google Scholar
  3. 3.
    Birgin, E. G., Chambouleyron, I., Martínez, J. M. 1999 Estimation of Optical Constants of Thin Films Using Unconstrained Optimization Journal of Computational Physics151862880CrossRefGoogle Scholar
  4. 4.
    Birgin, E. G., Evtushenko, Y. G. 1998 Automatic Differentiation and Spectral Projected Gradient Methods for Optimal Control Problems Optimization Methods and Software10125146MathSciNetGoogle Scholar
  5. 5.
    Raydan, M. 1997 The Barzilai and Borwein Gradient Method for the Large-Scale Unconstrained Minimization Problem SIAM Journal on Optimization72633CrossRefGoogle Scholar
  6. 6.
    Bielschowsky, R. H., Friedlander, A., Gomes, F. A. M., Martínez, J. M., Raydan, M. 1997 An Adaptive Algorithm for Bound-Constrained Quadratic Minimization Investigación Operativa767102Google Scholar
  7. 7.
    Friedlander, A., Martínez, J. M., Raydan, M. 1995 A New Method for Large-Scale Box Constrained Convex Quadratic Minimization Problems Optimization Methods and Software55774Google Scholar
  8. 8.
    Birgin, E. G., Martínez, J. M., Raydan, M. 2000 Nonmonotone Spectral Projected Gradient Methods on Convex Sets SIAM Journal of Optimization1011961211CrossRefGoogle Scholar
  9. 9.
    Luengo, F., Glunt, W., Hayden, T. L., Raydan, M. 2002 Preconditioned Spectral Gradient Method Numerical Algorithms30241258CrossRefGoogle Scholar
  10. 10.
    Molina, B., Raydan, M. 1996 The Preconditioned Barzilai-Borwein Method for the Numerical Solution of Partial Differential Equations Numerical Algorithms134560Google Scholar
  11. 11.
    Grippo, L., Lampariello, F., Lucidi, S. 1986 A Nonmonotone Line Search Technique for Newton’s Method SIAM Journal on Numerical Analysis23707716CrossRefGoogle Scholar
  12. 12.
    Birgin, E. G., Martínez, J. M., Raydan, M. 2003 Inexact Spectral Projected Gradient Methods on Convex Sets IMA Journal of Numerical Analysis23539559CrossRefGoogle Scholar
  13. 13.
    Bertsekas, D. P. 1995Nonlinear ProgrammingAthena ScientificBelmont, MassachusettsGoogle Scholar
  14. 14.
    Martínez, J. M. 200) BOX-QUACAN and the Implementation of Augmented Lagrangian Algorithms for Minimization with Inequality Constraints Computational and Applied Mathematics193156Google Scholar
  15. 15.
    Murphy, F. H. 1974 A Class of Exponential Penalty Functions SIAM Journal on Control12679687CrossRefGoogle Scholar
  16. 16.
    Martínez, J. M., Pilotta, E. A., and Raydan, M., Spectral Gradient Methods for Linearly Constrained Optimization, Technical Report, Department of Applied Mathematics, University of Campinas, Campinas, SP, Brazil, 2004; see also www.ime.unicamp.br/~martinez.Google Scholar
  17. 17.
    Duff, I. S., Reid, J. K. 1082MA27: A Set of Fortran Subroutines for Solving Sparse Symmetric Sets of Linear EquationsComputer Science and System Division, AERE HarwellOxford, EnglandTechnical Report R-10533Google Scholar
  18. 18.
    Gould, N. I. M. 1991 An Algorithm for Large-Scale Quadratic Programming IMA Journal of Numerical Analysis11299324Google Scholar
  19. 19.
    Conn, A. R., Gould, N. I. M., Toint, P. L. 1991 A Globally Convergent Augmented Lagrangian Algorithm for Optimization witn General Constraints and Simple Bounds SIAM Journal on Numerical Analysis28545572CrossRefGoogle Scholar
  20. 20.
    20 Conn, A. R., Gould, N. I. M., Toint, P. L. 1992Lancelot: A Fortran Package for Large-Scale Nonlinear Optimization (Release A)Springer VerlagNew York, NYSpringer Series in Computational MathematicsGoogle Scholar
  21. 21.
    Vanderbei, R. J. 1994LOQO: An Interior-Point Code for Quadratic ProgrammingPrinceton UniversityPrinceton New JerseyTechnical Report SOR-94-15Google Scholar
  22. 22.
    Vanderbei, R. J., Shanno, D. F. 1977An Interior-Point Algorithm for Nonconvex Nonlinear ProgrammingStatistics and Operations Research, Princeton UniversityPrinceton, New JerseyTechnical Report SOR-97-21Google Scholar
  23. 23.
    Krejić, N., Martínez, J. M., Mello, M. P., Pilotta, E. A. 2000 Validation of an Augmented Lagrangian Algorithm with a Gauss-Newton Hessian Approximation Using a Set of Hard-Spheres Problems Journal of Computational Optimization and Applications16247263CrossRefGoogle Scholar
  24. 24.
    Fletcher, R. 1987Practical Methods of OptimizationWileyNew York, NYGoogle Scholar
  25. 25.
    Gill, P. E., Murray, W., Wrigth, M. H. 1981Practical OptimizationAcademic PressLondon, EnglandGoogle Scholar

Copyright information

© Springer Science+Business Media, Inc. 2005

Authors and Affiliations

  • J. M. Martínez
    • 1
  • E. A. Pilotta
    • 2
  • M. Raydan
    • 3
  1. 1.Professor, IMECC-UNICAMPCampinasBrazil
  2. 2.Associate Professor, Universidad de CordobaCordobaArgentina
  3. 3.Professor,Universidad Central de VenezuelaCaracasenezuela

Personalised recommendations