Abstract
Linearly constrained optimization problems with simple bounds are considered in the present work. First, a preconditioned spectral gradient method is defined for the case in which no simple bounds are present. This algorithm can be viewed as a quasi-Newton method in which the approximate Hessians satisfy a weak secant equation. The spectral choice of steplength is embedded into the Hessian approximation and the whole process is combined with a nonmonotone line search strategy. The simple bounds are then taken into account by placing them in an exponential penalty term that modifies the objective function. The exponential penalty scheme defines the outer iterations of the process. Each outer iteration involves the application of the previously defined preconditioned spectral gradient method for linear equality constrained problems. Therefore, an equality constrained convex quadratic programming problem needs to be solved at every inner iteration. The associated extended KKT matrix remains constant unless the process is reinitiated. In ordinary inner iterations, only the right-hand side of the KKT system changes. Therefore, suitable sparse factorization techniques can be applied and exploited effectively. Encouraging numerical experiments are presented.
Similar content being viewed by others
References
J. Barzilai J. M. Borwein (1988) ArticleTitle Two-Point Stepsize Gradient Methods IMA Journal of Numerical Analysis 8 141–148
M. Raydan (1993) ArticleTitle On the Barzilai and Borwein Choice of Steplength for the Gradient Method IMA Journal of Numerical Analysis 13 321–326
E. G. Birgin I. Chambouleyron J. M. Martínez (1999) ArticleTitle Estimation of Optical Constants of Thin Films Using Unconstrained Optimization Journal of Computational Physics 151 862–880 Occurrence Handle10.1006/jcph.1999.6224 Occurrence Handle1:CAS:528:DyaK1MXjtVKksbg%3D
E. G. Birgin Y. G. Evtushenko (1998) ArticleTitle Automatic Differentiation and Spectral Projected Gradient Methods for Optimal Control Problems Optimization Methods and Software 10 125–146 Occurrence HandleMR1754734
M. Raydan (1997) ArticleTitle The Barzilai and Borwein Gradient Method for the Large-Scale Unconstrained Minimization Problem SIAM Journal on Optimization 7 26–33 Occurrence Handle10.1137/S1052623494266365
R. H. Bielschowsky A. Friedlander F. A. M. Gomes J. M. Martínez M. Raydan (1997) ArticleTitle An Adaptive Algorithm for Bound-Constrained Quadratic Minimization Investigación Operativa 7 67–102
A. Friedlander J. M. Martínez M. Raydan (1995) ArticleTitle A New Method for Large-Scale Box Constrained Convex Quadratic Minimization Problems Optimization Methods and Software 5 57–74
E. G. Birgin J. M. Martínez M. Raydan (2000) ArticleTitle Nonmonotone Spectral Projected Gradient Methods on Convex Sets SIAM Journal of Optimization 10 1196–1211 Occurrence Handle10.1137/S1052623497330963
F. Luengo W. Glunt T. L. Hayden M. Raydan (2002) ArticleTitle Preconditioned Spectral Gradient Method Numerical Algorithms 30 241–258 Occurrence Handle10.1023/A:1020181927999
B. Molina M. Raydan (1996) ArticleTitle The Preconditioned Barzilai-Borwein Method for the Numerical Solution of Partial Differential Equations Numerical Algorithms 13 45–60
L. Grippo F. Lampariello S. Lucidi (1986) ArticleTitle A Nonmonotone Line Search Technique for Newton’s Method SIAM Journal on Numerical Analysis 23 707–716 Occurrence Handle10.1137/0723046
E. G. Birgin J. M. Martínez M. Raydan (2003) ArticleTitle Inexact Spectral Projected Gradient Methods on Convex Sets IMA Journal of Numerical Analysis 23 539–559 Occurrence Handle10.1093/imanum/23.4.539
D. P. Bertsekas (1995) Nonlinear Programming Athena Scientific Belmont, Massachusetts
J. M. Martínez (200)) ArticleTitle BOX-QUACAN and the Implementation of Augmented Lagrangian Algorithms for Minimization with Inequality Constraints Computational and Applied Mathematics 19 31–56
F. H. Murphy (1974) ArticleTitle A Class of Exponential Penalty Functions SIAM Journal on Control 12 679–687 Occurrence Handle10.1137/0312052
Martínez, J. M., Pilotta, E. A., and Raydan, M., Spectral Gradient Methods for Linearly Constrained Optimization, Technical Report, Department of Applied Mathematics, University of Campinas, Campinas, SP, Brazil, 2004; see also www.ime.unicamp.br/~martinez.
I. S. Duff J. K. Reid (1082) MA27: A Set of Fortran Subroutines for Solving Sparse Symmetric Sets of Linear Equations Computer Science and System Division, AERE Harwell Oxford, England
N. I. M. Gould (1991) ArticleTitle An Algorithm for Large-Scale Quadratic Programming IMA Journal of Numerical Analysis 11 299–324
A. R. Conn N. I. M. Gould P. L. Toint (1991) ArticleTitle A Globally Convergent Augmented Lagrangian Algorithm for Optimization witn General Constraints and Simple Bounds SIAM Journal on Numerical Analysis 28 545–572 Occurrence Handle10.1137/0728030
A. R. 20 Conn N. I. M. Gould P. L. Toint (1992) Lancelot: A Fortran Package for Large-Scale Nonlinear Optimization (Release A) NumberInSeriesVol.17 Springer Verlag New York, NY
R. J. Vanderbei (1994) LOQO: An Interior-Point Code for Quadratic Programming Princeton University Princeton New Jersey
R. J. Vanderbei D. F. Shanno (1977) An Interior-Point Algorithm for Nonconvex Nonlinear Programming Statistics and Operations Research, Princeton University Princeton, New Jersey
N. Krejić J. M. Martínez M. P. Mello E. A. Pilotta (2000) ArticleTitle Validation of an Augmented Lagrangian Algorithm with a Gauss-Newton Hessian Approximation Using a Set of Hard-Spheres Problems Journal of Computational Optimization and Applications 16 247–263 Occurrence Handle10.1023/A:1008716329104
R. Fletcher (1987) Practical Methods of Optimization Wiley New York, NY
P. E. Gill W. Murray M. H. Wrigth (1981) Practical Optimization Academic Press London, England
Author information
Authors and Affiliations
Additional information
This research was supported by FAPESP Grant 2001-04597-4 and Grant 903724-6, FINEP and FAEP-UNICAMP, and the Scientific Computing Center of UCV. The authors thank two anonymous referees whose comments helped us to improve the final version of this paper.
Rights and permissions
About this article
Cite this article
Martínez, J.M., Pilotta, E.A. & Raydan, M. Spectral Gradient Methods for Linearly Constrained Optimization. J Optim Theory Appl 125, 629–651 (2005). https://doi.org/10.1007/s10957-005-2093-3
Issue Date:
DOI: https://doi.org/10.1007/s10957-005-2093-3