Skip to main content
SpringerLink
Log in

Recent Developments in Algorithms and Software for Trust Region Methods

  • Chapter
Mathematical Programming The State of the Art

Abstract

Trust region methods are an important class of iterative methods for the solution of nonlinear optimization problems. Algorithms in this class have been proposed for the solution of systems of nonlinear equations, nonlinear estimation problems, unconstrained and constrained optimization, nondifferentiable optimization, and large scale optimization. Interest in trust region methods derives, in part, from the availability of strong convergence results and from the development of software for these methods which is reliable, efficient, and amazingly free of ad-hoc decisions. In this paper we survey the theoretical and practical results available for trust region methods and discuss the relevance of these results to the implementation of trust region methods.

Work supported in part by the Applied Mathematical Sciences Research Program (KC-04-02) of the Office of Energy Research of the U.S. Department of Energy under Contract W-31-109-Eng-38.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Anderson, D. H. and Osborne, M. R. [ 1977 ]. Discrete, nonlinear approximation problems in polyhedral norms, Numer. Math. 28, 157–170.

    Article  MathSciNet  MATH  Google Scholar 

  • Bjorck, A. and Elfving, T. [ 1979 ]. Accelerated projection methods for computing pseudoinverse solutions of systems of linear equations, BIT 19, 145–163.

    Article  MathSciNet  Google Scholar 

  • Broyden, C. G. [ 1965 ]. A class of methods for solving nonlinear simultaneous equations, Math. Comp. 19, 577–593.

    Article  MathSciNet  MATH  Google Scholar 

  • Buckley, A. and LeNir, A. [ 1982 ]. QN-like variable storage conjugate gradients, University of Arizona, Management and Information Systems Department, Report 2, Tucson, Arizona.

    Google Scholar 

  • Cline, A. K., Moler, C. B., Stewart, G. W. and Wilkinson, J. H. [ 1979 ]. An estimate for the condition number of a matrix, SIAM J. Numer. Anal. 16, 368–375.

    Article  MathSciNet  MATH  Google Scholar 

  • Davidon, W. C. [ 1980 ]. Conic approximations and collinear scalings for optimizers, SIAM J. Numer. Anal. 17, 268–281.

    Article  MathSciNet  MATH  Google Scholar 

  • Dembo, R. S. [ 1982 ]. Large scale nonlinear optimization, Nonlinear Optimization 1981, M. J. D. Powell, ed., Academic Press.

    Google Scholar 

  • Dembo, R. S., Eisenstat, S. C. and Steihaug, T. [ 1982 ]. Inexact Newton methods, SIAM J. Numer. Anal. 19, 400–408.

    Article  MathSciNet  MATH  Google Scholar 

  • Dembo, R. S. and Steihaug, T. [ 1980 ]. Truncated Newton algorithms for large scale unconstrained optimization, Yale University, School of Organization and Management, Report 48, New Haven, Connecticut.

    Google Scholar 

  • Dennis, J. E., Gay, D. M. and Welsch, R. E. [ 1981 ]. An adaptive nonlinear least squares algorithm, ACM Trans. Math. Software 7, 348–368.

    Article  MATH  Google Scholar 

  • Dennis, J. E. and Mei, H. H. W. [ 1979 ]. Two new unconstrained optimization algorithms which use function and gradient values, J. Optim. Theory Appl. 28, 453–482.

    Article  MathSciNet  MATH  Google Scholar 

  • Deuflhard, P. [ 1974 ]. A modified Newton method for the solution of illconditioned systems of nonlinear equations with application to multiple shooting, Numer. Math. 22, 289–315.

    Article  MathSciNet  MATH  Google Scholar 

  • Fletcher, R. [ 1971 ]. A modified Marquardt subroutine for nonlinear least squares, Atomic Energy Research Establishment, Report R6799, Harwell, England.

    Google Scholar 

  • Fletcher, R. [ 1972 ]. An algorithm for solving linearly constained optimization problems, Math. Programming 2, 133–165.

    Article  MathSciNet  MATH  Google Scholar 

  • Fletcher, R. [ 1980 ]. Practical Methods of Optimization, Volume 1: Unconstrained Optimization, John Wiley & Sons.

    MATH  Google Scholar 

  • Fletcher, R. [ 1981 ]. Practical Methods of Optimization, Volume 2: Constrained Optimization, John Wiley & Sons.

    MATH  Google Scholar 

  • Fletcher, R. [ 1982 ]. Methods for nonlinear constraints, Nonlinear Optimization 1981, M. J. D. Powell, ed., Academic Press.

    Google Scholar 

  • Fletcher, R. and Jackson, M. [ 1974 ]. Minimization of a quadratic function of many variables subject to only lower and upper bounds, J. Inst. Math. Appl. 14, 159–174.

    Google Scholar 

  • Gay, D. M. [ 1980 ]. Subroutines for unconstrained minimization using a model/trust region approach, Massachusetts Institute of Technology, Center for Computational Research in Economics and Management Report 18, Cambridge, Massachusetts.

    Google Scholar 

  • Gay, D. M. [ 1981 ]. Computing optimal locally constrained steps, SIAM J. Sci. Stat. Comput. 2, 186–197.

    Article  MathSciNet  MATH  Google Scholar 

  • Gay, D. M. [ 1982 ]. On convergence testing in model/trust region algorithms for unconstrained optimization, manuscript in preparation.

    Google Scholar 

  • Gill, P. E. and Murray, W. [ 1979 ]. Conjugate-gradient methods for large-scale nonlinear optimization, Stanford University, Systems Optimization Laboratory, Report SOL 79–15, Stanford, California.

    Google Scholar 

  • Goldfeld, S., Quandt, R. and Trotter, H. [ 1966 ]. Maximization by quadratic hill climbing, Econometrica 34, 541–551.

    Article  MathSciNet  MATH  Google Scholar 

  • Griffith, R. E. and Stewart, R. A. [ 1961 ]. A nonlinear programming technique for the optimization of continuous processing systems, Management Sci. 7, 379–392.

    Article  MathSciNet  MATH  Google Scholar 

  • Hanson, R. J. and Hiebert, K. L. [ 1981 ]. A sparse linear programming subprogram, Sandia Laboratories, Report Sand 81 - 0297, Albuquerque, New Mexico.

    Google Scholar 

  • Hebden, M. D. [ 1973 ]. An algorithm for minimization using exact second derivatives, Atomic Energy Research Establishment, Report T. P. 515, Harwell, England.

    Google Scholar 

  • Hestenes, M. R. and Steifel, E. [ 1952 ]. Methods of conjugate gradients for solving linear systems, J. Res. Bur. Standards 49, 409–436.

    MATH  Google Scholar 

  • Levenberg, K. [ 1944 ]. A method for the solution of certain nonlinear problems in least squares, Quart. Appl. Math. 2, 164–168.

    MathSciNet  MATH  Google Scholar 

  • Madsen, K. [ 1975 ]. An algorithm for minimax solution of overdetermined systems of nonlinear equations, J. Inst. Math. Appl. 16, 321–328.

    Article  MathSciNet  MATH  Google Scholar 

  • Marquardt, D. W. [ 1963 ]. An algorithm for least squares estimation of nonlinear parameters, SIAM J. Appl. Math. 11, 431–441.

    Article  MathSciNet  MATH  Google Scholar 

  • Moré, J. J. [ 1978 ]. The Levenberg-Marquardt algorithm: Implementation and theory, Proceedings of the Dundee Conference on Numerical Analysis, G. A. Watson, ed., Springer-Verlag.

    Google Scholar 

  • Moré, J. J. [ 1982 ]. Notes on optimization software, Nonlinear Optimization 1981, M. J. D. Powell, ed., Academic Press.

    Google Scholar 

  • Moré, J. J., Garbow, B. S. and Hillstrom, K. E. [ 1980 ]. User guide for MINPACK-1, Argonne National Laboratory, Report ANL-80-74, Argonne, Illinois.

    Google Scholar 

  • Moré, J. J. and Sorensen, D. C. [ 1981 ]. Computing a trust region step, Argonne National Laboratory, Report ANL-81-83, Argonne, Illinois.

    Google Scholar 

  • Munksgaard, N. [ 1980 ]. Solving sparse symmetric sets of linear equations by preconditioned conjugate gradients, ACM Trans. Math. Software 6, 206–219.

    Article  MATH  Google Scholar 

  • Nazareth, L. and Nocedal, J. [ 1982 ]. Conjugate direction methods with variable storage, Math. Programming 23, 326–340.

    Article  MathSciNet  MATH  Google Scholar 

  • Paige, C. C. and Saunders, M. A. [ 1975 ]. Solution of sparse indefinite systems of linear equations, SIAM J. Numer. Anal. 12, 617–629.

    Article  MathSciNet  MATH  Google Scholar 

  • Paige, C. C. and Saunders, M. A. [ 1978 ]. A bidiagonalization algorithm for sparse linear equations and least squares problems, Stanford University, Department of Operations Research, Report 19, Stanford, California.

    Google Scholar 

  • Paige, C. C. and Saunders, M. A. [ 1982 ]. LSQR: An algorithm for sparse linear equations and linear squares, ACM Trans. Math. Software 8, 43–71.

    Article  MathSciNet  MATH  Google Scholar 

  • Powell, M. J. D. [ 1970 a]. A hybrid method for nonlinear equations, Numerical Methods for Nonlinear Algebraic Equations, P. Rabinowitz, ed., Gordon and Breach.

    Google Scholar 

  • Powell, M. J. D. [ 1970 b]. A new algorithm for unconstrained optimization, Nonlinear Programming, J. B. Rosen, O. L. Mangasarian and K. Ritter, eds., Academic Press.

    Google Scholar 

  • Powell, M. J. D. [ 1975 ]. Convergence properties of a class of minimization algorithms, Nonlinear Programming 2, O. L. Mangasarian, R. R. Meyer and S. M. Robinson, eds., Academic Press.

    Google Scholar 

  • Powell, M. J. D. [ 1978 ]. A fast algorithm for nonlinearly constrained optimization calculations, Proceedings of the Dundee Conference on Numerical Analysis, G. A. Watson, ed., Springer-Verlag.

    Google Scholar 

  • Reinsch, C. H. [ 1971 ]. Smoothing by spline functions II, Numer. Math. 16, 451–454.

    Article  MathSciNet  Google Scholar 

  • Saunders, M. A. [ 1979 ]. Sparse least squares by conjugate gradients: A comparison of preconditioning methods, Stanford University, Department of Operations Research, Report 5, Stanford, California.

    Google Scholar 

  • Schultz, G. A., Schnabel, R. S. and Byrd, R. H. [ 1982 ]. A family of trust region based algorithms for unconstrained minimization with strong global convergence properties, University of Colorado, Department of Computer Science, Report 216, Boulder, Colorado.

    Google Scholar 

  • Sorensen, D. C. [1982 a]. Newton’s method with a model trust region modification, SIAM J. Numer. Anal. 19, 409–426.

    Google Scholar 

  • Sorensen, D. C. [ 1982 b]. Trust region methods for unconstrained optimization, Nonlinear Optimization 1981, M. J. D. Powell, ed., Academic Press.

    Google Scholar 

  • Steihaug, T. [ 1981 ]. The conjugate gradient method and trust regions in large scale optimization, Rice University, Department of Mathematical Sciences, Report 1, Houston, Texas.

    Google Scholar 

  • Thomas, S. W. [ 1975 ]. Sequential estimation techniques for quasi-Newton algorithms, Ph. D. dissertation, Cornell University, Ithaca, New York.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mathematics and Computer Science Division, Argonne Nat. Lab., 9700 S Cass Avenue, Argonne, IL, 60439, USA

    J. J. Moré

Authors
  1. J. J. Moré
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institut für Ökonometrie und Operations Research, Universität Bonn, Nassestraße 2, D-5300, Bonn 1, Germany

    Achim Bachem  & Bernhard Korte  & 

  2. Lehrstuhl für Angewandte Mathematik II, Universität Augsburg, Memminger Straße 6, D-8900, Augsburg, Germany

    Martin Grötschel

Rights and permissions

Reprints and permissions

Copyright information

© 1983 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Moré, J.J. (1983). Recent Developments in Algorithms and Software for Trust Region Methods. In: Bachem, A., Korte, B., Grötschel, M. (eds) Mathematical Programming The State of the Art. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-68874-4_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-68874-4_11

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-68876-8

  • Online ISBN: 978-3-642-68874-4

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation