Advertisement

Generating Conjugate Directions Using Limited Second Derivatives

  • Andreas Griewank
  • Manfred Fruth
Conference paper
Part of the Progress in Systems and Control Theory book series (PSCT, volume 19)

Abstract

Gradients of objective functions in very many variables can be evaluated cheaply by the reverse, or adjoint, mode of automatic differentiation. One can simultaneously evaluate arbitrary Hessian-vector products with only a constant increase in complexity. These limited second derivatives can be used within a truncated Newton code or to improve nonlinear conjugate gradient codes with respect to the aspects: stepsize prediction, search direction conjugacy, restart criteria. We report preliminary results with an experimental conjugate gradient implementation.

Keywords

Automatic Differentiation Trial Point Established Point Truncate Newton Method Adjoint Code 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    C. Bischof, A. Carle, G. Corliss, A. Griewank, and P. Hovland; ADIFOR: Generating derivative codes from Fortran programs, Scient. Prog. 1, 1992, 11–29.Google Scholar
  2. [2]
    R. Fletcher and C. Reeves; Function minimization by conjugate gradients, Comp. J. 7, 1964, 149–154.MathSciNetMATHCrossRefGoogle Scholar
  3. [3]
    J. Gilbert and J. Nocedal; Global convergence properties of conjugate gradient methods for optimization, INRIA Rapport de Recherche n∘1268, Le Chesnay, 1991.Google Scholar
  4. [4]
    A. Griewank; Achieving logarithmic growth of temporal and spatial complexity in reverse automatic differentiation, Optim. Meth. Soft. 1, 1992), 35–54.CrossRefGoogle Scholar
  5. [5]
    A. Griewank, D. Juedes, and J. Utke, ADOL-C, a package for the automatic differentiation of algorithms written in C/C++, ACM Trans. Math. Soft., 1994, to appear.Google Scholar
  6. [6]
    E. Polak and G. Ribière; Note sur la convergence de méthods de directions conjuguées, Rev. Franç, d’Inform. Rech. Opér. 16, 1969, 35–43.Google Scholar
  7. [7]
    D. Shanno and K. Phua; Remark on algorithm 500: minimization of unconstrained multivariate functions, ACM Trans. Math. Soft. 6. 1980, 618–622.CrossRefGoogle Scholar
  8. [8]
    O. Talagrand and P. Courtier; Variational assimilation of meteorological observations with the adjoint vorticity equation. I: Theory, Q.J.R. Meteor. Soc. 113, 1987, 1311–1328.CrossRefGoogle Scholar
  9. [9]
    A. Griewank and P. Toint; On the unconstrained optimization of partially separable objective functions, in Nonlinear Optimization 1981 Ed. by M. Powell), Academic, London, 1981, 301–312.Google Scholar
  10. [10]
    P. Wolfe; Convergence conditions for ascent methods, SIAM Review 11, 1969, 226–235.MathSciNetMATHCrossRefGoogle Scholar
  11. [11]
    P. Wolfe; Convergence conditions for ascent methods II: some corrections, SIAM Review 13, 1971, 185–188.MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 1995

Authors and Affiliations

  • Andreas Griewank
    • 1
  • Manfred Fruth
    • 2
  1. 1.Institute of Scientific ComputingTechnical University DresdenGermany
  2. 2.Mathematics and Computer Science DivisionArgonne National LaboratoryUSA

Personalised recommendations