Abstract
Modern methods for numerical optimization calculate (or approximate) the matrix of second derivatives, the Hessian matrix, at each iteration. The recent axrival of robust software for automatic differentiation allows for the possibility of automatically computing the Hessian matrix, and the gradient, given a code to evaluate the objective function itself. However, for large-scale problems direct application of automatic differentiation may be unacceptably expensive. Recent work has shown that this cost can be dramatically reduced in the presence of sparsity. In this paper we show that forstructured problems it is possible to apply automatic differentiation tools in an economical way — even in the absence of sparsity in the Hessian.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Averick, B. M., Moré, J. J., Bischof, C. H., Carle, A., and Griewank, A. (1994). Computing large sparse Jacobian matrices using automatic differentiation.SIAM Journal on Scientific Computing, 15: 285–294.
Bischof, C. H., Bouaricha, A., Khademi, P. M., and Moré, J. J. (1995). Computing gradients in large-scale optimization using automatic differentiation. Preprint MCS-P488-0195, Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, Ill.
Bischof, C. H., Carle, A., Corliss, G. F., and Griewank, A. (1992). ADIFOR: Automatic differentiation in a source translation environment. In Wang, P. S., editor,Proceedings of the International Symposium on Symbolic and Algebraic Computation, pages 294–302, New York. ACM Press.
Broyden, C. (1965). A class of methods for solving nonlinear simultaneous equations.Mathematics of Computation, 19: 577–593.
Coleman, T. F. and Cai, J. Y. (1986). The cyclic coloring problem and estimation of sparse Hessian matrices.SIAM J. Alg. Disc. Meth., 7: 221–235.
Coleman, T. F. and Verma, A. (1995). The efficient computation of sparse Jacobian matrices using automatic differentiation. Tech. Report TR95-1557, Computer Science Department, Cornell University.
Coleman, T. F. and Verma, A. (1996). Structure and efficient Jacobian calculation. In Berz, M., Bischof, C., Corliss, G., and Griewank, A., editors,Computational Differentiation: Techniques, Applications, and Tools, pages 149–159. SIAM, Philadelphia, Penn.
Griewank, A., Juedes, D., and Utke, J. (1996). ADOL- C, a package for the automatic differentiation of algorithms written in C/C++.ACM Trans. Math. Software, 22 (2): 131–167.
Hossain, A. K. M. and Steihaug, T. (1995). Computing a sparse Jacobian matrix by rows and columns. Tech. Report 109, Department of Informatics, University of Bergen, Bergen.
matlab.MATLAB 4.2c for UNIX,The Mathworks, Inc., 24 Prime Park Way, Natick, Massachusetts 01760.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1998 Kluwer Academic Publishers
About this chapter
Cite this chapter
Coleman, T.F., Verma, A. (1998). Structure and Efficient Hessian Calculation. In: Yuan, Yx. (eds) Advances in Nonlinear Programming. Applied Optimization, vol 14. Springer, Boston, MA. https://doi.org/10.1007/978-1-4613-3335-7_3
Download citation
DOI: https://doi.org/10.1007/978-1-4613-3335-7_3
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4613-3337-1
Online ISBN: 978-1-4613-3335-7
eBook Packages: Springer Book Archive