Abstract
Every numerical function evaluation can be represented as a directed acyclic graph (DAG), beginning at the initial input variable settings, and terminating at the output or corresponding function value(s). The “reverse mode” of automatic differentiation (AD) generates a “tape” which is a representation of this underlying DAG. In this work we illustrate that a directed edge separator in this underlying DAG can yield space and time efficiency gains in the application of AD. Use of directed edge separators to increase AD efficiency in different ways than proposed here has been suggested by other authors (Bischof and Haghighat, Hierarchical approaches to automatic differentiation. In: Berz M, Bischof C, Corliss G, Griewank A (eds) Computational differentiation: techniques, applications, and tools, SIAM, Philadelphia, PA, pp 83–94, 1996; Bücker and Rasch, ACM Trans Math Softw 29(4):440–457, 2003). In contrast to these previous works, our focus here is primarily on space. Furthermore, we explore two simple algorithms to find good directed edge separators, and show how these ideas can be applied recursively to great advantage. Initial numerical experiments are presented.
This work was supported in part by Ophelia Lazaridis University Research Chair (held by Thomas F. Coleman), the National Sciences and Engineering Research Concil of Canada and the Natural Science Foundation of China (Project No: 11101310).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bischof, C.H., Bouaricha, A., Khademi, P., Moré, J.J.: Computing gradients in large-scale optimization using automatic differentiation. INFORMS J. Computing 9, 185–194 (1997)
Bischof, C.H., Haghighat, M.R.: Hierarchical approaches to automatic differentiation. In: M. Berz, C. Bischof, G. Corliss, A. Griewank (eds.) Computational Differentiation: Techniques, Applications, and Tools, pp. 83–94. SIAM, Philadelphia, PA (1996)
Bischof, C.H., Khademi, P.M., Bouaricha, A., Carle, A.: Efficient computation of gradients and Jacobians by dynamic exploitation of sparsity in automatic differentiation. Optimization Methods and Software 7, 1–39 (1997)
Bücker, H.M., Rasch, A.: Modeling the performance of interface contraction. ACM Transactions on Mathematical Software 29(4), 440–457 (2003). DOI http://doi.acm.org/10.1145/962437.962442
Cayuga Research Associates, L.: ADMAT-2.0 Users Guide (2009). URL http://www.cayugaresearch.com/
Coleman, T.F., Jonsson, G.F.: The efficient computation of structured gradients using automatic differentiation. SIAM Journal on Scientific Computing 20(4), 1430–1437 (1999). DOI 10.1137/S1064827597320794
Coleman, T.F., Santosa, F., Verma, A.: Efficient calculation of Jacobian and adjoint vector products in wave propagational inverse problem using automatic differentiation. J. Comp. Phys. 157, 234–255 (2000)
Coleman, T.F., Verma, A.: Structure and efficient Jacobian calculation. In: M. Berz, C. Bischof, G. Corliss, A. Griewank (eds.) Computational Differentiation: Techniques, Applications, and Tools, pp. 149–159. SIAM, Philadelphia, PA (1996)
Coleman, T.F., Verma, A.: The efficient computation of sparse Jacobian matrices using automatic differentiation. SIAM J. Sci. Comput. 19(4), 1210–1233 (1998). DOI 10.1137/S1064827595295349. URL http://link.aip.org/link/?SCE/19/1210/1
Coleman, T.F., Verma, A.: Structure and efficient Hessian calculation. In: Y. Yuan (ed.) Proceedings of the 1996 International Conference on Nonlinear Programming, pp. 57–72. Kluwer Academic Publishers (1998)
Coleman, T.F., Xiong, X.: New graph approaches to the determination of Jacobian and Hessian matrices, and Newton steps, via automatic differentiation (in preparation)
Coleman, T.F., Xu, W.: Fast (structured) Newton computations. SIAM Journal on Scientific Computing 31(2), 1175–1191 (2008). DOI 10.1137/070701005. URL http://link.aip.org/link/?SCE/31/1175/1
Ford, L., Fulkerson, D.: Maximal flow through a network. Canadian Journal of Mathematics 8, 399–404 (1956)
Griewank, A.: Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. No. 19 in Frontiers in Appl. Math. SIAM, Philadelphia, PA (2000)
Rall, L.B.: Automatic Differentiation: Techniques and Applications, Lecture Notes in Computer Science, vol. 120. Springer, Berlin (1981). DOI 10.1007/3-540-10861-0
Tadjouddine, E.M.: Vertex-ordering algorithms for automatic differentiation of computer codes. The Computer Journal 51(6), 688–699 (2008). DOI 10.1093/comjnl/bxm115. URL http://comjnl.oxfordjournals.org/cgi/content/abstract/51/6/688
Xu, W., Coleman, T.F.: Efficient (partial) determination of derivative matrices via automatic differentiation (to appear in SIAM journal on Scientific Computing, 2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Coleman, T.F., Xiong, X., Xu, W. (2012). Using Directed Edge Separators to Increase Efficiency in the Determination of Jacobian Matrices via Automatic Differentiation. In: Forth, S., Hovland, P., Phipps, E., Utke, J., Walther, A. (eds) Recent Advances in Algorithmic Differentiation. Lecture Notes in Computational Science and Engineering, vol 87. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30023-3_19
Download citation
DOI: https://doi.org/10.1007/978-3-642-30023-3_19
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-30022-6
Online ISBN: 978-3-642-30023-3
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)