Abstract
We discuss the design, implementation and performance of algorithms suitable for the efficient computation of sparse Jacobian and Hessian matrices using Automatic Differentiation via operator overloading on multicore architectures. The procedure for exploiting sparsity (for runtime and memory efficiency) in serial computation involves a number of steps. Using nonlinear optimization problems as test cases, we show that the algorithms involved in the various steps can be adapted to multithreaded computations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bücker, H.M., Rasch, A., Vehreschild, A.: Automatic generation of parallel code for Hessian computations. In: M.S. Mueller, B.M. Chapman, B.R. de Supinski, A.D. Malony, M. Voss (eds.) OpenMP Shared Memory Parallel Programming, Proceedings of the International Workshops IWOMP 2005 and IWOMP 2006, Eugene, OR, USA, June 1–4, 2005, and Reims, France, June 12–15, 2006, Lecture Notes in Computer Science, vol. 4315, pp. 372–381. Springer, Berlin / Heidelberg (2008). DOI 10.1007/978-3-540-68555-5_30
Bücker, H.M., Rasch, A., Wolf, A.: A class of OpenMP applications involving nested parallelism. In: Proceedings of the 19th ACM Symposium on Applied Computing, Nicosia, Cyprus, March 14–17, 2004, vol. 1, pp. 220–224. ACM Press, New York (2004). DOI 10.1145/967900.967948. URL http://doi.acm.org/10.1145/967900.967948
Conforti, D., Luca, L.D., Grandinetti, L., Musmanno, R.: A parallel implementation of automatic differentiation for partially separable functions using PVM. Parallel Computing 22, 643–656 (1996)
Fischer, H.: Automatic differentiation: Parallel computation of function, gradient and Hessian matrix. Parallel Computing 13, 101–110 (1990)
Gebremedhin, A.H., Manne, F., Pothen, A.: What color is your Jacobian? Graph coloring for computing derivatives. SIAM Review 47(4), 629–705 (2005)
Gebremedhin, A.H., Nguyen, D., Patwary, M., Pothen, A.: ColPack: Software for graph coloring and related problems in scientific computing. Tech. rep., Purdue University (2011)
Gebremedhin, A.H., Tarafdar, A., Manne, F., Pothen, A.: New acyclic and star coloring algorithms with applications to Hessian computation. SIAM J. Sci. Comput. 29, 1042–1072 (2007)
Kowarz, A.: Advanced concepts for Automatic Differentiation based on operator overloading (1998). PhD Thesis, TU Dresden
Kowarz, A., Walther, A.: Parallel derivative computation using ADOL-C. In: W. Nagel, R. Hoffmann, A. Koch (eds.) Proceedings of PASA 2008, Lecture Notes in Informatics, Vol. 124, pp. 83–92. Gesellschaft fr Informatik (2008)
Maurer, H., Mittelmann, H.: Optimization techniques for solving elliptic control problems with control and state constraints. II: Distributed control. Comput. Optim. Appl. 18(2), 141–160 (2001)
Utke, J., Hascoët, L., Heimbach, P., Hill, C., Hovland, P., Naumann, U.: Toward adjoinable MPI. In: Proceedings of the 10th IEEE International Workshop on Parallel and Distributed Scientific and Engineering, PDSEC-09 (2009). DOI http://doi.ieeecomputersociety. org/10.1109/IPDPS.2009.5161165
Wächter, A., Biegler, L.: On the implementation of a Primal-Dual Interior Point Filter Line Search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2006)
Walther, A., Griewank, A.: Getting started with ADOL-C. In: U. Naumann, O. Schenk (eds.) Combinatorial Scientific Computing. Chapman-Hall (2012). see also http://www.coin-or.org/projects/ADOL-C.xml
Acknowledgements
We thank the anonymous referees for their helpful comments. The experiments were performed on a computing facility hosted by the Paderborn Center for Parallel Computing (PC 2). The research is supported in part by the U.S. Department of Energy through the CSCAPES Institute grant DE-FC02-08ER25864 and by the U.S. National Science Foundation through grant CCF-0830645.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Letschert, B., Kulshreshtha, K., Walther, A., Nguyen, D., Gebremedhin, A., Pothen, A. (2012). Exploiting Sparsity in Automatic Differentiation on Multicore Architectures. In: Forth, S., Hovland, P., Phipps, E., Utke, J., Walther, A. (eds) Recent Advances in Algorithmic Differentiation. Lecture Notes in Computational Science and Engineering, vol 87. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30023-3_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-30023-3_14
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-30022-6
Online ISBN: 978-3-642-30023-3
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)