Prospects for Simulated Annealing Algorithms in Automatic Differentiation

  • Uwe Naumann
  • Peter Gottschling
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2264)


We present new ideas on how to make simulated annealing applicable to the combinatorial optimization problem of accumulating a Jacobian matrix of a given vector function using the minimal number of arithmetic operations. Building on vertex elimination in computational graphs we describe how simulated annealing can be used to find good approximations to the solution of this problem at a reasonable cost.


Accumulation of Jacobian matrices vertex elimination simulated annealing logarithmic cooling schedule inhomogeneous Markov chains 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    B. Averik and R. Carter and J. More, The Minpack-2 Test Problem Collection (Preliminary Version), Mathematical and Computer Science Division, Argonne National Laboratory, Technical Report 150, 1991.Google Scholar
  2. 2.
    A. Aho, R. Sethi, and J. Ullman, Compilers. Principles, Techniques, and Tools, Addison Wesley, 1986.Google Scholar
  3. 3.
    M. Berz, C. Bischof, G. Corliss, and A. Griewank, eds., Computational Differentiation: Techniques, Applications, and Tools, SIAM, 1996.Google Scholar
  4. 4.
    C. Bischof and A. Carle and P. Khademi and A. Maurer, The ADIFOR 2.0 System for Automatic Differentiation of Fortran 77 Programs, IEEE Comp. Sci. & Eng., 3, 3, 1996, 18–32.CrossRefGoogle Scholar
  5. 5.
    C. Bischof and M. Haghighat, Hierarchical Approaches to Automatic Differentiation, in [3], 83–94.Google Scholar
  6. 6.
    G. Corliss and C. Faure and A. Griewank and L. Hascoet and U. Naumann, eds., Automatic Differentiation-From Simulation to Optimization, LNCS, Springer, 2001, to appear.Google Scholar
  7. 7.
    G. Corliss and A. Griewank, eds., Automatic Differentiation: Theory, Implementation, and Application, SIAM, 1991.Google Scholar
  8. 8.
    A. Curtis, M. Powell and J. Reid, On the Estimation of Sparse Jacobian Matrices, J. Inst. Math. Appl., 13, 1974, 117–94.zbMATHGoogle Scholar
  9. 9.
    M. Garey and D. Johnson, Computers and Intractability-A Guide to the Theory of NP-completeness, W. H. Freeman and Company, 1979.Google Scholar
  10. 10.
    A. Griewank, Evaluating Derivatives, Principles and Techniques of Algorithmic Differentiation, Frontiers in Appl. Math., 19, SIAM, 2000.Google Scholar
  11. 11.
    A. Griewank and S. Reese, On the Calculation of Jacobian Matrices by the Markowitz Rule, in [7], 126–135.Google Scholar
  12. 12.
    B. Hajek,Cooling Schedules for Optimal Annealing, Mathem. Oper. Res., 13, 1998, 311–329.Google Scholar
  13. 13.
    K. Herley, Presentation at: Theory Institute on Combinatorial Challenges in Computational Differentiation, MCSD, Argonne National Laboratory, 1993.Google Scholar
  14. 14.
    R. Jessani and M. Putrino, Comparison of Single-and Dual-Pass Multiply-Add Fused Floating-Point Units, IEEE Transactions on Computers, 47, 9, 1998.Google Scholar
  15. 15.
    S. Lin, Bell System Technical Journal, 44, 1965, 2245–2269.zbMATHMathSciNetGoogle Scholar
  16. 16.
    W. Miller and C. Wrathall, Software for Roundoff Analysis of Matrix Algorithms, Academic Press, 1980.Google Scholar
  17. 17.
    N. Metropolis and A.W. Rosenbluth and M.N. Rosenbluth and A.H. Teller and E. Teller, Equation of State Calculations by Fast Computing Machines, J. Chem. Phys., 21, 1953, 1087–92.CrossRefGoogle Scholar
  18. 18.
    U. Naumann and B. Christianson, Differentiation Enabled Compiler Technology, MOD/DERA/EPSRC grant, GR/R55252.Google Scholar
  19. 19.
    U. Naumann, Efficient Calculation of Jacobian Matrices by Optimized Application of the Chain Rule to Computational Graphs Ph.D. thesis, Technical University Dresden, 1999.Google Scholar
  20. 20.
    U. Naumann, Optimized Jacobian Accumulation Techniques, in N. Mastorakis, Ed., Problems in Modern Applied Mathematics, WSES Press, 2000.Google Scholar
  21. 21.
    U. Naumann, An Enhanced Markowitz Rule for Accumulating Jacobians Efficiently, Proceedings of Algorithmy 2000, Slovak University of Technology, 2000.Google Scholar
  22. 22.
    U. Naumann, Elimination Techniques for Cheap Jacobians, in [6].Google Scholar
  23. 23.
    U. Naumann, Cheaper Jacobians by Simulated Annealing, to appear in SIAM J. Opt.Google Scholar
  24. 24.
    U. Naumann, Elimination Methods for Computational Graphs. On the Optimal Accumulation of Jacobian Matrices-Conceptual Framework., submitted to Math. Prog.Google Scholar
  25. 25.
    G. Newsam and J. Ramsdell, Estimation of Sparse Jacobian Matrices, SIAM J. Alg. Dis. Meth., 4, 1983, 404–417.zbMATHCrossRefMathSciNetGoogle Scholar
  26. 26.
    U. Naumann, S. A. Forth, M. Tadjouddine, and J. D. Pryce, A Standard Interface for Elimination Sequences in Automatic Differentiation, AMOR Report 2001/03, Cranfield Univ. (RMCS Shrivenham), 2001.Google Scholar
  27. 27.
    D. J. Rose and R. E. Tarjan, Algorithmic Aspects of Vertex Elimination on Directed Graphs, SIAM J. Appl. Math., 34, 1, 1978, 176–197.zbMATHCrossRefMathSciNetGoogle Scholar
  28. 28.
    M. Tadjouddine, S. A. Forth, and J. D. Pryce, AD Tools and Prospects for Optimal AD in CFD Flux Jacobian Calculations, in [6].Google Scholar
  29. 29.
    P. Van Laarhoven and E. Aarts, Simulated Annealing: Theory and Applications, Reidel, 1988.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Uwe Naumann
    • 1
  • Peter Gottschling
    • 1
  1. 1.Department of Computer ScienceUniversity of HertfordshireHatfieldUK

Personalised recommendations