Recomputations in Reverse Mode AD

Abstract

The main challenge of the reverse (or adjoint) mode of automatic differentiation (AD) is providing the accurate values of required variables to the derivative code. We discuss different strategies to tackle this challenge. The ability to generate efficient adjoint code is crucial for handling large scale applications. For challenging applications, efficient adjoint code must provide at least a fraction of the values of required variables through recomputations, but it is essential to avoid unnecessary recomputations. This is achieved by the Efficient Recomputation Algorithm implemented in the Tangent linear and Adjoint Model Compiler and in Transformation of Algorithms in Fortran, which are source-to-source translation AD tools for Fortran programs. We describe the algorithm and discuss possible improvements.