Inversion Based on Computational Simulations

  • K. M. Hanson
  • G. S. Cunningham
  • S. S. Saquib
Part of the Fundamental Theories of Physics book series (FTPH, volume 98)


A standard approach to solving inversion problems that involve many parameters uses gradient-based optimization to find the parameters that best match the data. We will discuss enabling techniques that facilitate application of this approach to large-scale computational simulations, which are the only way to investigate many complex physical phenomena. Such simulations may not seem to lend themselves to calculation of the gradient with respect to numerous parameters. However, adjoint differentiation allows one to efficiently compute the gradient of an objective function with respect to all the variables of a simulation. When combined with advanced gradient-based optimization algorithms, adjoint differentiation permits one to solve very large problems of optimization or parameter estimation. These techniques will be illustrated through the simulation of the time-dependent diffusion of infrared light through tissue, which has been used to perform optical tomography. The techniques discussed have a wide range of applicability to modeling including the optimization of models to achieve a desired design goal.

Key words

simulation inversion reconstruction adjoint differentiation sensitivity analysis optimization model validation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    S. S. Saquib, K. M. Hanson, and G. S. Cunningham, “Model-based image reconstruction from time-resolved diffusion data,” in Medical Imaging: Image Processing, K. M. Hanson, ed., Proc. SPIE, 3034, pp. 369–380, 1997.Google Scholar
  2. 2.
    G. Burgers, R. Giering, and M. Fischer, “Construction of the adjoint of the HOPE OGCM,” Ann. Geophysicae., C14, p. 390, 1996.Google Scholar
  3. 3.
    F. C. Vossepoel, G. Burgers, and P. J. vanLeeuwen, “Radar altimeter data assimilation in an intermediate ENSO model,” Ann. Geophysicae., C14, p. 429, 1996.Google Scholar
  4. 4.
    W. C. Thacker, “Automatic differentiation from an oceanographer’s perspective,” in Automatic Differentiation of Algorithms: Theory, Implementation, and Application, A. Griewank and G. F. Corliss, eds., pp. 191–201, SIAM, Philadelphia, 1991.Google Scholar
  5. 5.
    K. M. Hanson and G. S. Cunningham, “A computational approach to Bayesian inference,” in Computing Science and Statistics 27, M. M. Meyer and J. L. Rosenberger, eds., pp. 202–211, Interface Foundation, Fairfax Station, VA 22039–7460, 1996.Google Scholar
  6. 6.
    R. J. Henninger, P. J. Maudlin, and E. N. Harstad, “Differential sensitivity theory applied to the MESA2D code for multi-material problems,” in Shock Compression of Condensed Matter, S. C. Schmidt, et al., ed., pp. 283–287, Amer. Inst. Physics, Woodbury, NY, 1996.Google Scholar
  7. 7.
    R. J. Henninger, P. J. Maudlin, and M. L. Rightley, “Accuracy of differential sensitivity for one-dimensional shock problems,” in Shock Compression of Condensed Matter, S. C. Schmidt, et al., ed., Amer. Inst. Physics, Woodbury, NY, 1998 (to be published).Google Scholar
  8. 8.
    M. L. J. Rightley, R. J. Henninger, and K. M. Hanson, “Adjoint differentiation of hydrodynamic codes,” in CNLS Newsletter, Center for Nonlinear Studies, Los Alamos National Laboratory, Nov., 1997 (WWW:
  9. 9.
    X. L. Battle, G. S. Cunningham, and K. M. Hanson, “3D tomographic reconstruction using geometrical models,” in Medical Imaging: Image Processing, K. M. Hanson, ed., Proc. SPIE, 3034, pp. 346–357, 1997.Google Scholar
  10. 10.
    D. A. Harville, Matrix Algebra from a Statistician’s Perspective, Springer, New York, 1997.zbMATHCrossRefGoogle Scholar
  11. 11.
    C. Bischof, A. Carle, P. Khademi, and A. Mauer, “The ADIFOR 2.0 system for the automatic differentiation of FORTRAN 77 programs,” Tech. Rep. ANL-MCS-P481-1194, Argonne National Laboratory, 1995.Google Scholar
  12. 12.
    J. E. Horwedel, E. M. Oblow, B. A. Worley, and F. G. Pin, “GRESS 3.0 Gradient Enhanced Software System,” Tech. Rep. PSR-231, RSIC Peripheral Shielding Routine Collection, Oak Ridge National Laboratory, 1994.Google Scholar
  13. 13.
    R. Giering, “Tangent linear and adjoint model compiler,” Tech. Rep. TAMC 4.7, Max-Planck-Institut fuer Meteorologie, 1997 (e-mail: Scholar
  14. 14.
    W. C. Davidon, “Variable metric method for minimization,” Tech. Rep. ANL-5990, Argonne National Laboratory, 1966.Google Scholar
  15. 15.
    R. Fletcher, “A new approach to variable metric algorithms,” Computer Journal, 13, pp. 317–322, 1970.zbMATHCrossRefGoogle Scholar
  16. 16.
    D. C. Liu and J. Nocedal, “On the limited memory BFGS methods for large scale optimization,” Math. Programming, 45, pp. 503–528, 1989.MathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    S. G. Nash, “Preconditioning of truncated-Newton methods,” SIAM J. Sci. Stat. Gomput., 6, pp. 599–616, 1985.MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    S. G. Nash and J. Nocedal, “A numerical study of the limited memory BFGS method and the truncated-Newton method for large scale optimization,” Tech. Rep. NAM 02, Northwestern University–Dep’t. Elec. Eng. and Comp. Sci., 1989.Google Scholar
  19. 19.
    C. A. Bouman and K. Sauer, “A generalized Gaussian image model for edge-preserving MAP estimation,” IEEE Trans, on Image Proc., 2, pp. 296–310, 1993.CrossRefGoogle Scholar
  20. 20.
    S. S. Saquib, C. A. Bouman, and K. Sauer, “ML parameter estimation for Markov random fields, with applications to Bayesian tomography,” Tech. Rep. TR-ECE 95-24, School of Electrical and Computer Engineering, Purdue University, 1995.Google Scholar
  21. 21.
    M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour models,” Inter. J. Comp. Vision, 1, pp. 321–331, 1988.CrossRefGoogle Scholar
  22. 22.
    G. S. Cunningham, I. Koyfman, and K. M. Hanson, “Improved convergence of gradient-based reconstructions using multi-scale models,” in Image Processing, M. H. Loew and K. M. Hanson, eds., Proc. SPIE, 2710, pp. 145–155, 1996.Google Scholar
  23. 23.
    K. M. Hanson, G. S. Cunningham, and R. J. McKee, “Uncertainty assessment for reconstructions based on deformable models,” Int. J. Imaging Systems and Technology 8, pp 506–512, 1997.CrossRefGoogle Scholar
  24. 24.
    G. S. Cunningham, K. M. Hanson, and X. L. Battle, “Three-dimensional reconstructions from low-count SPECT data using deformable models,” Opt. Express, 1998 (to be published).Google Scholar
  25. 25.
    K. M. Hanson, R. L. Bilisoly, and G. S. Cunningham, “Kinky tomographic reconstruction,” in Image Processing, M. H. Loew and K. M. Hanson, eds., Proc. SPIE, 2710, pp. 156–166, 1996.Google Scholar
  26. 26.
    J. Besag, P. Green, D. Higdon, and K. Mengersen, “Bayesian computation and stochastic systems,” Stat. Sci., 10, pp. 3–66, 1995.MathSciNetzbMATHCrossRefGoogle Scholar
  27. 27.
    W. R. Gilks, S. Richardson, and D. J. Spiegelhalter, Markov Chain Monte Carlo in Practice, Chapman and Hall, London, 1996.zbMATHGoogle Scholar
  28. 28.
    R. M. Neal, Bayesian Learning for Neural Networks, Springer, New York, 1996.zbMATHCrossRefGoogle Scholar
  29. 29.
    J. Skilling, D. R. T. Robinson, and S. F. Gull, “Probabilistic displays,” in Maximum Entropy and Bayesian Methods, W. T. Grandy, Jr. and L. H. Shick, eds., pp. 365–368, Kluwer Academic, Dordrecht, 1991.CrossRefGoogle Scholar
  30. 30.
    A. H. Hielscher, S. S. Saquib, and K. M. Hanson, “Model-based iterative image reconstruction for photon migration tomography,” in Methods for Solving Ill-Posed Inverse Imaging Problems: Medical and Nonmedical Applications, R. L. Barbour, M. J. Carvlin, and M. A. Fiddy, eds., Proc. SPIE, 3171, pp. 106–117, 1997.Google Scholar
  31. 31.
    A. H. Hielscher, A. Klose, D. M. Catarious, Jr., and K. M. Hanson, “Tomographic imaging of breast and brain tissue by time-resolved, model-based, iterative image reconstruction,” in Proc. Conf. on Advances in Optical Imaging and Photon Migration, Optical Society of America, Orlando, 1998 (to be published).Google Scholar
  32. 32.
    K. M. Hanson and G. S. Cunningham, “Posterior sampling with improved efficiency,” in Medical Imaging: Image Processing, K. M. Hanson, ed., Proc. SPIE, 3338, 1998 (to be published).Google Scholar
  33. 33.
    R. J. Henninger, P. J. Maudlin, M. L. Rightley, and K. M. Hanson, “Application of forward and adjoint techniques to hydrocode sensitivity analysis,” in Proc. 9th Nuclear Explosives Code Developers’ Conference, F. Graviana, et al., ed., Lawrence Livermore National Laboratory, 1996 (proceedings are classified; article available from authors as report LA-CP-96–235.Google Scholar
  34. 34.
    W. L. Oberkampf, M. N. Sindir, and A. T. Conlisk, “Guide for the verification and validation of computational fluid dynamics simulations,” Tech. Rep. G-077–98, Amer. Inst. Aeronautics and Astronautics, 1998.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 1998

Authors and Affiliations

  • K. M. Hanson
    • 1
    • 2
  • G. S. Cunningham
    • 1
    • 2
  • S. S. Saquib
    • 1
    • 2
  1. 1.Los Alamos National LaboratoryUSA
  2. 2.Los AlamosUSA

Personalised recommendations