Maximum Entropy and Bayesian Methods pp 121-135 | Cite as

# Inversion Based on Computational Simulations

## Abstract

A standard approach to solving inversion problems that involve many parameters uses gradient-based optimization to find the parameters that best match the data. We will discuss enabling techniques that facilitate application of this approach to large-scale computational simulations, which are the only way to investigate many complex physical phenomena. Such simulations may not seem to lend themselves to calculation of the gradient with respect to numerous parameters. However, adjoint differentiation allows one to efficiently compute the gradient of an objective function with respect to all the variables of a simulation. When combined with advanced gradient-based optimization algorithms, adjoint differentiation permits one to solve very large problems of optimization or parameter estimation. These techniques will be illustrated through the simulation of the time-dependent diffusion of infrared light through tissue, which has been used to perform optical tomography. The techniques discussed have a wide range of applicability to modeling including the optimization of models to achieve a desired design goal.

## Key words

simulation inversion reconstruction adjoint differentiation sensitivity analysis optimization model validation## Preview

Unable to display preview. Download preview PDF.

## References

- 1.S. S. Saquib, K. M. Hanson, and G. S. Cunningham, “Model-based image reconstruction from time-resolved diffusion data,” in
*Medical Imaging: Image Processing*, K. M. Hanson, ed.,*Proc. SPIE*,**3034**, pp. 369–380, 1997.Google Scholar - 2.G. Burgers, R. Giering, and M. Fischer, “Construction of the adjoint of the HOPE OGCM,”
*Ann. Geophysicae.*,**C14**, p. 390, 1996.Google Scholar - 3.F. C. Vossepoel, G. Burgers, and P. J. vanLeeuwen, “Radar altimeter data assimilation in an intermediate ENSO model,”
*Ann. Geophysicae.*,**C14**, p. 429, 1996.Google Scholar - 4.W. C. Thacker, “Automatic differentiation from an oceanographer’s perspective,” in
*Automatic Differentiation of Algorithms: Theory, Implementation, and Application*, A. Griewank and G. F. Corliss, eds., pp. 191–201, SIAM, Philadelphia, 1991.Google Scholar - 5.K. M. Hanson and G. S. Cunningham, “A computational approach to Bayesian inference,” in
*Computing Science and Statistics***27**, M. M. Meyer and J. L. Rosenberger, eds., pp. 202–211, Interface Foundation, Fairfax Station, VA 22039–7460, 1996.Google Scholar - 6.R. J. Henninger, P. J. Maudlin, and E. N. Harstad, “Differential sensitivity theory applied to the MESA2D code for multi-material problems,” in
*Shock Compression of Condensed Matter*, S. C. Schmidt, et al., ed., pp. 283–287, Amer. Inst. Physics, Woodbury, NY, 1996.Google Scholar - 7.R. J. Henninger, P. J. Maudlin, and M. L. Rightley, “Accuracy of differential sensitivity for one-dimensional shock problems,” in
*Shock Compression of Condensed Matter*, S. C. Schmidt, et al., ed., Amer. Inst. Physics, Woodbury, NY, 1998 (to be published).Google Scholar - 8.M. L. J. Rightley, R. J. Henninger, and K. M. Hanson, “Adjoint differentiation of hydrodynamic codes,” in
*CNLS Newsletter*, Center for Nonlinear Studies, Los Alamos National Laboratory, Nov., 1997 (WWW:http://cnls.lanl.gov/Publications/newsletters.html). - 9.X. L. Battle, G. S. Cunningham, and K. M. Hanson, “3D tomographic reconstruction using geometrical models,” in
*Medical Imaging: Image Processing*, K. M. Hanson, ed.,*Proc. SPIE*,**3034**, pp. 346–357, 1997.Google Scholar - 10.D. A. Harville,
*Matrix Algebra from a Statistician’s Perspective*, Springer, New York, 1997.zbMATHCrossRefGoogle Scholar - 11.C. Bischof, A. Carle, P. Khademi, and A. Mauer, “The ADIFOR 2.0 system for the automatic differentiation of FORTRAN 77 programs,” Tech. Rep. ANL-MCS-P481-1194, Argonne National Laboratory, 1995.Google Scholar
- 12.J. E. Horwedel, E. M. Oblow, B. A. Worley, and F. G. Pin, “GRESS 3.0 Gradient Enhanced Software System,” Tech. Rep. PSR-231, RSIC Peripheral Shielding Routine Collection, Oak Ridge National Laboratory, 1994.Google Scholar
- 13.R. Giering, “Tangent linear and adjoint model compiler,” Tech. Rep. TAMC 4.7, Max-Planck-Institut fuer Meteorologie, 1997 (e-mail: giering@dkrz.de)Google Scholar
- 14.W. C. Davidon, “Variable metric method for minimization,” Tech. Rep. ANL-5990, Argonne National Laboratory, 1966.Google Scholar
- 15.R. Fletcher, “A new approach to variable metric algorithms,”
*Computer Journal*,**13**, pp. 317–322, 1970.zbMATHCrossRefGoogle Scholar - 16.D. C. Liu and J. Nocedal, “On the limited memory BFGS methods for large scale optimization,”
*Math. Programming*,**45**, pp. 503–528, 1989.MathSciNetzbMATHCrossRefGoogle Scholar - 17.S. G. Nash, “Preconditioning of truncated-Newton methods,”
*SIAM J. Sci. Stat. Gomput.*,**6**, pp. 599–616, 1985.MathSciNetzbMATHCrossRefGoogle Scholar - 18.S. G. Nash and J. Nocedal, “A numerical study of the limited memory BFGS method and the truncated-Newton method for large scale optimization,” Tech. Rep. NAM 02, Northwestern University–Dep’t. Elec. Eng. and Comp. Sci., 1989.Google Scholar
- 19.C. A. Bouman and K. Sauer, “A generalized Gaussian image model for edge-preserving MAP estimation,”
*IEEE Trans, on Image Proc*.,**2**, pp. 296–310, 1993.CrossRefGoogle Scholar - 20.S. S. Saquib, C. A. Bouman, and K. Sauer, “ML parameter estimation for Markov random fields, with applications to Bayesian tomography,” Tech. Rep. TR-ECE 95-24, School of Electrical and Computer Engineering, Purdue University, 1995.Google Scholar
- 21.M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour models,”
*Inter. J. Comp. Vision*,**1**, pp. 321–331, 1988.CrossRefGoogle Scholar - 22.G. S. Cunningham, I. Koyfman, and K. M. Hanson, “Improved convergence of gradient-based reconstructions using multi-scale models,” in
*Image Processing*, M. H. Loew and K. M. Hanson, eds.,*Proc. SPIE*,**2710**, pp. 145–155, 1996.Google Scholar - 23.K. M. Hanson, G. S. Cunningham, and R. J. McKee, “Uncertainty assessment for reconstructions based on deformable models,”
*Int. J. Imaging Systems and Technology***8**, pp 506–512, 1997.CrossRefGoogle Scholar - 24.G. S. Cunningham, K. M. Hanson, and X. L. Battle, “Three-dimensional reconstructions from low-count SPECT data using deformable models,”
*Opt. Express*, 1998 (to be published).Google Scholar - 25.K. M. Hanson, R. L. Bilisoly, and G. S. Cunningham, “Kinky tomographic reconstruction,” in
*Image Processing*, M. H. Loew and K. M. Hanson, eds.,*Proc. SPIE*,**2710**, pp. 156–166, 1996.Google Scholar - 26.J. Besag, P. Green, D. Higdon, and K. Mengersen, “Bayesian computation and stochastic systems,”
*Stat. Sci.*,**10**, pp. 3–66, 1995.MathSciNetzbMATHCrossRefGoogle Scholar - 27.W. R. Gilks, S. Richardson, and D. J. Spiegelhalter,
*Markov Chain Monte Carlo in Practice*, Chapman and Hall, London, 1996.zbMATHGoogle Scholar - 28.R. M. Neal,
*Bayesian Learning for Neural Networks*, Springer, New York, 1996.zbMATHCrossRefGoogle Scholar - 29.J. Skilling, D. R. T. Robinson, and S. F. Gull, “Probabilistic displays,” in
*Maximum Entropy and Bayesian Methods*, W. T. Grandy, Jr. and L. H. Shick, eds., pp. 365–368, Kluwer Academic, Dordrecht, 1991.CrossRefGoogle Scholar - 30.A. H. Hielscher, S. S. Saquib, and K. M. Hanson, “Model-based iterative image reconstruction for photon migration tomography,” in
*Methods for Solving Ill-Posed Inverse Imaging Problems: Medical and Nonmedical Applications*, R. L. Barbour, M. J. Carvlin, and M. A. Fiddy, eds.,*Proc. SPIE*,**3171**, pp. 106–117, 1997.Google Scholar - 31.A. H. Hielscher, A. Klose, D. M. Catarious, Jr., and K. M. Hanson, “Tomographic imaging of breast and brain tissue by time-resolved, model-based, iterative image reconstruction,” in
*Proc. Conf. on Advances in Optical Imaging and Photon Migration*, Optical Society of America, Orlando, 1998 (to be published).Google Scholar - 32.K. M. Hanson and G. S. Cunningham, “Posterior sampling with improved efficiency,” in
*Medical Imaging: Image Processing*, K. M. Hanson, ed.,*Proc. SPIE*,**3338**, 1998 (to be published).Google Scholar - 33.R. J. Henninger, P. J. Maudlin, M. L. Rightley, and K. M. Hanson, “Application of forward and adjoint techniques to hydrocode sensitivity analysis,” in
*Proc. 9th Nuclear Explosives Code Developers’ Conference*, F. Graviana, et al., ed., Lawrence Livermore National Laboratory, 1996 (proceedings are classified; article available from authors as report LA-CP-96–235.Google Scholar - 34.W. L. Oberkampf, M. N. Sindir, and A. T. Conlisk, “Guide for the verification and validation of computational fluid dynamics simulations,” Tech. Rep. G-077–98, Amer. Inst. Aeronautics and Astronautics, 1998.Google Scholar