Inexact Trust-Region Methods for PDE-Constrained Optimization

  • Drew P. Kouri
  • Denis RidzalEmail author
Part of the The IMA Volumes in Mathematics and its Applications book series (IMA, volume 163)


Numerical solution of optimization problems with partial differential equation (PDE) constraints typically requires inexact objective function and constraint evaluations, derivative approximations, and the use of iterative linear system solvers. Over the last 30 years, trust-region methods have been extended to rigorously, robustly, and efficiently handle various sources of inexactness in the optimization process. In this chapter, we review some of the recent advances, discuss their key algorithmic contributions, and present numerical examples that demonstrate how inexact computations can be exploited to enable the solution of large-scale PDE-constrained optimization problems.



This work was supported by DARPA EQUiPS grant SNL 014150709 and the DOE NNSA ASC ATDM program.

Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia LLC, a wholly owned subsidiary of Honeywell International Inc. for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the paper do not necessarily represent the views of the U.S. Department of Energy or the United States Government.


  1. 1.
    N. Alexandrov. Robustness properties of a trust-region framework for managing approximation models in engineering optimization. In Proceedings from the AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, Work-in-progress Paper AIAA–96–4102-CP, pages 1056–1059, 1996.Google Scholar
  2. 2.
    N. Alexandrov. A trust–region framework for managing approximations in constrained optimization problems. In Proceedings of the First ISSMO/NASA Internet Conference on Approximation and Fast Reanalysis Techniques in Engineering Optimization, June 14–27, 1998, 1998.Google Scholar
  3. 3.
    N. Alexandrov and J. E. Dennis. Multilevel algorithms for nonlinear optimization. In J. Borggaard, J. Burkardt, M. D. Gunzburger, and J. Peterson, editors, Optimal Design and Control, pages 1–22, Basel, Boston, Berlin, 1995. Birkhäuser Verlag.Google Scholar
  4. 4.
    N. Alexandrov, J. E. Dennis Jr., R. M. Lewis, and V. Torczon. A trust region framework for managing the use of approximation models in optimization. Structural Optimization, 15: 16–23, 1998. Appeared also as ICASE report 97–50.Google Scholar
  5. 5.
    E. Arian, M. Fahl, and E. W. Sachs. Trust–region proper orthogonal decomposition for flow control. Technical Report 2000–25, ICASE, NASA Langley Research Center, Hampton VA 23681–2299, 2000.Google Scholar
  6. 6.
    I. Babuška, F. Nobile, and R. Tempone. A stochastic collocation method for elliptic partial differential equations with random input data. SIAM Rev., 52(2):317–355, 2010.MathSciNetzbMATHGoogle Scholar
  7. 7.
    V. Barthelmann, E. Novak, and K. Ritter. High dimensional polynomial interpolation on sparse grids. Adv. Comput. Math., 12(4):273–288, 2000. Multivariate polynomial interpolation.Google Scholar
  8. 8.
    F. Bastin, C. Cirillo, and Ph. L. Toint. An adaptive Monte Carlo algorithm for computing mixed logit estimators. Comput. Manag. Sci., 3(1):55–79, 2006.MathSciNetzbMATHGoogle Scholar
  9. 9.
    M. Benzi, G. H Golub, and J. Liesen. Numerical solution of saddle point problems. Acta Numerica, 14(1):1–137, 2005.MathSciNetzbMATHGoogle Scholar
  10. 10.
    R. G. Carter. Numerical optimization in Hilbert space using inexact function and gradient evaluations. Technical Report 89–45, ICASE, Langley, VA, 1989.Google Scholar
  11. 11.
    R. G. Carter. On the global convergence of trust region algorithms using inexact gradient information. SIAM J. Numer. Anal., 28:251–265, 1991.MathSciNetzbMATHGoogle Scholar
  12. 12.
    R. G. Carter. Numerical experience with a class of algorithms for nonlinear optimization using inexact function and gradient information. SIAM Journal on Scientific Computing, 14(2):368–388, 1993.MathSciNetzbMATHGoogle Scholar
  13. 13.
    A. R. Conn, N. I. M. Gould, and Ph. L. Toint. Trust–Region Methods. SIAM, Philadelphia, 2000.zbMATHGoogle Scholar
  14. 14.
    J. E. Dennis, M. El-Alem, and M. C. Maciel. A Global Convergence Theory for General Trust–Region–Based Algorithms for Equality Constrained Optimization. SIAM J. Optimization, 7:177–207, 1997.MathSciNetzbMATHGoogle Scholar
  15. 15.
    J. E. Dennis and V. Torczon. Approximation model managemet for optimization. In Proceedings from the AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, Work-in-progress Paper AIAA–96–4099-CP, pages 1044–1046, 1996.Google Scholar
  16. 16.
    J. E. Dennis and V. Torczon. Managing approximation models in optimization. In N. Alexandrov and M. Y. Hussaini, editors, Multidisciplinary Design Optimization. State of the Art, pages 330–347, Philadelphia, 1997. SIAM.Google Scholar
  17. 17.
    J. E. Dennis, Jr. and R. B. Schnabel. Numerical Methods for Nonlinear Equations and Unconstrained Optimization. SIAM, Philadelphia, 1996.zbMATHGoogle Scholar
  18. 18.
    M. Fahl and E.W. Sachs. Reduced order modelling approaches to PDE–constrained optimization based on proper orthogonal decompostion. In L. T. Biegler, O. Ghattas, M. Heinkenschloss, and B. van Bloemen Waanders van Bloemen Waanders, editors, Large-Scale PDE-Constrained Optimization, Lecture Notes in Computational Science and Engineering, Vol. 30, Heidelberg, 2003. Springer-Verlag.Google Scholar
  19. 19.
    T. Gerstner and M. Griebel. Dimension-adaptive tensor-product quadrature. Computing, 71(1):65–87, 2003.MathSciNetzbMATHGoogle Scholar
  20. 20.
    M. Heinkenschloss and D. Ridzal. A matrix-free trust-region SQP method for equality constrained optimization. SIAM Journal on Optimization, 24(3):1507–1541, 2014.MathSciNetzbMATHGoogle Scholar
  21. 21.
    M. Heinkenschloss and L. N. Vicente. Analysis of inexact trust–region SQP algorithms. SIAM J. Optimization, 12:283–302, 2001.MathSciNetzbMATHGoogle Scholar
  22. 22.
    M. Hinze, R. Pinnau, M. Ulbrich, and S. Ulbrich. Optimization with Partial Differential Equations, volume 23 of Mathematical Modelling, Theory and Applications. Springer Verlag, Heidelberg, New York, Berlin, 2009.zbMATHGoogle Scholar
  23. 23.
    K. Ito and S. S. Ravindran. Optimal control of thermally convected fluid flows. SIAM J. on Scientific Computing, 19:1847–1869, 1998.MathSciNetzbMATHGoogle Scholar
  24. 24.
    C.T. Kelley and E.W. Sachs. Truncated newton methods for optimization with inaccurate functions and gradients. Journal of Optimization Theory and Applications, 116(1):83–98, 2003.MathSciNetzbMATHGoogle Scholar
  25. 25.
    D. P. Kouri, M. Heinkenschloss, D. Ridzal, and B. G. van Bloemen Waanders. A trust-region algorithm with adaptive stochastic collocation for PDE optimization under uncertainty. SIAM Journal on Scientific Computing, 35(4):A1847–A1879, 2013.MathSciNetzbMATHGoogle Scholar
  26. 26.
    D. P. Kouri, M. Heinkenschloss, D. Ridzal, and B. G. van Bloemen Waanders. Inexact objective function evaluations in a trust-region algorithm for PDE-constrained optimization under uncertainty. SIAM Journal on Scientific Computing, 36(6):A3011–A3029, 2014.MathSciNetzbMATHGoogle Scholar
  27. 27.
    D. P. Kouri, G. von Winckel, and D. Ridzal. ROL: Rapid Optimization Library., 2017.
  28. 28.
    L. Lubkoll, A. Schiela, and M. Weiser. An affine covariant composite step method for optimization with PDEs as equality constraints. Optimization Methods and Software, 32(5):1132–1161, 2017.MathSciNetzbMATHGoogle Scholar
  29. 29.
    J. J. Moré. Recent developments in algorithms and software for trust region methods. In A. Bachem, M. Grötschel, and B. Korte, editors, Mathematical Programming, The State of The Art, pages 258–287. Springer Verlag, Berlin, Heidelberg, New-York, 1983.Google Scholar
  30. 30.
    M. F. Murphy, G. H. Golub, and A. J. Wathen. A note on preconditioning for indefinite linear systems. SIAM Journal on Scientific Computing, 21(6):1969–1972, 2000.MathSciNetzbMATHGoogle Scholar
  31. 31.
    E. Novak and K. Ritter. High-dimensional integration of smooth functions over cubes. Numer. Math., 75(1):79–97, 1996.MathSciNetzbMATHGoogle Scholar
  32. 32.
    E. O. Omojokun. Trust region algorithms for optimization with nonlinear equality and inequality constraints. PhD thesis, Department of Computer Science, University of Colorado, Boulder, Colorado, 1989.Google Scholar
  33. 33.
    T. Rees, H. S. Dollar, and A. J. Wathen. Optimal Solvers for PDE-Constrained Optimization. SIAM Journal on Scientific Computing, 32(1):271–298, 2010.MathSciNetzbMATHGoogle Scholar
  34. 34.
    T. Rees, M. Stoll, and A. Wathen. All-at-once preconditioning in PDE-constrained optimization. Kybernetika, 46(2):341–360, 2010.MathSciNetzbMATHGoogle Scholar
  35. 35.
    T. Rees and A. J. Wathen. Preconditioning iterative methods for the optimal control of the Stokes equation. SIAM J. Sci. Comput, 33(5), 2010.MathSciNetzbMATHGoogle Scholar
  36. 36.
    D. Ridzal. Preconditioning of a Full-Space Trust-Region SQP Algorithm for PDE-constrained Optimization. In Report No. 04/2013: Numerical Methods for PDE Constrained Optimization with Uncertain Data. Mathematisches Forschungsinstitut Oberwolfach, 2013.Google Scholar
  37. 37.
    S. A. Smoljak. Quadrature and interpolation formulae on tensor products of certain function classes. Soviet Math. Dokl., 4:240–243, 1963.Google Scholar
  38. 38.
    M. Stoll. One-shot solution of a time-dependent time-periodic PDE-constrained optimization problem. IMA Journal of Numerical Analysis, 34(4):1554–1577, 2014.MathSciNetzbMATHGoogle Scholar
  39. 39.
    M. Stoll and A. Wathen. All-at-once solution of time-dependent Stokes control. Journal of Computational Physics, 232(1):498–515, 2013.MathSciNetGoogle Scholar
  40. 40.
    Ph. L. Toint. Global convergence of a class of trust-region methods for nonconvex minimization in Hilbert space. IMA Journal of Numerical Analysis, 8:231–252, 1988.MathSciNetzbMATHGoogle Scholar
  41. 41.
    S. Ulbrich and J. C. Ziems. Adaptive multilevel trust-region methods for time-dependent PDE-constrained optimization. Portugaliae Mathematica, 74(1):37–67, 2017.MathSciNetzbMATHGoogle Scholar
  42. 42.
    D. Xiu and J. S. Hesthaven. High-order collocation methods for differential equations with random inputs. SIAM J. Sci. Comput., 27(3):1118–1139 (electronic), 2005.MathSciNetzbMATHGoogle Scholar
  43. 43.
    J. C. Ziems and S. Ulbrich. Adaptive multilevel inexact SQP methods for PDE-constrained optimization. SIAM Journal on Optimization, 21(1):1–40, 2011.MathSciNetzbMATHGoogle Scholar
  44. 44.
    J. Carsten Ziems. Adaptive Multilevel Inexact SQP-Methods for PDE-Constrained Optimization with Control Constraints. SIAM Journal on Optimization, 23(2):1257–1283, 2013.MathSciNetzbMATHGoogle Scholar

Copyright information

© National Technology & Engineering Solutions of Sandia, LLC. Under the terms of Contract DE-NA0003525, there is a non-exclusive license for use of this work by or on behalf of the U.S. Government 2018

Authors and Affiliations

  1. 1.Center for Computing ResearchSandia National LaboratoriesAlbuquerqueUSA

Personalised recommendations