Encyclopedia of Systems and Control

Living Edition
| Editors: John Baillieul, Tariq Samad

Characteristics in Optimal Control Computation

Living reference work entry
DOI: https://doi.org/10.1007/978-1-4471-5102-9_100056-1


A review of characteristics-based approaches to optimal control computation is presented for nonlinear deterministic systems described by ordinary differential equations. We recall Pontryagin’s principle which gives necessary optimality conditions for open-loop control strategies. The related framework of generalized characteristics for first-order Hamilton–Jacobi–Bellman equations is discussed as a theoretical tool that bridges the gap between the necessary and sufficient optimality conditions. We point out widely used numerical techniques for obtaining optimal open-loop control strategies (in particular for state-constrained problems) and how indirect (characteristics based) and direct methods may reasonably be combined. A possible transition from open-loop to feedback constructions is also described. Moreover, we discuss approaches for attenuating the curse of dimensionality for certain classes of Hamilton–Jacobi–Bellman equations.


Optimal control Pontryagin’s principle Hamilton–Jacobi–Bellman equations Method of characteristics Indirect and direct numerical approaches State constraints Curse of dimensionality 
This is a preview of subscription content, log in to check access.


  1. Afanas’ev VN, Kolmanovskii VB, Nosov VR (1996) Mathematical theory of control systems design. Springer Science + Business Media, DordrechtCrossRefGoogle Scholar
  2. Bardi M, Capuzzo-Dolcetta I (2008) Optimal control and viscosity solutions of Hamilton–Jacobi–Bellman equations. Birkhäuser, BostonzbMATHGoogle Scholar
  3. Basco V, Cannarsa P, Frankowska H (2018) Necessary conditions for infinite horizon optimal control problems with state constraints. Math Control Relat F 8(3&4):535–555MathSciNetCrossRefGoogle Scholar
  4. Bellman R (1957) Dynamic programming. Princeton University Press, PrincetonzbMATHGoogle Scholar
  5. Benson DA, Huntington GT, Thorvaldsen TP, Rao AV (2006) Direct trajectory optimization and costate estimation via an orthogonal collocation method. J Guid Control Dyn 29 (6): 1435–1440CrossRefGoogle Scholar
  6. Bernhard P (1977) Singular surfaces in differential games: an introduction. In: Hagedorn P, Knobloch HW, Olsder GJ (eds) Differential games and applications. Lecture notes in control and information sciences, vol 3, pp 1–33. Springer, Berlin, HeidelbergGoogle Scholar
  7. Bokanowski O, Desilles A, Zidani H, Zhao J (2017) User’s guide for the ROC-HJ solver: reachability, optimal control, and Hamilton–Jacobi equations, Version 2.3. http://uma.ensta-paristech.fr/soft/ROC-HJ/
  8. Bonnard B, Faubourg L, Launay G, Trélat E (2003) Optimal control with state constraints and the space shuttle re-entry problem. J Dyn Control Syst 9 (2): 155–199MathSciNetCrossRefGoogle Scholar
  9. Cesari L (1983) Optimization—theory and applications, problems with ordinary differential equations. Applications of mathematics, vol 17. Springer, New YorkGoogle Scholar
  10. Chow YT, Darbon J, Osher S, Yin W (2017) Algorithm for overcoming the curse of dimensionality for time-dependent non-convex Hamilton–Jacobi equations arising from optimal control and differential games problems. J Sci Comput 73 (2–3): 617–643MathSciNetCrossRefGoogle Scholar
  11. Clarke FH, Ledyaev YS, Stern RJ, Wolenski PR (1998) Nonsmooth analysis and control theory. Springer, New YorkzbMATHGoogle Scholar
  12. Cristiani E, Martinon P (2010) Initialization of the shooting method via the Hamilton–Jacobi–Bellman approach. J Optim Theory Appl 146 (2): 321–346MathSciNetCrossRefGoogle Scholar
  13. Darbon J, Osher S (2016) Algorithms for overcoming the curse of dimensionality for certain Hamilton–Jacobi equations arising in control theory and elsewhere. Res Math Sci 3: 19MathSciNetCrossRefGoogle Scholar
  14. Fleming WH, Soner HM (2006) Controlled Markov processes and viscosity solutions. Springer, New YorkzbMATHGoogle Scholar
  15. Grüne L (2014) Numerical methods for nonlinear optimal control problems. In: Baillieul J, Samad T (eds) Encyclopedia of systems and control. Springer, LondonGoogle Scholar
  16. Hartl RF, Sethi SP, Vickson RG (1995) A survey of the maximum principles for optimal control problems with state constraints. SIAM Rev 37 (2): 181–218MathSciNetCrossRefGoogle Scholar
  17. Huntington GT, Benson DA, Rao AV (2007) Optimal configuration of tetrahedral spacecraft formations. J Aeronaut Sci 55 (2): 141–169Google Scholar
  18. Jensen M, Smears I (2013) On the convergence of finite element methods for Hamilton–Jacobi–Bellman equations. SIAM J Numer Anal 51 (1): 137–162MathSciNetCrossRefGoogle Scholar
  19. Kang W, Wilcox LC (2017) Mitigating the curse of dimensionality: sparse grid characteristics method for optimal feedback control and HJB equations. Comput Optim Appl 68 (2): 289–315MathSciNetCrossRefGoogle Scholar
  20. Kushner HJ, Dupuis P (2001) Numerical methods for stochastic control problems in continuous time. Springer, New YorkCrossRefGoogle Scholar
  21. Maurer H (1977) On optimal control problems with bounded state variables and control appearing linearly. SIAM J Control Optim 15 (3): 345–362MathSciNetCrossRefGoogle Scholar
  22. McEneaney WM (2006) Max-plus methods in nonlinear control and estimation. Birkhäuser, BostonzbMATHGoogle Scholar
  23. McEneaney WM, Kluberg J (2009) Convergence rate for a curse-of-dimensionality-free method for a class of HJB PDEs. SIAM J Control Optim 48 (5): 3052–3079MathSciNetCrossRefGoogle Scholar
  24. Melikyan AA (1998) Generalized characteristics of first order PDEs: application in optimal control and differential games. Birkhäuser, BostonCrossRefGoogle Scholar
  25. Michalska H, Mayne DQ (1993) Robust receding horizon control of constrained nonlinear systems. IEEE Trans Automat Contr 38 (11): 1623–1633MathSciNetCrossRefGoogle Scholar
  26. Osher S, Fedkiw R (2003) Level set methods and dynamic implicit surfaces. Springer, New YorkCrossRefGoogle Scholar
  27. Pesch HJ (1994) A practical guide to the solution of real-life optimal control problems. Control Cybern 23 (1–2): 7–60MathSciNetzbMATHGoogle Scholar
  28. Pontryagin LS, Boltyansky VG, Gamkrelidze RV, Mishchenko EF (1964) The mathematical theory of optimal processes. Macmillan, New YorkGoogle Scholar
  29. Rao AV (2010) A survey of numerical methods for optimal control. Adv Astron Sci 135 (1): 497–528Google Scholar
  30. Schättler H (2014) Optimal control with state space constraints. In: Baillieul J, Samad T (eds) Encyclopedia of systems and control. Springer, LondonGoogle Scholar
  31. Schättler H, Ledzewicz U (2015) Optimal control for mathematical models of cancer therapies: an application of geometric methods. Interdisciplinary applied mathematics, vol 42. Springer, New YorkGoogle Scholar
  32. Subbotin AI (1995) Generalized solutions of first-order PDEs: the dynamical optimization perspective. Birkhäuser, BostonCrossRefGoogle Scholar
  33. Subbotina NN (2006) The method of characteristics for Hamilton–Jacobi equations and applications to dynamical optimization. J Math Sci 135 (3): 2955–3091MathSciNetCrossRefGoogle Scholar
  34. Trélat E (2012) Optimal control and applications to aerospace: some results and challenges. J Optim Theory Appl 154 (3): 713–758MathSciNetCrossRefGoogle Scholar
  35. Wang L (2009) Model predictive control system design and implementation using MATLAB. Springer, LondonGoogle Scholar
  36. Yegorov I, Dower P (2018) Perspectives on characteristics based curse-of-dimensionality-free numerical approaches for solving Hamilton–Jacobi equations. Appl Math Opt. To appear. https://doi.org/10.1007/s00245-018-9509-6
  37. Yong J, Zhou XY (1999) Stochastic controls: Hamiltonian systems and HJB equations. Springer, New YorkCrossRefGoogle Scholar

Authors and Affiliations

  1. 1.Department of MathematicsNorth Dakota State UniversityFargoUSA
  2. 2.Department of Electrical and Electronic EngineeringThe University of MelbourneMelbourneAustralia

Section editors and affiliations

  • Michael Cantoni
    • 1
  1. 1.Department of Electrical & Electronic EngineeringThe University of MelbourneParkvilleAustralia