Skip to main content
SpringerLink
Log in
Book cover

Encyclopedia of Systems and Control pp 1–10Cite as

Characteristics in Optimal Control Computation

  • Living reference work entry
  • First Online:

  • 282 Accesses

Abstract

A review of characteristics-based approaches to optimal control computation is presented for nonlinear deterministic systems described by ordinary differential equations. We recall Pontryagin’s principle which gives necessary optimality conditions for open-loop control strategies. The related framework of generalized characteristics for first-order Hamilton–Jacobi–Bellman equations is discussed as a theoretical tool that bridges the gap between the necessary and sufficient optimality conditions. We point out widely used numerical techniques for obtaining optimal open-loop control strategies (in particular for state-constrained problems) and how indirect (characteristics based) and direct methods may reasonably be combined. A possible transition from open-loop to feedback constructions is also described. Moreover, we discuss approaches for attenuating the curse of dimensionality for certain classes of Hamilton–Jacobi–Bellman equations.

This is a preview of subscription content, log in via an institution.

Bibliography

  • Afanas’ev VN, Kolmanovskii VB, Nosov VR (1996) Mathematical theory of control systems design. Springer Science + Business Media, Dordrecht

    Book  Google Scholar 

  • Bardi M, Capuzzo-Dolcetta I (2008) Optimal control and viscosity solutions of Hamilton–Jacobi–Bellman equations. Birkhäuser, Boston

    MATH  Google Scholar 

  • Basco V, Cannarsa P, Frankowska H (2018) Necessary conditions for infinite horizon optimal control problems with state constraints. Math Control Relat F 8(3&4):535–555

    Article  MathSciNet  Google Scholar 

  • Bellman R (1957) Dynamic programming. Princeton University Press, Princeton

    MATH  Google Scholar 

  • Benson DA, Huntington GT, Thorvaldsen TP, Rao AV (2006) Direct trajectory optimization and costate estimation via an orthogonal collocation method. J Guid Control Dyn 29 (6): 1435–1440

    Article  Google Scholar 

  • Bernhard P (1977) Singular surfaces in differential games: an introduction. In: Hagedorn P, Knobloch HW, Olsder GJ (eds) Differential games and applications. Lecture notes in control and information sciences, vol 3, pp 1–33. Springer, Berlin, Heidelberg

    Google Scholar 

  • Bokanowski O, Desilles A, Zidani H, Zhao J (2017) User’s guide for the ROC-HJ solver: reachability, optimal control, and Hamilton–Jacobi equations, Version 2.3. http://uma.ensta-paristech.fr/soft/ROC-HJ/

  • Bonnard B, Faubourg L, Launay G, Trélat E (2003) Optimal control with state constraints and the space shuttle re-entry problem. J Dyn Control Syst 9 (2): 155–199

    Article  MathSciNet  Google Scholar 

  • Cesari L (1983) Optimization—theory and applications, problems with ordinary differential equations. Applications of mathematics, vol 17. Springer, New York

    Google Scholar 

  • Chow YT, Darbon J, Osher S, Yin W (2017) Algorithm for overcoming the curse of dimensionality for time-dependent non-convex Hamilton–Jacobi equations arising from optimal control and differential games problems. J Sci Comput 73 (2–3): 617–643

    Article  MathSciNet  Google Scholar 

  • Clarke FH, Ledyaev YS, Stern RJ, Wolenski PR (1998) Nonsmooth analysis and control theory. Springer, New York

    MATH  Google Scholar 

  • Cristiani E, Martinon P (2010) Initialization of the shooting method via the Hamilton–Jacobi–Bellman approach. J Optim Theory Appl 146 (2): 321–346

    Article  MathSciNet  Google Scholar 

  • Darbon J, Osher S (2016) Algorithms for overcoming the curse of dimensionality for certain Hamilton–Jacobi equations arising in control theory and elsewhere. Res Math Sci 3: 19

    Article  MathSciNet  Google Scholar 

  • Fleming WH, Soner HM (2006) Controlled Markov processes and viscosity solutions. Springer, New York

    MATH  Google Scholar 

  • Grüne L (2014) Numerical methods for nonlinear optimal control problems. In: Baillieul J, Samad T (eds) Encyclopedia of systems and control. Springer, London

    Google Scholar 

  • Hartl RF, Sethi SP, Vickson RG (1995) A survey of the maximum principles for optimal control problems with state constraints. SIAM Rev 37 (2): 181–218

    Article  MathSciNet  Google Scholar 

  • Huntington GT, Benson DA, Rao AV (2007) Optimal configuration of tetrahedral spacecraft formations. J Aeronaut Sci 55 (2): 141–169

    Google Scholar 

  • Jensen M, Smears I (2013) On the convergence of finite element methods for Hamilton–Jacobi–Bellman equations. SIAM J Numer Anal 51 (1): 137–162

    Article  MathSciNet  Google Scholar 

  • Kang W, Wilcox LC (2017) Mitigating the curse of dimensionality: sparse grid characteristics method for optimal feedback control and HJB equations. Comput Optim Appl 68 (2): 289–315

    Article  MathSciNet  Google Scholar 

  • Kushner HJ, Dupuis P (2001) Numerical methods for stochastic control problems in continuous time. Springer, New York

    Book  Google Scholar 

  • Maurer H (1977) On optimal control problems with bounded state variables and control appearing linearly. SIAM J Control Optim 15 (3): 345–362

    Article  MathSciNet  Google Scholar 

  • McEneaney WM (2006) Max-plus methods in nonlinear control and estimation. Birkhäuser, Boston

    MATH  Google Scholar 

  • McEneaney WM, Kluberg J (2009) Convergence rate for a curse-of-dimensionality-free method for a class of HJB PDEs. SIAM J Control Optim 48 (5): 3052–3079

    Article  MathSciNet  Google Scholar 

  • Melikyan AA (1998) Generalized characteristics of first order PDEs: application in optimal control and differential games. Birkhäuser, Boston

    Book  Google Scholar 

  • Michalska H, Mayne DQ (1993) Robust receding horizon control of constrained nonlinear systems. IEEE Trans Automat Contr 38 (11): 1623–1633

    Article  MathSciNet  Google Scholar 

  • Osher S, Fedkiw R (2003) Level set methods and dynamic implicit surfaces. Springer, New York

    Book  Google Scholar 

  • Pesch HJ (1994) A practical guide to the solution of real-life optimal control problems. Control Cybern 23 (1–2): 7–60

    MathSciNet  MATH  Google Scholar 

  • Pontryagin LS, Boltyansky VG, Gamkrelidze RV, Mishchenko EF (1964) The mathematical theory of optimal processes. Macmillan, New York

    Google Scholar 

  • Rao AV (2010) A survey of numerical methods for optimal control. Adv Astron Sci 135 (1): 497–528

    Google Scholar 

  • Schättler H (2014) Optimal control with state space constraints. In: Baillieul J, Samad T (eds) Encyclopedia of systems and control. Springer, London

    Google Scholar 

  • Schättler H, Ledzewicz U (2015) Optimal control for mathematical models of cancer therapies: an application of geometric methods. Interdisciplinary applied mathematics, vol 42. Springer, New York

    Google Scholar 

  • Subbotin AI (1995) Generalized solutions of first-order PDEs: the dynamical optimization perspective. Birkhäuser, Boston

    Book  Google Scholar 

  • Subbotina NN (2006) The method of characteristics for Hamilton–Jacobi equations and applications to dynamical optimization. J Math Sci 135 (3): 2955–3091

    Article  MathSciNet  Google Scholar 

  • Trélat E (2012) Optimal control and applications to aerospace: some results and challenges. J Optim Theory Appl 154 (3): 713–758

    Article  MathSciNet  Google Scholar 

  • Wang L (2009) Model predictive control system design and implementation using MATLAB. Springer, London

    Google Scholar 

  • Yegorov I, Dower P (2018) Perspectives on characteristics based curse-of-dimensionality-free numerical approaches for solving Hamilton–Jacobi equations. Appl Math Opt. To appear. https://doi.org/10.1007/s00245-018-9509-6

  • Yong J, Zhou XY (1999) Stochastic controls: Hamiltonian systems and HJB equations. Springer, New York

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, North Dakota State University, Fargo, ND, USA

    Ivan Yegorov

  2. Department of Electrical and Electronic Engineering, The University of Melbourne, Melbourne, VIC, Australia

    Peter M Dower

Authors
  1. Ivan Yegorov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peter M Dower
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ivan Yegorov .

Editor information

Editors and Affiliations

  1. Electrical and Computer Engineering, Boston University, Boston, MA, USA

    John Baillieul

  2. Automation and Control Solutions, Honeywell, Golden Valley, MN, USA

    Tariq Samad

Section Editor information

  1. Department of Electrical & Electronic Engineering, The University of Melbourne, Parkville, VIC, Australia

    Michael Cantoni

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer-Verlag London Ltd., part of Springer Nature

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Yegorov, I., Dower, P.M. (2020). Characteristics in Optimal Control Computation. In: Baillieul, J., Samad, T. (eds) Encyclopedia of Systems and Control. Springer, London. https://doi.org/10.1007/978-1-4471-5102-9_100056-1

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-5102-9_100056-1

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-5102-9

  • Online ISBN: 978-1-4471-5102-9

  • eBook Packages: Springer Reference EngineeringReference Module Computer Science and Engineering

Publish with us

Policies and ethics

Search

Navigation