Skip to main content
SpringerLink
Log in

The Dynamic Programming Approach

  • Chapter
  • First Online:
Dynamics and Control of Trajectory Tubes

Part of the book series: Systems & Control: Foundations & Applications ((SCFA,volume 85))

  • 1085 Accesses

Abstract

This chapter describes general schemes of the Dynamic Programming approach. It introduces the notion of value function and its role in these schemes. They are dealt with under either classical conditions or directional differentiability of related functions, leaving more complicated cases to later chapters. Here the emphasis is on indicating solutions to forward and backward reachability problems for “linear-convex” systems and the design of closed-loop control strategies for optimal target and time-optimal feedback problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 19.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 29.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 29.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The general nondifferentiable case for the value function is discussed later in Sect. 5.1.

  2. 2.

    Problems of reachability under unknown but bounded disturbances are beyond the scope of this book, along the approaches of which they are treated in papers [133, 176, 183].

  3. 3.

    With additional information on \(\mathcal{P}(t)\) (see Remark 1.5.3), in degenerate cases the control u ∗(t) may be written down in more detail.

  4. 4.

    This example is animated in the toolbox [132].

  5. 5.

    A closed set \(\mathcal{Q}\) is said to be reachable in finite time if the intersection \(\mathcal{Q}\cap \mathbf{X}_{+}(t,x)\not =\varnothing \) for some τ > t. Here X +(t, x) is the total (forward) reachability set from position {t, x}. 

References

  1. Bardi, M., Capuzzo Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton–Jacobi–Bellman Equations. SCFA. Birkhäuser, Boston (1997)

    Book  MATH  Google Scholar 

  2. Bellman, R.: Introduction to the Mathematical Theory of Controlled Processes, vol. 1/2. Academic, New York (1967/1971)

    Google Scholar 

  3. Bertsekas, D.P.: Dynamic Programming and Optimal Control, vol. 1/2. Athena Scientific, Belmont (1996/2012)

    Google Scholar 

  4. Blagodatskih, V.I, Filippov, A.F.: Differential inclusions and optimal control. Proc. Steklov Math. Inst. 169, 194–252 (1985)

    Google Scholar 

  5. Bliss, G.A.: Calculus of Variations. Mathematical Association of America, Chicago (1944)

    Google Scholar 

  6. Bolza, O.: Lectures on Calculus of Variations. Hafer Pub. Co, New York (1946). Dover reprint

    Google Scholar 

  7. Crandall, M.G., Lions, P.-L.: Viscosity solutions of Hamilton–Jacobi equations. Trans. Am. Math. Soc. 277(1), 1–41 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  8. Demyanov, V.F., Malozemov, V.N.: Introduction to Minimax. Wiley, New York (1974)

    Google Scholar 

  9. Fan, K.: Existence theorems and extreme solutions for inequalities concerning convex functions or linear transformations. Mathematische Zeitschrift 68(1), 205–216 (1957)

    Article  MathSciNet  MATH  Google Scholar 

  10. Filippov, A.F.: On certain questions in the theory of optimal control. SIAM J. Control. 1, 76–84 (1962)

    MATH  Google Scholar 

  11. Gelfand, I.M., Shilov, G.E.: Generalized Functions. Dover, New York (1991)

    Google Scholar 

  12. Krasovski, N.N.: Rendezvous Game Problems. National Technical Information Service, Springfield (1971)

    Google Scholar 

  13. Kurzhanskiy, A.A., Varaiya, P.: Ellipsoidal Toolbox. University of California at Berkeley. http://code.google.com/p/ellipsoids/ (2005)

  14. Kurzhanskiy, A.A., Varaiya, P.: Ellipsoidal techniques for reachability analysis of discrete-time linear systems. IEEE Trans. Automat. Control 52(1), 26–38 (2007)

    Article  MathSciNet  Google Scholar 

  15. Kurzhanski, A.B., Filippova, T.F.: On the theory of trajectory tubes: a mathematical formalism for uncertain dynamics, viability and control. In: Advances in Nonlinear Dynamics and Control. Progress in Systems and Control Theory, vol. 17, pp. 122–188. Birkhäuser, Boston (1993)

    Google Scholar 

  16. Kurzhanski, A.B., Vályi, I.: Ellipsoidal Calculus for Estimation and Control. SCFA. Birkhäuser, Boston (1997)

    Book  MATH  Google Scholar 

  17. Kurzhanski, A.B., Varaiya, P.: On the reachability problem under persistent disturbances. Dokl. Math. 61(3), 3809–3814 (2000)

    MathSciNet  Google Scholar 

  18. Kurzhanski, A.B., Varaiya, P.: On reachability under uncertainty. SIAM J. Control Optim. 41(1), 181–216 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  19. Panasyuk, V.I., Panasyuk, V.I.: On one equation generated by a differential inclusion. Math. Notes 27(3), 429–437 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  20. Rockafellar, R.T.: Convex Analysis, 2nd edn. Princeton University Press, Princeton (1999)

    Google Scholar 

  21. Rockafellar, R.T., Wets, R.J.: Variational Analysis. Springer, Berlin (2005)

    Google Scholar 

  22. Subbotin, A.I.: Generalized Solutions of First-Order PDE’s. The Dynamic Optimization Perspective. SCFA. Birkhäuser, Boston (1995)

    Book  Google Scholar 

  23. Whittaker, E.T.: Analytical Dynamics. Dover, New York (1944)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computational Mathematics and Cybernetics, Moscow State (Lomonosov) University, Moscow, Russia

    Alexander B. Kurzhanski

  2. Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, CA, USA

    Alexander B. Kurzhanski & Pravin Varaiya

Authors
  1. Alexander B. Kurzhanski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pravin Varaiya
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Kurzhanski, A.B., Varaiya, P. (2014). The Dynamic Programming Approach. In: Dynamics and Control of Trajectory Tubes. Systems & Control: Foundations & Applications, vol 85. Birkhäuser, Cham. https://doi.org/10.1007/978-3-319-10277-1_2

Download citation

Publish with us

Policies and ethics

Search

Navigation