The Dynamic Programming Approach

Part of the Systems & Control: Foundations & Applications book series (SCFA, volume 85)


This chapter describes general schemes of the Dynamic Programming approach. It introduces the notion of value function and its role in these schemes. They are dealt with under either classical conditions or directional differentiability of related functions, leaving more complicated cases to later chapters. Here the emphasis is on indicating solutions to forward and backward reachability problems for “linear-convex” systems and the design of closed-loop control strategies for optimal target and time-optimal feedback problems.


Dynamic programming Value function HJB equation Reachability Linear-convex systems Colliding tubes Terminal control Time-optimal control 


  1. 16.
    Bardi, M., Capuzzo Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton–Jacobi–Bellman Equations. SCFA. Birkhäuser, Boston (1997)CrossRefzbMATHGoogle Scholar
  2. 22.
    Bellman, R.: Introduction to the Mathematical Theory of Controlled Processes, vol. 1/2. Academic, New York (1967/1971)Google Scholar
  3. 24.
    Bertsekas, D.P.: Dynamic Programming and Optimal Control, vol. 1/2. Athena Scientific, Belmont (1996/2012)Google Scholar
  4. 26.
    Blagodatskih, V.I, Filippov, A.F.: Differential inclusions and optimal control. Proc. Steklov Math. Inst. 169, 194–252 (1985)Google Scholar
  5. 29.
    Bliss, G.A.: Calculus of Variations. Mathematical Association of America, Chicago (1944)Google Scholar
  6. 33.
    Bolza, O.: Lectures on Calculus of Variations. Hafer Pub. Co, New York (1946). Dover reprintGoogle Scholar
  7. 50.
    Crandall, M.G., Lions, P.-L.: Viscosity solutions of Hamilton–Jacobi equations. Trans. Am. Math. Soc. 277(1), 1–41 (1983)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 61.
    Demyanov, V.F., Malozemov, V.N.: Introduction to Minimax. Wiley, New York (1974)Google Scholar
  9. 72.
    Fan, K.: Existence theorems and extreme solutions for inequalities concerning convex functions or linear transformations. Mathematische Zeitschrift 68(1), 205–216 (1957)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 75.
    Filippov, A.F.: On certain questions in the theory of optimal control. SIAM J. Control. 1, 76–84 (1962)zbMATHGoogle Scholar
  11. 87.
    Gelfand, I.M., Shilov, G.E.: Generalized Functions. Dover, New York (1991)Google Scholar
  12. 121.
    Krasovski, N.N.: Rendezvous Game Problems. National Technical Information Service, Springfield (1971)Google Scholar
  13. 132.
    Kurzhanskiy, A.A., Varaiya, P.: Ellipsoidal Toolbox. University of California at Berkeley. (2005)
  14. 133.
    Kurzhanskiy, A.A., Varaiya, P.: Ellipsoidal techniques for reachability analysis of discrete-time linear systems. IEEE Trans. Automat. Control 52(1), 26–38 (2007)MathSciNetCrossRefGoogle Scholar
  15. 158.
    Kurzhanski, A.B., Filippova, T.F.: On the theory of trajectory tubes: a mathematical formalism for uncertain dynamics, viability and control. In: Advances in Nonlinear Dynamics and Control. Progress in Systems and Control Theory, vol. 17, pp. 122–188. Birkhäuser, Boston (1993)Google Scholar
  16. 174.
    Kurzhanski, A.B., Vályi, I.: Ellipsoidal Calculus for Estimation and Control. SCFA. Birkhäuser, Boston (1997)CrossRefzbMATHGoogle Scholar
  17. 176.
    Kurzhanski, A.B., Varaiya, P.: On the reachability problem under persistent disturbances. Dokl. Math. 61(3), 3809–3814 (2000)MathSciNetGoogle Scholar
  18. 183.
    Kurzhanski, A.B., Varaiya, P.: On reachability under uncertainty. SIAM J. Control Optim. 41(1), 181–216 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 224.
    Panasyuk, V.I., Panasyuk, V.I.: On one equation generated by a differential inclusion. Math. Notes 27(3), 429–437 (1980)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 237.
    Rockafellar, R.T.: Convex Analysis, 2nd edn. Princeton University Press, Princeton (1999)Google Scholar
  21. 238.
    Rockafellar, R.T., Wets, R.J.: Variational Analysis. Springer, Berlin (2005)Google Scholar
  22. 247.
    Subbotin, A.I.: Generalized Solutions of First-Order PDE’s. The Dynamic Optimization Perspective. SCFA. Birkhäuser, Boston (1995)CrossRefGoogle Scholar
  23. 269.
    Whittaker, E.T.: Analytical Dynamics. Dover, New York (1944)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Faculty of Computational Mathematics and CyberneticsMoscow State (Lomonosov) UniversityMoscowRussia
  2. 2.Electrical Engineering and Computer SciencesUniversity of California, BerkeleyBerkeleyUSA

Personalised recommendations