Abstract
In this chapter we turn to study another powerful approach to solving optimal control problems, namely, the method of dynamic programming. Dynamic programming, originated by R. Bellman in the early 1950s, is a mathematical technique for making a sequence of interrelated decisions, which can be applied to many optimization problems (including optimal control problems). The basic idea of this method applied to optimal controls is to consider a family of optimal control problems with different initial times and states, to establish relationships among these problems via the so-called Hamilton-Jacobi-Bellman equation (HJB, for short), which is a nonlinear first-order (in the deterministic case) or second-order (in the stochastic case) partial differential equation. If the HJB equation is solvable (either analytically or numerically), then one can obtain an optimal feedback control by taking the maximizer/minimizer of the Hamiltonian or generalized Hamiltonian involved in the HJB equation. This is the so-called verification technique. Note that this approach actually gives solutions to the whole family of problems (with different initial times and states), and in particular, the original problem.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer Science+Business Media New York
About this chapter
Cite this chapter
Yong, J., Zhou, X.Y. (1999). Dynamic Programming and HJB Equations. In: Stochastic Controls. Applications of Mathematics, vol 43. Springer, New York, NY. https://doi.org/10.1007/978-1-4612-1466-3_4
Download citation
DOI: https://doi.org/10.1007/978-1-4612-1466-3_4
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4612-7154-3
Online ISBN: 978-1-4612-1466-3
eBook Packages: Springer Book Archive