Abstract
This chapter contains a brief discussion of the basic mathematical ideas behind dynamic programming methods for optimal control of Markov processes. It is based on lectures given by the author at the Summer School on Computational Finance held at Smolenice Castle, Slovakia, in September 2014.
The key theoretical ideas behind the approach are outlined in a somewhat abstract setting. We hope that this will help the reader to understand the key points of the dynamic programming principle and Hamilton-Jacobi-Bellman equations and their potential as tools to solve a large array of optimization problems, without paying too much attention to the technical difficulties that often arise in concrete applications.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bovier A.: Markov processes. Lecture Notes Summer 2012, Bonn, Institute for Applied Mathematics, University of Bonn (2012). http://wt.iam.uni-bonn.de/bovier/lecture-notes/
Dumitrescu, R., Quenez, M.-C., Sulem, A.: A weak dynamic programming principle for combined optimal stopping/stochastic control with \(\mathcal{E}^{f}\)-expectations. SIAM J. Control Optim. 54(4), 2090–2115 (2015)
de Vallière, D., Kabanov, Y., Lépinette, E.: Consumption-investment problem with transaction costs for Lévy-driven price process. Finance Stochast. 20, 705–740 (2016)
Ethier, S.N., Kurtz, T.G.: Markov Processes. Characterization and Convergence. Wiley-Interscience, New York (2005)
El Karoui, N., Tan, X.: Capacities, measurable selection and dynamic programming. Part I: abstract framework. arXiv:1310.3363v1 (2013)
Fleming, W.H., Soner, H.M.: Controlled Markov Processes and Viscosity Solutions, 2nd edn. Springer, New York (2006)
Goldys, B., Wu, W.: Dynamic programming principle for stochastic control problems driven by general Lévy noise. Stoch. Anal. Appl. 34(6), 1083–1093 (2016)
Rogers, L.C.G., Williams, D.: Diffusions, Markov Processes, and Martingales. Volume 1: Foundations, 2nd edn. Wiley, New York (1994)
Touzi, N.: Optimal Stochastic Control, Stochastic Target Problems, and Backward SDE. Springer, New York (2013)
Yong, J., Zhou, X.Y.: Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer, New York (1999)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Guerra, M. (2017). Stochastic Dynamic Programming and Control of Markov Processes. In: Ehrhardt, M., Günther, M., ter Maten, E. (eds) Novel Methods in Computational Finance. Mathematics in Industry(), vol 25. Springer, Cham. https://doi.org/10.1007/978-3-319-61282-9_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-61282-9_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-61281-2
Online ISBN: 978-3-319-61282-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)