Abstract
Martingale problems provide a powerful method for characterizing Markov processes. Stochastic control problems can also be formulated naturally as martingale problems (see for example Fleming (1983)), and our goal here is to exploit this formulation to give a general existence theorem for optimal solutions of a stochastic control problem in the finite time horizon and discounted cases (Sections 1,3), to construct the Nisio semigroup (Section 2), and to give conditions in terms of the generator of the Nisio semigroup under which the optimal solution can be approximated by solutions in which the control is piecewise constant (Sections 2,3). The proof of this last result involves showing that the value function of an appropriately defined discounted control problem is in the domain of the generator of the Nisio semigroup.
Research presented here was supported in part by the National Science Foundation contract DMS-8401360.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
Crandall, Michael G. and Liggett, Thomas M. (1971). Generation of semigroups of nonlinear transformations on general Banach spaces. Amer. J. Math. 93, 265–298.
Crandall, Michael G. and Lions, Pierre-Louis (1983). Viscosity solutions of Hamilton-Jacobi equations. Trans. Amer. Math. Soc. 277, 1–42.
El Karoui, Nicole, Lepeltier, Jean-Pierre, and Marchal, Bernard (1982). Nisio semigroup associated to the control of Markov processes. Stochastic differential systems (Bad Honnef, 1982). Lect. Notes Control Inf. Sci. 43, 294–301.
Ethier, Stewart N. and Kurtz, Thomas G. (1986). Markov Processes: Characterization and Convergence. Wiley, New York.
Fleming, Wendell H. (1984). Optimal control of Markov processes. Proceedings of the International Congress of Mathematicians, Vol. 1, 2, 71–84. PWN, Warsaw.
Lions, Pierre-Louis (1983). Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations II. Viscosity solutions and uniqueness. Comm. Partial Differential Equations 8, 1229–1276.
Nisio, Makiko (1976). On a nonlinear semigroup attached to optimal stochastic control. Pub. RIMS Kyoto Univ. 13, 513–537
Nisio, Makiko (1981). Lectures on Stochastic Control Theory. ISI Lecture Notes 9. Macmillan, New Delhi.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1987 Springer-Verlag
About this paper
Cite this paper
Kurtz, T.G. (1987). Martingale problems for controlled processes. In: Germani, A. (eds) Stochastic Modelling and Filtering. Lecture Notes in Control and Information Sciences, vol 91. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0009051
Download citation
DOI: https://doi.org/10.1007/BFb0009051
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-17575-9
Online ISBN: 978-3-540-47461-6
eBook Packages: Springer Book Archive