Abstract
We study zero-sum stochastic differential games with player dynamics governed by a nondegenerate controlled diffusion process. Under the assumption of uniform stability, we establish the existence of a solution to the Isaac’s equation for the ergodic game and characterize the optimal stationary strategies. The data is not assumed to be bounded, nor do we assume geometric ergodicity. Thus our results extend previous work in the literature. We also study a relative value iteration scheme that takes the form of a parabolic Isaac’s equation. Under the hypothesis of geometric ergodicity we show that the relative value iteration converges to the elliptic Isaac’s equation as time goes to infinity. We use these results to establish convergence of the relative value iteration for risk-sensitive control problems under an asymptotic flatness assumption.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arapostathis A, Borkar VS (2012) A relative value iteration algorithm for nondegenerate controlled diffusions. SIAM J Control Optim 50(4):1886–1902
Arapostathis A, Borkar VS, Ghosh MK (2011) Ergodic control of diffusion processes. Encyclopedia of mathematics and its applications, vol 143. Cambridge University Press, Cambridge
Basak GK, Bhattacharya RN (1992) Stability in distribution for a class of singular diffusions. Ann Probab 20(1):312–321
Beneš VE (1970) Existence of optimal strategies based on specified information, for a class of stochastic decision problems. SIAM J Control 8:179–188
Borkar VS, Ghosh MK (1992) Stochastic differential games: occupation measure based approach. J Optim Theory Appl 73(2):359–385
Borkar VS, Suresh Kumar K (2010) Singular perturbations in risk-sensitive stochastic control. SIAM J Control Optim 48(6):3675–3697
Fleming WH, McEneaney WM (1995) Risk-sensitive control on an infinite time horizon. SIAM J Control Optim 33(6):1881–1915
Gilbarg D, Trudinger NS (1983) Elliptic partial differential equations of second order, 2nd edn. Grundlehren der Mathematischen Wissenschaften, vol 224. Springer, Berlin
Gruber M (1984) Harnack inequalities for solutions of general second order parabolic equations and estimates of their Hölder constants. Math Z 185(1):23–43
Ladyženskaja OA, Solonnikov VA, Ural′ceva NN (1967) Linear and quasilinear equations of parabolic type. Translated from the Russian by S. Smith. Translations of Mathematical Monographs, Vol. 23. American Mathematical Society, Providence, R.I.
Meyn SP, Tweedie RL (1993) Stability of Markovian processes. III. Foster-Lyapunov criteria for continuous-time processes. Adv Appl Probab 25(3):518–548
White DJ (1963) Dynamic programming, Markov chains, and the method of successive approximations. J Math Anal Appl 6:373–376
Whittle P (1990) Risk-sensitive optimal control. Wiley-Interscience Series in Systems and Optimization. Wiley, Chichester
Acknowledgments
The work of Ari Arapostathis was supported in part by the Office of Naval Research under the Electric Ship Research and Development Consortium. The work of Vivek Borkar was supported in part by Grant #11IRCCSG014 from IRCC, IIT, Mumbai.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Arapostathis, A., Borkar, V.S., Kumar, K.S. (2013). Relative Value Iteration for Stochastic Differential Games. In: Křivan, V., Zaccour, G. (eds) Advances in Dynamic Games. Annals of the International Society of Dynamic Games, vol 13. Birkhäuser, Cham. https://doi.org/10.1007/978-3-319-02690-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-02690-9_1
Published:
Publisher Name: Birkhäuser, Cham
Print ISBN: 978-3-319-02689-3
Online ISBN: 978-3-319-02690-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)