Numerical Methods for Stochastic Differential Games: The Ergodic Cost Criterion
The Markov chain approximation method is a widely used, relatively easy to apply, and efficient family of methods for the bulk of stochastic control problems in continuous time, for reflected jump-diffusion type models. It is the most general method available, it has been shown to converge under broad conditions, via probabilistic methods, and there are good algorithms for solving the numerical problems, if the dimension is not too high (see the basic reference ). We consider a class of stochastic differential games with a reflected diffusion system model and ergodic cost criterion, where the controls for the two players are separated in the dynamics and cost function. The value of the game exists and the numerical method converges to this value as the discretization parameter goes to zero. The actual numerical method solves a stochastic game for a finite state Markov chain and ergodic cost criterion. The essential conditions are nondegeneracy of the diffusion and that a weak local consistency condition hold “almost everywhere” for the numerical approximations, just as for the control problem. The latter is close to a minimal condition. Such ergodic and “separated” game models occur in risk-sensitive and robust control.
KeywordsDifferential Game Dynamic Game Stochastic Game Stochastic Control Problem Stochastic Differential Game
Unable to display preview. Download preview PDF.
- M. Bardi, S. Bottacini, and M. Falcone. Convergence of discrete schemes for discontinuous value functions of pursuit-evasion games. In G.I. Olsder, editor, New Trends in Dynamic Games and Applications. Birkhäuser, Boston, 1995.Google Scholar
- M. Bardi, M. Falcone, and P. Soravia. Fully discrete schemes for the value function of pursuit-evasion games. In T. Basar and A. Haurie, editors, Advances in Stochastic Games and Applications. Birkhäuser, Boston, 199Google Scholar
- M. Bardi, M. Falcone, and P. Soravia. Numerical methods for pursuitevasion games via viscosity solutions. In M. Bardi, T.E.S. Raghavan, and T. Parthasarathy, editors, Stochastic and Differential Games: Theory and Numerical Methods. Birkh„auser, Boston, 1998.Google Scholar
- M. Bardi and P. Soravia. Approximation of differential games of pursuitevasion by discrete time games. In R.P. Hämäläinen and H.K. Ehtamo, editors, Differential Games: Developments in Modeling and Computation. Springer-Verlag, Berlin and New York, 1991.Google Scholar
- P. Billingsley. Convergence of Probability Measures; second edition. Wiley, New York, 1999.Google Scholar
- I. Karatzas and S.E. Shreve. Brownian Motion and Stochastic Calculus. Springer-Verlag, New York, 198Google Scholar
- H.J. Kushner. Heavy Traffic Analysis of Controlled Queueing and Communication Networks. Springer-Verlag, Berlin and New York, 2001.Google Scholar
- O. Pourtallier and M. Tidball. Approximation of the value function for a class of differential games with target. Research report 2942, INRIA, France, 1996.Google Scholar
- O. Pourtallier and B. Tolwinski. Discretization of Isaac’s equation. Report: INRIA, France, 1992.Google Scholar
- M. Tidball. Undiscounted zero-sum differential games with stopping times. In G.J. Olsder, editor, New Trends in Dynamic Games and Applications. Birkhäuser, Boston, 1995.Google Scholar
- M. Tidball and R.L.V. González. Zero-sum differential games with stopping times: Some results and about its numerical resolution. In T. Basar and A. Haurie, editors, Advances in Dynamic Games and Applications. Birkhäuser, Boston, 1994.Google Scholar