Some Numerical Methods

Abstract

In this chapter, we will explain some numerical methods to deal with two person stochastic differential games that are developed by Kushner ([107, 108] and [109]). Almost all of the materials of this chapter are extracted from these three excellent works of Kushner. The Markov chain approximation method is a powerful and more popularly used class of methods introduced in early 1990s by Kushner for the numerical solution of almost all of the standard forms of stochastic strategy problems [106, 109], Kushner and Ramachandran [113], and Kushner and Chamberlain [110]. The idea of the Markov chain approximation method is to first approximate the controlled diffusion dynamics by a suitable Markov chain on a finite state space with a discretization parameter h > 0, then approximate the payoff functions. One solves the game problem for the simpler chain model, and then proves that the value functions associated with equilibrium or δ -equilibrium strategies for the chain converge to the value functions associated with equilibrium or δ -equilibrium strategies for the diffusion model, as δ → 0. This is method is intuitive and it uses approximations which are “physically” close to the original problem. Extensions to approximations for two-person differential games with discounted, finite time, stopping time, and pursuit-evasion games were given in Kushner [108] for reflected diffusion models where the strategies for the two players are separated in the dynamics and payoff rate functions. An extension to two-player stochastic dynamic games with the same systems model, but where the payoff function is ergodic is given in Kushner [107].