Numerical Methods for Non-Zero-Sum Stochastic Differential Games: Convergence of the Markov Chain Approximation Method

  • Harold J. Kushner
Part of the The IMA Volumes in Mathematics and its Applications book series (IMA, volume 145)


The Markov chain approximation method is an efficient and popular collection of methods for the numerical solution of stochastic control problems in continuous time, for reflected-jump-diffusion-type models and the convergence proofs have been extended to zero-sum stochastic differential games. We apply it to a class of non-zero-sum stochastic differential games with a diffusion system model where the controls for the two players are separated, It is shown that equilibrium values for the approximating chain converge to equilibrium values for the original process and that any equilibrium value for the original process can be approximated by an ε-equilibrium for the chain for arbitrarily small ε > 0. The actual numerical algorithm is that for a stochastic game for a finite-state Markov chain.


Wiener Process Differential Game Convergence Proof Game Problem Stochastic Control Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    A. BENSOUSSAN AND A. FRIEDMAN. Nonzero-sum stochastic differential games with stopping times and free boundaries. Trans. Amer. Math Soc., 231: 275–327, 1977.MATHMathSciNetGoogle Scholar
  2. [2]
    I. BERKES AND W. PHILIPP. Approximation theorems for independent and weakly dependent random vectors. Ann. Probab., 7: 29–54, 1979.MATHCrossRefMathSciNetGoogle Scholar
  3. [3]
    R. BUCKDAHN, P. CARDALIAGUET, AND C. RAINIER. Nash equilibrium payoffs for nonzero-sum stochastic differential games. SIAM J. Control and Optimization, 43: 624–642, 2002.CrossRefGoogle Scholar
  4. [4]
    E. ALTMAN, O. POURTALLIER, A. HAURIE, AND F. MORESINO. Approximating Nash equilibria in nonzero-sum games. International Game Theory Review, 2: 155–172, 2000.MATHMathSciNetGoogle Scholar
  5. [5]
    R.J. ELLIOTT AND N.J. KALTON. Existence of value in differential games, Mem. AMS, 126. Amer. Math. Soc, Providence, RI, 1974.Google Scholar
  6. [6]
    S.N. ETHIER AND T.G. KURTZ. Markov Processes: Characterization and Convergence. Wiley, New York, 1986.MATHGoogle Scholar
  7. [7]
    W.F. FLEMING. Generalized solutions in optimal stochastic control. In P.T. Liu, E. Roxin, and R. Sternberg, editors, Differential Games and Control Theory: III, pp. 147–165. Marcel Dekker, 1977.Google Scholar
  8. [8]
    W.H. FLEMING AND P.E. SOUGANIDIS. On the existence of value functions for two-player zero-sum differential games. Indiana Univ. Math. J., 38: 293–314, 1989.MATHCrossRefMathSciNetGoogle Scholar
  9. [9]
    A. FRIEDMAN. Stochastic differential games. J. Differential Equations, 11: 79–108, 1972.MATHCrossRefMathSciNetGoogle Scholar
  10. [10]
    S. HAMADENE. Points d’équilibre dans les jeux stochastiques de somme non nulle. C. R. Acad. Sei. Paris Sér. I Math, 318: 251–256, 1994.MATHMathSciNetGoogle Scholar
  11. [11]
    A.B. HAURIE, J.B. KRAWCZYK, AND M. ROCHE. Monitoring cooperative equilibria in a stochastic differential game. J. of Optimiz. Theory and Applic, 81: 73–95, 1994.MATHCrossRefMathSciNetGoogle Scholar
  12. [12]
    A.B. HAURIE AND F. MORESENO. Computing equilibria in stochastic games of intergenerational equity. In International Conference on Dynamic Games. International Society of Dynamic Games, 2004.Google Scholar
  13. [13]
    H.J. KUSHNER. Probability Methods for Approximations in Stochastic Control and for Elliptic Equations. Academic Press, New York, 1977.MATHGoogle Scholar
  14. [14]
    H.J. KUSHNER. Numerical methods for stochastic control problems in continuous time. SIAM J. Control Optim., 28: 999–1048, 1990.MATHCrossRefMathSciNetGoogle Scholar
  15. [15]
    H.J. KUSHNER. Numerical methods for stochastic differential games. SIAM J. Control Optim., pp. 457–486, 2002.Google Scholar
  16. [16]
    H.J. KUSHNER. Numerical approximations for stochastic differential games: The ergodic case. SIAM J. Control Optim., 42: 1911–1933, 2004.MATHCrossRefMathSciNetGoogle Scholar
  17. [17]
    H.J. KUSHNER. Numerical methods for stochastic differential games: The ergodic cost criterion. In S. Jorgensen, M. Quincampoix, and T. Vincent, editors, Annals of the International Society of Dynamic Games, 2005. To appear.Google Scholar
  18. [18]
    H.J. KUSHNER AND P. DUPUIS. Numerical Methods for Stochastic Control Problems in Continuous Time. Springer-Verlag, Berlin and New York, 1992. Second edition, 2001.MATHGoogle Scholar
  19. [19]
    K.M. RAMACHANDRAN. Stochastic differential games and applications. In Handbook of Stochastic Analysis and Applications, pp. 473–532. Marcel Dekker, New York, 2002.Google Scholar

Copyright information

© Springer Science + Business Media, LLC 2008

Authors and Affiliations

  • Harold J. Kushner
    • 1
  1. 1.Applied Mathematics DepartmentBrown UniversityProvidence

Personalised recommendations