Ergodic Problems in Differential Games

  • Olivier Alvarez
  • Martino Bardi
Part of the Annals of the International Society of Dynamic Games book series (AISDG, volume 9)


We present and study a notion of ergodicity for deterministic zero-sum differential games that extends the one in classical ergodic control theory to systems with two conflicting controllers.We show its connections with the existence of a constant and uniform long-time limit of the value function of finite horizon games, and characterize this property in terms of Hamilton-Jacobi-Isaacs equations.We also give several sufficient conditions for ergodicity and describe some extensions of the theory to stochastic differential games.


Differential Game Stochastic Game Cell Problem Dynamic Programming Principle Stochastic Differential Game 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    O. Alvarez, Homogenization of Hamilton-Jacobi equations in perforated sets, J. Differential Equations 159 (1999), 543–577.MATHCrossRefMathSciNetGoogle Scholar
  2. [2]
    O. Alvarez and M. Bardi, Viscosity solutions methods for singular perturbations in deterministic and stochastic control, SIAM J. Control Optim. 40 (2001), 1159–1188.MATHCrossRefMathSciNetGoogle Scholar
  3. [3]
    O. Alvarez and M. Bardi, Singular perturbations of degenerate parabolic PDEs: a general convergence result, Arch. Rational Mech. Anal. 170 (2003), 17–61.MATHCrossRefMathSciNetGoogle Scholar
  4. [4]
    O. Alvarez and M. Bardi, Ergodicity, stabilization, and singular perturbations of Bellman-Isaacs equations, Preprint, University of Padova, to appear.Google Scholar
  5. [5]
    O. Alvarez, M. Bardi, and C. Marchi, Multiscale problems and homogenization for second-order Hamilton-Jacobi equations, Preprint, University of Padova, submitted.Google Scholar
  6. [6]
    M. Arisawa, Ergodic problem for the Hamilton-Jacobi-Bellman equation II, Ann. Inst. H. Poincaré Anal. Non Linéaire 15 (1998), 1–24.MATHCrossRefMathSciNetGoogle Scholar
  7. [7]
    M. Arisawa and P.-L. Lions, On ergodic stochastic control, Comm. Partial Differential Equations 23 (1998), 2187–2217.MATHCrossRefMathSciNetGoogle Scholar
  8. [8]
    Z. Artstein and V. Gaitsgory, The value function of singularly perturbed control systems, Appl. Math. Optim. 41 (2000), 425–445.MATHCrossRefMathSciNetGoogle Scholar
  9. [9]
    M. Bardi and I. Capuzzo-Dolcetta, Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations, Birkhäuser, Boston, 1997.MATHGoogle Scholar
  10. [10]
    A. Bensoussan, Perturbation methods in optimal control, Wiley/Gauthiers-Villars, Chichester, 1988.MATHGoogle Scholar
  11. [11]
    R. Buckdahn and N. Ichihara, Limit theorems for controlled backward SDEs and homogenization of Hamilton-Jacobi-Bellman equations, Appl. Math. Optim. 51 (2005), 1–33.CrossRefMathSciNetGoogle Scholar
  12. [12]
    D. A. Carlson and A. B. Haurie, A turnpike theory for infinite horizon openloop differential games with decoupled controls, New trends in dynamic games and applications (G.J. Olsder, ed.),Ann. Internat. Soc. Dynam. Games, no. 3, Birkhäuser Boston, 1995, pp. 353–376.Google Scholar
  13. [13]
    D. A. Carlson, A. B. Haurie, and A. Leizarowitz, Infinite horizon optimal control: Deterministic and stochastic systems, Springer-Verlag, Berlin, 1991.MATHGoogle Scholar
  14. [14]
    F. Colonius and W. Kliemann, The dynamics of control, Birkhäuser, Boston, 2000.Google Scholar
  15. [15]
    L. Evans, The perturbed test function method for viscosity solutions of nonlinear P.D.E., Proc. Roy. Soc. Edinburgh Sect. A 111 (1989), 359–375.MathSciNetGoogle Scholar
  16. [16]
    L. Evans, Periodic homogenisation of certain fully nonlinear partial differential equations, Proc. Roy. Soc. Edinburgh Sect. A 120 (1992), 245–265.MathSciNetGoogle Scholar
  17. [17]
    L. Evans and P. E. Souganidis, Differential games and representation formulas for solutions of Hamilton-Jacobi-Isaacs equations, Indiana Univ. Math. J. 33 (1984), 773–797.CrossRefMathSciNetGoogle Scholar
  18. [18]
    W. H. Fleming and W. M. McEneaney, Risk-sensitive control on an infinite time horizon, SIAM J. Control Optim. 33 (1995), 1881–1915.MATHCrossRefMathSciNetGoogle Scholar
  19. [19]
    W. H. Fleming and P. E. Souganidis, On the existence of value functions of two-players, zero-sum stochastic differential games, Indiana Univ. Math. J. 38 (1989), 293–314.MATHCrossRefMathSciNetGoogle Scholar
  20. [20]
    V. Gaitsgory, Limit Hamilton-Jacobi-Isaacs equations for singularly perturbed zero-sum differential games, J. Math. Anal. Appl. 202 (1996), 862–899.MATHCrossRefMathSciNetGoogle Scholar
  21. [21]
    M. K. Ghosh and K. S. M. Rao, Differential games with ergodic payoff, SIAM J. Control Optim. 43 (2005), 2020–2035.MATHCrossRefMathSciNetGoogle Scholar
  22. [22]
    L. Grüne, On the relation between discounted and average optimal value functions, J. Differential Equations 148 (1998), 65–99.MATHCrossRefMathSciNetGoogle Scholar
  23. [23]
    H. Ishii, Homogenization of the Cauchy problem for Hamilton-Jacobi equations, Stochastic analysis, control, optimization and applications. A volume in honor of Wendell H. Fleming (W.M. McEneaney, G. Yin, and Q. Zhang, eds.), Birkhäuser, Boston, 1999, pp. 305–324.Google Scholar
  24. [24]
    Y. Kabanov and S. Pergamenshchikov, Optimal control of singularly perturbed linear stochastic systems, Stochastics Stochastics Rep. 36 (1991), 109–135.MATHMathSciNetGoogle Scholar
  25. [25]
    P. V. Kokotović, H. K. Khalil, and J. O’Reilly, Singular perturbation methods in control: analysis and design, Academic Press, London, 1986.Google Scholar
  26. [26]
    H. J. Kushner, Weak convergence methods and singularly perturbed stochastic control and filtering problems, Birkhäuser, Boston, 1990.MATHGoogle Scholar
  27. [27]
    H. J. Kushner, Numerical approximations for stochastic differential games: the ergodic case, SIAM J. Control Optim. 42 (2004), 1911–1933.MATHCrossRefMathSciNetGoogle Scholar
  28. [28]
    P. L. Lions, Neumann type boundary conditions for Hamilton-Jacobi equations, Duke Math. J. 52 (1985), 793–820.MATHCrossRefMathSciNetGoogle Scholar
  29. [29]
    P. L. Lions, G. Papanicolaou, and S. R. S. Varadhan, Homogenization of Hamilton-Jacobi equations, Unpublished, 1986.Google Scholar
  30. [30]
    C. Marchi, Homogenization for fully nonllinear parabolic equations, Nonlinear Anal. 60 (2005), 411–428.MATHCrossRefMathSciNetGoogle Scholar
  31. [31]
    M. Quincampoix and F. Watbled, Averaging method for discontinuous Mayer’s problem of singularly perturbed control systems, Nonlinear Anal. 54 (2003), 819–837.MATHCrossRefMathSciNetGoogle Scholar
  32. [32]
    P. Soravia, Pursuit-evasion problems and viscosity solutions of Isaacs equations, SIAM J. Control Optim. 31 (1993), 604–623.MATHCrossRefMathSciNetGoogle Scholar
  33. [33]
    P. Soravia, Stability of dynamical systems with competitive controls: the degenerate case, J. Math. Anal. Appl. 191 (1995), 428–449.MATHCrossRefMathSciNetGoogle Scholar
  34. [34]
    S. Sorin, New approaches and recent advances in two-person zero-sum repeated games, Advances in dynamic games (A.S. Nowak and K. Szajowski, (eds.), Ann. Internat. Soc. Dynam. Games, no. 7, Birkhäuser, Boston, 2005, pp. 67–93.Google Scholar
  35. [35]
    N. N. Subbotina, Asymptotic properties of minimax solutions of Isaacs-Bellman equations in differential games with fast and slow motions, J. Appl. Math. Mech. 60 (1996), 883–890.MATHCrossRefMathSciNetGoogle Scholar
  36. [36]
    N. N. Subbotina, Asymptotics for singularly perturbed differential games, Game theory and applications, Vol. VII, Nova Sci. Publ., Huntington, NY, 2001, pp. 175–196.Google Scholar
  37. [37]
    A. Swiech, Another approach to the existence of value functions of stochastic differential games, J. Math. Anal. Appl. 204 (1996), 884–897.MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Birkhäuser Boston 2007

Authors and Affiliations

  • Olivier Alvarez
    • 1
  • Martino Bardi
    • 2
  1. 1.UMR 60-85Université de RouenMont-Saint Aignan CedexFrance
  2. 2.Dipartimento di Matematica Pura ed ApplicataUniversità di PadovaPadovaItaly

Personalised recommendations