Constrained Expected Average Stochastic Games for Continuous-Time Jump Processes

  • Qingda WeiEmail author


In this paper we study the nonzero-sum constrained stochastic games for continuous-time jump processes with denumerable states and possibly unbounded transition rates. The optimality criterion under consideration is the expected average payoff criterion and the payoff functions of the players are allowed to be unbounded. Under the reasonable conditions, we introduce an approximating sequence of the auxiliary game models and show the existence of stationary constrained Nash equilibria for these approximating game models via employing the average occupation measures and constructing a suitable multifunction. Moreover, we obtain that any limit point of the stationary constrained Nash equilibria for the approximating sequence of the game models is a constrained Nash equilibrium for the original game model. Furthermore, we use a controlled birth and death system to illustrate our main results.


Nonzero-sum games Expected average payoff criterion Constrained Nash equilibrium Continuous-time jump process 

Mathematics Subject Classification

91A15 91A25 



I am greatly indebted to the referees for the valuable comments and suggestions which have greatly improved the presentation. The research was supported by National Natural Science Foundation of China (Grant No. 11601166).

Compliance with Ethical Standards

Conflict of interest

I declare that no conflict of interest exists in this paper.


  1. 1.
    Aliprantis, C., Border, K.: Infinite Dimensional Analysis. Springer, New York (2006)zbMATHGoogle Scholar
  2. 2.
    Altman, E., Shwartz, A.: Constrained Markov Games: Nash Equilibria. Ann Int Soc Dynam Games, vol. 5, pp. 213–221. Birkhäuser, Boston (2000)zbMATHGoogle Scholar
  3. 3.
    Alvarez-Mena, J., Hernández-Lerma, O.: Convergence of the optimal values of constrained Markov control processes. Math. Methods Oper. Res. 55, 461–484 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Alvarez-Mena, J., Hernández-Lerma, O.: Existence of Nash equilibria for constrained stochastic games. Math. Methods Oper. Res. 63, 261–285 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Bertsekas, D.P., Shreve, S.E.: Stochastic Optimal Control: The Discrete-Time Case. Athena Scientific, Belmont (1996)zbMATHGoogle Scholar
  6. 6.
    Föllmer, H., Schied, A.: Stochastic Finance: An Introduction in Discrete Time. Walter de Gruyter, Berlin (2004)CrossRefzbMATHGoogle Scholar
  7. 7.
    Guo, X.P., Hernández-Lerma, O.: Continuous-Time Markov Decision Processes: Theory and Applications. Springer, Berlin (2009)CrossRefzbMATHGoogle Scholar
  8. 8.
    Guo, X.P., Song, X.Y.: Discounted continuous-time constrained Markov decision processes in Polish spaces. Ann. Appl. Probab. 21, 2016–2049 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Guo, X.P., Huang, Y.H., Song, X.Y.: Linear programming and constrained average optimality for general continuous-time Markov decision processes in history-dependent policies. SIAM J. Control Optim. 50, 23–47 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Hernández-Lerma, O., Lasserre, J.B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer, New York (1996)CrossRefzbMATHGoogle Scholar
  11. 11.
    Kitaev, M.Y., Rykov, V.V.: Controlled Queueing Systems. CRC Press, Boca Raton (1995)zbMATHGoogle Scholar
  12. 12.
    Piunovskiy, A., Zhang, Y.: Discounted continuous-time Markov decision processes with unbounded rates: the convex analytic approach. SIAM J. Control Optim. 49, 2032–2061 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Prieto-Rumeau, T., Hernández-Lerma, O.: Ergodic control of continuous-time Markov chains with pathwise constraints. SIAM J. Control Optim. 47, 1888–1908 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Wei, Q.D., Chen, X.: Strong average optimality criterion for continuous-time Markov decision processes. Kybernetika 50, 950–977 (2014)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Wei, Q.D., Chen, X.: Constrained stochastic games with the average payoff criteria. Oper. Res. Lett. 43, 83–88 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Wei, Q.D., Chen, X.: Stochastic games for continuous-time jump processes under finite-horizon payoff criterion. Appl. Math. Optim. 74, 273–301 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Zhang, W.Z.: Continuous-time constrained stochastic games under the discounted cost criteria. Appl. Math. Optim. 77, 275–296 (2018)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Zhang, W.Z., Guo, X.P.: Nonzero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates. Sci. China Math. 55, 2405–2416 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Zhang, W.Z., Huang, Y.H., Guo, X.P.: Nonzero-sum constrained discrete-time Markov games: the case of unbounded costs. TOP 22, 1074–1102 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Zhang, W.Z., Wang, B.F., Chen, D.W.: Continuous-time constrained stochastic games with average criteria. Oper. Res. Lett. 46, 109–115 (2018)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Economics and FinanceHuaqiao UniversityQuanzhouPeople’s Republic of China

Personalised recommendations