Semi-stationary Equilibrium Strategies in Non-cooperative N-person Semi-Markov Games

  • Prasenjit MondalEmail author
  • Sagnik Sinha
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 302)


For a limiting ratio average (undiscounted) non-cooperative N-person semi-Markov game with finite state and action spaces, we prove that the solutions in the game where all players are restricted to semi-stationary strategies (that depend only on the initial state and the current state) are solutions for the unrestricted game. Furthermore, we consider zero-sum two-person semi-Markov games with action independent transitions (where the transition probabilities are independent of the actions of the players in each state) and prove the existence of an optimal semi-stationary strategy for each player. An example is provided to show that the semi-stationary optimal strategies cannot be strengthened further for such class of games.


Semi-Markov games Limiting ratio average payoff Nash equilibrium Action independent transitions Optimal semi-stationary strategies 

Mathemetics Subject Classification (2000)

Primary: 91A15 Secondary: 60G99 



The authors wish to thank the unknown referees who have patiently gone through this paper and whose suggestions have improved its presentation and readability considerably.


  1. 1.
    Bertsekas, D.P., Shreve, S.E.: Stochastic optimal control: The discrete time case. Academic Press NY, 139 (1978)Google Scholar
  2. 2.
    Blackwell, D., Ferguson, T.S.: The big match. Ann. Math. Statist 39(1), 159–163 (1968)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Doob, J.L.: Stochastic Processes. Willey ISBN 0-471-52369 (1953)Google Scholar
  4. 4.
    Feinberg, E.A.: Constrained Semi-Markov decision processes with average rewards. Math. Methods Operat. Res. 39(3), 257–288 (1994)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Ghosh, M.K., Sinha, S.: Non-cooperative N-person semi-Markov games on a denumerable state space. Comp. Appl. Math. 21, 833–847 (2002)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Jaskiewicz, A.: Zero-sum semi-Markov games. SIAM journal on control and optimization 41(3), 723–739 (2002)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Jianyong, L., Xiaobo, Z.: On average reward semi-Markov decision processes with a general multichain structure. Math. Operat. Res. 29(2), 339–352 (2004)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Lal, A.K., Sinha, S.: Zero-sum two-person semi-Markov games. J. Appl. Prob. 29(1), 56–72 (1992)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Loomis, L.H.: On a theorem of von Neumann. Proc. Nat. Acad. Sci. 32, 213–215 (1946)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Mondal, P., Sinha, S.: Ordered field property in a subclass of finite SeR-SIT Semi-Markov Games. Int. Game Theor. Rev. 15(4) (2013). 1340026-1-1340026-20MathSciNetCrossRefGoogle Scholar
  11. 11.
    Mondal, P., Sinha, S.: Ordered Field Property for Semi-Markov Games when One Player Controls Transition Probabilities and Transition Times, International Game Theory Review, 17(2). (2015). 1540022-1-1540022-26MathSciNetCrossRefGoogle Scholar
  12. 12.
    Mondal, P., Sinha, S., Neogy, S.K., Das, A.K.: On Discounted AR-AT Semi-Markov games and its complementarity formulations. Int. J. Game Theor. 45(3), 567–583 (2016)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Mondal, P.: Linear Programming and Zero-Sum Two-Person Undiscounted Semi-Markov Games. Asia-Pacific Journal of Operational Research, 32(6), 1550043-1-1550043-20 (2015)Google Scholar
  14. 14.
    Mondal, P.: On undiscounted Semi-Markov decision processes with absorbing states. Math. Methods Oper. Res. 83(2), 161–177 (2016)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Mondal, P.: On zero-sum two-person undiscounted Semi-Markov games with a multichain structure. Advanc. Appl. Probab. 49(3), 826–849 (2017)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Polowczuk, W.: Nonzero-sum semi-Markov games with countable state spaces. Applicationes Mathematicae 27(4), 395–402 (2000)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Shapley, L.: Stochastic Games. Proc. Natl. Acad. Sci. USA 39(10), 1095–1100 (1953)CrossRefGoogle Scholar
  18. 18.
    Sinha, S., Mondal, P.: Semi-Markov decision processes with limiting ratio average rewards. J. Math. Anal. Appl. 455(1), 864–871 (2017)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Stern, M.A.: On Stochastic Games with Limiting Average Payoff. University of Illinois at Chicago Circle, Chicago, Illinois, PhD Thesis (1975)Google Scholar
  20. 20.
    Vega-Amaya, O.: Zero-sum average semi-Markov games: fixed-point solutions of the Shapley equation. SIAM J. Control Optimizat. 42(5), 1876–1894 (2003)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  1. 1.Mathematics DepartmentGovernment General Degree CollegeBankuraIndia
  2. 2.Mathematics DepartmentJadavpur UniversityKolkataIndia

Personalised recommendations