Nonzero-Sum Stochastic Games

  • Anna JaśkiewiczEmail author
  • Andrzej S. Nowak
Reference work entry


This chapter describes a number of results obtained in the last 60 years on the theory of nonzero-sum discrete-time stochastic games. We provide an overview of almost all basic streams of research in this area such as the existence of stationary Nash and correlated equilibria in models on countable and general state spaces, the existence of subgame-perfect equilibria, algorithms, stopping games, and the existence of uniform equilibria. Our survey incorporates several examples of games studied in operations research and economics. In particular, separate sections are devoted to intergenerational games, dynamic Cournot competition and game models of resource extraction. The provided reference list includes not only seminal papers that commenced research in various directions but also exposes recent advances in this field.


Nonzero-sum game Stochastic game Discounted payoff Limit-average payoff Markov perfect equilibrium Subgame-perfect equilibrium Intergenerational altruism Uniform equilibrium Stopping game 



We thank Tamer Başar and Georges Zaccour for inviting us to write this chapter and their help. We also thank Elżbieta Ferenstein, János Flesch, Eilon Solan, Yeneng Sun, Krzysztof Szajowski and two reviewers for their comments on an earlier version of this survey.


  1. Abreu D, Pearce D, Stacchetti E (1986) Optimal cartel equilibria with imperfect monitoring. J Econ Theory 39:251–269MathSciNetzbMATHCrossRefGoogle Scholar
  2. Abreu D, Pearce D, Stacchetti E (1990) Toward a theory of discounted repeated games with imperfect monitoring. Econometrica 58:1041–1063MathSciNetzbMATHCrossRefGoogle Scholar
  3. Adlakha S, Johari R (2013) Mean field equilibrium in dynamic games with strategic complementarities. Oper Res 61:971–989MathSciNetzbMATHCrossRefGoogle Scholar
  4. Aliprantis C, Border K (2006) Infinite dimensional analysis: a Hitchhiker’s guide. Springer, New YorkzbMATHGoogle Scholar
  5. Alj A, Haurie A (1983) Dynamic equilibria in multigenerational stochastic games. IEEE Trans Autom Control 28:193–203zbMATHCrossRefGoogle Scholar
  6. Alós-Ferrer C, Ritzberger K (2015) Characterizing existence of equilibrium for large extensive form games: a necessity result. Econ Theory 63:407–430MathSciNetzbMATHCrossRefGoogle Scholar
  7. Alós-Ferrer C, Ritzberger K (2016) Equilibrium existence for large perfect information games. J Math Econ 62:5–18MathSciNetzbMATHCrossRefGoogle Scholar
  8. Altman E (1996) Non-zero-sum stochastic games in admission, service and routing control in queueing systems. Queueing Syst Theory Appl 23:259–279MathSciNetzbMATHCrossRefGoogle Scholar
  9. Altman E, Avrachenkov K, Bonneau N, Debbah M, El-Azouzi R, Sadoc Menasche D (2008) Constrained cost-coupled stochastic games with independent state processes. Oper Res Lett 36:160–164MathSciNetzbMATHCrossRefGoogle Scholar
  10. Altman E, Avrachenkov K, Marquez R, Miller G (2005) Zero-sum constrained stochastic games with independent state processes. Math Methods Oper Res 62:375–386MathSciNetzbMATHCrossRefGoogle Scholar
  11. Altman E, Hordijk A, Spieksma FM (1997) Contraction conditions for average and α-discount optimality in countable state Markov games with unbounded rewards. Math Oper Res 22: 588–618MathSciNetzbMATHCrossRefGoogle Scholar
  12. Amir R (1996a) Continuous stochastic games of capital accumulation with convex transitions. Games Econ Behavior 15:132–148MathSciNetzbMATHCrossRefGoogle Scholar
  13. Amir R (1996b) Strategic intergenerational bequests with stochastic convex production. Econ Theory 8:367–376zbMATHCrossRefGoogle Scholar
  14. Amir R (2003) Stochastic games in economics: the lattice-theoretic approach. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 443–453zbMATHCrossRefGoogle Scholar
  15. Artstein Z (1989) Parametrized integration of multifunctions with applications to control and optimization. SIAM J Control Optim 27:1369–1380MathSciNetzbMATHCrossRefGoogle Scholar
  16. Aumann RJ (1974) Subjectivity and correlation in randomized strategies. J Math Econ 1:67–96MathSciNetzbMATHCrossRefGoogle Scholar
  17. Aumann RJ (1987) Correlated equilibrium as an expression of Bayesian rationality. Econometrica 55:1–18MathSciNetzbMATHCrossRefGoogle Scholar
  18. Balbus Ł, Jaśkiewicz A, Nowak AS (2014) Robust Markov perfect equilibria in a dynamic choice model with quasi-hyperbolic discounting. In: Haunschmied J et al (eds) Dynamic games in economics, dynamic modeling and econometrics in economics and finance 16. Springer, Berlin/Heidelberg, pp 1–22zbMATHGoogle Scholar
  19. Balbus Ł, Jaśkiewicz A, Nowak AS (2015a) Existence of stationary Markov perfect equilibria in stochastic altruistic growth economies. J Optim Theory Appl 165:295–315MathSciNetzbMATHCrossRefGoogle Scholar
  20. Balbus Ł, Jaśkiewicz A, Nowak AS (2015b) Stochastic bequest games. Games Econ Behavior 90:247–256MathSciNetzbMATHCrossRefGoogle Scholar
  21. Balbus Ł, Jaśkiewicz A, Nowak AS (2015c) Bequest games with unbounded utility functions. J Math Anal Appl 427:515–524MathSciNetzbMATHCrossRefGoogle Scholar
  22. Balbus Ł, Jaśkiewicz A, Nowak AS (2016) Non-paternalistic intergenerational altruism revisited. J Math Econ 63:27–33MathSciNetzbMATHCrossRefGoogle Scholar
  23. Balbus Ł, Nowak AS (2004) Construction of Nash equilibria in symmetric stochastic games of capital accumulation. Math Methods Oper Res 60:267–277MathSciNetzbMATHCrossRefGoogle Scholar
  24. Balbus Ł, Nowak AS (2008) Existence of perfect equilibria in a class of multigenerational stochastic games of capital accumulation. Automatica 44:1471–1479MathSciNetzbMATHCrossRefGoogle Scholar
  25. Balbus Ł, Reffett K, Woźny Ł (2012) Stationary Markovian equilibria in altruistic stochastic OLG models with limited commitment. J Math Econ 48:115–132zbMATHCrossRefGoogle Scholar
  26. Balbus Ł, Reffett K, Woźny Ł (2013a) A constructive geometrical approach to the uniqueness of Markov stationary equilibrium in stochastic games of intergenerational altruism. J Econ Dyn Control 37:1019–1039MathSciNetCrossRefGoogle Scholar
  27. Balbus Ł, Reffett K, Woźny Ł (2013b) Markov stationary equilibria in stochastic supermodular games with imperfect private and public information. Dyn Games Appl 3:187–206MathSciNetzbMATHCrossRefGoogle Scholar
  28. Balbus Ł, Reffett K, Woźny Ł (2014a) Constructive study of Markov equilibria in stochastic games with complementarities. J Econ Theory 150:815–840MathSciNetzbMATHCrossRefGoogle Scholar
  29. Balbus Ł, Jaśkiewicz A, Nowak AS, Woźny L (2017) A note on Markov perfect equilibria in a class of non-stationary bequest games. J Math Anal Appl 456:394–401MathSciNetzbMATHCrossRefGoogle Scholar
  30. Balbus Ł, Jaśkiewicz A, Nowak AS (2018, in press) Markov perfect equilibria in a dynamic decision model with quasi-hyperbolic discounting. Ann Oper ResGoogle Scholar
  31. Barelli P, Duggan J (2014) A note on semi-Markov perfect equilibria in discounted stochastic games. J Econ Theory 15:596–604MathSciNetzbMATHCrossRefGoogle Scholar
  32. Başar T, Olsder GJ (1995) Dynamic noncooperative game theory. Academic, New YorkzbMATHGoogle Scholar
  33. Berg K (2016) Elementary subpaths in discounted stochastic games. Dyn Games Appl 6(3): 304–323MathSciNetzbMATHCrossRefGoogle Scholar
  34. Bernheim D, Ray D (1983) Altruistic growth economies I. Existence of bequest equilibria. Technical report no 419, Institute for Mathematical Studies in the Social Sciences, Stanford UniversityGoogle Scholar
  35. Bernheim D, Ray D (1986) On the existence of Markov-consistent plans under production uncertainty. Rev Econ Stud 53:877–882MathSciNetzbMATHCrossRefGoogle Scholar
  36. Bernheim D, Ray D (1987) Economic growth with intergenerational altruism. Rev Econ Stud 54:227–242MathSciNetzbMATHCrossRefGoogle Scholar
  37. Bernheim D, Ray D (1989) Markov perfect equilibria in altruistic growth economies with production uncertainty. J Econ Theory 47:195–202MathSciNetzbMATHCrossRefGoogle Scholar
  38. Bhattacharya R, Majumdar M (2007) Random dynamical systems: theory and applications. Cambridge University Press, CambridgezbMATHCrossRefGoogle Scholar
  39. Billingsley P (1968) Convergence of probability measures. Wiley, New YorkzbMATHGoogle Scholar
  40. Blackwell D (1965) Discounted dynamic programming. Ann Math Stat 36:226–235MathSciNetzbMATHCrossRefGoogle Scholar
  41. Blackwell D (1969) Infinite Gδ -games with imperfect information. Zastosowania Matematyki (Appl Math) 10:99–101MathSciNetzbMATHGoogle Scholar
  42. Borkar VS, Ghosh MK (1993) Denumerable state stochastic games with limiting average payoff. J Optim Theory Appl 76:539–560MathSciNetzbMATHCrossRefGoogle Scholar
  43. Browder FE (1960) On continuity of fixed points under deformations of continuous mappings. Summa Brasiliensis Math 4:183–191MathSciNetzbMATHGoogle Scholar
  44. Carlson D, Haurie A (1996) A turnpike theory for infinite-horizon open-loop competitive processes. SIAM J Control Optim 34:1405–1419MathSciNetzbMATHCrossRefGoogle Scholar
  45. Castaing C, Valadier M (1977) Convex analysis and measurable multifunctions. Lecture notes in mathematics, vol 580. Springer, New YorkzbMATHGoogle Scholar
  46. Chatterjee S, Eyigungor B (2016) Continuous Markov equilibria with quasi-geometric discounting. J Econ Theory 163:467–494MathSciNetzbMATHCrossRefGoogle Scholar
  47. Chiarella C, Kemp MC, Van Long N, Okuguchi K (1984) On the economics of international fisheries. Int Econ Rev 25:85–92MathSciNetzbMATHCrossRefGoogle Scholar
  48. Cingiz K, Flesch J, Herings JJP, Predtetchinski A (2016) Doing it now, later, or never. Games Econ Behavior 97:174–185MathSciNetzbMATHCrossRefGoogle Scholar
  49. Cole HL, Kocherlakota N (2001) Dynamic games with hidden actions and hidden states. J Econ Theory 98:114–126MathSciNetzbMATHCrossRefGoogle Scholar
  50. Cottle RW, Pang JS, Stone RE (1992) The linear complementarity problem. Academic, New YorkzbMATHGoogle Scholar
  51. Curtat LO (1996) Markov equilibria of stochastic games with complementarities. Games Econ Behavior 17:177–199MathSciNetzbMATHCrossRefGoogle Scholar
  52. Doraszelski U, Escobar JF (2010) A theory of regular Markov perfect equilibria in dynamic stochastic games: genericity, stability, and purification. Theor Econ 5:369–402MathSciNetzbMATHCrossRefGoogle Scholar
  53. Doraszelski U, Pakes A (2007) A framework for applied dynamic analysis in IO. In: Amstrong M, Porter RH (eds) Handbook of industrial organization, vol 3. North Holland, Amsterdam/ London, pp 1887–1966Google Scholar
  54. Doraszelski U, Satterthwaite M (2010) Computable Markov-perfect industry dynamics. Rand J Econ 41:215–243CrossRefGoogle Scholar
  55. Dubins LE, Savage LJ (2014) Inequalities for stochastic processes. Dover, New YorkGoogle Scholar
  56. Duffie D, Geanakoplos J, Mas-Colell A, McLennan A (1994) Stationary Markov equilibria. Econometrica 62:745–781MathSciNetzbMATHCrossRefGoogle Scholar
  57. Duggan J (2012) Noisy stochastic games. Econometrica 80:2017–2046MathSciNetzbMATHCrossRefGoogle Scholar
  58. Dutta PK (1995) A folk theorem for stochastic games. J Econ Theory 66:1–32MathSciNetzbMATHCrossRefGoogle Scholar
  59. Dutta PK, Sundaram RK (1992) Markovian equilibrium in class of stochastic games: existence theorems for discounted and undiscounted models. Econ Theory 2:197–214MathSciNetzbMATHCrossRefGoogle Scholar
  60. Dutta PK, Sundaram RK (1993) The tragedy of the commons? Econ Theory 3:413–426MathSciNetzbMATHCrossRefGoogle Scholar
  61. Dutta PK, Sundaram RK (1998) The equilibrium existence problem in general Markovian games. In: Majumdar M (ed) Organizations with incomplete information. Cambridge University Press, Cambridge, pp 159–207CrossRefGoogle Scholar
  62. Dynkin EB (1969) The game variant of a problem on optimal stopping. Sov Math Dokl 10:270–274zbMATHGoogle Scholar
  63. Dynkin EB, Evstigneev IV (1977) Regular conditional expectations of correspondences. Theory Probab Appl 21:325–338zbMATHCrossRefGoogle Scholar
  64. Eaves B (1972) Homotopies for computation of fixed points. Math Program 3:1–22MathSciNetzbMATHCrossRefGoogle Scholar
  65. Eaves B (1984) A course in triangulations for solving equations with deformations. Springer, BerlinzbMATHCrossRefGoogle Scholar
  66. Elliott RJ, Kalton NJ, Markus L (1973) Saddle-points for linear differential games. SIAM J Control Optim 11:100–112MathSciNetzbMATHCrossRefGoogle Scholar
  67. Enns EG, Ferenstein EZ (1987) On a multi-person time-sequential game with priorities. Seq Anal 6:239–256zbMATHCrossRefGoogle Scholar
  68. Ericson R, Pakes A (1995) Markov-perfect industry dynamics: a framework for empirical work. Rev Econ Stud 62:53–82zbMATHCrossRefGoogle Scholar
  69. Escobar JF (2013) Equilibrium analysis of dynamic models of imperfect competition. Int J Ind Organ 31:92–101CrossRefGoogle Scholar
  70. Federgruen A (1978) On N-person stochastic games with denumerable state space. Adv Appl Probab 10:452–471MathSciNetzbMATHCrossRefGoogle Scholar
  71. Ferenstein E (2007) Randomized stopping games and Markov market games. Math Methods Oper Res 66:531–544MathSciNetzbMATHCrossRefGoogle Scholar
  72. Filar JA, Schultz T, Thuijsman F, Vrieze OJ (1991) Nonlinear programming and stationary equilibria in stochastic games. Math Program 50:227–237MathSciNetzbMATHCrossRefGoogle Scholar
  73. Filar JA, Vrieze K (1997) Competitive Markov decision processes. Springer, New YorkzbMATHGoogle Scholar
  74. Fink AM (1964) Equilibrium in a stochastic n-person game. J Sci Hiroshima Univ Ser A-I Math 28:89–93MathSciNetzbMATHGoogle Scholar
  75. Flesch J, Schoenmakers G, Vrieze K (2008) Stochastic games on a product state space. Math Oper Res 33:403–420MathSciNetzbMATHCrossRefGoogle Scholar
  76. Flesch J, Schoenmakers G, Vrieze K (2009) Stochastic games on a product state space: the periodic case. Int J Game Theory 38:263–289MathSciNetzbMATHCrossRefGoogle Scholar
  77. Flesch J, Thuijsman F, Vrieze OJ (1997) Cyclic Markov equilibrium in stochastic games. Int J Garne Theory 26:303–314zbMATHCrossRefGoogle Scholar
  78. Flesch J, Thuijsman F, Vrieze OJ (2003) Stochastic games with non-observable actions. Math Methods Oper Res 58:459–475MathSciNetzbMATHCrossRefGoogle Scholar
  79. Flesch J, Kuipers J, Mashiah-Yaakovi A, Schoenmakers G, Solan E, Vrieze K (2010a) Perfect-information games with lower-semicontinuous payoffs. Math Oper Res 35:742–755MathSciNetzbMATHCrossRefGoogle Scholar
  80. Flesch J, Kuipers J, Schoenmakers G, Vrieze K (2010b) Subgame-perfection in positive recursive games with perfect information. Math Oper Res 35:193–207MathSciNetzbMATHCrossRefGoogle Scholar
  81. Flesch J, Kuipers J, Mashiah-Yaakovi A, Schoenmakers G, Shmaya E, Solan E, Vrieze K (2014) Non-existence of subgame-perfect 𝜖-equilibrium in perfect-information games with infinite horizon. Int J Game Theory 43:945–951MathSciNetzbMATHCrossRefGoogle Scholar
  82. Flesch J, Thuijsman F, Vrieze OJ (2007) Stochastic games with additive transitions. Eur J Oper Res 179:483–497MathSciNetzbMATHCrossRefGoogle Scholar
  83. Flesch J, Predtetchinski A (2015) On refinements of subgame perfect 𝜖 -equilibrium. Int J Game Theory 45:523–542MathSciNetzbMATHCrossRefGoogle Scholar
  84. Forges E (1986) An approach to communication equilibria. Econometrica 54:1375–1385MathSciNetzbMATHCrossRefGoogle Scholar
  85. Forges F (1992) Repeated games of incomplete information: nonzero-sum. In: Aumann RJ, Hart S (eds) Handbook of game theory, vol 1. North Holland, Amsterdam, pp 155–177zbMATHGoogle Scholar
  86. Forges F (2009) Correlated equilibria and communication in games. In: Meyers RA (ed) Encyclopedia of complexity and systems science. Springer, New York, pp 1587–1596CrossRefGoogle Scholar
  87. Fudenberg D, Levine D (1983) Subgame-perfect equilibria of finite and infinite horizon games. J Econ Theory 31:251–268MathSciNetzbMATHCrossRefGoogle Scholar
  88. Fudenberg D, Tirole J (1991) Game theory. MIT Press, CambridgezbMATHGoogle Scholar
  89. Fudenberg D, Yamamoto Y (2011) The folk theorem for irreducible stochastic games with imperfect public monitoring. J Econ Theory 146:1664–1683MathSciNetzbMATHCrossRefGoogle Scholar
  90. Gale D (1967) On optimal development in a multisector economy. Rev Econ Stud 34:1–19CrossRefGoogle Scholar
  91. Glicksberg IL (1952) A further generalization of the Kakutani fixed point theorem with application to Nash equilibrium points. Proc Am Math Soc 3:170–174MathSciNetzbMATHGoogle Scholar
  92. Govindan S, Wilson R (2003) A global Newton method to compute Nash equilibria. J Econ Theory 110:65–86MathSciNetzbMATHCrossRefGoogle Scholar
  93. Govindan S, Wilson R (2009) Global Newton method for stochastic games. J Econ Theory 144:414–421MathSciNetzbMATHCrossRefGoogle Scholar
  94. Haller H, Lagunoff R (2000) Genericity and Markovian behavior in stochastic games. Econometrica 68:1231–1248MathSciNetzbMATHCrossRefGoogle Scholar
  95. Hamadène S, Hassani M (2014) The multi-player nonzero-sum Dynkin game in discrete time. Math Methods Oper Res 79:179–194MathSciNetzbMATHCrossRefGoogle Scholar
  96. Harris C (1985) Existence and characterization of perfect equilibrium in games of perfect information. Econometrica 53:613–628MathSciNetzbMATHCrossRefGoogle Scholar
  97. Harris C, Laibson D (2001) Dynamic choices of hyperbolic consumers. Econometrica 69:935–957MathSciNetzbMATHCrossRefGoogle Scholar
  98. Harris C, Reny PJ, Robson A (1995) The existence of subgame-perfect equilibrium in continuous games with almost perfect information: a case for public randomization. Econometrica 63: 507–544MathSciNetzbMATHCrossRefGoogle Scholar
  99. Harsanyi JC (1973a) Oddness of the number of equilibrium points: a new proof. Int J Game Theory 2:235–250MathSciNetzbMATHCrossRefGoogle Scholar
  100. Harsanyi JC (1973b) Games with randomly disturbed payoffs: a new rationale for mixed-strategy equilibrium points. Int J Game Theory 2:1–23MathSciNetzbMATHCrossRefGoogle Scholar
  101. Haurie A, Krawczyk JB, Zaccour G (2012) Games and dynamic games. World Scientific, SingaporezbMATHCrossRefGoogle Scholar
  102. He W, Sun Y (2017) Stationary Markov perfect equilibria in discounted stochastic games. J Econ Theory 169:35–61MathSciNetzbMATHCrossRefGoogle Scholar
  103. Heller Y (2012) Sequential correlated equilibria in stopping games. Oper Res 60:209–224MathSciNetzbMATHCrossRefGoogle Scholar
  104. Herings JJP, Peeters RJAP (2004) Stationary equilibria in stochastic games: structure, selection, and computation. J Econ Theory 118:32–60MathSciNetzbMATHGoogle Scholar
  105. Herings JJP, Peeters RJAP (2010) Homotopy methods to compute equilibria in game theory. Econ Theory 42:119–156MathSciNetzbMATHCrossRefGoogle Scholar
  106. Himmelberg CJ (1975) Measurable relations. Fundam Math 87:53–72MathSciNetzbMATHCrossRefGoogle Scholar
  107. Himmelberg CJ, Parthasarathy T, Raghavan TES, Van Vleck FS (1976) Existence of p-equilibrium and optimal stationary strategies in stochastic games. Proc Am Math Soc 60:245–251MathSciNetzbMATHGoogle Scholar
  108. Hopenhayn H, Prescott E (1992) Stochastic monotonicity and stationary distributions for dynamic economies. Econometrica 60:1387–1406MathSciNetzbMATHCrossRefGoogle Scholar
  109. Hörner J, Sugaya T, Takahashi S, Vieille N (2011) Recursive methods in discounted stochastic games: an algorithm for δ → 1 and a folk theorem. Econometrica 79:1277–1318MathSciNetzbMATHCrossRefGoogle Scholar
  110. Hörner J, Takahashi S, Vieille N (2014) On the limit perfect public equilibrium payoff set in repeated and stochastic game. Games Econ Behavior 85:70–83MathSciNetzbMATHCrossRefGoogle Scholar
  111. Horst U (2005) Stationary equilibria in discounted stochastic games with weakly interacting players. Games Econ Behavior 51:83–108MathSciNetzbMATHCrossRefGoogle Scholar
  112. Jaśkiewicz A, Nowak AS (2006) Approximation of noncooperative semi-Markov games. J Optim Theory Appl 131:115–134MathSciNetzbMATHCrossRefGoogle Scholar
  113. Jaśkiewicz A, Nowak AS (2014a) Stationary Markov perfect equilibria in risk sensitive stochastic overlapping generations models. J Econ Theory 151:411–447MathSciNetzbMATHCrossRefGoogle Scholar
  114. Jaśkiewicz A, Nowak AS (2014b) Robust Markov perfect equilibria. J Math Anal Appl 419: 1322–1332MathSciNetzbMATHCrossRefGoogle Scholar
  115. Jaśkiewicz A, Nowak AS (2015a) On pure stationary almost Markov Nash equilibria in nonzero-sum ARAT stochastic games. Math Methods Oper Res 81:169–179MathSciNetzbMATHCrossRefGoogle Scholar
  116. Jaśkiewicz A, Nowak AS (2015b) Stochastic games of resource extraction. Automatica 54: 310–316MathSciNetzbMATHCrossRefGoogle Scholar
  117. Jaśkiewicz A, Nowak AS (2016) Stationary almost Markov perfect equilibria in discounted stochastic games. Math Oper Res 41:430–441MathSciNetzbMATHCrossRefGoogle Scholar
  118. Jaśkiewicz A, Nowak AS (2018a) Zero-sum stochastic games. In: Basar T, Zaccour G (eds) Handbook of dynamic game theory, Birkhäuser, BaselGoogle Scholar
  119. Jaśkiewicz A, Nowak AS (2018b, in press) On symmetric stochastic games of resource extraction with weakly continuous transitions. TOPGoogle Scholar
  120. Kakutani S (1941) A generalization of Brouwer’s fixed point theorem. Duke Math J 8:457–459MathSciNetzbMATHCrossRefGoogle Scholar
  121. Kiefer YI (1971) Optimal stopped games. Theory Probab Appl 16:185–189CrossRefGoogle Scholar
  122. Kitti M (2016) Subgame-perfect equilibria in discounted stochastic games. J Math Anal Appl 435:253–266MathSciNetzbMATHCrossRefGoogle Scholar
  123. Klein E, Thompson AC (1984) Theory of correspondences. Wiley, New YorkzbMATHGoogle Scholar
  124. Kohlberg E, Mertens JF (1986) On the strategic stability of equilibria. Econometrica 54:1003–1037MathSciNetzbMATHCrossRefGoogle Scholar
  125. Krasnosielska-Kobos A (2016) Construction of Nash equilibrium based on multiple stopping problem in multi-person game. Math Methods Oper Res (2016) 83:53–70MathSciNetzbMATHCrossRefGoogle Scholar
  126. Krasnosielska-Kobos A, Ferenstein E (2013) Construction of Nash equilibrium in a game version of Elfving’s multiple stopping problem. Dyn Games Appl 3:220–235MathSciNetzbMATHCrossRefGoogle Scholar
  127. Krishnamurthy N, Parthasarathy T, Babu S (2012) Existence of stationary equilibrium for mixtures of discounted stochastic games. Curr Sci 103:1003–1013Google Scholar
  128. Krishnamurthy N, Parthasarathy T, Ravindran G (2012) Solving subclasses of multi-player stochastic games via linear complementarity problem formulations – a survey and some new results. Optim Eng 13:435–457MathSciNetzbMATHCrossRefGoogle Scholar
  129. Kuipers J, Flesch J, Schoenmakers G, Vrieze K (2016) Subgame-perfection in recursive perfect information games, where each player controls one state. Int J Game Theory 45:205–237MathSciNetzbMATHCrossRefGoogle Scholar
  130. Kuratowski K, Ryll-Nardzewski C (1965) A general theorem on selectors. Bull Polish Acad Sci (Ser Math) 13:397–403MathSciNetzbMATHGoogle Scholar
  131. Küenle HU (1994) On Nash equilibrium solutions in nonzero-sum stochastic games with complete information. Int J Game Theory 23:303–324MathSciNetzbMATHCrossRefGoogle Scholar
  132. Küenle HU (1999) Equilibrium strategies in stochastic games with additive cost and transition structure and Borel state and action spaces. Int Game Theory Rev 1:131–147MathSciNetzbMATHCrossRefGoogle Scholar
  133. Leininger W (1986) The existence of perfect equilibria in model of growth with altruism between generations. Rev Econ Stud 53:349–368MathSciNetzbMATHCrossRefGoogle Scholar
  134. Lemke CE (1965) Bimatrix equilibrium points and mathematical programming. Manag Sci 11:681–689MathSciNetzbMATHCrossRefGoogle Scholar
  135. Lemke CE, Howson JT Jr (1964) Equilibrium points of bimatrix games. SIAM J Appl Math 12:413–423MathSciNetzbMATHCrossRefGoogle Scholar
  136. Levhari D, Mirman L (1980) The great fish war: an example using a dynamic Coumot-Nash solution. Bell J Econ 11:322–334CrossRefGoogle Scholar
  137. Levy YJ (2013) Discounted stochastic games with no stationary Nash equilibrium: two examples. Econometrica 81:1973–2007MathSciNetzbMATHCrossRefGoogle Scholar
  138. Levy YJ, McLennan A (2015) Corrigendum to: discounted stochastic games with no stationary Nash equilibrium: two examples. Econometrica 83:1237–1252MathSciNetCrossRefGoogle Scholar
  139. Maitra A, Sudderth W (2003) Borel stay-in-a-set games. Int J Game Theory 32:97–108MathSciNetzbMATHCrossRefGoogle Scholar
  140. Maitra A, Sudderth WD (2007) Subgame-perfect equilibria for stochastic games. Math Oper Res 32:711–722MathSciNetzbMATHCrossRefGoogle Scholar
  141. Majumdar MK, Sundaram R (1991) Symmetric stochastic games of resource extraction. The existence of non-randomized stationary equilibrium. In: Raghavan et al (eds) Stochastic games and related topics. Kluwer, Dordrecht, pp 175–190CrossRefGoogle Scholar
  142. Maliar, L, Maliar, S (2016) Ruling out multiplicity of smooth equilibria in dynamic games: a hyperbolic discounting example. Dyn Games Appl 6:243–261MathSciNetzbMATHCrossRefGoogle Scholar
  143. Mas-Colell A (1974) A note on a theorem of F. Browder. Math Program 6:229–233MathSciNetzbMATHCrossRefGoogle Scholar
  144. Mashiah-Yaakovi A (2009) Periodic stopping games. Int J Game Theory 38:169–181MathSciNetzbMATHCrossRefGoogle Scholar
  145. Mashiah-Yaakovi A (2014) Subgame-perfect equilibria in stopping games. Int J Game Theory 43:89–135MathSciNetzbMATHCrossRefGoogle Scholar
  146. Mashiah-Yaakovi A (2015) Correlated equilibria in stochastic games with Borel measurable payoffs. Dyn Games Appl 5:120–135MathSciNetzbMATHCrossRefGoogle Scholar
  147. Maskin E, Tirole J (2001) Markov perfect equilibrium: I. Observable actions. J Econ Theory 100:191–219MathSciNetzbMATHCrossRefGoogle Scholar
  148. Mertens JF (2002) Stochastic games. In: Aumann RJ, Hart S (eds) Handbook of game theory with economic applications, vol 3. North Holland, Amsterdam/London, pp 1809–1832Google Scholar
  149. Mertens JF (2003) A measurable measurable choice theorem. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 107–130CrossRefGoogle Scholar
  150. Mertens JF, Neyman A (1981) Stochastic games. Int J Game Theory 10:53–56MathSciNetzbMATHCrossRefGoogle Scholar
  151. Mertens JF, Parthasarathy T (1991) Nonzero-sum stochastic games. In: Raghavan et al (eds) Stochastic games and related topics. Kluwer, Dordrecht, pp 145–148CrossRefGoogle Scholar
  152. Mertens JF, Parthasarathy T (2003) Equilibria for discounted stochastic games. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 131–172zbMATHCrossRefGoogle Scholar
  153. Mertens JF, Sorin S, Zamir S (2015) Repeated games. Cambridge University Press, CambridgezbMATHCrossRefGoogle Scholar
  154. Milgrom P, Roberts J (1990) Rationalizability, learning and equilibrium in games with strategic complementarities. Econometrica 58:1255–1277MathSciNetzbMATHCrossRefGoogle Scholar
  155. Milgrom P, Shannon C (1994) Monotone comparative statics. Econometrica 62:157–180MathSciNetzbMATHCrossRefGoogle Scholar
  156. Mohan SR, Neogy SK, Parthasarathy T (1997) Linear complementarity and discounted polystochastic game when one player controls transitions. In: Ferris MC, Pang JS (eds) Proceedings of the international conference on complementarity problems. SIAM, Philadelphia, pp 284–294zbMATHGoogle Scholar
  157. Mohan SR, Neogy SK, Parthasarathy T (2001) Pivoting algorithms for some classes of stochastic games: a survey. Int Game Theory Rev 3:253–281MathSciNetzbMATHCrossRefGoogle Scholar
  158. Monderer D, Shapley LS (1996) Potential games. Games Econ Behavior 14:124–143MathSciNetzbMATHCrossRefGoogle Scholar
  159. Montrucchio L (1987) Lipschitz continuous policy functions for strongly concave optimization problems. J Math Econ 16:259–273MathSciNetzbMATHCrossRefGoogle Scholar
  160. Morimoto H (1986) Nonzero-sum discrete parameter stochastic games with stopping times. Probab Theory Relat Fields 72:155–160MathSciNetzbMATHCrossRefGoogle Scholar
  161. Nash JF (1950) Equilibrium points in n -person games. Proc Nat Acad Sci USA 36:48–49MathSciNetzbMATHCrossRefGoogle Scholar
  162. Neveu J (1975) Discrete-parameter martingales. North-Holland, AmsterdamzbMATHGoogle Scholar
  163. Neyman A (2017) Continuous-time stochastic games. Games Econ Behavior 104:92–130MathSciNetzbMATHCrossRefGoogle Scholar
  164. Neyman A, Sorin S (eds) (2003) Stochastic games and applications. Kluwer, DordrechtzbMATHGoogle Scholar
  165. Nowak AS (1985) Existence of equilibrium stationary strategies in discounted noncooperative stochastic games with uncountable state space. J Optim Theory Appl 45:591–602MathSciNetzbMATHCrossRefGoogle Scholar
  166. Nowak AS (1987) Nonrandomized strategy equilibria in noncooperative stochastic games with additive transition and reward structure. J Optim Theory Appl 52:429–441MathSciNetzbMATHCrossRefGoogle Scholar
  167. Nowak AS (2003a) N-person stochastic games: extensions of the finite state space case and correlation. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 93–106zbMATHCrossRefGoogle Scholar
  168. Nowak AS (2003b) On a new class of nonzero-sum discounted stochastic games having stationary Nash equilibrium point. Int J Game Theory 32:121–132MathSciNetzbMATHCrossRefGoogle Scholar
  169. Nowak AS (2006a) On perfect equilibria in stochastic models of growth with intergenerational altruism. Econ Theory 28:73–83MathSciNetzbMATHCrossRefGoogle Scholar
  170. Nowak AS (2006b) A multigenerational dynamic game of resource extraction. Math Social Sci 51:327–336MathSciNetzbMATHCrossRefGoogle Scholar
  171. Nowak AS (2006c) A note on equilibrium in the great fish war game. Econ Bull 17(2):1–10MathSciNetGoogle Scholar
  172. Nowak AS (2007) On stochastic games in economics. Math Methods Oper Res 66:513–530MathSciNetzbMATHCrossRefGoogle Scholar
  173. Nowak AS (2008) Equilibrium in a dynamic game of capital accumulation with the overtaking criterion. Econ Lett 99:233–237MathSciNetzbMATHCrossRefGoogle Scholar
  174. Nowak AS (2010) On a noncooperative stochastic game played by internally cooperating generations. J Optim Theory Appl 144:88–106MathSciNetzbMATHCrossRefGoogle Scholar
  175. Nowak AS, Altman E (2002) ε-Equilibria for stochastic games with uncountable state space and unbounded costs. SIAM J Control Optim 40:1821–1839MathSciNetzbMATHCrossRefGoogle Scholar
  176. Nowak AS, Jaśkiewicz A (2005) Nonzero-sum semi-Markov games with the expected average payoffs. Math Methods Oper Res 62:23–40MathSciNetzbMATHCrossRefGoogle Scholar
  177. Nowak AS, Raghavan TES (1992) Existence of stationary correlated equilibria with symmetric information for discounted stochastic games. Math Oper Res 17:519–526MathSciNetzbMATHCrossRefGoogle Scholar
  178. Nowak AS, Raghavan TES (1993) A finite step algorithm via a bimatrix game to a single controller non-zero sum stochastic game. Math Program Ser A 59:249–259MathSciNetzbMATHCrossRefGoogle Scholar
  179. Nowak AS, Szajowski K (1999) Nonzero-sum stochastic games. In: Ann Int Soc Dyn Games, vol 4. Birkhäuser, Boston, pp 297–343Google Scholar
  180. Nowak AS, Wiȩcek P (2007) On Nikaido-Isoda type theorems for discounted stochastic games. J Math Anal Appl 332:1109–1118Google Scholar
  181. Ohtsubo Y (1987) A nonzero-sum extension of Dynkin’s stopping problem. Math Oper Res 12:277–296MathSciNetzbMATHCrossRefGoogle Scholar
  182. Ohtsubo Y (1991) On a discrete-time nonzero-sum Dynkin problem with monotonicity. J Appl Probab 28:466–472MathSciNetzbMATHCrossRefGoogle Scholar
  183. Parthasarathy T (1973) Discounted, positive, and noncooperative stochastic games. Int J Game Theory 2:25–37MathSciNetzbMATHCrossRefGoogle Scholar
  184. Parthasarathy T, Sinha S (1989) Existence of stationary equilibrium strategies in nonzero-sum discounted stochastic games with uncountable state space and state-independent transitions. Int J Game Theory 18:189–194zbMATHCrossRefGoogle Scholar
  185. Peleg B, Yaari M (1973) On the existence of consistent course of action when tastes are changing. Rev Econ Stud 40:391–401zbMATHCrossRefGoogle Scholar
  186. Peski M, Wiseman T (2015) A folk theorem for stochastic games with infrequent state changes. Theoret Econ 10:131–173MathSciNetCrossRefzbMATHGoogle Scholar
  187. Phelps E, Pollak R (1968) On second best national savings and game equilibrium growth. Rev Econ Stud 35:195–199CrossRefGoogle Scholar
  188. Pollak R (1968) Consistent planning. Rev Econ Stud 35:201–208CrossRefGoogle Scholar
  189. Potters JAM, Raghavan TES, Tijs SH (2009) Pure equilibrium strategies for stochastic games via potential functions. In: Advances in dynamic games and their applications. Annals of the international society of dynamic games, vol 10. Birkhäuser, Boston, pp 433–444Google Scholar
  190. Purves RA, Sudderth WD (2011) Perfect information games with upper semicontinuous payoffs. Math Oper Res 36:468–473MathSciNetzbMATHCrossRefGoogle Scholar
  191. Puterman ML (1994) Markov decision processes: discrete stochastic dynamic programming. Wiley, HobokenzbMATHCrossRefGoogle Scholar
  192. Raghavan TES, Syed Z (2002) Computing stationary Nash equilibria of undiscounted single-controller stochastic games. Math Oper Res 27:384–400MathSciNetzbMATHCrossRefGoogle Scholar
  193. Raghavan TES, Tijs SH, Vrieze OJ (1985) On stochastic games with additive reward and transition structure. J Optim Theory Appl 47:451–464MathSciNetzbMATHCrossRefGoogle Scholar
  194. Ramsey FP (1928) A mathematical theory of savings. Econ J 38:543–559CrossRefGoogle Scholar
  195. Ray D (1987) Nonpaternalistic intergenerational altruism. J Econ Theory 40:112–132zbMATHCrossRefGoogle Scholar
  196. Reny PJ, Robson A (2002) Existence of subgame-perfect equilibrium with public randomization: a short proof. Econ Bull 3(24):1–8Google Scholar
  197. Rieder U (1979) Equilibrium plans for non-zero sum Markov games. In: Moeschlin O, Pallaschke D (eds) Game theory and related topics. North-Holland, Amsterdam, pp 91–102Google Scholar
  198. Rogers PD (1969) Non-zero-sum stochastic games. Ph.D. dissertation, report 69–8, Univ of CaliforniaGoogle Scholar
  199. Rosen JB (1965) Existence and uniqueness of equilibrium points for concave n-person games. Econometrica 33:520–534MathSciNetzbMATHCrossRefGoogle Scholar
  200. Rosenberg D, Solan E, Vieille N (2001) Stopping games with randomized strategies. Probab Theory Relat Fields 119:433–451MathSciNetzbMATHCrossRefGoogle Scholar
  201. Rubinstein A (1979) Equilibrium in supergames with the overtaking criterion. J Econ Theory 21:1–9MathSciNetzbMATHCrossRefGoogle Scholar
  202. Secchi P, Sudderth WD (2002a) Stay-in-a-set games. Int J Game Theory 30:479–490MathSciNetzbMATHCrossRefGoogle Scholar
  203. Secchi P, Sudderth WD (2002b) N-person stochastic games with upper semi-continuous payoffs. Int J Game Theory 30:491–502MathSciNetzbMATHCrossRefGoogle Scholar
  204. Secchi P, Sudderth.WD (2005) A simple two-person stochastic game with money. In: Annals of the international society of dynamic games, vol 7. Birkhäuser, Boston, pp 39–66Google Scholar
  205. Selten R (1975) Re-examination of the perfectness concept for equilibrium points in extensive games. Int J Game Theory 4:25–55zbMATHCrossRefGoogle Scholar
  206. Shapley LS (1953) Stochastic games. Proc Nat Acad Sci USA 39:1095–1100MathSciNetzbMATHCrossRefGoogle Scholar
  207. Shubik M, Whitt W (1973) Fiat money in an economy with one non-durable good and no credit: a non-cooperative sequential game. In: Blaquière A (ed) Topics in differential games. North-Holland, Amsterdam, pp 401–448Google Scholar
  208. Shmaya E, Solan E (2004) Two player non-zero sum stopping games in discrete time. Ann Probab 32:2733–2764MathSciNetzbMATHCrossRefGoogle Scholar
  209. Shmaya E, Solan E, Vieille N (2003) An applications of Ramsey theorem to stopping games. Games Econ Behavior 42:300–306MathSciNetzbMATHCrossRefGoogle Scholar
  210. Simon RS (2007) The structure of nonzero-sum stochastic games. Adv Appl Math 38:1–26MathSciNetCrossRefGoogle Scholar
  211. Simon R (2012) A topological approach to quitting games. Math Oper Res 37:180–195MathSciNetzbMATHCrossRefGoogle Scholar
  212. Simon RS (2016) The challenge of nonzero-sum stochastic games. Int J Game Theory 45:191–204zbMATHCrossRefGoogle Scholar
  213. Sleet C, Yeltekin S (2016) On the computation of value correspondences for dynamic games. Dyn Games Appl 6:174–186MathSciNetzbMATHCrossRefGoogle Scholar
  214. Smale S (1976) A convergent process of price adjustment and global Newton methods. J Math Econ 3:107–120MathSciNetzbMATHCrossRefGoogle Scholar
  215. Sobel MJ (1871) Non-cooperative stochastic games. Ann Math Stat 42:1930–1935zbMATHCrossRefGoogle Scholar
  216. Solan E (1998) Discounted stochastic games. Math Oper Res 23:1010–1021MathSciNetzbMATHCrossRefGoogle Scholar
  217. Solan E (1999) Three-person absorbing games. Math Oper Res 24:669–698MathSciNetzbMATHCrossRefGoogle Scholar
  218. Solan E (2001) Characterization of correlated equilibria in stochastic games. Int J Game Theory 30:259–277MathSciNetzbMATHCrossRefGoogle Scholar
  219. Solan E (2017, in press) Acceptable strategy profiles in stochastic games. Games Econ BehaviorGoogle Scholar
  220. Solan E, Vieille N (2001) Quitting games. Math Oper Res 26:265–285MathSciNetzbMATHCrossRefGoogle Scholar
  221. Solan E, Vieille N (2002) Correlated equilibrium in stochastic games. Games Econ Behavior 38:362–399MathSciNetzbMATHCrossRefGoogle Scholar
  222. Solan E, Vieille N (2003) Deterministic multi-player Dynkin games. J Math Econ 39:911–929MathSciNetzbMATHCrossRefGoogle Scholar
  223. Solan E, Vieille N (2010) Computing uniformly optimal strategies in two-player stochastic games. Econ Theory 42:237–253MathSciNetzbMATHCrossRefGoogle Scholar
  224. Solan E, Ziliotto B (2016) Stochastic games with signals. In: Annals of the international society of dynamic games, vol 14. Birkhäuser, Boston, pp 77–94zbMATHGoogle Scholar
  225. Sorin S (1986) Asymptotic properties of a nonzero-sum stochastic games. Int J Game Theory 15:101–107MathSciNetzbMATHCrossRefGoogle Scholar
  226. Spence M (1976) Product selection, fixed costs, and monopolistic competition. Rev Econ Stud 43:217–235zbMATHCrossRefGoogle Scholar
  227. Stachurski J (2009) Economic dynamics: theory and computation. MIT Press, Cambridge, MAzbMATHGoogle Scholar
  228. Stokey NL, Lucas RE, Prescott E (1989) Recursive methods in economic dynamics. Harvard University Press, CambridgeGoogle Scholar
  229. Strotz RH (1956) Myopia and inconsistency in dynamic utility maximization. Rev Econ Stud 23:165–180CrossRefGoogle Scholar
  230. Sundaram RK (1989a) Perfect equilibrium in a class of symmetric dynamic games. J Econ Theory 47:153–177zbMATHCrossRefGoogle Scholar
  231. Sundaram RK (1989b) Perfect equilibrium in a class of symmetric dynamic games. Corrigendum. J Econ Theory 49:385–187zbMATHCrossRefGoogle Scholar
  232. Szajowski K (1994) Markov stopping game with random priority. Z Oper Res 39:69–84MathSciNetzbMATHGoogle Scholar
  233. Szajowski K (1995) Optimal stopping of a discrete Markov process by two decision makers. SIAM J Control Optim 33:1392–1410MathSciNetzbMATHCrossRefGoogle Scholar
  234. Takahashi M (1964) Equilibrium points of stochastic non-cooperative n-person games. J Sci Hiroshima Univ Ser A-I Math 28:95–99MathSciNetzbMATHGoogle Scholar
  235. Thuijsman F, Raghavan TES (1997) Perfect information stochastic games and related classes. Int J Game Theory 26:403–408MathSciNetzbMATHCrossRefGoogle Scholar
  236. Topkis D (1978) Minimizing a submodular function on a lattice. Oper Res 26:305–321MathSciNetzbMATHCrossRefGoogle Scholar
  237. Topkis D (1998) Supermodularity and complementarity. Princeton University Press, PrincetonGoogle Scholar
  238. Valadier M (1994) Young measures, weak and strong convergence and the Visintin-Balder theorem. Set-Valued Anal 2:357–367MathSciNetzbMATHCrossRefGoogle Scholar
  239. Van Long N (2011) Dynamic games in the economics of natural resources: a survey. Dyn Games Appl 1:115–148MathSciNetzbMATHCrossRefGoogle Scholar
  240. Vieille N (2000a) Two-player stochastic games I: a reduction. Isr J Math 119:55–92MathSciNetzbMATHCrossRefGoogle Scholar
  241. Vieille N (2000b) Two-player stochastic games II: the case of recursive games. Israel J Math 119:93–126MathSciNetzbMATHCrossRefGoogle Scholar
  242. Vieille N (2002) Stochastic games: Recent results. In: Aumann RJ, Hart S (eds) Handbook of game theory with economic applications, vol 3. North Holland, Amsterdam/London, pp 1833–1850Google Scholar
  243. Vives X (1990) Nash equilibrium with strategic complementarities. J Math Econ 19:305–321MathSciNetzbMATHCrossRefGoogle Scholar
  244. von Weizsäcker CC (1965) Existence of optimal programs of accumulation for an infinite horizon. Rev Econ Stud 32:85–104CrossRefGoogle Scholar
  245. Vrieze OJ, Thuijsman F (1989) On equilibria in repeated games with absorbing states. Int J Game Theory 18:293–310MathSciNetzbMATHCrossRefGoogle Scholar
  246. Whitt W (1980) Representation and approximation of noncooperative sequential games. SIAM J Control Optim 18:33–48MathSciNetzbMATHCrossRefGoogle Scholar
  247. Wiecek P (2009) Pure equilibria in a simple dynamic model of strategic market game. Math Methods Oper Res 69:59–79MathSciNetzbMATHCrossRefGoogle Scholar
  248. Wiecek P (2012) N-person dynamic strategic market games. Appl Math Optim 65:147–173MathSciNetzbMATHCrossRefGoogle Scholar
  249. Yasuda M (1985) On a randomised strategy in Neveu’s stopping problem. Stoch Process Appl 21:159–166zbMATHCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Faculty of Pure and Applied MathematicsWrocław University of Science and TechnologyWrocławPoland
  2. 2.Faculty of Mathematics, Computer Science and EconometricsUniversity of Zielona GóraZielona GóraPoland

Personalised recommendations