Advertisement

Nonzero-Sum Stochastic Games

  • Anna Jaśkiewicz
  • Andrzej S. Nowak
Reference work entry

Abstract

This chapter describes a number of results obtained in the last 60 years on the theory of nonzero-sum discrete-time stochastic games. We provide an overview of almost all basic streams of research in this area such as the existence of stationary Nash and correlated equilibria in models on countable and general state spaces, the existence of subgame-perfect equilibria, algorithms, stopping games, and the existence of uniform equilibria. Our survey incorporates several examples of games studied in operations research and economics. In particular, separate sections are devoted to intergenerational games, dynamic Cournot competition and game models of resource extraction. The provided reference list includes not only seminal papers that commenced research in various directions but also exposes recent advances in this field.

Keywords

Nonzero-sum game Stochastic game Discounted payoff Limit-average payoff Markov perfect equilibrium Subgame-perfect equilibrium Intergenerational altruism Uniform equilibrium Stopping game 

Notes

Acknowledgements

We thank Tamer Başar and Georges Zaccour for inviting us to write this chapter and their help. We also thank Elżbieta Ferenstein, János Flesch, Eilon Solan, Yeneng Sun, Krzysztof Szajowski and two reviewers for their comments on an earlier version of this survey.

References

  1. Abreu D, Pearce D, Stacchetti E (1986) Optimal cartel equilibria with imperfect monitoring. J Econ Theory 39:251–269MathSciNetMATHCrossRefGoogle Scholar
  2. Abreu D, Pearce D, Stacchetti E (1990) Toward a theory of discounted repeated games with imperfect monitoring. Econometrica 58:1041–1063MathSciNetMATHCrossRefGoogle Scholar
  3. Adlakha S, Johari R (2013) Mean field equilibrium in dynamic games with strategic complementarities. Oper Res 61:971–989MathSciNetMATHCrossRefGoogle Scholar
  4. Aliprantis C, Border K (2006) Infinite dimensional analysis: a Hitchhiker’s guide. Springer, New YorkMATHGoogle Scholar
  5. Alj A, Haurie A (1983) Dynamic equilibria in multigenerational stochastic games. IEEE Trans Autom Control 28:193–203MATHCrossRefGoogle Scholar
  6. Alós-Ferrer C, Ritzberger K (2015) Characterizing existence of equilibrium for large extensive form games: a necessity result. Econ Theory 63:407–430MathSciNetMATHCrossRefGoogle Scholar
  7. Alós-Ferrer C, Ritzberger K (2016) Equilibrium existence for large perfect information games. J Math Econ 62:5–18MathSciNetMATHCrossRefGoogle Scholar
  8. Altman E (1996) Non-zero-sum stochastic games in admission, service and routing control in queueing systems. Queueing Syst Theory Appl 23:259–279MathSciNetMATHCrossRefGoogle Scholar
  9. Altman E, Avrachenkov K, Bonneau N, Debbah M, El-Azouzi R, Sadoc Menasche D (2008) Constrained cost-coupled stochastic games with independent state processes. Oper Res Lett 36:160–164MathSciNetMATHCrossRefGoogle Scholar
  10. Altman E, Avrachenkov K, Marquez R, Miller G (2005) Zero-sum constrained stochastic games with independent state processes. Math Methods Oper Res 62:375–386MathSciNetMATHCrossRefGoogle Scholar
  11. Altman E, Hordijk A, Spieksma FM (1997) Contraction conditions for average and α-discount optimality in countable state Markov games with unbounded rewards. Math Oper Res 22: 588–618MathSciNetMATHCrossRefGoogle Scholar
  12. Amir R (1996a) Continuous stochastic games of capital accumulation with convex transitions. Games Econ Behavior 15:132–148MathSciNetMATHCrossRefGoogle Scholar
  13. Amir R (1996b) Strategic intergenerational bequests with stochastic convex production. Econ Theory 8:367–376MATHCrossRefGoogle Scholar
  14. Amir R (2003) Stochastic games in economics: the lattice-theoretic approach. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 443–453MATHCrossRefGoogle Scholar
  15. Artstein Z (1989) Parametrized integration of multifunctions with applications to control and optimization. SIAM J Control Optim 27:1369–1380MathSciNetMATHCrossRefGoogle Scholar
  16. Aumann RJ (1974) Subjectivity and correlation in randomized strategies. J Math Econ 1:67–96MathSciNetMATHCrossRefGoogle Scholar
  17. Aumann RJ (1987) Correlated equilibrium as an expression of Bayesian rationality. Econometrica 55:1–18MathSciNetMATHCrossRefGoogle Scholar
  18. Balbus Ł, Jaśkiewicz A, Nowak AS (2014) Robust Markov perfect equilibria in a dynamic choice model with quasi-hyperbolic discounting. In: Haunschmied J et al (eds) Dynamic games in economics, dynamic modeling and econometrics in economics and finance 16. Springer, Berlin/Heidelberg, pp 1–22MATHGoogle Scholar
  19. Balbus Ł, Jaśkiewicz A, Nowak AS (2015a) Existence of stationary Markov perfect equilibria in stochastic altruistic growth economies. J Optim Theory Appl 165:295–315MathSciNetMATHCrossRefGoogle Scholar
  20. Balbus Ł, Jaśkiewicz A, Nowak AS (2015b) Stochastic bequest games. Games Econ Behavior 90:247–256MathSciNetMATHCrossRefGoogle Scholar
  21. Balbus Ł, Jaśkiewicz A, Nowak AS (2015c) Bequest games with unbounded utility functions. J Math Anal Appl 427:515–524MathSciNetMATHCrossRefGoogle Scholar
  22. Balbus Ł, Jaśkiewicz A, Nowak AS (2016) Non-paternalistic intergenerational altruism revisited. J Math Econ 63:27–33MathSciNetMATHCrossRefGoogle Scholar
  23. Balbus Ł, Nowak AS (2004) Construction of Nash equilibria in symmetric stochastic games of capital accumulation. Math Methods Oper Res 60:267–277MathSciNetMATHCrossRefGoogle Scholar
  24. Balbus Ł, Nowak AS (2008) Existence of perfect equilibria in a class of multigenerational stochastic games of capital accumulation. Automatica 44:1471–1479MathSciNetMATHCrossRefGoogle Scholar
  25. Balbus Ł, Reffett K, Woźny Ł (2012) Stationary Markovian equilibria in altruistic stochastic OLG models with limited commitment. J Math Econ 48:115–132MATHCrossRefGoogle Scholar
  26. Balbus Ł, Reffett K, Woźny Ł (2013a) A constructive geometrical approach to the uniqueness of Markov stationary equilibrium in stochastic games of intergenerational altruism. J Econ Dyn Control 37:1019–1039MathSciNetCrossRefGoogle Scholar
  27. Balbus Ł, Reffett K, Woźny Ł (2013b) Markov stationary equilibria in stochastic supermodular games with imperfect private and public information. Dyn Games Appl 3:187–206MathSciNetMATHCrossRefGoogle Scholar
  28. Balbus Ł, Reffett K, Woźny Ł (2014a) Constructive study of Markov equilibria in stochastic games with complementarities. J Econ Theory 150:815–840MathSciNetMATHCrossRefGoogle Scholar
  29. Balbus Ł, Jaśkiewicz A, Nowak AS, Woźny L (2017) A note on Markov perfect equilibria in a class of non-stationary bequest games. J Math Anal Appl 456:394–401MathSciNetMATHCrossRefGoogle Scholar
  30. Balbus Ł, Jaśkiewicz A, Nowak AS (2018, in press) Markov perfect equilibria in a dynamic decision model with quasi-hyperbolic discounting. Ann Oper ResGoogle Scholar
  31. Barelli P, Duggan J (2014) A note on semi-Markov perfect equilibria in discounted stochastic games. J Econ Theory 15:596–604MathSciNetMATHCrossRefGoogle Scholar
  32. Başar T, Olsder GJ (1995) Dynamic noncooperative game theory. Academic, New YorkMATHGoogle Scholar
  33. Berg K (2016) Elementary subpaths in discounted stochastic games. Dyn Games Appl 6(3): 304–323MathSciNetMATHCrossRefGoogle Scholar
  34. Bernheim D, Ray D (1983) Altruistic growth economies I. Existence of bequest equilibria. Technical report no 419, Institute for Mathematical Studies in the Social Sciences, Stanford UniversityGoogle Scholar
  35. Bernheim D, Ray D (1986) On the existence of Markov-consistent plans under production uncertainty. Rev Econ Stud 53:877–882MathSciNetMATHCrossRefGoogle Scholar
  36. Bernheim D, Ray D (1987) Economic growth with intergenerational altruism. Rev Econ Stud 54:227–242MathSciNetMATHCrossRefGoogle Scholar
  37. Bernheim D, Ray D (1989) Markov perfect equilibria in altruistic growth economies with production uncertainty. J Econ Theory 47:195–202MathSciNetMATHCrossRefGoogle Scholar
  38. Bhattacharya R, Majumdar M (2007) Random dynamical systems: theory and applications. Cambridge University Press, CambridgeMATHCrossRefGoogle Scholar
  39. Billingsley P (1968) Convergence of probability measures. Wiley, New YorkMATHGoogle Scholar
  40. Blackwell D (1965) Discounted dynamic programming. Ann Math Stat 36:226–235MathSciNetMATHCrossRefGoogle Scholar
  41. Blackwell D (1969) Infinite Gδ -games with imperfect information. Zastosowania Matematyki (Appl Math) 10:99–101MathSciNetMATHGoogle Scholar
  42. Borkar VS, Ghosh MK (1993) Denumerable state stochastic games with limiting average payoff. J Optim Theory Appl 76:539–560MathSciNetMATHCrossRefGoogle Scholar
  43. Browder FE (1960) On continuity of fixed points under deformations of continuous mappings. Summa Brasiliensis Math 4:183–191MathSciNetMATHGoogle Scholar
  44. Carlson D, Haurie A (1996) A turnpike theory for infinite-horizon open-loop competitive processes. SIAM J Control Optim 34:1405–1419MathSciNetMATHCrossRefGoogle Scholar
  45. Castaing C, Valadier M (1977) Convex analysis and measurable multifunctions. Lecture notes in mathematics, vol 580. Springer, New YorkMATHGoogle Scholar
  46. Chatterjee S, Eyigungor B (2016) Continuous Markov equilibria with quasi-geometric discounting. J Econ Theory 163:467–494MathSciNetMATHCrossRefGoogle Scholar
  47. Chiarella C, Kemp MC, Van Long N, Okuguchi K (1984) On the economics of international fisheries. Int Econ Rev 25:85–92MathSciNetMATHCrossRefGoogle Scholar
  48. Cingiz K, Flesch J, Herings JJP, Predtetchinski A (2016) Doing it now, later, or never. Games Econ Behavior 97:174–185MathSciNetMATHCrossRefGoogle Scholar
  49. Cole HL, Kocherlakota N (2001) Dynamic games with hidden actions and hidden states. J Econ Theory 98:114–126MathSciNetMATHCrossRefGoogle Scholar
  50. Cottle RW, Pang JS, Stone RE (1992) The linear complementarity problem. Academic, New YorkMATHGoogle Scholar
  51. Curtat LO (1996) Markov equilibria of stochastic games with complementarities. Games Econ Behavior 17:177–199MathSciNetMATHCrossRefGoogle Scholar
  52. Doraszelski U, Escobar JF (2010) A theory of regular Markov perfect equilibria in dynamic stochastic games: genericity, stability, and purification. Theor Econ 5:369–402MathSciNetMATHCrossRefGoogle Scholar
  53. Doraszelski U, Pakes A (2007) A framework for applied dynamic analysis in IO. In: Amstrong M, Porter RH (eds) Handbook of industrial organization, vol 3. North Holland, Amsterdam/ London, pp 1887–1966Google Scholar
  54. Doraszelski U, Satterthwaite M (2010) Computable Markov-perfect industry dynamics. Rand J Econ 41:215–243CrossRefGoogle Scholar
  55. Dubins LE, Savage LJ (2014) Inequalities for stochastic processes. Dover, New YorkGoogle Scholar
  56. Duffie D, Geanakoplos J, Mas-Colell A, McLennan A (1994) Stationary Markov equilibria. Econometrica 62:745–781MathSciNetMATHCrossRefGoogle Scholar
  57. Duggan J (2012) Noisy stochastic games. Econometrica 80:2017–2046MathSciNetMATHCrossRefGoogle Scholar
  58. Dutta PK (1995) A folk theorem for stochastic games. J Econ Theory 66:1–32MathSciNetMATHCrossRefGoogle Scholar
  59. Dutta PK, Sundaram RK (1992) Markovian equilibrium in class of stochastic games: existence theorems for discounted and undiscounted models. Econ Theory 2:197–214MathSciNetMATHCrossRefGoogle Scholar
  60. Dutta PK, Sundaram RK (1993) The tragedy of the commons? Econ Theory 3:413–426MathSciNetMATHCrossRefGoogle Scholar
  61. Dutta PK, Sundaram RK (1998) The equilibrium existence problem in general Markovian games. In: Majumdar M (ed) Organizations with incomplete information. Cambridge University Press, Cambridge, pp 159–207CrossRefGoogle Scholar
  62. Dynkin EB (1969) The game variant of a problem on optimal stopping. Sov Math Dokl 10:270–274MATHGoogle Scholar
  63. Dynkin EB, Evstigneev IV (1977) Regular conditional expectations of correspondences. Theory Probab Appl 21:325–338MATHCrossRefGoogle Scholar
  64. Eaves B (1972) Homotopies for computation of fixed points. Math Program 3:1–22MathSciNetMATHCrossRefGoogle Scholar
  65. Eaves B (1984) A course in triangulations for solving equations with deformations. Springer, BerlinMATHCrossRefGoogle Scholar
  66. Elliott RJ, Kalton NJ, Markus L (1973) Saddle-points for linear differential games. SIAM J Control Optim 11:100–112MathSciNetMATHCrossRefGoogle Scholar
  67. Enns EG, Ferenstein EZ (1987) On a multi-person time-sequential game with priorities. Seq Anal 6:239–256MATHCrossRefGoogle Scholar
  68. Ericson R, Pakes A (1995) Markov-perfect industry dynamics: a framework for empirical work. Rev Econ Stud 62:53–82MATHCrossRefGoogle Scholar
  69. Escobar JF (2013) Equilibrium analysis of dynamic models of imperfect competition. Int J Ind Organ 31:92–101CrossRefGoogle Scholar
  70. Federgruen A (1978) On N-person stochastic games with denumerable state space. Adv Appl Probab 10:452–471MathSciNetMATHCrossRefGoogle Scholar
  71. Ferenstein E (2007) Randomized stopping games and Markov market games. Math Methods Oper Res 66:531–544MathSciNetMATHCrossRefGoogle Scholar
  72. Filar JA, Schultz T, Thuijsman F, Vrieze OJ (1991) Nonlinear programming and stationary equilibria in stochastic games. Math Program 50:227–237MathSciNetMATHCrossRefGoogle Scholar
  73. Filar JA, Vrieze K (1997) Competitive Markov decision processes. Springer, New YorkMATHGoogle Scholar
  74. Fink AM (1964) Equilibrium in a stochastic n-person game. J Sci Hiroshima Univ Ser A-I Math 28:89–93MathSciNetMATHGoogle Scholar
  75. Flesch J, Schoenmakers G, Vrieze K (2008) Stochastic games on a product state space. Math Oper Res 33:403–420MathSciNetMATHCrossRefGoogle Scholar
  76. Flesch J, Schoenmakers G, Vrieze K (2009) Stochastic games on a product state space: the periodic case. Int J Game Theory 38:263–289MathSciNetMATHCrossRefGoogle Scholar
  77. Flesch J, Thuijsman F, Vrieze OJ (1997) Cyclic Markov equilibrium in stochastic games. Int J Garne Theory 26:303–314MATHCrossRefGoogle Scholar
  78. Flesch J, Thuijsman F, Vrieze OJ (2003) Stochastic games with non-observable actions. Math Methods Oper Res 58:459–475MathSciNetMATHCrossRefGoogle Scholar
  79. Flesch J, Kuipers J, Mashiah-Yaakovi A, Schoenmakers G, Solan E, Vrieze K (2010a) Perfect-information games with lower-semicontinuous payoffs. Math Oper Res 35:742–755MathSciNetMATHCrossRefGoogle Scholar
  80. Flesch J, Kuipers J, Schoenmakers G, Vrieze K (2010b) Subgame-perfection in positive recursive games with perfect information. Math Oper Res 35:193–207MathSciNetMATHCrossRefGoogle Scholar
  81. Flesch J, Kuipers J, Mashiah-Yaakovi A, Schoenmakers G, Shmaya E, Solan E, Vrieze K (2014) Non-existence of subgame-perfect 𝜖-equilibrium in perfect-information games with infinite horizon. Int J Game Theory 43:945–951MathSciNetMATHCrossRefGoogle Scholar
  82. Flesch J, Thuijsman F, Vrieze OJ (2007) Stochastic games with additive transitions. Eur J Oper Res 179:483–497MathSciNetMATHCrossRefGoogle Scholar
  83. Flesch J, Predtetchinski A (2015) On refinements of subgame perfect 𝜖 -equilibrium. Int J Game Theory 45:523–542MathSciNetMATHCrossRefGoogle Scholar
  84. Forges E (1986) An approach to communication equilibria. Econometrica 54:1375–1385MathSciNetMATHCrossRefGoogle Scholar
  85. Forges F (1992) Repeated games of incomplete information: nonzero-sum. In: Aumann RJ, Hart S (eds) Handbook of game theory, vol 1. North Holland, Amsterdam, pp 155–177MATHGoogle Scholar
  86. Forges F (2009) Correlated equilibria and communication in games. In: Meyers RA (ed) Encyclopedia of complexity and systems science. Springer, New York, pp 1587–1596CrossRefGoogle Scholar
  87. Fudenberg D, Levine D (1983) Subgame-perfect equilibria of finite and infinite horizon games. J Econ Theory 31:251–268MathSciNetMATHCrossRefGoogle Scholar
  88. Fudenberg D, Tirole J (1991) Game theory. MIT Press, CambridgeMATHGoogle Scholar
  89. Fudenberg D, Yamamoto Y (2011) The folk theorem for irreducible stochastic games with imperfect public monitoring. J Econ Theory 146:1664–1683MathSciNetMATHCrossRefGoogle Scholar
  90. Gale D (1967) On optimal development in a multisector economy. Rev Econ Stud 34:1–19CrossRefGoogle Scholar
  91. Glicksberg IL (1952) A further generalization of the Kakutani fixed point theorem with application to Nash equilibrium points. Proc Am Math Soc 3:170–174MathSciNetMATHGoogle Scholar
  92. Govindan S, Wilson R (2003) A global Newton method to compute Nash equilibria. J Econ Theory 110:65–86MathSciNetMATHCrossRefGoogle Scholar
  93. Govindan S, Wilson R (2009) Global Newton method for stochastic games. J Econ Theory 144:414–421MathSciNetMATHCrossRefGoogle Scholar
  94. Haller H, Lagunoff R (2000) Genericity and Markovian behavior in stochastic games. Econometrica 68:1231–1248MathSciNetMATHCrossRefGoogle Scholar
  95. Hamadène S, Hassani M (2014) The multi-player nonzero-sum Dynkin game in discrete time. Math Methods Oper Res 79:179–194MathSciNetMATHCrossRefGoogle Scholar
  96. Harris C (1985) Existence and characterization of perfect equilibrium in games of perfect information. Econometrica 53:613–628MathSciNetMATHCrossRefGoogle Scholar
  97. Harris C, Laibson D (2001) Dynamic choices of hyperbolic consumers. Econometrica 69:935–957MathSciNetMATHCrossRefGoogle Scholar
  98. Harris C, Reny PJ, Robson A (1995) The existence of subgame-perfect equilibrium in continuous games with almost perfect information: a case for public randomization. Econometrica 63: 507–544MathSciNetMATHCrossRefGoogle Scholar
  99. Harsanyi JC (1973a) Oddness of the number of equilibrium points: a new proof. Int J Game Theory 2:235–250MathSciNetMATHCrossRefGoogle Scholar
  100. Harsanyi JC (1973b) Games with randomly disturbed payoffs: a new rationale for mixed-strategy equilibrium points. Int J Game Theory 2:1–23MathSciNetMATHCrossRefGoogle Scholar
  101. Haurie A, Krawczyk JB, Zaccour G (2012) Games and dynamic games. World Scientific, SingaporeMATHCrossRefGoogle Scholar
  102. He W, Sun Y (2017) Stationary Markov perfect equilibria in discounted stochastic games. J Econ Theory 169:35–61MathSciNetMATHCrossRefGoogle Scholar
  103. Heller Y (2012) Sequential correlated equilibria in stopping games. Oper Res 60:209–224MathSciNetMATHCrossRefGoogle Scholar
  104. Herings JJP, Peeters RJAP (2004) Stationary equilibria in stochastic games: structure, selection, and computation. J Econ Theory 118:32–60MathSciNetMATHGoogle Scholar
  105. Herings JJP, Peeters RJAP (2010) Homotopy methods to compute equilibria in game theory. Econ Theory 42:119–156MathSciNetMATHCrossRefGoogle Scholar
  106. Himmelberg CJ (1975) Measurable relations. Fundam Math 87:53–72MathSciNetMATHCrossRefGoogle Scholar
  107. Himmelberg CJ, Parthasarathy T, Raghavan TES, Van Vleck FS (1976) Existence of p-equilibrium and optimal stationary strategies in stochastic games. Proc Am Math Soc 60:245–251MathSciNetMATHGoogle Scholar
  108. Hopenhayn H, Prescott E (1992) Stochastic monotonicity and stationary distributions for dynamic economies. Econometrica 60:1387–1406MathSciNetMATHCrossRefGoogle Scholar
  109. Hörner J, Sugaya T, Takahashi S, Vieille N (2011) Recursive methods in discounted stochastic games: an algorithm for δ → 1 and a folk theorem. Econometrica 79:1277–1318MathSciNetMATHCrossRefGoogle Scholar
  110. Hörner J, Takahashi S, Vieille N (2014) On the limit perfect public equilibrium payoff set in repeated and stochastic game. Games Econ Behavior 85:70–83MathSciNetMATHCrossRefGoogle Scholar
  111. Horst U (2005) Stationary equilibria in discounted stochastic games with weakly interacting players. Games Econ Behavior 51:83–108MathSciNetMATHCrossRefGoogle Scholar
  112. Jaśkiewicz A, Nowak AS (2006) Approximation of noncooperative semi-Markov games. J Optim Theory Appl 131:115–134MathSciNetMATHCrossRefGoogle Scholar
  113. Jaśkiewicz A, Nowak AS (2014a) Stationary Markov perfect equilibria in risk sensitive stochastic overlapping generations models. J Econ Theory 151:411–447MathSciNetMATHCrossRefGoogle Scholar
  114. Jaśkiewicz A, Nowak AS (2014b) Robust Markov perfect equilibria. J Math Anal Appl 419: 1322–1332MathSciNetMATHCrossRefGoogle Scholar
  115. Jaśkiewicz A, Nowak AS (2015a) On pure stationary almost Markov Nash equilibria in nonzero-sum ARAT stochastic games. Math Methods Oper Res 81:169–179MathSciNetMATHCrossRefGoogle Scholar
  116. Jaśkiewicz A, Nowak AS (2015b) Stochastic games of resource extraction. Automatica 54: 310–316MathSciNetMATHCrossRefGoogle Scholar
  117. Jaśkiewicz A, Nowak AS (2016) Stationary almost Markov perfect equilibria in discounted stochastic games. Math Oper Res 41:430–441MathSciNetMATHCrossRefGoogle Scholar
  118. Jaśkiewicz A, Nowak AS (2018a) Zero-sum stochastic games. In: Basar T, Zaccour G (eds) Handbook of dynamic game theory, Birkhäuser, BaselGoogle Scholar
  119. Jaśkiewicz A, Nowak AS (2018b, in press) On symmetric stochastic games of resource extraction with weakly continuous transitions. TOPGoogle Scholar
  120. Kakutani S (1941) A generalization of Brouwer’s fixed point theorem. Duke Math J 8:457–459MathSciNetMATHCrossRefGoogle Scholar
  121. Kiefer YI (1971) Optimal stopped games. Theory Probab Appl 16:185–189CrossRefGoogle Scholar
  122. Kitti M (2016) Subgame-perfect equilibria in discounted stochastic games. J Math Anal Appl 435:253–266MathSciNetMATHCrossRefGoogle Scholar
  123. Klein E, Thompson AC (1984) Theory of correspondences. Wiley, New YorkMATHGoogle Scholar
  124. Kohlberg E, Mertens JF (1986) On the strategic stability of equilibria. Econometrica 54:1003–1037MathSciNetMATHCrossRefGoogle Scholar
  125. Krasnosielska-Kobos A (2016) Construction of Nash equilibrium based on multiple stopping problem in multi-person game. Math Methods Oper Res (2016) 83:53–70MathSciNetMATHCrossRefGoogle Scholar
  126. Krasnosielska-Kobos A, Ferenstein E (2013) Construction of Nash equilibrium in a game version of Elfving’s multiple stopping problem. Dyn Games Appl 3:220–235MathSciNetMATHCrossRefGoogle Scholar
  127. Krishnamurthy N, Parthasarathy T, Babu S (2012) Existence of stationary equilibrium for mixtures of discounted stochastic games. Curr Sci 103:1003–1013Google Scholar
  128. Krishnamurthy N, Parthasarathy T, Ravindran G (2012) Solving subclasses of multi-player stochastic games via linear complementarity problem formulations – a survey and some new results. Optim Eng 13:435–457MathSciNetMATHCrossRefGoogle Scholar
  129. Kuipers J, Flesch J, Schoenmakers G, Vrieze K (2016) Subgame-perfection in recursive perfect information games, where each player controls one state. Int J Game Theory 45:205–237MathSciNetMATHCrossRefGoogle Scholar
  130. Kuratowski K, Ryll-Nardzewski C (1965) A general theorem on selectors. Bull Polish Acad Sci (Ser Math) 13:397–403MathSciNetMATHGoogle Scholar
  131. Küenle HU (1994) On Nash equilibrium solutions in nonzero-sum stochastic games with complete information. Int J Game Theory 23:303–324MathSciNetMATHCrossRefGoogle Scholar
  132. Küenle HU (1999) Equilibrium strategies in stochastic games with additive cost and transition structure and Borel state and action spaces. Int Game Theory Rev 1:131–147MathSciNetMATHCrossRefGoogle Scholar
  133. Leininger W (1986) The existence of perfect equilibria in model of growth with altruism between generations. Rev Econ Stud 53:349–368MathSciNetMATHCrossRefGoogle Scholar
  134. Lemke CE (1965) Bimatrix equilibrium points and mathematical programming. Manag Sci 11:681–689MathSciNetMATHCrossRefGoogle Scholar
  135. Lemke CE, Howson JT Jr (1964) Equilibrium points of bimatrix games. SIAM J Appl Math 12:413–423MathSciNetMATHCrossRefGoogle Scholar
  136. Levhari D, Mirman L (1980) The great fish war: an example using a dynamic Coumot-Nash solution. Bell J Econ 11:322–334CrossRefGoogle Scholar
  137. Levy YJ (2013) Discounted stochastic games with no stationary Nash equilibrium: two examples. Econometrica 81:1973–2007MathSciNetMATHCrossRefGoogle Scholar
  138. Levy YJ, McLennan A (2015) Corrigendum to: discounted stochastic games with no stationary Nash equilibrium: two examples. Econometrica 83:1237–1252MathSciNetCrossRefGoogle Scholar
  139. Maitra A, Sudderth W (2003) Borel stay-in-a-set games. Int J Game Theory 32:97–108MathSciNetMATHCrossRefGoogle Scholar
  140. Maitra A, Sudderth WD (2007) Subgame-perfect equilibria for stochastic games. Math Oper Res 32:711–722MathSciNetMATHCrossRefGoogle Scholar
  141. Majumdar MK, Sundaram R (1991) Symmetric stochastic games of resource extraction. The existence of non-randomized stationary equilibrium. In: Raghavan et al (eds) Stochastic games and related topics. Kluwer, Dordrecht, pp 175–190CrossRefGoogle Scholar
  142. Maliar, L, Maliar, S (2016) Ruling out multiplicity of smooth equilibria in dynamic games: a hyperbolic discounting example. Dyn Games Appl 6:243–261MathSciNetMATHCrossRefGoogle Scholar
  143. Mas-Colell A (1974) A note on a theorem of F. Browder. Math Program 6:229–233MathSciNetMATHCrossRefGoogle Scholar
  144. Mashiah-Yaakovi A (2009) Periodic stopping games. Int J Game Theory 38:169–181MathSciNetMATHCrossRefGoogle Scholar
  145. Mashiah-Yaakovi A (2014) Subgame-perfect equilibria in stopping games. Int J Game Theory 43:89–135MathSciNetMATHCrossRefGoogle Scholar
  146. Mashiah-Yaakovi A (2015) Correlated equilibria in stochastic games with Borel measurable payoffs. Dyn Games Appl 5:120–135MathSciNetMATHCrossRefGoogle Scholar
  147. Maskin E, Tirole J (2001) Markov perfect equilibrium: I. Observable actions. J Econ Theory 100:191–219MathSciNetMATHCrossRefGoogle Scholar
  148. Mertens JF (2002) Stochastic games. In: Aumann RJ, Hart S (eds) Handbook of game theory with economic applications, vol 3. North Holland, Amsterdam/London, pp 1809–1832Google Scholar
  149. Mertens JF (2003) A measurable measurable choice theorem. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 107–130CrossRefGoogle Scholar
  150. Mertens JF, Neyman A (1981) Stochastic games. Int J Game Theory 10:53–56MathSciNetMATHCrossRefGoogle Scholar
  151. Mertens JF, Parthasarathy T (1991) Nonzero-sum stochastic games. In: Raghavan et al (eds) Stochastic games and related topics. Kluwer, Dordrecht, pp 145–148CrossRefGoogle Scholar
  152. Mertens JF, Parthasarathy T (2003) Equilibria for discounted stochastic games. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 131–172MATHCrossRefGoogle Scholar
  153. Mertens JF, Sorin S, Zamir S (2015) Repeated games. Cambridge University Press, CambridgeMATHCrossRefGoogle Scholar
  154. Milgrom P, Roberts J (1990) Rationalizability, learning and equilibrium in games with strategic complementarities. Econometrica 58:1255–1277MathSciNetMATHCrossRefGoogle Scholar
  155. Milgrom P, Shannon C (1994) Monotone comparative statics. Econometrica 62:157–180MathSciNetMATHCrossRefGoogle Scholar
  156. Mohan SR, Neogy SK, Parthasarathy T (1997) Linear complementarity and discounted polystochastic game when one player controls transitions. In: Ferris MC, Pang JS (eds) Proceedings of the international conference on complementarity problems. SIAM, Philadelphia, pp 284–294MATHGoogle Scholar
  157. Mohan SR, Neogy SK, Parthasarathy T (2001) Pivoting algorithms for some classes of stochastic games: a survey. Int Game Theory Rev 3:253–281MathSciNetMATHCrossRefGoogle Scholar
  158. Monderer D, Shapley LS (1996) Potential games. Games Econ Behavior 14:124–143MathSciNetMATHCrossRefGoogle Scholar
  159. Montrucchio L (1987) Lipschitz continuous policy functions for strongly concave optimization problems. J Math Econ 16:259–273MathSciNetMATHCrossRefGoogle Scholar
  160. Morimoto H (1986) Nonzero-sum discrete parameter stochastic games with stopping times. Probab Theory Relat Fields 72:155–160MathSciNetMATHCrossRefGoogle Scholar
  161. Nash JF (1950) Equilibrium points in n -person games. Proc Nat Acad Sci USA 36:48–49MathSciNetMATHCrossRefGoogle Scholar
  162. Neveu J (1975) Discrete-parameter martingales. North-Holland, AmsterdamMATHGoogle Scholar
  163. Neyman A (2017) Continuous-time stochastic games. Games Econ Behavior 104:92–130MathSciNetMATHCrossRefGoogle Scholar
  164. Neyman A, Sorin S (eds) (2003) Stochastic games and applications. Kluwer, DordrechtMATHGoogle Scholar
  165. Nowak AS (1985) Existence of equilibrium stationary strategies in discounted noncooperative stochastic games with uncountable state space. J Optim Theory Appl 45:591–602MathSciNetMATHCrossRefGoogle Scholar
  166. Nowak AS (1987) Nonrandomized strategy equilibria in noncooperative stochastic games with additive transition and reward structure. J Optim Theory Appl 52:429–441MathSciNetMATHCrossRefGoogle Scholar
  167. Nowak AS (2003a) N-person stochastic games: extensions of the finite state space case and correlation. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 93–106MATHCrossRefGoogle Scholar
  168. Nowak AS (2003b) On a new class of nonzero-sum discounted stochastic games having stationary Nash equilibrium point. Int J Game Theory 32:121–132MathSciNetMATHCrossRefGoogle Scholar
  169. Nowak AS (2006a) On perfect equilibria in stochastic models of growth with intergenerational altruism. Econ Theory 28:73–83MathSciNetMATHCrossRefGoogle Scholar
  170. Nowak AS (2006b) A multigenerational dynamic game of resource extraction. Math Social Sci 51:327–336MathSciNetMATHCrossRefGoogle Scholar
  171. Nowak AS (2006c) A note on equilibrium in the great fish war game. Econ Bull 17(2):1–10Google Scholar
  172. Nowak AS (2007) On stochastic games in economics. Math Methods Oper Res 66:513–530MathSciNetMATHCrossRefGoogle Scholar
  173. Nowak AS (2008) Equilibrium in a dynamic game of capital accumulation with the overtaking criterion. Econ Lett 99:233–237MathSciNetMATHCrossRefGoogle Scholar
  174. Nowak AS (2010) On a noncooperative stochastic game played by internally cooperating generations. J Optim Theory Appl 144:88–106MathSciNetMATHCrossRefGoogle Scholar
  175. Nowak AS, Altman E (2002) ε-Equilibria for stochastic games with uncountable state space and unbounded costs. SIAM J Control Optim 40:1821–1839MathSciNetMATHCrossRefGoogle Scholar
  176. Nowak AS, Jaśkiewicz A (2005) Nonzero-sum semi-Markov games with the expected average payoffs. Math Methods Oper Res 62:23–40MathSciNetMATHCrossRefGoogle Scholar
  177. Nowak AS, Raghavan TES (1992) Existence of stationary correlated equilibria with symmetric information for discounted stochastic games. Math Oper Res 17:519–526MathSciNetMATHCrossRefGoogle Scholar
  178. Nowak AS, Raghavan TES (1993) A finite step algorithm via a bimatrix game to a single controller non-zero sum stochastic game. Math Program Ser A 59:249–259MathSciNetMATHCrossRefGoogle Scholar
  179. Nowak AS, Szajowski K (1999) Nonzero-sum stochastic games. In: Ann Int Soc Dyn Games, vol 4. Birkhäuser, Boston, pp 297–343Google Scholar
  180. Nowak AS, Wiȩcek P (2007) On Nikaido-Isoda type theorems for discounted stochastic games. J Math Anal Appl 332:1109–1118Google Scholar
  181. Ohtsubo Y (1987) A nonzero-sum extension of Dynkin’s stopping problem. Math Oper Res 12:277–296MathSciNetMATHCrossRefGoogle Scholar
  182. Ohtsubo Y (1991) On a discrete-time nonzero-sum Dynkin problem with monotonicity. J Appl Probab 28:466–472MathSciNetMATHCrossRefGoogle Scholar
  183. Parthasarathy T (1973) Discounted, positive, and noncooperative stochastic games. Int J Game Theory 2:25–37MathSciNetMATHCrossRefGoogle Scholar
  184. Parthasarathy T, Sinha S (1989) Existence of stationary equilibrium strategies in nonzero-sum discounted stochastic games with uncountable state space and state-independent transitions. Int J Game Theory 18:189–194MATHCrossRefGoogle Scholar
  185. Peleg B, Yaari M (1973) On the existence of consistent course of action when tastes are changing. Rev Econ Stud 40:391–401MATHCrossRefGoogle Scholar
  186. Peski M, Wiseman T (2015) A folk theorem for stochastic games with infrequent state changes. Theoret Econ 10:131–173MathSciNetCrossRefGoogle Scholar
  187. Phelps E, Pollak R (1968) On second best national savings and game equilibrium growth. Rev Econ Stud 35:195–199CrossRefGoogle Scholar
  188. Pollak R (1968) Consistent planning. Rev Econ Stud 35:201–208CrossRefGoogle Scholar
  189. Potters JAM, Raghavan TES, Tijs SH (2009) Pure equilibrium strategies for stochastic games via potential functions. In: Advances in dynamic games and their applications. Annals of the international society of dynamic games, vol 10. Birkhäuser, Boston, pp 433–444Google Scholar
  190. Purves RA, Sudderth WD (2011) Perfect information games with upper semicontinuous payoffs. Math Oper Res 36:468–473MathSciNetMATHCrossRefGoogle Scholar
  191. Puterman ML (1994) Markov decision processes: discrete stochastic dynamic programming. Wiley, HobokenMATHCrossRefGoogle Scholar
  192. Raghavan TES, Syed Z (2002) Computing stationary Nash equilibria of undiscounted single-controller stochastic games. Math Oper Res 27:384–400MathSciNetMATHCrossRefGoogle Scholar
  193. Raghavan TES, Tijs SH, Vrieze OJ (1985) On stochastic games with additive reward and transition structure. J Optim Theory Appl 47:451–464MathSciNetMATHCrossRefGoogle Scholar
  194. Ramsey FP (1928) A mathematical theory of savings. Econ J 38:543–559CrossRefGoogle Scholar
  195. Ray D (1987) Nonpaternalistic intergenerational altruism. J Econ Theory 40:112–132MATHCrossRefGoogle Scholar
  196. Reny PJ, Robson A (2002) Existence of subgame-perfect equilibrium with public randomization: a short proof. Econ Bull 3(24):1–8Google Scholar
  197. Rieder U (1979) Equilibrium plans for non-zero sum Markov games. In: Moeschlin O, Pallaschke D (eds) Game theory and related topics. North-Holland, Amsterdam, pp 91–102Google Scholar
  198. Rogers PD (1969) Non-zero-sum stochastic games. Ph.D. dissertation, report 69–8, Univ of CaliforniaGoogle Scholar
  199. Rosen JB (1965) Existence and uniqueness of equilibrium points for concave n-person games. Econometrica 33:520–534MathSciNetMATHCrossRefGoogle Scholar
  200. Rosenberg D, Solan E, Vieille N (2001) Stopping games with randomized strategies. Probab Theory Relat Fields 119:433–451MathSciNetMATHCrossRefGoogle Scholar
  201. Rubinstein A (1979) Equilibrium in supergames with the overtaking criterion. J Econ Theory 21:1–9MathSciNetMATHCrossRefGoogle Scholar
  202. Secchi P, Sudderth WD (2002a) Stay-in-a-set games. Int J Game Theory 30:479–490MathSciNetMATHCrossRefGoogle Scholar
  203. Secchi P, Sudderth WD (2002b) N-person stochastic games with upper semi-continuous payoffs. Int J Game Theory 30:491–502MathSciNetMATHCrossRefGoogle Scholar
  204. Secchi P, Sudderth.WD (2005) A simple two-person stochastic game with money. In: Annals of the international society of dynamic games, vol 7. Birkhäuser, Boston, pp 39–66Google Scholar
  205. Selten R (1975) Re-examination of the perfectness concept for equilibrium points in extensive games. Int J Game Theory 4:25–55MATHCrossRefGoogle Scholar
  206. Shapley LS (1953) Stochastic games. Proc Nat Acad Sci USA 39:1095–1100MathSciNetMATHCrossRefGoogle Scholar
  207. Shubik M, Whitt W (1973) Fiat money in an economy with one non-durable good and no credit: a non-cooperative sequential game. In: Blaquière A (ed) Topics in differential games. North-Holland, Amsterdam, pp 401–448Google Scholar
  208. Shmaya E, Solan E (2004) Two player non-zero sum stopping games in discrete time. Ann Probab 32:2733–2764MathSciNetMATHCrossRefGoogle Scholar
  209. Shmaya E, Solan E, Vieille N (2003) An applications of Ramsey theorem to stopping games. Games Econ Behavior 42:300–306MathSciNetMATHCrossRefGoogle Scholar
  210. Simon RS (2007) The structure of nonzero-sum stochastic games. Adv Appl Math 38:1–26MathSciNetCrossRefGoogle Scholar
  211. Simon R (2012) A topological approach to quitting games. Math Oper Res 37:180–195MathSciNetMATHCrossRefGoogle Scholar
  212. Simon RS (2016) The challenge of nonzero-sum stochastic games. Int J Game Theory 45:191–204MATHCrossRefGoogle Scholar
  213. Sleet C, Yeltekin S (2016) On the computation of value correspondences for dynamic games. Dyn Games Appl 6:174–186MathSciNetMATHCrossRefGoogle Scholar
  214. Smale S (1976) A convergent process of price adjustment and global Newton methods. J Math Econ 3:107–120MathSciNetMATHCrossRefGoogle Scholar
  215. Sobel MJ (1871) Non-cooperative stochastic games. Ann Math Stat 42:1930–1935MATHCrossRefGoogle Scholar
  216. Solan E (1998) Discounted stochastic games. Math Oper Res 23:1010–1021MathSciNetMATHCrossRefGoogle Scholar
  217. Solan E (1999) Three-person absorbing games. Math Oper Res 24:669–698MathSciNetMATHCrossRefGoogle Scholar
  218. Solan E (2001) Characterization of correlated equilibria in stochastic games. Int J Game Theory 30:259–277MathSciNetMATHCrossRefGoogle Scholar
  219. Solan E (2017, in press) Acceptable strategy profiles in stochastic games. Games Econ BehaviorGoogle Scholar
  220. Solan E, Vieille N (2001) Quitting games. Math Oper Res 26:265–285MathSciNetMATHCrossRefGoogle Scholar
  221. Solan E, Vieille N (2002) Correlated equilibrium in stochastic games. Games Econ Behavior 38:362–399MathSciNetMATHCrossRefGoogle Scholar
  222. Solan E, Vieille N (2003) Deterministic multi-player Dynkin games. J Math Econ 39:911–929MathSciNetMATHCrossRefGoogle Scholar
  223. Solan E, Vieille N (2010) Computing uniformly optimal strategies in two-player stochastic games. Econ Theory 42:237–253MathSciNetMATHCrossRefGoogle Scholar
  224. Solan E, Ziliotto B (2016) Stochastic games with signals. In: Annals of the international society of dynamic games, vol 14. Birkhäuser, Boston, pp 77–94MATHGoogle Scholar
  225. Sorin S (1986) Asymptotic properties of a nonzero-sum stochastic games. Int J Game Theory 15:101–107MathSciNetMATHCrossRefGoogle Scholar
  226. Spence M (1976) Product selection, fixed costs, and monopolistic competition. Rev Econ Stud 43:217–235MATHCrossRefGoogle Scholar
  227. Stachurski J (2009) Economic dynamics: theory and computation. MIT Press, Cambridge, MAMATHGoogle Scholar
  228. Stokey NL, Lucas RE, Prescott E (1989) Recursive methods in economic dynamics. Harvard University Press, CambridgeGoogle Scholar
  229. Strotz RH (1956) Myopia and inconsistency in dynamic utility maximization. Rev Econ Stud 23:165–180CrossRefGoogle Scholar
  230. Sundaram RK (1989a) Perfect equilibrium in a class of symmetric dynamic games. J Econ Theory 47:153–177MATHCrossRefGoogle Scholar
  231. Sundaram RK (1989b) Perfect equilibrium in a class of symmetric dynamic games. Corrigendum. J Econ Theory 49:385–187MATHCrossRefGoogle Scholar
  232. Szajowski K (1994) Markov stopping game with random priority. Z Oper Res 39:69–84MathSciNetMATHGoogle Scholar
  233. Szajowski K (1995) Optimal stopping of a discrete Markov process by two decision makers. SIAM J Control Optim 33:1392–1410MathSciNetMATHCrossRefGoogle Scholar
  234. Takahashi M (1964) Equilibrium points of stochastic non-cooperative n-person games. J Sci Hiroshima Univ Ser A-I Math 28:95–99MathSciNetMATHGoogle Scholar
  235. Thuijsman F, Raghavan TES (1997) Perfect information stochastic games and related classes. Int J Game Theory 26:403–408MathSciNetMATHCrossRefGoogle Scholar
  236. Topkis D (1978) Minimizing a submodular function on a lattice. Oper Res 26:305–321MathSciNetMATHCrossRefGoogle Scholar
  237. Topkis D (1998) Supermodularity and complementarity. Princeton University Press, PrincetonGoogle Scholar
  238. Valadier M (1994) Young measures, weak and strong convergence and the Visintin-Balder theorem. Set-Valued Anal 2:357–367MathSciNetMATHCrossRefGoogle Scholar
  239. Van Long N (2011) Dynamic games in the economics of natural resources: a survey. Dyn Games Appl 1:115–148MathSciNetMATHCrossRefGoogle Scholar
  240. Vieille N (2000a) Two-player stochastic games I: a reduction. Isr J Math 119:55–92MathSciNetMATHCrossRefGoogle Scholar
  241. Vieille N (2000b) Two-player stochastic games II: the case of recursive games. Israel J Math 119:93–126MathSciNetMATHCrossRefGoogle Scholar
  242. Vieille N (2002) Stochastic games: Recent results. In: Aumann RJ, Hart S (eds) Handbook of game theory with economic applications, vol 3. North Holland, Amsterdam/London, pp 1833–1850Google Scholar
  243. Vives X (1990) Nash equilibrium with strategic complementarities. J Math Econ 19:305–321MathSciNetMATHCrossRefGoogle Scholar
  244. von Weizsäcker CC (1965) Existence of optimal programs of accumulation for an infinite horizon. Rev Econ Stud 32:85–104CrossRefGoogle Scholar
  245. Vrieze OJ, Thuijsman F (1989) On equilibria in repeated games with absorbing states. Int J Game Theory 18:293–310MathSciNetMATHCrossRefGoogle Scholar
  246. Whitt W (1980) Representation and approximation of noncooperative sequential games. SIAM J Control Optim 18:33–48MathSciNetMATHCrossRefGoogle Scholar
  247. Wiecek P (2009) Pure equilibria in a simple dynamic model of strategic market game. Math Methods Oper Res 69:59–79MathSciNetMATHCrossRefGoogle Scholar
  248. Wiecek P (2012) N-person dynamic strategic market games. Appl Math Optim 65:147–173MathSciNetMATHCrossRefGoogle Scholar
  249. Yasuda M (1985) On a randomised strategy in Neveu’s stopping problem. Stoch Process Appl 21:159–166MATHCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Faculty of Pure and Applied MathematicsWrocław University of Science and TechnologyWrocławPoland
  2. 2.Faculty of Mathematics, Computer Science and EconometricsUniversity of Zielona GóraZielona GóraPoland

Personalised recommendations