Skip to main content
SpringerLink
Log in

Stochastic Games

  • Reference work entry
Computational Complexity

Article Outline

Glossary

Definition of the Subject

Strategies, Evaluations and Equilibria

Zero-Sum Games

Multi‐Player Games

Correlated Equilibrium

Imperfect Monitoring

Algorithms

Additional and Future Directions

Acknowledgments

Bibliography

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 1,500.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 1,399.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    That is, at each stage player 2 plays L with probability \( { \frac{1}{2} } \) and R with probability \( { \frac{1}{2} } \).

  2. 2.

    Mertens and Neyman's [38]result actually holds in every stochastic game that satisfies a proper condition, which is always satisfied when the state and action spaces arefinite.

Abbreviations

A stochastic game:

A  repeated interaction between several participants in which the underlying state of the environment changes stochastically, and it depends on the decisions of the participants.

A strategy:

A  rule that dictates how a participant in an interaction makes his decisions as a function of the observed behavior of the other participants and of the evolution of the environment.

Evaluation of stage payoffs:

The way that a participant in a repeated interaction evaluates the stream of stage payoffs that he receives (or stage costs that he pays) along the interaction.

An equilibrium:

A  collection of strategies, one for each player, such that each player maximizes (or minimizes, in case of stage costs) his evaluation of stage payoffs given the strategies of the other players.

A correlated equilibrium:

An equilibrium in an extended game in which at the outset of the game each player receives a private signal, and the vector of private signals is chosen according to a known joint probability distribution. In the extended game, a strategy of a player depends, in addition to past play, on the signal he received.

Bibliography

Primary Literature

  1. Altman E (2005) Applications of dynamic games in queues. Adv Dyn Games 7:309–342

    Article  MathSciNet  Google Scholar 

  2. Altman E, Gaitsgory VA (1995) A hybrid (differential‐stochastic) zero-sum game with fast stochastic part. Ann Int Soc Dyn Games 3:47–59

    MathSciNet  Google Scholar 

  3. Altman E, Solan E (2007) Games with constraints with networking applications. Preprint

    Google Scholar 

  4. Altman E, Avrachenkov K, Marquez R, Miller G (2005) Zero-sum constrained stochastic games with independent state processes. Math Methods Oper Res 62:375–386

    Article  MathSciNet  MATH  Google Scholar 

  5. Altman E, Avrachenkov K, Bonneau N, Debbah M, El‐Azouzi R, Sadoc Menasche D (2008) Constrained cost‐coupled stochastic games with independent state processes. Oper Res Lett 36:160–164

    Google Scholar 

  6. Amir R (1996) Continuous stochastic games of capital accumulation with convex transitions. Games Econ Behav 15:111–131

    Article  MathSciNet  MATH  Google Scholar 

  7. Aumann RJ (1974) Subjectivity and correlation in randomized strategies. J Math Econ 1:67–96

    Article  MathSciNet  MATH  Google Scholar 

  8. Aumann RJ (1987) Correlated equilibrium as an expression of bayesian rationality. Econometrica 55:1–18

    Article  MathSciNet  MATH  Google Scholar 

  9. Başar T, Olsder GJ (1995) Dynamic noncooperative game theory. Academic Press, New York

    Google Scholar 

  10. Bewley T, Kohlberg E (1976) The asymptotic theory of stochastic games. Math Oper Res 1:197–208

    Article  MathSciNet  MATH  Google Scholar 

  11. Blackwell D (1956) An analog of the minimax theorem for vector payoffs. Pac J Math 6:1–8

    Article  MathSciNet  MATH  Google Scholar 

  12. Blackwell D, Ferguson TS (1968) The big match. Ann Math Stat 39:159–163

    Article  MathSciNet  MATH  Google Scholar 

  13. Chari V, Kehoe P (1990) Sustainable plans. J Political Econ 98:783–802

    Article  Google Scholar 

  14. Chatterjee K, Majumdar R, Henzinger TA (2008) Stochastic limit‐average games are in EXPTIME. Int J Game Theory 37:219–234

    Article  MathSciNet  MATH  Google Scholar 

  15. Coulomb JM (2003) Absorbing games with a signalling structure. In: Neyman A, Sorin S (eds) Stochastic games and applications. NATO Science Series. Kluwer, Dordrecht, pp 335–355

    Chapter  Google Scholar 

  16. Coulomb JM (2003) Games with a recursive structure. In: Neyman A, Sorin S (eds) Stochastic games and applications. NATO Science Series. Kluwer, Dordrecht, pp 427–442

    Chapter  Google Scholar 

  17. Dutta P, Sundaram RK (1992) Markovian equilibrium in a class of stochastic games: Existence theorems for discounted and undiscounted models. Econ Theory 2:197–214

    Article  MathSciNet  MATH  Google Scholar 

  18. Dutta P, Sundaram RK (1993) The tragedy of the commons? Econ Theory 3:413–426

    Article  MathSciNet  MATH  Google Scholar 

  19. Filar JA, Vrieze K (1996) Competitive Markov decision processes. Springer

    Google Scholar 

  20. Fink AM (1964) Equilibrium in a stochastic n‑person game. J Sci Hiroshima Univ 28:89–93

    MathSciNet  MATH  Google Scholar 

  21. Flesch J, Thuijsman F, Vrieze K (1997) Cyclic Markov equilibria in stochastic games. Int J Game Th 26:303–314

    Article  MathSciNet  MATH  Google Scholar 

  22. Flesch J, Thuijsman F, Vrieze OJ (2003) Stochastic games with non‐observable actions. Math Meth Oper Res 58:459–475

    Article  MathSciNet  MATH  Google Scholar 

  23. Flesch J, Schoenmakers G, Vrieze K (2008) Stochastic games on a product state space. Math Oper Res 33:403–420

    Article  MathSciNet  MATH  Google Scholar 

  24. Flesch J, Thuijsman F, Vrieze OJ (2007) Stochastic games with additive transitions. Europ J Oper Res 179:483–497

    Article  MathSciNet  MATH  Google Scholar 

  25. Forges F (1990) Universal mechanisms. Econometrica 58:1341–1364

    Article  MathSciNet  MATH  Google Scholar 

  26. Fortnow L, Kimmel P (1998) Beating a finite automaton in the big match. In: Proceedings of the 7th conference on theoretical aspects of rationality and knowledge. Morgan Kaufmann, San Francisco, pp 225–234

    Google Scholar 

  27. Gillette D (1957) Stochastic games with zero stop probabilities, contributions to the theory of games, vol 3. Princeton University Press, Princeton

    Google Scholar 

  28. Herings JJP, Peeters RJAP (2004) Stationary equilibria in stochastic games: Structure, selection, and computation. J Econ Theory 118:32–60

    Article  MathSciNet  MATH  Google Scholar 

  29. Jaskiewicz A, Nowak AS (2006) Zero-sum ergodic stochastic games with Feller transition probabilities. SIAM J Control Optim 45:773–789

    Article  MathSciNet  MATH  Google Scholar 

  30. Kakutani S (1941) A generalization of Brouwer's fixed point theorem. Duke Math J 8:457–459

    Article  MathSciNet  Google Scholar 

  31. Kohlberg E (1974) Repeated games with absorbing states. Ann Stat 2:724–738

    Article  MathSciNet  MATH  Google Scholar 

  32. Krausz A, Rieder U (1997) Markov games with incomplete information. Math Meth Oper Res 46:263–279

    Article  MathSciNet  MATH  Google Scholar 

  33. Levhari D, Mirman L (1980) The great fish war: An example using a dynamic Cournot–Nash solution. Bell J Econ 11(1):322–334

    Article  MathSciNet  Google Scholar 

  34. Maitra A, Sudderth W (1998) Finitely additive stochastic games with Borel measurable payoffs. Int J Game Theory 27:257–267

    Article  MathSciNet  MATH  Google Scholar 

  35. Martin DA (1998) The determinacy of Blackwell games. J Symb Logic 63:1565–1581

    Article  MATH  Google Scholar 

  36. Mertens JF (1987) Repeated games. In: Proceedings of the international congress of mathematicians, American Mathematical Society, Berkeley, California, pp 1528–1577

    Google Scholar 

  37. Mertens JF, Neyman A (1981) Stochastic games. Int J Game Th 10:53–66

    Article  MathSciNet  MATH  Google Scholar 

  38. Mertens JF, Parthasarathy T (1987) Equilibria for discounted stochastic games, CORE Discussion Paper No. 8750. (Also published in Stochastic Games and Applications, Neyman A, Sorin S (eds), NATO Science Series, Kluwer, 131–172)

    Google Scholar 

  39. Mertens JF, Sorin S, Zamir S (1994) Repeated games, CORE Discussion Paper 9420-9422

    Google Scholar 

  40. Milman E (2006) Approachable sets of vector payoffs in stochastic games. Games Econ Behav 56:135–147

    Article  MathSciNet  MATH  Google Scholar 

  41. Neyman A, Sorin S (2003) Stochastic games and applications. NATO Science Series. Kluwer

    Google Scholar 

  42. Nowak AS (1985) Existence of equilibrium stationary strategies in discounted noncooperative stochastic games with uncountable state space. J Optim Theory Appl 45:591–620

    Article  MathSciNet  MATH  Google Scholar 

  43. Nowak AS (2003) N‑person stochastic games: Extensions of the finite state space case and correlation. In: Neyman A, Sorin S (eds) Stochastic games and applications. NATO Science Series. Kluwer, Dordrecht, pp 93–106

    Chapter  Google Scholar 

  44. Nowak AS (2003) On a new class of nonzero‐sum discounted stochastic games having stationary Nash equilibrium points. Int J Game Theory 32:121–132

    Article  MATH  Google Scholar 

  45. Nowak AS (2003) Zero-sum stochastic games with Borel state spaces. In: Neyman A, Sorin S (eds) Stochastic games and applications. NATO Science Series. Kluwer, Dordrecht, pp 77–91

    Chapter  Google Scholar 

  46. Nowak AS, Raghavan TES (1991) Existence of stationary correlated equilibria with symmetric information for discounted stochastic games. Math Oper Res 17:519–526

    Article  MathSciNet  Google Scholar 

  47. Phelan C, Stacchetti E (2001) Sequential equilibria in a Ramsey tax model. Econometrica 69:1491–1518

    Article  MathSciNet  MATH  Google Scholar 

  48. Puterman ML (1994) Markov decision processes: Discrete stochastic dynamic programming. Wiley, Hoboken

    Book  MATH  Google Scholar 

  49. Raghavan TES, Syed Z (2002) Computing stationary Nash equilibria of undiscounted single‐controller stochastic games. Math Oper Res 27:384–400

    Article  MathSciNet  MATH  Google Scholar 

  50. Raghavan TES, Syed Z (2003) A policy improvement type algorithm for solving zero-sum two‐person stochastic games of perfect information. Math Program Ser A 95:513–532

    Article  MathSciNet  MATH  Google Scholar 

  51. Renault J (2006) The value of Markov chain games with lack of information on one side. Math Oper Res 31:490–512

    Article  MathSciNet  MATH  Google Scholar 

  52. Renault J (2007) The value of repeated games with an informed controller. Preprint

    Google Scholar 

  53. Renault J (2007) Uniform value in dynamic programming. Preprint

    Google Scholar 

  54. Rosenberg D, Solan E, Vieille N (2002) Blackwell optimality in Markov decision processes with partial observation. Ann Statists 30:1178–1193

    Article  MathSciNet  MATH  Google Scholar 

  55. Rosenberg D, Solan E, Vieille N (2004) Stochastic games with a single controller and incomplete information. SIAM J Control Optim 43:86–110

    Article  MathSciNet  MATH  Google Scholar 

  56. Rosenberg D, Solan E, Vieille N (2006) Protocol with no acknowledgement. Oper Res, forthcoming

    Google Scholar 

  57. Sagduyu YE, Ephremides A (2003) Power control and rate adaptation as stochastic games for random access. Proc 42nd IEEE Conf Decis Control 4:4202–4207

    Google Scholar 

  58. Savani R, von Stengel B (2004) Exponentially many steps for finding a Nash equilibrium in a bimatrix game. Proc 45th Ann IEEE Symp Found Comput Sci 2004:258–267

    Google Scholar 

  59. Shapley LS (1953) Stochastic games. Proc Nat Acad Sci USA 39:1095–1100

    Article  MathSciNet  MATH  Google Scholar 

  60. Simon RS (2003) The structure of non-zero-sum stochastic games. Adv Appl Math 38:1–26

    Article  Google Scholar 

  61. Solan E (1998) Discounted stochastic games. Math Oper Res 23:1010–1021

    Article  MathSciNet  MATH  Google Scholar 

  62. Solan E (1999) Three‐person absorbing games. Math Oper Res 24:669–698

    Article  MathSciNet  MATH  Google Scholar 

  63. Solan E (2001) Characterization of correlated equilibria in stochastic games. Int J Game Theory 30:259–277

    Article  MathSciNet  MATH  Google Scholar 

  64. Solan E, Vieille N (2001) Quitting games. Math Oper Res 26:265–285

    Article  MathSciNet  MATH  Google Scholar 

  65. Solan E, Vieille N (2002) Correlated equilibrium in stochastic games. Games Econ Behav 38:362–399

    Article  MathSciNet  MATH  Google Scholar 

  66. Solan E, Vieille N (2007) Calculating uniform optimal strategies and equilibria in two‐player stochastic games. Preprint

    Google Scholar 

  67. Solan E, Vohra R (2002) Correlated equilibrium payoffs and public signalling in absorbing games. Int J Game Theory 31:91–122

    Article  MathSciNet  MATH  Google Scholar 

  68. Sorin S (1984) Big match with lack of information on one side (part 1). Int J Game Theory 13:201–255

    Article  MathSciNet  MATH  Google Scholar 

  69. Sorin S (1985) Big match with lack of information on one side (part 2). Int J Game Theory 14:173–204

    Article  MathSciNet  MATH  Google Scholar 

  70. Sorin S (1986) Asymptotic properties of a non‐zerosum stochastic games. Int J Game Theory 15:101–107

    Article  MathSciNet  MATH  Google Scholar 

  71. Sorin S (2002) A first course on zero-sum repeated games. Mathématiques et Applications, vol 37. Springer

    Google Scholar 

  72. Sorin S (2003) Stochastic gameswith incomplete information. In: Neyman A, Sorin S (eds) Stochastic Games and Applications. NATO Science Series. Kluwer, Berlin, pp 375–395

    Chapter  Google Scholar 

  73. Sorin S, Zamir S (1991) Big match with lack of information on one side (part 3). In: Raghavan TES et al (eds) Stochastic games and related topics. Kluwer, pp 101–112

    Google Scholar 

  74. Takahashi M (1962) Stochastic games with infinitely many strategies. J Sci Hiroshima Univ Ser A-I 26:123–134

    MATH  Google Scholar 

  75. Thuijsman F, Raghavan TES (1997) Perfect information stochastic games and related classes. Int J Game Theory 26:403–408

    Article  MathSciNet  MATH  Google Scholar 

  76. Vieille N (2000) Equilibrium in 2‑person stochastic games I: A Reduction. Israel J Math 119:55–91

    Article  MathSciNet  MATH  Google Scholar 

  77. Vieille N (2000) Equilibrium in 2‑person stochastic games II: The case of recursive games. Israel J Math 119:93–126

    Article  MathSciNet  MATH  Google Scholar 

  78. Vrieze OJ, Thuijsman F (1989) On equilibria in repeated games with absorbing states. Int J Game Theory 18:293–310

    Article  MathSciNet  MATH  Google Scholar 

  79. Vrieze OJ, Tijs SH (1982) Fictitious play applied to sequences of games and discounted stochastic games. Int J Game Theory 12:71–85

    Article  MathSciNet  Google Scholar 

  80. Whitt W (1980) Representation and approximation of noncooperative sequential games. SIAM J Control Optim 18:33–48

    Article  MathSciNet  MATH  Google Scholar 

Books and Reviews

  1. Başar T, Olsder GJ (1995) Dynamic noncooperative game theory. Academic

    Google Scholar 

  2. Filar JA, Vrieze K (1996) Competitive Markov decision processes. Springer

    Google Scholar 

  3. Maitra AP, Sudderth WD (1996) Discrete gambling and stochastic games. Springer

    Google Scholar 

  4. Mertens JF (2002) Stochastic games. In: Aumann RJ, Hart S (eds) Handbook of game theory with economic applications, vol 3. Elsevier, pp 1809–1832

    Google Scholar 

  5. Mertens JF, Sorin S, Zamir S (1994) Repeated games. CORE Discussion Paper 9420-9422

    Google Scholar 

  6. Raghavan TES, Shapley LS (1991) Stochastic games and related topics: In honor of Professor L.S. Shapley. Springer

    Google Scholar 

  7. Vieille N (2002) Stochastic games: Recent results. In: Aumann RJ, Hart S (eds) Handbook of game theory with economic applications, vol 3. Elsevier, pp 1833–1850

    Google Scholar 

Download references

Acknowledgments

I thank Eitan Altman, János Flesch, Yuval Heller, Jean‐Jacques Herings, AyalaMashiach‐Yakovi, Andrzej Nowak, Ronald Peeters, T.E.S. Raghavan, Jérôme Renault, Nahum Shimkin, Robert Simon, Sylvain Sorin, WilliamSudderth, and Frank Thuijmsman, for their comments on an earlier version of the entry.

Author information

Authors and Affiliations

  1. The School of Mathematical Sciences, Tel Aviv University, Tel Aviv, Israel

    Eilon Solan

Authors
  1. Eilon Solan
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. RAMTECH LIMITED, 122 Escalle Lane, Larkspur, CA, 94939, USA

    Robert A. Meyers Ph. D. (Editor-in-Chief) (Editor-in-Chief)

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag

About this entry

Cite this entry

Solan, E. (2012). Stochastic Games. In: Meyers, R. (eds) Computational Complexity. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-1800-9_191

Download citation

Publish with us

Policies and ethics

Search

Navigation