Skip to main content
SpringerLink
Log in

Perturbations of Markov Chains with Applications to Stochastic Games

  • Conference paper
Stochastic Games and Applications

Part of the book series: NATO Science Series ((ASIC,volume 570))

Abstract

In this chapter we will review several topics that are used extensively in the study of n-player stochastic games. These tools were used in the proof of several results on non-zero-sum stochastic games.

Most of the results presented here appeared in [17],[16], and a few appeared in [12],[13].

The first main issue is Markov chains where the transition rule is a Puiseux probability distribution. We define the notion of communicating sets and construct a hierarchy on the collection of these sets. We then relate these concepts to stochastic games, and show several conditions that enable the players to control the exit distribution from communicating sets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Avsar, Z.M. and Baykal-Gürsoy, M. (1999) A decomposition approach for undiscounted two-person zero-sum stochastic games, Mathematical Methods in Operations Research 3, 483–500.

    Article  Google Scholar 

  2. Bather, J. (1973) Optimal decision procedures for finite Markov chains. Part III: General convex systems, Advances in Applied Probability 5, 541–553.

    Article  MathSciNet  MATH  Google Scholar 

  3. Bewley, T. and Kohlberg, E. (1976) The asymptotic theory of stochastic games, Mathematics of Operations Research 1, 197–208.

    Article  MathSciNet  MATH  Google Scholar 

  4. Coulomb, J.M. (2002) Stochastic games without perfect monitoring, mimeo.

    Google Scholar 

  5. Eaves, B.C. and Rothblum, U.G. (1989) A theory on extending algorithms for parametric problems, Mathematics of Operations Research 14, 502–533.

    Article  MathSciNet  MATH  Google Scholar 

  6. Freidlin, M. and Wentzell, A. (1984) Random Perturbations of Dynamical Systems, Springer-Verlag, Berlin.

    Book  MATH  Google Scholar 

  7. Mertens, J.F. and Neyman, A. (1981) Stochastic games, International Journal of Game Theory 10, 53–66.

    Article  MathSciNet  MATH  Google Scholar 

  8. Neyman, A. (2003) Stochastic games: Existence of the minmax, in A. Neyman and S. Sorin (eds.), Stochastic Games and Applications, NATO Science Series C, Mathematical and Physical Sciences, Vol. 570, Kluwer Academic Publishers, Dordrecht, Chapter 11, pp. 173–193.

    Google Scholar 

  9. Rosenberg, D., Solan, E. and Vieille, N. (2002) Stochastic games with imperfect monitoring, Discussion Paper 1341, The Center for Mathematical Studies in Economics and Management Sciences, Northwestern University.

    Google Scholar 

  10. Rosenberg, D., Solan, E. and Vieille, N. (2002) On the maxmin value of stochastic games with imperfect monitoring, Discussion Paper 1344, The Center for Mathematical Studies in Economics and Management Sciences, Northwestern University.

    Google Scholar 

  11. Ross, K.W. and Varadarajan, R. (1991) Multichain Markov decision processes with a sample path constraint: A decomposition approach, Mathematics of Operations Research 16, 195–207.

    Article  MathSciNet  MATH  Google Scholar 

  12. Solan, E. (1999) Three-person absorbing games, Mathematics of Operations Research 24, 669–698.

    Article  MathSciNet  MATH  Google Scholar 

  13. Solan, E. (2000) Stochastic games with two non-absorbing states, Israel Journal of Mathematics 119, 29–54.

    Article  MathSciNet  MATH  Google Scholar 

  14. Solan, E. and Vieille, N. (2002), Correlated equilibrium in stochastic games, Games and Economic Behavior 38, 362–399.

    Article  MathSciNet  MATH  Google Scholar 

  15. Thuijsman, F. (2003) Repeated games with absorbing states, in A. Neyman and S. Sorin (eds.), Stochastic Games and Applications, NATO Science Series C, Mathematical and Physical Sciences, Vol. 570, Kluwer Academic Publishers, Dordrecht, Chapter 13, pp. 205–213.

    Google Scholar 

  16. Vieille, N. (2000) Equilibrium in two-person stochastic games II: The case of recursive games, Israel Journal of Mathematics 119, 93–126.

    Article  MathSciNet  MATH  Google Scholar 

  17. Vieille, N. (2000) Small perturbations and stochastic games, Israel Journal of Mathematics 119, 127–142.

    Article  MathSciNet  MATH  Google Scholar 

  18. Vrieze, O.J. and Thuijsman, F. (1989) On equilibria in repeated games with absorbing states, International Journal of Game Theory 18, 293–310.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Northwestern University, Evanston, USA

    Eilon Solan

  2. Tel Aviv University, Tel Aviv, Israel

    Eilon Solan

Authors
  1. Eilon Solan
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute of Mathematics and Center for Rationality and Interactive Decision Theory, Hebrew University of Jerusalem, Jerusalem, Israel

    Abraham Neyman

  2. Université Pierre et Marie Curie and École Polytechnique, Paris, France

    Sylvain Sorin

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer Science+Business Media New York

About this paper

Cite this paper

Solan, E. (2003). Perturbations of Markov Chains with Applications to Stochastic Games. In: Neyman, A., Sorin, S. (eds) Stochastic Games and Applications. NATO Science Series, vol 570. Springer, Dordrecht. https://doi.org/10.1007/978-94-010-0189-2_17

Download citation

  • DOI: https://doi.org/10.1007/978-94-010-0189-2_17

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-1-4020-1493-2

  • Online ISBN: 978-94-010-0189-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation