Skip to main content
SpringerLink
Log in
Book cover

International Conference on Integer Programming and Combinatorial Optimization

IPCO 2010: Integer Programming and Combinatorial Optimization pp 341–354Cite as

A Pumping Algorithm for Ergodic Stochastic Mean Payoff Games with Perfect Information

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 6080))

Abstract

In this paper, we consider two-person zero-sum stochastic mean payoff games with perfect information, or BWR-games, given by a digraph G = (V = V B  ∪ V W  ∪ V R , E), with local rewards \(r: E \to {\mathbb R}\), and three types of vertices: black V B , white V W , and random V R . The game is played by two players, White and Black: When the play is at a white (black) vertex v, White (Black) selects an outgoing arc (v,u). When the play is at a random vertex v, a vertex u is picked with the given probability p(v,u). In all cases, Black pays White the value r(v,u). The play continues forever, and White aims to maximize (Black aims to minimize) the limiting mean (that is, average) payoff. It was recently shown in [7] that BWR-games are polynomially equivalent with the classical Gillette games, which include many well-known subclasses, such as cyclic games, simple stochastic games (SSG′s), stochastic parity games, and Markov decision processes. In this paper, we give a new algorithm for solving BWR-games in the ergodic case, that is when the optimal values do not depend on the initial position. Our algorithm solves a BWR-game by reducing it, using a potential transformation, to a canonical form in which the optimal strategies of both players and the value for every initial position are obvious, since a locally optimal move in it is optimal in the whole game. We show that this algorithm is pseudo-polynomial when the number of random nodes is constant. We also provide an almost matching lower bound on its running time, and show that this bound holds for a wider class of algorithms. Let us add that the general (non-ergodic) case is at least as hard as SSG′s, for which no pseudo-polynomial algorithm is known.

This research was partially supported by DIMACS, Center for Discrete Mathematics and Theoretical Computer Science, Rutgers University, and by the Scientific Grant-in-Aid from Ministry of Education, Science, Sports and Culture of Japan.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Andersson, D., Miltersen, P.B.: The complexity of solving stochastic games on graphs. In: Dong, Y., Du, D.-Z., Ibarra, O. (eds.) ISAAC 2009. LNCS, vol. 5878, pp. 112–121. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  2. Beffara, E., Vorobyov, S.: Adapting Gurvich-Karzanov-Khachiyan’s algorithm for parity games: Implementation and experimentation. Technical Report 2001-020, Department of Information Technology, Uppsala University (2001), https://www.it.uu.se/research/reports/#2001

  3. Beffara, E., Vorobyov, S.: Is randomized Gurvich-Karzanov-Khachiyan’s algorithm for parity games polynomial? Technical Report 2001-025, Department of Information Technology, Uppsala University (2001), https://www.it.uu.se/research/reports/#2001

  4. Björklund, H., Sandberg, S., Vorobyov, S.: A combinatorial strongly sub-exponential strategy improvement algorithm for mean payoff games. DIMACS Technical Report 2004-05, DIMACS, Rutgers University (2004)

    Google Scholar 

  5. Björklund, H., Vorobyov, S.: Combinatorial structure and randomized subexponential algorithms for infinite games. Theoretical Computer Science 349(3), 347–360 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  6. Björklund, H., Vorobyov, S.: A combinatorial strongly sub-exponential strategy improvement algorithm for mean payoff games. Discrete Applied Mathematics 155(2), 210–229 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  7. Boros, E., Elbassioni, K., Gurvich, V., Makino, K.: Every stochastic game with perfect information admits a canonical form. RRR-09-2009, RUTCOR. Rutgers University (2009)

    Google Scholar 

  8. Boros, E., Elbassioni, K., Gurvich, V., Makino, K.: A pumping algorithm for ergodic stochastic mean payoff games with perfect information. RRR-19-2009, RUTCOR. Rutgers University (2009)

    Google Scholar 

  9. Boros, E., Gurvich, V.: Why chess and back gammon can be solved in pure positional uniformly optimal strategies? RRR-21-2009, RUTCOR. Rutgers University (2009)

    Google Scholar 

  10. Chatterjee, K., Henzinger, T.A.: Reduction of stochastic parity to stochastic mean-payoff games. Inf. Process. Lett. 106(1), 1–7 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  11. Chatterjee, K., Jurdziński, M., Henzinger, T.A.: Quantitative stochastic parity games. In: SODA ’04: Proceedings of the fifteenth annual ACM-SIAM symposium on Discrete algorithms, pp. 121–130. Society for Industrial and Applied Mathematics, Philadelphia (2004)

    Google Scholar 

  12. Condon, A.: The complexity of stochastic games. Information and Computation 96, 203–224 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  13. Condon, A.: An algorithm for simple stochastic games. In: Advances in computational complexity theory. DIMACS series in discrete mathematics and theoretical computer science, vol. 13 (1993)

    Google Scholar 

  14. Dhingra, V., Gaubert, S.: How to solve large scale deterministic games with mean payoff by policy iteration. In: Valuetools ’06: Proceedings of the 1st international conference on Performance evaluation methodolgies and tools, vol. 12. ACM, New York (2006)

    Google Scholar 

  15. Eherenfeucht, A., Mycielski, J.: Positional strategies for mean payoff games. International Journal of Game Theory 8, 109–113 (1979)

    Article  MathSciNet  Google Scholar 

  16. Friedmann, O.: An exponential lower bound for the parity game strategy improvement algorithm as we know it. In: Symposium on Logic in Computer Science, pp. 145–156 (2009)

    Google Scholar 

  17. Gillette, D.: Stochastic games with zero stop probabilities. In: Dresher, M., Tucker, A.W., Wolfe, P. (eds.) Contribution to the Theory of Games III. Annals of Mathematics Studies, vol. 39, pp. 179–187. Princeton University Press, Princeton (1957)

    Google Scholar 

  18. Gimbert, H., Horn, F.: Simple stochastic games with few random vertices are easy to solve. In: Amadio, R.M. (ed.) FOSSACS 2008. LNCS, vol. 4962, pp. 5–19. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  19. Gurvich, V., Karzanov, A., Khachiyan, L.: Cyclic games and an algorithm to find minimax cycle means in directed graphs. USSR Computational Mathematics and Mathematical Physics 28, 85–91 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  20. Halman, N.: Simple stochastic games, parity games, mean payoff games and discounted payoff games are all LP-type problems. Algorithmica 49(1), 37–50 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  21. Hoffman, A.J., Karp, R.M.: On nonterminating stochastic games. Management Science, Series A 12(5), 359–370 (1966)

    MathSciNet  Google Scholar 

  22. Jurdziński, M.: Deciding the winner in parity games is in UP ∩ co-UP. Inf. Process. Lett. 68(3), 119–124 (1998)

    Article  Google Scholar 

  23. Jurdziński, M.: Games for Verification: Algorithmic Issues. PhD thesis, Faculty of Science, University of Aarhus, USA (2000)

    Google Scholar 

  24. Jurdziński, M., Paterson, M., Zwick, U.: A deterministic subexponential algorithm for solving parity games. In: SODA ’06: Proceedings of the seventeenth annual ACM-SIAM symposium on Discrete algorithm, pp. 117–123. ACM, New York (2006)

    Chapter  Google Scholar 

  25. Karp, R.M.: A characterization of the minimum cycle mean in a digraph. Discrete Math. 23, 309–311 (1978)

    MATH  MathSciNet  Google Scholar 

  26. Karzanov, A.V., Lebedev, V.N.: Cyclical games with prohibition. Mathematical Programming 60, 277–293 (1993)

    Article  MathSciNet  Google Scholar 

  27. Kratsch, D., McConnell, R.M., Mehlhorn, K., Spinrad, J.P.: Certifying algorithms for recognizing interval graphs and permutation graphs. In: SODA ’03: Proceedings of the fourteenth annual ACM-SIAM symposium on Discrete algorithms, pp. 158–167. Society for Industrial and Applied Mathematics, Philadelphia (2003)

    Google Scholar 

  28. Liggett, T.M., Lippman, S.A.: Stochastic games with perfect information and time-average payoff. SIAM Review 4, 604–607 (1969)

    Article  MathSciNet  Google Scholar 

  29. Littman, M.L.: Algorithm for sequential decision making, CS-96-09. PhD thesis, Dept. of Computer Science, Brown Univ., USA (1996)

    Google Scholar 

  30. Mine, H., Osaki, S.: Markovian decision process. American Elsevier Publishing Co., New York (1970)

    Google Scholar 

  31. Moulin, H.: Extension of two person zero sum games. Journal of Mathematical Analysis and Application 5(2), 490–507 (1976)

    Article  MathSciNet  Google Scholar 

  32. Moulin, H.: Prolongement des jeux à deux joueurs de somme nulle. Bull. Soc. Math. France, Memoire 45 (1976)

    Google Scholar 

  33. Pisaruk, N.N.: Mean cost cyclical games. Mathematics of Operations Research 24(4), 817–828 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  34. Vöge, J., Jurdzinski, M.: A discrete strategy improvement algorithm for solving parity games. In: Emerson, E.A., Sistla, A.P. (eds.) CAV 2000. LNCS, vol. 1855, pp. 202–215. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  35. Vorobyov, S.: Cyclic games and linear programming. Discrete Applied Mathematics 156(11), 2195–2231 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  36. Zwick, U., Paterson, M.: The complexity of mean payoff games on graphs. Theoretical Computer Science 158(1-2), 343–359 (1996)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. RUTCOR, Rutgers University, 640 Bartholomew Road, Piscataway, NJ, 08854-8003

    Endre Boros & Vladimir Gurvich

  2. Max-Planck-Institut für Informatik, 66123, Saarbrücken, Germany

    Khaled Elbassioni

  3. Graduate School of Information Science and Technology, University of Tokyo, Tokyo, 113-8656, Japan

    Kazuhisa Makino

Authors
  1. Endre Boros
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Khaled Elbassioni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vladimir Gurvich
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kazuhisa Makino
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute of Mathematics, École Polytechnique Féderale de Lausanne, 1015, Lausanne, Switzerland

    Friedrich Eisenbrand

  2. McGill University, 805 Sherbrooke West, H3A 2K6, Montreal, Quebec, Canada

    F. Bruce Shepherd

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Boros, E., Elbassioni, K., Gurvich, V., Makino, K. (2010). A Pumping Algorithm for Ergodic Stochastic Mean Payoff Games with Perfect Information. In: Eisenbrand, F., Shepherd, F.B. (eds) Integer Programming and Combinatorial Optimization. IPCO 2010. Lecture Notes in Computer Science, vol 6080. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13036-6_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-13036-6_26

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-13035-9

  • Online ISBN: 978-3-642-13036-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation