Abstract
The study of simple stochastic games (SSGs) was initiated by Condon for analyzing the computational power of randomized space-bounded alternating Turing machines. The game is played by two players, MAX and MIN, on a directed multigraph, and when the play terminates at a sink s, MAX wins from MIN a payoff p(s) ∈ [0,1]. Condon showed that the SSG value problem, which given a SSG asks whether the expected payoff won by MAX exceeds 1/2 when both players use their optimal strategies, is in NP ∩ coNP. However, the exact complexity of this problem remains open as it is not known whether the problem is in P or is hard for some natural complexity class. In this paper, we study the computational complexity of a strategy improvement algorithm by Hoffman and Karp for this problem. The Hoffman-Karp algorithm converges to optimal strategies of a given SSG, but no nontrivial bounds were previously known on its running time. We show a bound of O(2n/n) on the convergence time of this algorithm, and a bound of O(20.78 n) on a randomized variant. These are the first non-trivial upper bounds on the convergence time of these strategy improvement algorithms.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Condon, A.: The complexity of stochastic games. Information and Computation 96, 203–224 (1992)
Condon, A.: On algorithms for simple stochastic games. In: Cai, J. (ed.) Advances in Computational Complexity Theory. DIMACS series in DM&TCS, vol. 13, pp. 51–73. American Mathematical Society, Providence (1993)
Derman, C.: Finite State Markovian Decision Processes. In: Mathematics in Science and Engineering, vol. 67. Academic Press, New York (1970)
Friedmann, O.: An exponential lower bound for the parity game strategy improvement algorithm as we know it. In: Proceedings of the 24th IEEE Symposium on LICS. IEEE Computer Society, Los Alamitos (2009) (to appear)
Gimbert, H., Horn, F.: Solving simple stochastic games with few random vertices. In: Foundations of Software Science and Comput. Structures (2009)
Halman, N.: Simple stochastic games, parity games, mean payoff games and DPGs are all LP-type problems. Algorithmica 49(1), 37–50 (2007)
Hoffman, A., Karp, R.: On nonterminating stochastic games. Management Science 12, 359–370 (1966)
Howard, R.: Dynamic Programming and Markov Processes. MIT Press, Cambridge (1960)
Jukna, S.: Extremal Combinatorics. Springer, Heidelberg (2001)
Ludwig, W.: A subexponential randomized algorithm for the simple stochastic game problem. Information and Computation 117(1), 151–155 (1995)
Melekopoglou, M., Condon, A.: On the complexity of the policy improvement algorithm for Markov decision processes. ORSA Journal of Computing 6(2), 188–192 (1994)
Mansour, Y., Singh, S.: On the complexity of Policy Iteration. In: Proceedings of the 15th Conference on UAI, July 1999, pp. 401–408 (1999)
Shapley, L.: Stochastic games. In: Proceedings of National Academy of Sciences (USA), vol. 39, pp. 1095–1100 (1953)
Somla, R.: New algorithms for solving simple stochastic games. Electronic Notes in Theoretical Computer Science 119(1), 51–65 (2005)
Vöge, J., Jurdziński, M.: A discrete strategy improvement algorithm for solving parity games. In: Emerson, E.A., Sistla, A.P. (eds.) CAV 2000. LNCS, vol. 1855, pp. 202–215. Springer, Heidelberg (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tripathi, R., Valkanova, E., Kumar, V.S.A. (2010). On Strategy Improvement Algorithms for Simple Stochastic Games. In: Calamoneri, T., Diaz, J. (eds) Algorithms and Complexity. CIAC 2010. Lecture Notes in Computer Science, vol 6078. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13073-1_22
Download citation
DOI: https://doi.org/10.1007/978-3-642-13073-1_22
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13072-4
Online ISBN: 978-3-642-13073-1
eBook Packages: Computer ScienceComputer Science (R0)