Abstract
We give polynomial-time algorithms for computing the values of Markov decision processes (MDPs) with limsup and liminf objectives. A real-valued reward is assigned to each state, and the value of an infinite path in the MDP is the limsup (resp. liminf) of all rewards along the path. The value of an MDP is the maximal expected value of an infinite path that can be achieved by resolving the decisions of the MDP. Using our result on MDPs, we show that turn-based stochastic games with limsup and liminf objectives can be solved in NP ∩ coNP.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Chatterjee, K., Henzinger, T.A.: Value iteration. In: Grumberg, O., Veith, H. (eds.) 25 Years of Model Checking. LNCS, vol. 5000, pp. 107–138. Springer, Heidelberg (2008)
Chatterjee, K., Jurdziński, M., Henzinger, T.A.: Simple stochastic parity games. In: Baaz, M., Makowsky, J.A. (eds.) CSL 2003. LNCS, vol. 2803, pp. 100–113. Springer, Heidelberg (2003)
Chatterjee, K., Jurdziński, M., Henzinger, T.A.: Quantitative stochastic parity games. In: Ian Munro, J. (ed.) SODA 2004, pp. 121–130. SIAM, Philadelphia (2004)
Courcoubetis, C., Yannakakis, M.: Markov decision processes and regular events. In: Paterson, M. (ed.) ICALP 1990. LNCS, vol. 443, pp. 336–349. Springer, Heidelberg (1990)
de Alfaro, L.: Formal Verification of Probabilistic Systems. PhD thesis, Stanford University (1997)
Filar, J., Vrieze, K.: Competitive Markov Decision Processes. Springer, Heidelberg (1997)
Gimbert, H.: Jeux positionnels. PhD thesis, Université Paris 7 (2006)
Gimbert, H., Zielonka, W.: Games where you can play optimally without any memory. In: Abadi, M., de Alfaro, L. (eds.) CONCUR 2005. LNCS, vol. 3653, pp. 428–442. Springer, Heidelberg (2005)
Gimbert, H., Zielonka, W.: Perfect information stochastic priority games. In: Arge, L., Cachin, C., Jurdziński, T., Tarlecki, A. (eds.) ICALP 2007. LNCS, vol. 4596, pp. 850–861. Springer, Heidelberg (2007)
Liggett, T.A., Lippman, S.A.: Stochastic games with perfect information and time average payoff. SIAM Review 11, 604–607 (1969)
Maitra, A., Sudderth, W. (eds.): Discrete Gambling and Stochastic Games. Stochastic Modelling and Applied Probability, vol. 32. Springer, Heidelberg (1996)
Martin, D.A.: The determinacy of Blackwell games. J. Symb. Log. 63(4), 1565–1581 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chatterjee, K., Henzinger, T.A. (2009). Probabilistic Systems with LimSup and LimInf Objectives. In: Archibald, M., Brattka, V., Goranko, V., Löwe, B. (eds) Infinity in Logic and Computation. ILC 2007. Lecture Notes in Computer Science(), vol 5489. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03092-5_4
Download citation
DOI: https://doi.org/10.1007/978-3-642-03092-5_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03091-8
Online ISBN: 978-3-642-03092-5
eBook Packages: Computer ScienceComputer Science (R0)