Abstract
We consider a two-person zero-sum stochastic game with an infinite-time horizon. The payoff is a linear combination of expected total discounted rewards with different discount factors. For a model with a countable state space and compact action sets, we characterize the set of persistently optimal (subgame perfect) policies. For a model with finite state and action sets and with perfect information, we prove the existence of an optimal pure Markov policy, which is stationary from some epoch onward, and we describe an algorithm to compute such a policy. We provide an example that shows that an optimal policy, which is stationary after some step, may not exist for weighted discounted sequential games with finite state and action sets and without the perfect information assumption. We also provide examples of similar phenomena of nonstationary behavior for the following two classes of problems with weighted discounted criteria: (i) models with one controller and with finite state and compact action sets, and (ii) nonzero-sum games with perfect information and with finite state and action sets.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Altman, E., A. Hordijk, and F. M. Spieksma. Contraction Conditions for Average and α-Discounted Optimality in Countable State, Markov Games with Unbounded Payoffs. Mathematics of Operations Research, 22, No. 3, 588–618,1997.
Billingsley, P. Convergence of Probability Measures. Wiley, New York, 1968.
Feinberg, E. A. Controlled Markov Processes with Arbitrary Numerical Criteria. Theory Probability and its Applications, 27,486–503, 1982.
Feinberg, E. A. and A. Shwartz. Markov Decision Models with Weighted Discounted Criteria. Mathematics oj Operations Research, 19,152–168, 1994.
Feinberg, E. A. and A. Shwartz. Constrained Markov Decision Models with Weighted Discounted Criteria. Mathematics of Operations Research, 20,302–320, 1994.
Feinberg, E. A. and A. Shwartz Constrained Dynamic Programming with Two Discount Factors: Applications and an Algorithm. IEEE Transactions on Automatic Control. 44, 628–630,1990.
Federgruen, A. On N-Person Stochastic Games with Denumerable State Space. Advances in Applied Probability, 10,452–471, 1978.
Fernández-Gaucherand, E., M. K. Ghosh, and S. I. Marcus. Controlled Markov Processes on the Infinite Planning Horizon: Weighted and Overtaking Cost Criteria. ZOR-Methods and Models of Operations Research, 39, 131–155, 1994.
Filar, J. A. and O. J. Vrieze. Weighted Reward Criteria in Competitive Markov Decision Processes. ZOR-Methods and Models of Operations Research, 36, 343–358, 1992.
Filar, J. and O.J. Vrieze. Competitive Markov Decision Processes. Springer-Verlag, New York, 1996.
Gillette, D. Stochastic Games with Zero Stop Probabilities. Contribution to the Theory of Games, vol. III (M. Dresner, A. W Tucker, and P. Wolfe, eds.). Princeton University Press, Princeton, NJ, pp. 179-187, 1957.
Golabi, K., R. B. Kulkarni, and G. B. Way. A Statewide Pavement Management System. Interfaces, 12, 5–21, 1982.
Himmelberg, C. J., T. Parthasarathy, and T. E. S. Raghavan. Existence of p-Equilibrium and Optimal Stationary Strategies in Stochastic Games. Proceedings of the American Mathematical Society, 60,245–251, 1976.
Karlin, S. Mathematical Methods and Theory in Games, Programming, and Economics. Volume II: The Theory of Infinite Games. Addison-Wesley, New York, 1959
Krass, D., J. A. Filar, and S. S. Sinha. A Weighted Markov Decision Process. Operations Research, 40, 1180–1187, 1992.
Küenle, H.-U. Stochastiche Spiele und Entscheidungesmodelle. Tebuner-Texte, Leipzig, Band 89, 1986.
Kumar, P. R. and T. H. Shiau. Existence of Value and Randomized Strategies in Zero-Sum Discrete-Time Stochastic Dynamic Games. SIAM Journal on Control and Optimization, 19, 617–634, 1981.
Maitra, A., and W. D. Sudderth. Discrete Gambling and Stochastic Games. Springer-Verlag, New York, 1996.
Nowak, A. S. On Zero-Sum Stochastic Games with General State Space I. Probability and Mathematical Statistics, 4, 13–32, 1984.
Nowak, A. S. Universally Measurable Strategies in Stochastic Games. Annals of Probability, 13,269–287, 1985.
Nowak, A. S. Semicontinuous Nonstationary Stochastic Games. Journal of Mathematical Analysis and Applications, 117, 84–99, 1986.
Parthasarathy, K. R. Probability Measures on Metric Spaces. Academic Press, New York, 1967.
Parthasarathy, T. and E. S. Raghavan. Some Topics in Two-Person Games. Elsevier, New York, 1971.
Reiman, M. I. and A. Shwartz Call Admission: A New Approach to Quality of Service. CC Pub. 216, Technion, and Bell Labs Manuscript, 1997.
Shapley L. S. Stochastic Games. Proceedings of the National Academy of Sciences U.S.A., 39, 1095–1100, 1953.
Shapley L. S. and R. N. Snow. Basic Solutions of Discrete Games. Contribution to the Theory of Games, I (H. W. Kuhn and A. W. Tucker, eds.). Princeton University Press, Princeton, NJ, pp. 27–35, 1957.
Sion, M. On General Minimax Theorems. Pacific Journal of Mathematics, 8,171–176, 1958.
Winden, C. V. and R. Dekker. Markov Decision Models for Building Maintenance: A Feasibility Study. Report 9473/A, ERASMUS University Rotterdam, The Netherlands, 1994.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer Science+Business Media New York
About this paper
Cite this paper
Altman, E., Feinberg, E.A., Shwartz, A. (2000). Weighted Discounted Stochastic Games with Perfect Information. In: Filar, J.A., Gaitsgory, V., Mizukami, K. (eds) Advances in Dynamic Games and Applications. Annals of the International Society of Dynamic Games, vol 5. Birkhäuser, Boston, MA. https://doi.org/10.1007/978-1-4612-1336-9_17
Download citation
DOI: https://doi.org/10.1007/978-1-4612-1336-9_17
Publisher Name: Birkhäuser, Boston, MA
Print ISBN: 978-1-4612-7100-0
Online ISBN: 978-1-4612-1336-9
eBook Packages: Springer Book Archive