Abstract
We study a dynamic optimal routing problem, where a controller has to decide to which of two queues should arriving customers (representing packets, messages, calls, etc.) be sent. The service rate in each queue may depend on the state of the system, may change in time, and is unknown to the controller. The goal of the controller is to design a strategy that guarantees the best performance under the worst case service conditions. The payoff is composed of a holding cost, an admission cost, and a cost that depends on the quality of the service. We consider both the finite and infinite horizon discounted cost. The problem is studied in the framework of zero-sum Markov games where the server, called player 1, is assumed to play against the router, called player 2. Each player is assumed to have the information of all previous actions of both players as well as the current and past states of the system. We show that there exist pure optimal strategies for both players. A value iteration algorithm is used to establish properties of the value of the game, which are related to supermodularity and to convexity. This is then shown to imply the existence of optimal strategies described by monotone switching curves for both players.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
E. Altman, “Flow control using the theory of zero-sum Markov games,”IEEE Trans. Automatic Control 39 (4), pp. 814–818, 1994.
E. Altman, “ Monotonicity of optimal policies in a zero sum game: a flow control model,”Adv. Dyn. Games Appl.,1269–286,1994.
E. Altman, A. Hordijk, and F. M. Spieksma, “Contraction conditions for average and cr-discounted optimality in countable state Markov games with unbounded rewards,”MOR 22 (3), 1997.
E. Altman and G. Koole, “Stochastic scheduling games with Markov decision arrival processes,”J. Computers Math. Appl.3rd special issue on Differential Games, pp. 141–148, 1993.
E. Altman and N. Shimkin, “Individually optimal dynamic routing in a processor sharing system: Stochastic game analysis,” EE Pub No. 849, August 1992. To appear inOperations Res.1998.
E. Altman and N. Shimkin, “Worst-case and Nash routing policies in parallel queues with uncertain service allocations,” IMA Preprint Series No. 1120, Institute for Mathematics and Applications, University of Minnesota, Minneapolis, 1993.
B. Hajek. “Optimal control of two interacting service stations,”IEEE Trans. Automatic Control 29 (6), pp. 491–499, 1984.
R. Hassin and M. Haviv. Equilibrium strategies and the value of information in a two line queuing system with threshold jockeyingCommun. Statist. - Stochastic Models 10 (2), pp. 415–435, 1994.
M. Haviv. Stable strategies for processor sharing systemsEuropean J. Operations Res. 52, pp. 103–106, 1991.
M. T. Hsiao and A. A. Lazar, “A game theoretic approach to decentralized flow control of Markovian queuing Networks,”Performance‘87, Courtois and Latouche, Eds., pp. 55–73, 1988.
Y. A. Korilis and A. Lazar, “On the existence of equilibria in noncooperative optimal flow control,”J. Assoc. Comput. Mach. 42, May 1995.
A. S. Nowak, “On zero-sum stochastic games with general state space I.”Prob. Math. Stat. IVFasc. 1, pp. 13–32, 1984.
T.E.S. Raghavan and J.A. Filar, “Algorithms for stochastic games — A survey,”Zeitschrift für OR 35pp. 437–472, 1991.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer Science+Business Media New York
About this chapter
Cite this chapter
Altman, E. (1999). A Markov Game Approach for Optimal Routing Into a Queuing Network. In: Bardi, M., Raghavan, T.E.S., Parthasarathy, T. (eds) Stochastic and Differential Games. Annals of the International Society of Dynamic Games, vol 4. Birkhäuser, Boston, MA. https://doi.org/10.1007/978-1-4612-1592-9_9
Download citation
DOI: https://doi.org/10.1007/978-1-4612-1592-9_9
Publisher Name: Birkhäuser, Boston, MA
Print ISBN: 978-1-4612-7208-3
Online ISBN: 978-1-4612-1592-9
eBook Packages: Springer Book Archive