Abstract
A user facing a multi-user resource-sharing system considers a vector of performance measures (e.g. response times to various tasks). Acceptable performance is defined through a set in the space of performance vectors. Can the user obtain a (time-average) performance vector which approaches this desired set? We consider the worst-case scenario, where other users may, for selfish reasons, try to exclude his vector from the desired set. For a controlled Markov model of the system, we give a sufficient condition for approachability, and construct appropriate policies. Under certain recurrence conditions, a complete characterization of approachability is then provided for convex sets. The mathematical formulation leads to a theory of approachability for stochastic games. A simple queueing example is analyzed to illustrate the applicability of this approach.
Research performed in part while this author was visiting the Systems Research Center, University of Maryland, College Park, where he was supported in part through NSF Grant NSF ECS-83-51836.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
E. Altman and A. Shwartz, “Non stationary policies for controlled Markov chains,” EE Pub. 633, Technion, June 1987.
E. Altman and A. Shwartz, “Markov decision problems and state-action frequencies,” SIAM J. Control and Optimization 29 No. 4, July 1991.
R. J. Aumann and M. Maschler, Game theoretic aspects of gradual disarmament. Chapter V, Report to the U.S. Arms Control and Disarmament Agency, Contract S.T.80, prepared by Mathematica, Inc., Princeton, N.J., 1996.
D. Blackwell, “An analogue for the minimax theorem for vector payoffs,” Pacific J. Math., 6, pp. 1–8, 1956.
D. Blackwell, “Controlled random walks,” Proc. Internat. Congress Math., 3, pp. 336–338, 1954.
D. Blackwell, “On multi-component attrition games,” Naval Res. Log. Quart., pp. 210–216, 1954.
V. S. Borkar, “Control of Markov chains with long-run average cost criterion,” in Stochastic Differential Systems, Stochastic Control and Application, W. Fleming and P.L. Lions, eds., IMA Vol. 10, Springer-Verlag, pp. 57–77, 1988.
C. Derman, Finite State Markovian Decision Processes, Academic Press, New-York, 1970.
A. Federgruen, “On N-person games with denumerable state space,” Adv. Appl. Prob. 10, pp. 452–471, 1978.
D. Gillette, “Stochastic games with zero stop probabilities,” in Contributions to the Theory of Games, III, (Annals of Math. Studies 39), M. Dresher et al., editors, Princeton Univ. Press, pp. 179–188, 1957.
S. Hart, “Nonzero-sum two-person repeated games with incomplete information,” Math. of Oper. Res., 10, pp. 117–153, 1985.
J. F. Hannan, “Approximation to Bayes risk in repeated play,’ in Contributions to the Theory of Games, III, (Annals of Math. Studies 39), M. Dresher et al., editors, Princeton Univ. Press, pp. 97–139, 1957.
T. F. Hou, “Weak approachability in a two-person game,” Ann. Math. Statist., 40, pp. 789–813, 1969.
T. F. Hou, “Approachability in a two-person game,” Ann. Math. Stat., 42, pp. 735–744, 1971.
M. Katz, “Infinitely repeatable games,” Pac. J. Math., 10, pp. 879–885, 1960.
R. D. Luce and H. Raiffa, Games and Decisions, Wiley, New-York, 1957.
A. Maitra and T. Parthasarathy, “On stochastic games, II,” Journal of Optimization Theory and Applications, Vol. 8, pp. 155–160, 1971.
T. Parthasarathy and M. Stern, “Markov games: a survey,” in Differential Games and Control Theory, P.L.E. Roxin and R. Sternberg, eds., Marcel Dekker, 1977.
T. E. S. Raghavan and J.A. Filar, “Algorithms for stochastic games — a survey,” Preprint, June 1989.
S. M. Ross, Applied Probability Models with Optimization Applications, Holden-Day, San Francisco, 1970.
H. Sackrowitz, “A note on approachability in a two-person game,” Ann. Math. Stat., 43, pp. 1017–1019, 1972.
L. I. Sennott, “Average cost optimal stationary policies in infinite state Markov decision processes with unbounded costs,” Operations Research, Vol. 37, pp. 626–633, 1989.
A. N. Shiryayev, Probability, Springer-Verlag, 1984.
N. Shimkin and A. Shwartz, “Guaranteed performance regions for multi-use Markov models,” submitted to the IEEE Trans. Automat. Contr., 1991.
J. Sorin, An Introduction to Two-Person-Zero-Sum Repeated Games with Incomplete Information. IMSSS-Economics TR-312, Stanford University, memo.
M. A. Stern, On Stochastic Games with Limiting Average Pay-Off. Ph.D. dissertation, submitted to the University of Illinois, Circle Campus, Chicago, 1975.
N. Vieille, “Weak approachability,” Preprint, 1991.
J. Walrand, An Introduction to Queueing Networks, Prentice-Hall, New Jersey, 1988.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1992 Springer-Verlag
About this paper
Cite this paper
Shimkin, N., Shwartz, A. (1992). Guaranteed performance regions for multi-user Markov models. In: Duncan, T.E., Pasik-Duncan, B. (eds) Stochastic Theory and Adaptive Control. Lecture Notes in Control and Information Sciences, vol 184. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0113259
Download citation
DOI: https://doi.org/10.1007/BFb0113259
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-55962-7
Online ISBN: 978-3-540-47327-5
eBook Packages: Springer Book Archive