An Optimal Adaptive Scheme for Two Competing Queues with Constraints
Two types of traffic, e. g., voice and data, share a single synchronous and noisy communication channel. This situation is modelled as a system of two discrete-time queues with geometric service requirements which compete for the attention of a single server. The problem is cast as one in Markov decision theory with long-run average cost and constraint. An optimal strategy is identified that possesses a simple structure, and its implementation is discussed in terms of an adaptive algorithm of the stochastic-approximations type. The proposed algorithm is extremely simple, recursive and easily implementable, with no a priori knowledge of the actual values of the statistical parameters. The derivation of the results combines martingale arguments, results on Markov chains, O.D.E. characterization of the limit of stochastic approximations and methods from weak convergence. The ideas developed here are of independent interest and should prove useful in studying broad classes of constrained Markov decision problems.
KeywordsStochastic Approximation Queue Size Optimal Bias Uniform Integrability Stochastic Approximation Algorithm
Unable to display preview. Download preview PDF.
- J. S. Baras, A. J. Dorsey, and A. M. Makowski, “Discrete time competing queues with geometric service requirements: stability, parameter estimation and adaptive control,” SIAM J. Control Opt., Under revision. Invited paper to the ORSA/TIMS National Meeting, San-Francisco, Cali-fornia, (May 1984).Google Scholar
- D. P. Heyman and M. J. Sobel, Stochastic Models in Operations Research, Volume II: Stochastic Optimization, MacGraw-Hill, New York (1984).Google Scholar
- P. Nain and K. W. Ross, Optimal Priority Assignment with Constraints, Rapport de Recherche No. 346, INRIA - Rocquencourt, France (November 1984).Google Scholar
- K. W. Ross, Constrained Markov Decision Processes with Queueing Applications, Ph. D. thesis, Computer, Information and Control Engineering, University of Michigan (1985).Google Scholar
- S. M. Ross, Introduction to Stochastic Dynamic Programming, Academic Press (1984).Google Scholar
- A. Shwartz and A. M. Makowsii, Adaptive schemes for server allocation in systems of compet-ing queues with constraints, Systems Research Report, In preparation, Systems Research Center, University of Maryland at Colleg Park. (1986)Google Scholar