Abstract
Queueing models are frequently helpful in the analysis and control of communication, manufacturing, and transportation systems. The theory of Markov decision processes and the inductive techniques of dynamic programming have been used to develop normative models for optimal control of admission, servicing, routing, and scheduling of jobs in queues and networks of queues. We review some of these models, beginning with single-facility models and then progressing to models for networks of queues. We emphasize the use of induction on a sequence of successive approximations of the optimal value function (value iteration) to establish the form of optimal control policies.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
I. Adler and P. Naor, “Social optimization versus self-optimization in waiting lines, ” Tech. Rpt. No. 4, De- partment of Operations Research, Stanford University (1974).
K. Balachandran and M. Schaefer, “Class dominance characteristics at a service facility,” Econometrica 47, 515–519 (1979).
R. Bellman, Dynamic Programming. Princeton: Princeton University Press (1957).
B. Bengtsson, “On some control problems for queues,” Linkøping Studies in Science and Technology, Dissertation No. 87, Department of Electrical Engineering, Linkøping University, Sweden (1982).
D. Bertsekas, Dynamic Programming and Stochastic Control. New York: Academic Press (1976).
T. Crabill, “Optimal control of a service facility with variable exponential service time and constant arrival rate,” Management Sci. 18, 560–566 (1972).
T. Crabill, “Optimal control of a maintenance system with variable service rates,” Operations Res. 22, 736–745 (1974).
T. Crabill, D. Gross, and M. Magazine, “A classified bibliography of research on optimal design and control of queues,” Operations Res. 25, 219–232 (1977).
E. Davis, “Optimal control of arrivals to a two-server queueing system with separate queues,” Ph.D. dissertation, Program in Operations Research, N.C. State University, Raleigh (1977).
B. Doshi, “Continuous-time control of the arrival process in an M/G/1 queue,” Stoch. Proc. Appl. 5, 265–284 (1977).
B. Doshi, “Optimal control of the service rate in an M/G/1 queueing system,” Adv. Appi. Prob. 10, 682–701 (1978).
A. Ephremides, P. Varaiya, and J. Walrand, “A simple dynamic routing problem,” IEEE Trans. Auto. Control AC-25, 690–693 (1980).
W. Farrell, “Optimal switching policies in a nonhomogeneous exponential queueing system,” Pb.D. dissertation, Graduate School of Management, University of California, Los Angeles (1976).
W. Fleming and R. Rishel, Deterministic and Stochastic Optimal Control. Berlin: Springer-Verlag (1975).
R. Gallisch, “On monotone optimal policies in a queueing system of M/G/i type with controllable service-time distribution,” Adv. Appl. Prob. 11, 870–887 (1979).
H. Ghoneim, “Optimal control of arrivals to a network of two queues in series,” Ph.D. dissertation, Program in Operations Research, N.C. State University, Raleigh (1980).
H. Ghoneim and S. Stidham, “Optimal control of arrivals to two queues in series, ” Euro. J. Operations Res. 21, 399–409 (1983).
B. Hajek, “Optimal control of two interacting service stations. IEEE Trans. Auto. Control AC-29, 491–499 (1982).
B. Hajek, “Extremal splittings of point processes,” Math. of Operations Res. 10, 543–556 (1983).
J.M. Harrison, “Brownian models of queueing networks with heterogeneous customer populations,” Proc. IMA Workshop on Stochastic Differential Systems. Berlin: Springer-Verlag (1987) (this volume).
W. Helm and K.-H. Waldmann, “Optimal control of arrivals to multiserver queues in a random environment,” J. Appl. Prob. 21, 602–615 (1984).
K. Jo, “Optimal service-rate control of exponential systems,” J. Operat. Res. Soc. Japan 26, 147–165 (1983).
K. Jo and S. Stidham, “Optimal service-rate control of M/G/1 queueing systems using phase methods,” Adv. Appl. Prob. 15, 616–637 (1983).
S.G. Johansen and S. Stidham, “Control of arrivals to a stochastic input-output system,” Adv. Appl. Prob. 12, 972–999 (1980).
S.G. Johansen and S. Stidham, “The optimal pricing of admission to a GI/M/c queue,” Department of Operations Research, Aarhus University, Denmark (1983).
P. Kakamanu, “Continuous-time Markov decision models with applications to optimization problems,” Tech. Rpt. No. 63, Department of Operations Research, Cornell University (1969).
N.C. Knudsen, “Individual and social optimization in a multi-server queue with a general cost-benefit structure,” Econometrica 40, 515–528 (1972).
N.C. Knudsen and S. Stidham, “Individual and social optimization in a birth-death congestion system with a general cost-benefit structure, ” Tech. Rpt. No. 76–8, Department of Industrial Engineering, N, C. State University, Raleigh (1976).
H.-J. Langen, “Applying the method of phases in the optimization of queueing systems,” Adv. Appl. Prob. 14, 122–142 (1982).
A. Lazar, “Optimal control of a class of queueing networks in equilibrium,” IEEE Trans. Auto. Control AC-28, 1001–1007 (1983).
S. Lippman, “Applying a new device in the optimization of exponential queueing systems,” Operations Res. 23, 687–710 (1975).
S. Lippman and S. Stidham, “Individual versus social optimization in exponential congestion systems,” Operations Res. 25, 233–247 (1977).
F. Lu and R. Serfozo, “M/M/1 queueing decision processes with monotone hysteretic optimal policies,” Operations Res. 32, 1116–1132 (1984).
M. Moustafa, “Optimal Scheduling in Networks of Queues,” Ph.D. Dissertation (in preparation), Program in Operations Research, N.C. State University, Raleigh (1987).
W. Mitchell, “Optimal service-rate selection in an M/G/i queue,” Siam J. Appl. Math. 24, 19–35 (1973).
P. Naor, “On the regulation of queue size by levying tolls,” Econometrica 37, 15–24 (1969).
Z. Rosberg, P. Varaiya, J. Walrand, “Optimal control of service in tandem queues,” IEEE Trans. Auto. Control AC-27, 600–610 (1982).
S. Ross, Applied Probability Models with Optimization Applications. San Francisco: Holden-Day (1970).
R. Rue and M. Rosenshine, “Optimal control of entry of many classes of customers to an M/M/1 queue,” Naval Res. Logist. Quart. 28, 489–495 (1981).
H. Sabeti, “Optimal selection of service rates in queueing with di fferent costs, ” J. Operat. Res. Soc. Japan 16, 15–35 (1973).
M. Schäl, “Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal,” Z. Wahrscheinlichkeitstheorie verw. Geb. 32, 179–196 (1975).
R. Schassberger, “A note on optimal service selection in a single-server queue,” Management Sci. 21, 1326–1331 (1975).
R. Serfozo, “An equivalence between discrete-and continuous-time Markov decision processes,” Operations Res. 27, 616–620 (1979).
R. Serfozo, “Optimal control of random walks, birth and death processes, and queues,” Adv. Appl. Prob. 13, 61–83 (1981)
A. Simonovits, “Self-and social optimization in queues,” Studia Scient. Math. Hungarica 11, 131–138 (1976).
M. Sobel, “Optimal operation of queues,” in Mathematical Methods in Queueing Theory, A. B. Clarke (ed.), Berlin: Springer-Verlag, 231–261 (1974).
M. Sobel, “The optimality of full-service policies,” Operations Res. 30, 636–649 (1982).
S. Stidham, “Socially and individually optimal control of arrivals to a GI/M/1 queue,” Management Sci. 24, 1598–1610 (1978).
S. Stidham, “On the convergence of successive approximations in dynamic programming with non-zero terminal reward,” Z. Operations Res. 25, 57–77 (1981).
S. Stidham, “Optimal control of admission, routing, and service in queues and networks of queues: a tutorial review,” Proc. ARO Workshop: Analytical and Computational Issues in Logistics R and D, George Washington University, 330–377 (1984).
S. Stidham, “Optimal control of admission to a queueing system,” IEEE Trans. Auto. Control AC-30 705–713 (1985).
S. Stidham and N.U. Prabhu, “Optimal control of queueing systems,” in Mathematical Methods in Queueing Theory, A.B. Clarke (ed.), Berlin: Springer-Verlag, 263–294 (1974).
S. Stidham and R. Weber, “Monotonic and insensitive optimal policies for control of arrival and service rates in queues,” submitted for publication (1986).
D. Topkis, “Minimizing a submodular function on a lattice,” Operations Res. 26, 305–321 (1978).
J. van der Wal, Stochastic Dynamic Programming, Mathematical Centre Tract 139, Amsterdam (1981).
R. Weber, “On the optimal assignment of customers to parallel servers,” J. Appl. Prob. 15, 406–413 (1978).
R. Weber and S. Stidham, “Control of service rates in net-works of queues,” J. Appl, Prob. 24, 000–000 (1987).
P. Whittle, Optimization Over Time, New York: Wiley (1983).
W. Winston, “Optimality of the shortest-line discipline,” J. Appl. Prob. 14, 181–189 (1977).
U. Yechiali, “On optimal balking rules and toll charges in a GI/M/1 queueing process,” Operations Res. 19, 349–370 (1971).
U. Yechiali, “Customers’ optimal joiningg rules for the GI/M/s queue,” Management Sci. 18, 434–443 (1971).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1988 Springer-Verlag New York Inc.
About this paper
Cite this paper
Stidham, S. (1988). Scheduling, Routing, and Flow Control in Stochastic Networks. In: Fleming, W., Lions, PL. (eds) Stochastic Differential Systems, Stochastic Control Theory and Applications. The IMA Volumes in Mathematics and Its Applications, vol 10. Springer, New York, NY. https://doi.org/10.1007/978-1-4613-8762-6_31
Download citation
DOI: https://doi.org/10.1007/978-1-4613-8762-6_31
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4613-8764-0
Online ISBN: 978-1-4613-8762-6
eBook Packages: Springer Book Archive