Skip to main content
SpringerLink
Log in

Scheduling, Routing, and Flow Control in Stochastic Networks

  • Conference paper
Book cover Stochastic Differential Systems, Stochastic Control Theory and Applications

Part of the book series: The IMA Volumes in Mathematics and Its Applications ((IMA,volume 10))

Abstract

Queueing models are frequently helpful in the analysis and control of communication, manufacturing, and transportation systems. The theory of Markov decision processes and the inductive techniques of dynamic programming have been used to develop normative models for optimal control of admission, servicing, routing, and scheduling of jobs in queues and networks of queues. We review some of these models, beginning with single-facility models and then progressing to models for networks of queues. We emphasize the use of induction on a sequence of successive approximations of the optimal value function (value iteration) to establish the form of optimal control policies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. I. Adler and P. Naor, “Social optimization versus self-optimization in waiting lines, ” Tech. Rpt. No. 4, De- partment of Operations Research, Stanford University (1974).

    Google Scholar 

  2. K. Balachandran and M. Schaefer, “Class dominance characteristics at a service facility,” Econometrica 47, 515–519 (1979).

    Article  MathSciNet  MATH  Google Scholar 

  3. R. Bellman, Dynamic Programming. Princeton: Princeton University Press (1957).

    MATH  Google Scholar 

  4. B. Bengtsson, “On some control problems for queues,” Linkøping Studies in Science and Technology, Dissertation No. 87, Department of Electrical Engineering, Linkøping University, Sweden (1982).

    Google Scholar 

  5. D. Bertsekas, Dynamic Programming and Stochastic Control. New York: Academic Press (1976).

    MATH  Google Scholar 

  6. T. Crabill, “Optimal control of a service facility with variable exponential service time and constant arrival rate,” Management Sci. 18, 560–566 (1972).

    Article  MathSciNet  MATH  Google Scholar 

  7. T. Crabill, “Optimal control of a maintenance system with variable service rates,” Operations Res. 22, 736–745 (1974).

    Article  MathSciNet  MATH  Google Scholar 

  8. T. Crabill, D. Gross, and M. Magazine, “A classified bibliography of research on optimal design and control of queues,” Operations Res. 25, 219–232 (1977).

    Article  MathSciNet  Google Scholar 

  9. E. Davis, “Optimal control of arrivals to a two-server queueing system with separate queues,” Ph.D. dissertation, Program in Operations Research, N.C. State University, Raleigh (1977).

    Google Scholar 

  10. B. Doshi, “Continuous-time control of the arrival process in an M/G/1 queue,” Stoch. Proc. Appl. 5, 265–284 (1977).

    Article  MathSciNet  MATH  Google Scholar 

  11. B. Doshi, “Optimal control of the service rate in an M/G/1 queueing system,” Adv. Appi. Prob. 10, 682–701 (1978).

    Article  MathSciNet  MATH  Google Scholar 

  12. A. Ephremides, P. Varaiya, and J. Walrand, “A simple dynamic routing problem,” IEEE Trans. Auto. Control AC-25, 690–693 (1980).

    Article  MathSciNet  Google Scholar 

  13. W. Farrell, “Optimal switching policies in a nonhomogeneous exponential queueing system,” Pb.D. dissertation, Graduate School of Management, University of California, Los Angeles (1976).

    Google Scholar 

  14. W. Fleming and R. Rishel, Deterministic and Stochastic Optimal Control. Berlin: Springer-Verlag (1975).

    MATH  Google Scholar 

  15. R. Gallisch, “On monotone optimal policies in a queueing system of M/G/i type with controllable service-time distribution,” Adv. Appl. Prob. 11, 870–887 (1979).

    Article  MathSciNet  MATH  Google Scholar 

  16. H. Ghoneim, “Optimal control of arrivals to a network of two queues in series,” Ph.D. dissertation, Program in Operations Research, N.C. State University, Raleigh (1980).

    Google Scholar 

  17. H. Ghoneim and S. Stidham, “Optimal control of arrivals to two queues in series, ” Euro. J. Operations Res. 21, 399–409 (1983).

    Article  MathSciNet  Google Scholar 

  18. B. Hajek, “Optimal control of two interacting service stations. IEEE Trans. Auto. Control AC-29, 491–499 (1982).

    MathSciNet  Google Scholar 

  19. B. Hajek, “Extremal splittings of point processes,” Math. of Operations Res. 10, 543–556 (1983).

    Article  MathSciNet  Google Scholar 

  20. J.M. Harrison, “Brownian models of queueing networks with heterogeneous customer populations,” Proc. IMA Workshop on Stochastic Differential Systems. Berlin: Springer-Verlag (1987) (this volume).

    Google Scholar 

  21. W. Helm and K.-H. Waldmann, “Optimal control of arrivals to multiserver queues in a random environment,” J. Appl. Prob. 21, 602–615 (1984).

    Article  MathSciNet  MATH  Google Scholar 

  22. K. Jo, “Optimal service-rate control of exponential systems,” J. Operat. Res. Soc. Japan 26, 147–165 (1983).

    MathSciNet  MATH  Google Scholar 

  23. K. Jo and S. Stidham, “Optimal service-rate control of M/G/1 queueing systems using phase methods,” Adv. Appl. Prob. 15, 616–637 (1983).

    Article  MathSciNet  MATH  Google Scholar 

  24. S.G. Johansen and S. Stidham, “Control of arrivals to a stochastic input-output system,” Adv. Appl. Prob. 12, 972–999 (1980).

    Article  MathSciNet  MATH  Google Scholar 

  25. S.G. Johansen and S. Stidham, “The optimal pricing of admission to a GI/M/c queue,” Department of Operations Research, Aarhus University, Denmark (1983).

    Google Scholar 

  26. P. Kakamanu, “Continuous-time Markov decision models with applications to optimization problems,” Tech. Rpt. No. 63, Department of Operations Research, Cornell University (1969).

    Google Scholar 

  27. N.C. Knudsen, “Individual and social optimization in a multi-server queue with a general cost-benefit structure,” Econometrica 40, 515–528 (1972).

    Article  MathSciNet  MATH  Google Scholar 

  28. N.C. Knudsen and S. Stidham, “Individual and social optimization in a birth-death congestion system with a general cost-benefit structure, ” Tech. Rpt. No. 76–8, Department of Industrial Engineering, N, C. State University, Raleigh (1976).

    Google Scholar 

  29. H.-J. Langen, “Applying the method of phases in the optimization of queueing systems,” Adv. Appl. Prob. 14, 122–142 (1982).

    Article  MathSciNet  MATH  Google Scholar 

  30. A. Lazar, “Optimal control of a class of queueing networks in equilibrium,” IEEE Trans. Auto. Control AC-28, 1001–1007 (1983).

    Article  MathSciNet  Google Scholar 

  31. S. Lippman, “Applying a new device in the optimization of exponential queueing systems,” Operations Res. 23, 687–710 (1975).

    Article  MathSciNet  MATH  Google Scholar 

  32. S. Lippman and S. Stidham, “Individual versus social optimization in exponential congestion systems,” Operations Res. 25, 233–247 (1977).

    Article  MathSciNet  MATH  Google Scholar 

  33. F. Lu and R. Serfozo, “M/M/1 queueing decision processes with monotone hysteretic optimal policies,” Operations Res. 32, 1116–1132 (1984).

    Article  MathSciNet  MATH  Google Scholar 

  34. M. Moustafa, “Optimal Scheduling in Networks of Queues,” Ph.D. Dissertation (in preparation), Program in Operations Research, N.C. State University, Raleigh (1987).

    Google Scholar 

  35. W. Mitchell, “Optimal service-rate selection in an M/G/i queue,” Siam J. Appl. Math. 24, 19–35 (1973).

    Article  MathSciNet  MATH  Google Scholar 

  36. P. Naor, “On the regulation of queue size by levying tolls,” Econometrica 37, 15–24 (1969).

    Article  MATH  Google Scholar 

  37. Z. Rosberg, P. Varaiya, J. Walrand, “Optimal control of service in tandem queues,” IEEE Trans. Auto. Control AC-27, 600–610 (1982).

    Article  MathSciNet  Google Scholar 

  38. S. Ross, Applied Probability Models with Optimization Applications. San Francisco: Holden-Day (1970).

    MATH  Google Scholar 

  39. R. Rue and M. Rosenshine, “Optimal control of entry of many classes of customers to an M/M/1 queue,” Naval Res. Logist. Quart. 28, 489–495 (1981).

    Article  MathSciNet  MATH  Google Scholar 

  40. H. Sabeti, “Optimal selection of service rates in queueing with di fferent costs, ” J. Operat. Res. Soc. Japan 16, 15–35 (1973).

    MathSciNet  MATH  Google Scholar 

  41. M. Schäl, “Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal,” Z. Wahrscheinlichkeitstheorie verw. Geb. 32, 179–196 (1975).

    Article  MATH  Google Scholar 

  42. R. Schassberger, “A note on optimal service selection in a single-server queue,” Management Sci. 21, 1326–1331 (1975).

    Article  MathSciNet  MATH  Google Scholar 

  43. R. Serfozo, “An equivalence between discrete-and continuous-time Markov decision processes,” Operations Res. 27, 616–620 (1979).

    Article  MathSciNet  MATH  Google Scholar 

  44. R. Serfozo, “Optimal control of random walks, birth and death processes, and queues,” Adv. Appl. Prob. 13, 61–83 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  45. A. Simonovits, “Self-and social optimization in queues,” Studia Scient. Math. Hungarica 11, 131–138 (1976).

    MathSciNet  MATH  Google Scholar 

  46. M. Sobel, “Optimal operation of queues,” in Mathematical Methods in Queueing Theory, A. B. Clarke (ed.), Berlin: Springer-Verlag, 231–261 (1974).

    Google Scholar 

  47. M. Sobel, “The optimality of full-service policies,” Operations Res. 30, 636–649 (1982).

    Article  MATH  Google Scholar 

  48. S. Stidham, “Socially and individually optimal control of arrivals to a GI/M/1 queue,” Management Sci. 24, 1598–1610 (1978).

    Article  MathSciNet  MATH  Google Scholar 

  49. S. Stidham, “On the convergence of successive approximations in dynamic programming with non-zero terminal reward,” Z. Operations Res. 25, 57–77 (1981).

    Article  MathSciNet  MATH  Google Scholar 

  50. S. Stidham, “Optimal control of admission, routing, and service in queues and networks of queues: a tutorial review,” Proc. ARO Workshop: Analytical and Computational Issues in Logistics R and D, George Washington University, 330–377 (1984).

    Google Scholar 

  51. S. Stidham, “Optimal control of admission to a queueing system,” IEEE Trans. Auto. Control AC-30 705–713 (1985).

    Article  MathSciNet  Google Scholar 

  52. S. Stidham and N.U. Prabhu, “Optimal control of queueing systems,” in Mathematical Methods in Queueing Theory, A.B. Clarke (ed.), Berlin: Springer-Verlag, 263–294 (1974).

    Google Scholar 

  53. S. Stidham and R. Weber, “Monotonic and insensitive optimal policies for control of arrival and service rates in queues,” submitted for publication (1986).

    Google Scholar 

  54. D. Topkis, “Minimizing a submodular function on a lattice,” Operations Res. 26, 305–321 (1978).

    Article  MathSciNet  MATH  Google Scholar 

  55. J. van der Wal, Stochastic Dynamic Programming, Mathematical Centre Tract 139, Amsterdam (1981).

    Google Scholar 

  56. R. Weber, “On the optimal assignment of customers to parallel servers,” J. Appl. Prob. 15, 406–413 (1978).

    Article  MATH  Google Scholar 

  57. R. Weber and S. Stidham, “Control of service rates in net-works of queues,” J. Appl, Prob. 24, 000–000 (1987).

    Google Scholar 

  58. P. Whittle, Optimization Over Time, New York: Wiley (1983).

    MATH  Google Scholar 

  59. W. Winston, “Optimality of the shortest-line discipline,” J. Appl. Prob. 14, 181–189 (1977).

    Article  MathSciNet  MATH  Google Scholar 

  60. U. Yechiali, “On optimal balking rules and toll charges in a GI/M/1 queueing process,” Operations Res. 19, 349–370 (1971).

    Article  MathSciNet  MATH  Google Scholar 

  61. U. Yechiali, “Customers’ optimal joiningg rules for the GI/M/s queue,” Management Sci. 18, 434–443 (1971).

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Curriculum in Operations Research and Systems Analysis, University of North Carolina, Smith Building 128A, Chapel Hill, NC, 27514, USA

    Shaler Stidham Jr.

Authors
  1. Shaler Stidham Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Division of Applied Mathematics, Brown University, 02912, Providence, Rhode Island, USA

    Wendell Fleming

  2. Ceremade, Universite Paris-Dauphine, Place de Lattre de Tassigny, 75775, Paris Cedex 16, France

    Pierre-Louis Lions

Rights and permissions

Reprints and permissions

Copyright information

© 1988 Springer-Verlag New York Inc.

About this paper

Cite this paper

Stidham, S. (1988). Scheduling, Routing, and Flow Control in Stochastic Networks. In: Fleming, W., Lions, PL. (eds) Stochastic Differential Systems, Stochastic Control Theory and Applications. The IMA Volumes in Mathematics and Its Applications, vol 10. Springer, New York, NY. https://doi.org/10.1007/978-1-4613-8762-6_31

Download citation

  • DOI: https://doi.org/10.1007/978-1-4613-8762-6_31

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4613-8764-0

  • Online ISBN: 978-1-4613-8762-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation