Skip to main content
SpringerLink
Log in

Dynamic Safety-Stocks for Asymptotic Optimality in Stochastic Networks

  • Published:
Queueing Systems Aims and scope Submit manuscript

Abstract

This paper concerns control of stochastic networks using state-dependent safety-stocks.

Three examples are considered: a pair of tandem queues; a simple routing model; and the Dai-Wang re-entrant line. In each case, a single policy is proposed that is independent of network load ρ•. The following conclusions are obtained for the controlled network in each case, where the finite constant K0 is independent of load: The policy is fluid-scale asymptotically optimal, and satisfies the bound

$$ eta_ast \le \eta \le \eta_\ast + K_0 \log(\eta_\ast),\quad 0 \le \rho_\bullet \le 1, $$
(2)

where η* is the optimal steady-state cost.

These results are based on the construction of an approximate solution to the average-cost dynamic programming equations using a perturbation of the value function for an associated fluid model.

Moreover, a new technique is introduced to obtain fluid-scale asymptotic optimality for general networks modeled in discrete time. The proposed policy is myopic with respect to a perturbation of the value function for the fluid model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Nicole Bäuerle, Optimal control of queueing networks: An approach via fluid models, Adv. in Appl. Probab. 34(2) (2002) 313–328.

    Article  Google Scholar 

  2. S.L. Bell and R.J. Williams, Dynamic scheduling of a system with two parallel servers in heavy traffic with complete resource pooling: Asymptotic optimality of a continuous review threshold policy, Ann. Appl. Probab. 11 (2001) 608–649.

    Article  Google Scholar 

  3. M. Bramson, State space collapse with application to heavy traffic limits for multiclass queueing networks, Queueing Systems: Theory and Applications 30 (1998) 89–148.

    Article  Google Scholar 

  4. H. Chen and D.D. Yao, Fundamentals of Queueing Networks: Performance, Asymptotics, and Optimization (Springer-Verlag, New York, 2001) Stochastic Modelling and Applied Probability.

  5. M. Chen, C. Pandit and S.P. Meyn, in search of sensitivity in network optimization, Queueing Systems: Theory and Applications 44(4) (2003) 313–363.

    Article  Google Scholar 

  6. D.P. Connors, G. Feigin and D. Yao, Scheduling semiconductor lines using a fluid network model, IEEE Trans. on Robotics and Automation 10 (1994) 88–98.

    Article  Google Scholar 

  7. J.G. Dai and W. Lin, Maximum pressure policies in stochastic processing networks, Operations Research, 53(2) to appear.

  8. J.G. Dai and Y. Wang, Nonexistence of Brownian models of certain multiclass queueing networks, Queueing Systems: Theory and Applications 13 (1993) 41–46.

    Article  MathSciNet  Google Scholar 

  9. A. Ephremides and L. Tassiulas, Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks, IEEE Trans. Automat. Control 37 (1992) 1936–1949.

    Article  MathSciNet  Google Scholar 

  10. A. Gajrat, A. Hordijk and A. Ridder, Large deviations analysis of the fluid approximation for a controllable tandem queue, Ann. Appl. Probab. 13 (2003) 1423–1448.

    Article  Google Scholar 

  11. L. Georgiadis, W. Szpankowski and L. Tassiulas, A scheduling policy with maximal stability region for ring networks with spatial reuse, Queueing Systems: Theory and Applications 19(1/2) (1995) 131–148.

    Article  MathSciNet  Google Scholar 

  12. P.W. Glynn and S.P. Meyn, A Liapounov bound for solutions of the Poisson equation, Ann. Probab. 24(2) (1996) 916–931.

    Article  Google Scholar 

  13. L. Green, A queueing system with general-use and limited-use servers, Oper. Res. 33(1) (1985) 168–182.

    Google Scholar 

  14. B. Hajek, Optimal control of two interacting service stations, IEEE Trans. Automat. Control, AC-29 (1984) 491–499.

    Article  Google Scholar 

  15. J. M. Harrison, Brownian Motion and Stochastic Flow Systems (Robert E. Krieger Publishing Co. Inc., Malabar, FL, 1990) Reprint of the 1985 original.

  16. J.M. Harrison and M.J. López, Heavy traffic resource pooling in parallel-server systems, Queueing Systems: Theory and Applications 33 (1999) 339–368.

    Article  MathSciNet  Google Scholar 

  17. S.G. Henderson, S.P. Meyn and V. B. Tadić, Performance evaluation and policy selection in multiclass networks, Discrete Event Dynamic Systems: Theory and Applications, 13(1/2) (2003) 149–189. Special issue on learning, optimization and decision making (invited).

    Google Scholar 

  18. F.P. Kelly and C.N. Laws, Dynamic routing in open queueing networks: Brownian models, cut constraints and resource pooling, QUESTA, 13 (1993) 47–86.

    Google Scholar 

  19. L. Kleinrock, Queueing Systems Volume 1: Theory (Wiley, 1975).

  20. G. Koole, On the pathwise optimal Bernoulli routing policy for homogeneous parallel servers, Math. Operations Res. 21 (1996) 469–476.

    Google Scholar 

  21. V.G. Kulkarni, Modeling and Analysis of Stochastic Systems (Texts in Statistical Science Series. Chapman and Hall Ltd., London, 1995).

  22. H.J. Kushner, Heavy Traffic Analysis of Controlled Queueing and Communication Networks (Springer-Verlag, New York, 2001). Stochastic Modelling and Applied Probability.

  23. C.N. Laws, Dynamic routing in queueing networks, PhD thesis, Cambridge University, Cambridge, UK, 1990.

  24. C. Maglaras, Dynamic scheduling in multiclass queueing networks: Stability under discrete-review policies, Queueing Systems: Theory and Applications 31 (1999) 171–206.

    Article  Google Scholar 

  25. A. Mandelbaum and A.L. Stolyar, Scheduling flexible servers with convex delay costs: heavy-traffic optimality of the generalized cμ-rule, Operations Res. 52(6) (2004) 836–855.

    Article  Google Scholar 

  26. S.P. Meyn, The policy iteration algorithm for average reward Markov decision processes with general state space, IEEE Trans. Automat. Control 42(12) (1997) 1663–1680.

    Article  Google Scholar 

  27. S.P. Meyn, Stability and optimization of queueing networks and their fluid models, in: Mathematics of Stochastic Manufacturing Systems (Williamsburg, VA, 1996) pp. 175–199. Amer. Math. Soc., Providence, RI, 1997.

  28. S.P. Meyn, Sequencing and routing in multiclass queueing networks. Part I: Feedback regulation, SIAM J. Control Optim. 40(3) (2001) 741–776.

    Article  Google Scholar 

  29. S.P. Meyn, Sequencing and routing in multiclass queueing networks. Part II: Workload relaxations, SIAM J. Control Optim. 42(1) (2003) 178–217.

    Article  Google Scholar 

  30. S.P. Meyn, Workload models for Stochastic networks: Value functions and performance evaluation, in IEEE Transactions on Automatic Control, July 2005.

  31. S.P. Meyn and R.L. Tweedie, Markov Chains and Stochastic Stability (Springer-Verlag, London, 1993). online: http://black.csl.uiuc.edu/meyn/pages/book.html.

  32. T. Osogami, M. Harchol-Balter, A. Scheller-Wolf and L. Zhang, Exploring threshold-base policies for load sharing, in: Proceedings of the Forty-second Annual Allerton Conference on Communication, Control, and Computing, 2004.

  33. S. Stidham, Jr. and R. Weber, A survey of Markov decision models for control of networks of queues, Queueing Systems: Theory and Applications 13(1/3) (1993) 291–314.

    Article  Google Scholar 

  34. A. L. Stolyar, Maxweight scheduling in a generalized switch: State space collapse and workload minimization in heavy traffic, Adv. Appl. Probab. 14(1) (2004) 1–53.

    Article  Google Scholar 

  35. V. Tadic and S.P. Meyn, Asymptotic properties of two time-scale stochastic approximation algorithms with constant step sizes, in: American Control Conference, 2003.

  36. L. Tassiulas and A. Ephremides, Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks, IEEE Trans. Automat. Control 37(12) (1992) 1936–1948.

    Article  MathSciNet  Google Scholar 

  37. Y.-C. Teh and A.R. Ward, Critical thresholds for dynamic routing in queueing networks, Queueing Systems: Theory and Applications 42(3) (2002) 297–316.

    Article  Google Scholar 

  38. M.H. Veatch, Fluid analysis of arrival routing, IEEE Trans. Automat. Control AC-46 (2001) 1254–1257.

    Article  MathSciNet  Google Scholar 

  39. G. Weiss, Optimal draining of a fluid re-entrant line, in: Volume 71 of IMA volumes in Mathematics and its Applications, eds. Frank Kelly and Ruth Williams (New York, Springer-Verlag, 1995) pp. 91–103.

  40. R.J. Williams, Diffusion approximations for open multiclass queueing networks: sufficient conditions involving state space collapse, Queueing Systems: Theory and Applications, 30(1/2) (1998) 27–88.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering and the Coordinated Sciences Laboratory, University of Illinois at Urbana-Champaign, Urbana, IL, 61801, USA

    Sean Meyn

Authors
  1. Sean Meyn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sean Meyn.

Additional information

AMS subject classification: Primary: 90B35, 68M20, 90B15; Secondary: 93E20, 60J20.

This paper is based upon work supported by the National Science Foundation under Award Nos. ECS 02 17836 and DMI 00 85165.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Meyn, S. Dynamic Safety-Stocks for Asymptotic Optimality in Stochastic Networks. Queueing Syst 50, 255–297 (2005). https://doi.org/10.1007/s11134-005-0732-x

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11134-005-0732-x

Keywords

Search

Navigation