Dynamic Policies

  • Xiaoqiang Cai
  • Xianyi Wu
  • Xian Zhou
Part of the International Series in Operations Research & Management Science book series (ISOR, volume 207)


This chapter is devoted to optimal dynamic policies. Section 7.1 discusses differences between optimal static and dynamic policies with emphasis on the impacts of different levels of information utilization. Section 7.2 treats optimal policies in the class of restricted dynamic policies for problems subject to random machine breakdowns under the total-loss model. Section 7.3 studies the optimal restricted dynamic policies for no-loss breakdown models. Section 7.4 deals with partial- loss breakdown models. Its focus is on restricted dynamic policies, but optimal static and nonpreemptive dynamic policies are also presented as by-products. The restricted dynamic policies in Sections 7.2–7.4 show the applications of Gittins index theory to stochastic scheduling. Section 7.5, on the other hand, is dedicated to unrestricted dynamic policies for parallel machine scheduling with exponentially distributed processing times, with optimal policies obtained by means of general Markovian decision processes.


  1. Adiri, I., Bruno, J., Frostig, E., & Rinnooy Kan, A. H. G. (1989). Single machine flowtime scheduling with a single breakdown. Acta Informatica, 26, 679–696.CrossRefGoogle Scholar
  2. Adiri, I., Frostig, E., & Rinnooy Kan, A. H. G. (1991). Scheduling on a single machine with a single breakdown to minimize stochastically the number of tardy jobs. Naval Research Logistics, 38, 261–271.CrossRefGoogle Scholar
  3. Birge, J., Frenk, J. B. G., Mittenthal, J., & Rinnooy Kan, A. H. G. (1990). Single-machine scheduling subject to stochastic breakdown. Naval Research Logistics, 37, 661–677.CrossRefGoogle Scholar
  4. Bruno, J. (1985). On scheduling tasks with exponential service times and in-tree precedence constraints. Acta Informatica, 22, 139–148.Google Scholar
  5. Cai, X.Q.i, & Zhou, X. (1999). Stochastic scheduling on parallel machine subject to random breakdowns to minimize expected costs for earliness and tardy cost. Operations Research, 47, 422–437.Google Scholar
  6. Cai, X.Q.i, & Zhou, X. (2000). Asymmetric earliness-tardiness scheduling with exponential processing times on an unreliable machine. Annals of Operations Research, 98, 313–331.Google Scholar
  7. Cai, X.Q.i, & Zhou, X. (2005). Single-machine scheduling with exponential processing times and general stochastic cost functions. Journal of Global Optimization, 31, 317–332.Google Scholar
  8. Cai, X.Q., Sun, X., & Zhou, X. (2003). Stochastic scheduling with breakdown-repeat machine breakdowns to minimize the expected weighted flowtime. Probability in the Engineering and Informational Sciences, 17, 467–485.CrossRefGoogle Scholar
  9. Cai, X.Q., Sun, X., & Zhou, X. (2004). Stochastic scheduling subject to machine breakdowns: The breakdown-repeat model with discounted reward and other criteria. Naval Research Logistics, 51, 800–817.CrossRefGoogle Scholar
  10. Cai, X.Q., Wu, X.Y., & Zhou, X. (2005). Dynamically optimal policies for stochastic scheduling subject to breakdown-repeat breakdowns. IEEE Transactions on Automation Science and Engineering, 2, 158–172.CrossRefGoogle Scholar
  11. Cai, X.Q., Wang, L., & Zhou, X. (2007b). Single-machine scheduling to stochastically minimize maximum lateness. Journal of Scheduling, 10(4), 293–301.CrossRefGoogle Scholar
  12. Cai, X.Q., Wu, X.Y., & Zhou, X. (2009a). Stochastic scheduling on parallel machines to minimize discounted holding costs. Journal of Scheduling, 12(4), 375–388.CrossRefGoogle Scholar
  13. Chang, C. S., Chao, X., Pinedo, M., & Weber, R. (1992). On the optimality of LEPT and c μ rules for machines in parallel. Journal of Applied Probability, 29, 667–681.CrossRefGoogle Scholar
  14. Denneberg, D. (1994). Non-additive measure and integral. Dordrecht/Boston: Kluwer.CrossRefGoogle Scholar
  15. Frostig, E. (1991). A note on stochastic scheduling on a single machine subject to breakdown – The preemptive repeat model. Probability in the Engineering and Informational Sciences, 5, 349–354.CrossRefGoogle Scholar
  16. Garey M. R., & Johnson, D. S. (1979). Computers and intractability: A guide to the theory of NP-completeness. New York: W. H. Freeman.Google Scholar
  17. Gittins, J. C. (1989). Multi-armed bandit allocation indices (Wiley-Interscience series in systems and optimization). Chichester: Wiley. ISBN:0-471-92059-2.Google Scholar
  18. Gittins, J. C., & Glazebrook, K. D. (1977). On Bayesian models in stochastic scheduling. Journal of Applied Probability, 14, 556–565.CrossRefGoogle Scholar
  19. Glazebrook, K. D. (1979). Scheduling tasks with exponential service times on parallel processors. Journal of Applied Probability, 16, 685–689.CrossRefGoogle Scholar
  20. Glazebrook, K. D. (1984). Scheduling stochastic jobs on a single machine subject to breakdowns. Naval Research Logistics Quarterly, 31, 251–264.CrossRefGoogle Scholar
  21. Glazebrook, K. D. (1987). Evaluating the effects of machine breakdowns in scheduling stochastic problems. Naval Research Logistics, 34, 319–335.CrossRefGoogle Scholar
  22. Hopkins, A. L., Smith, T. B., III & Lala, J. H. (1978). FTMP – A highly reliable fault-tolerant multiprocessor for aircraft. Proceedings of the IEEE, 66(10), 1221–1239.CrossRefGoogle Scholar
  23. Iijima, M., Komatsu, S., & Katoh, S. (1996). Hybrid just-in-time logistics systems and information networks for effective management in perishable food industries. International Journal of Production Economics, 44, 97–103.CrossRefGoogle Scholar
  24. Jang, W., & Klein, C. M. (2002). Minimizing the expected number of tardy jobs when processing times are normally distributed. Operation Research Letters, 30, 100–106.CrossRefGoogle Scholar
  25. Kämpke, T. (1987a). On the optimality of static priority policies in stochastic scheduling on parallel machines. Journal of Applied Probability, 24, 430–448.CrossRefGoogle Scholar
  26. Kämpke, T. (1987b). Necessary optimality conditions for priority policies in stochastic weighted flowtime scheduling problems. Advances in Applied Probability, 19(3), 749–750.CrossRefGoogle Scholar
  27. Liskyla-Peuralahti, J., Spies, M., & Tapaninen, U. (2011). Transport vulnerabilities and critical industries: Experiences from a Finnish stevedore strike. International Journal of Risk Assessment and Management, 15, 222–240.CrossRefGoogle Scholar
  28. Mandelbaum, A. (1987). Continuous multi-armed bandits and multiparameter processes. Annals of Probabability, 15(4), 1527–1556.CrossRefGoogle Scholar
  29. Merten, A. G., & Muller, M. E. (1972). Variance minimization in single machine sequencing problems. Management Science, 18, 518–528.CrossRefGoogle Scholar
  30. Pinedo, M. (1983). Stochastic scheduling with release dates and due dates. Operations Research, 31, 559–572.CrossRefGoogle Scholar
  31. Pinedo, M. (2002). Scheduling: Theory, algorithms, and systems (2nd ed.). Englewood Cliffs: Prentice Hall.Google Scholar
  32. Pinedo, M., & Weiss, G. (1987). The largest variance first policy in some stochastic scheduling problems. Operations Research, 35, 884–891.CrossRefGoogle Scholar
  33. Qi, X. D., Yin, G., & Birge, J. R. (2000a). Scheduling problems with random processing times under expected earliness/tardiness costs. Stochastic Analysis and Applications, 18, 453–473.CrossRefGoogle Scholar
  34. Raafat, F. (1991). Survey of literature on continuously deteriorating inventory models. Journal of the Operational Research Society, 42, 27–37.CrossRefGoogle Scholar
  35. Righter, R. (1994). Scheduling. In M. Shaked & J. G. Shanthikumar (Eds.), Stochastic orders and their applications. Boston: Academic.Google Scholar
  36. Ronconi, D. P., & Powell, W. B. (2010). Minimizing total tardiness in a stochastic single machine scheduling problem using approximate dynamic programming. Journal of Scheduling, 13, 597–607.CrossRefGoogle Scholar
  37. Townsend, W. (1978). The single-machine problem with quadratic penalty function of completion times: A branch-and-bound solution. Managment Science, 24, 530–534.CrossRefGoogle Scholar
  38. Wang, J. B., Wang, D., & Zhang, G. D. (2010). Single-machine scheduling with learning functions. Applied Mathematics and Computation, 216, 1280–1286.CrossRefGoogle Scholar
  39. Weber, R. R. (1982b). Scheduling jobs with stochastic processing requirements on parallel machines to minimize makespan or flowtime. Journal of Applied Probability, 19, 167–182.CrossRefGoogle Scholar
  40. Weber, R. R. (1992). On the Gittins index for multiarmed bandits. Annals of Probability, 2(4), 1024–1033.CrossRefGoogle Scholar
  41. Weber, R. R., Varaiya, P., & Walrand, J. (1986). Scheduling jobs with stochastically ordered processing times on parallel machines to minimize expected flowtime. Journal of Applied Probability, 23, 841–847.CrossRefGoogle Scholar
  42. Weiss, G. (1988). Branching bandit processes. Probability in Engineering and Information Science, 2, 269–278.CrossRefGoogle Scholar
  43. Weiss, G. (1990). Approximation results in parallel machines stochastic scheduling. Annals of Operations Research, 26, 195–242.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Xiaoqiang Cai
    • 1
  • Xianyi Wu
    • 2
  • Xian Zhou
    • 3
  1. 1.Department of Systems Engineering and Engineering ManagementThe Chinese University of Hong KongShatin, N.T.Hong Kong SAR
  2. 2.Department of Statistics and Actuarial ScienceEast China Normal UniversityShanghaiPeople’s Republic of China
  3. 3.Department of Applied Finance and Actuarial StudiesMacquarie UniversityNorth RydeAustralia

Personalised recommendations