Skip to main content
SpringerLink
Log in
Book cover

Dynamic and Stochastic Multi-Project Planning pp 19–28Cite as

Literature Review

  • Chapter
  • First Online:

  • 1066 Accesses

Part of the book series: Lecture Notes in Economics and Mathematical Systems ((LNE,volume 673))

Abstract

Dynamic programming is a general technique for solving sequential problems. The first comprehensive books on the topic have been written by Bellman [13] and Howard [62]. The most important methodologies for determining an optimal policy for a Markov decision process (MDP) are backward induction, value iteration (VI), policy iteration (PI) and linear programming. As MDPs are in discrete time where transitions have the same deterministic durations many results and methodologies, such as VI, cannot be directly applied to continuous-time Markov decision processes with exponentially distributed transition times.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Bibliography

  1. Adelman, D. (2004). A price-directed approach to stochastic inventory/routing. Operations Research, 52(4), 499–514.

    Article  Google Scholar 

  2. Adler, P. S., Mandelbaum, A., Nguyen, V., & Schwerer, E. (1995). From project to process management: An empirically-based framework for analyzing product development time. Management Science, 41(3), 458–484.

    Article  Google Scholar 

  3. Anavi-Isakow, S., & Golany, B. (2003). Managing multi-project environments through constant work-in-process. International Journal of Project Management, 21, 9–18.

    Article  Google Scholar 

  4. Anderson, E. J., & Nyrenda, J. C. (1990). Two new rules to minimize tardiness in a job shop. International Journal of Production Research, 28(12), 2277–2292.

    Article  Google Scholar 

  5. Ashtiani, B., Leus, R., & Aryanezhad, M.-B. (2011). New competitive results for the stochastic resource-constrained project-scheduling problem: Exploring the benefits of pre-processing. Journal of Scheduling, 14, 157–171.

    Article  Google Scholar 

  6. Azaron, A., Katagiri, H., Kato, K., & Sakawa, M. (2006). Longest path analysis in networks of queues: Dynamic scheduling problems. European Journal of Operational Research, 174, 132–149.

    Article  Google Scholar 

  7. Azaron, A., & Modarres, M. (2007). Project completion time in dynamic PERT networks with generating projects. Scientia Iranica, 14(1), 56–63.

    Google Scholar 

  8. Azaron, A., & Tavakkoli-Moghaddam, R. (2006). A multi-objective resource allocation in dynamic PERT networks. Applied Mathematics and Computation, 181, 163–174.

    Article  Google Scholar 

  9. Baccelli, F., Liu, Z., & Towsley, D. (1993). Extremal scheduling of parallel processing with and without real-time constraints. Journal of the Association for Computing Machinery, 40(5), 1209–1237.

    Article  Google Scholar 

  10. Ballestín, F., & Leus, R. (2009). Resource-constrained project scheduling for timely project completion with stochastic activity durations. Production and Operations Management, 18(4), 459–474.

    Article  Google Scholar 

  11. Bard, J., Balachandra, R., & Kaufman, P. (1988). An interactive approach to R&D project selection and termination. IEEE Transactions on Engineering Management, 35, 139–146.

    Article  Google Scholar 

  12. Bellman, R. (1957). Dynamic programming. Princeton: Princeton University Press.

    Google Scholar 

  13. Berman, E. B. (1964). Resource allocation in a PERT network under continuous activity time-cost functions. Management Science, 10(4), 734–745.

    Article  Google Scholar 

  14. Bertsekas, D. P., & Tsitsiklis, J. N. (1996). Neuro-dynamic programming. Belmont: Athena Scientific.

    Google Scholar 

  15. Boctor, F. F. (1990). Some efficient multi-heuristic procedures for resource-constrained project scheduling. European Journal of Operational Research, 49, 3–13.

    Article  Google Scholar 

  16. Brucker, P., Drexl, A., Möhring, R., Neumann, K., & Pesch, E. (1999). Resource-constrained project scheduling: Notation, classification, models, and methods. European Journal of Operational Research, 112, 3–41.

    Article  Google Scholar 

  17. Buss, A. H., & Rosenblatt, M. J. (1997). Activity delay in stochastic project networks. Operations Research, 45(1), 126–139.

    Article  Google Scholar 

  18. Choi, J., Realff, M. J., & Lee, J. H. (2004). Dynamic programming in a heuristically confined state space: A stochastic resource constrained project scheduling application. Computers and Chemical Engineering, 28, 1039–1058.

    Article  Google Scholar 

  19. Choi, J., Realff, M. J., & Lee, J. H. (2007). A Q-learning-based method applied to stochastic resource constrained project scheduling with new project arrivals. International Journal of Robust and Nonlinear Control, 17, 1214–1231.

    Article  Google Scholar 

  20. Cohen, I., Nguyen, V., & Shtub, A. (2004). Multi-project scheduling and control: A process-based comparative study of the critical chain methodology and some alternatives. Project Management Journal, 35(2), 39–50.

    Google Scholar 

  21. Cox, D. R., & Smith, W. L. (1961). Queues. London: Methuen.

    Google Scholar 

  22. Crabill, T. B. (1972). Optimal control of a service facility with variable exponential service times and constant arrival rate. Management Science, 18(9), 560–566.

    Article  Google Scholar 

  23. Creemers, S., Leus, R., & Lambrecht, M. (2010). Scheduling Markovian PERT networks to maximize the net present value. Operations Research Letters, 38, 51–56.

    Article  Google Scholar 

  24. Davis, E. W., & Patterson, J. H. (1975). A comparison of heuristic and optimum solutions in resource-constrained project scheduling. Management Science, 21, 944–955.

    Article  Google Scholar 

  25. De Boer, R. (1998). Resource-constrained multi-project management. PhD thesis, Universiteit Twente.

    Google Scholar 

  26. De Farias, D. P., & Van Roy, B. (2003). Approximate linear programming for average-cost dynamic programming. Advances in Neural Information Processing Systems, 15, 1619–1626.

    Google Scholar 

  27. De Farias, D. P., & Van Roy, B. (2003). The linear programming approach to approximate dynamic programming. Operations Research, 51(6), 850–865.

    Article  Google Scholar 

  28. De Farias, D. P., & Van Roy, B. (2004). On constraint sampling in the linear programming approach to approximate dynamic programming. Mathematics of Operations Research, 29(3), 462–478.

    Article  Google Scholar 

  29. Demeulemeester, E. L., & Herroelen, W. S. (2002). A research handbook (International series in operations research & management science). Boston: Kluwer Academic.

    Google Scholar 

  30. De Serres, I. (1991). Simultaneous optimization of flow control and scheduling in a single server queue with two job classes. Operations Research Letters, 10, 103–112.

    Article  Google Scholar 

  31. De Serres, I. (1991). Simultaneous optimization of flow control and scheduling in queues. PhD thesis, McGill University, Montreal.

    Google Scholar 

  32. Dumond, J., & Mabert, V. A. (1988). Evaluating project scheduling and due date assignment procedures: An experimental analysis. Management Science, 34(1), 101–118.

    Article  Google Scholar 

  33. Ebben, M. J. R., Hans, E. W., & Olde Weghuis, F. M. (2005). Workload based order acceptance in job shop environments. OR Spectrum, 27, 107–122.

    Article  Google Scholar 

  34. Feinberg, E. A., & Yang, F. (2010). Optimality of trunk reservation for an M/M/k/N queue with several customer types and holding costs. Technical report, State University of New York at Stony Brook.

    Google Scholar 

  35. Fernandez, A. A., Armacost, R. L., & Pet-Edwards, J. J. A. (1996). The role of the nonanticipativity constraint in commercial software for stochastic project scheduling. Computers Industrial Engineering, 31(1/2), 233–236.

    Article  Google Scholar 

  36. Gittins, J. C., & Jones, D. M. (1972). A dynamic allocation index for the sequential design of experiments. In J. Bolyaim (Ed.), Progress in statistics (European meeting of statisticians, Budapest) (Vol. 9). Budapest: Colloquium Mathematical Society.

    Google Scholar 

  37. Goldratt, E. M. (1997). Critical chain. Great Barrington: The North River Press.

    Google Scholar 

  38. Hans, E. W. (2001). Resource loading by branch-and-price techniques. PhD thesis, University of Twente.

    Google Scholar 

  39. Herbots, J., Herroelen, W., & Leus, R. (2007). Dynamic order acceptance and capacity planning on a single bottleneck resource. Naval Research Logistics, 54, 874–889.

    Article  Google Scholar 

  40. Herroelen, W., & Leus, R. (2005). Project scheduling under uncertainty: Survey and research potentials. European Journal of Operational Research, 165, 289–306.

    Article  Google Scholar 

  41. Herroelen, W., De Reyck, B., & Demeulemeester, E. (1998). Resource-constrained project scheduling: A survey of recent developments. Computers and Operations Research, 25(4), 279–302.

    Article  Google Scholar 

  42. Howard, R. (1960). Dynamic programming and Markov processes. Cambridge: MIT.

    Google Scholar 

  43. Ivanescu, C. V., Fransoo, J. C., & Bertrand, J. W. M. (2002). Makespan estimation and order acceptance in batch process industries when processing times are uncertain. OR Spectrum, 24, 467–495.

    Article  Google Scholar 

  44. Ivanescu, V. C., Fransoo, J. C., & Betrand, J. W. M. (2006). A hybrid policy for order acceptance in batch process industries. OR Spectrum, 28, 199–222.

    Article  Google Scholar 

  45. Kavadias, S., & Loch, C. H. (2003). Optimal project sequencing with recourse at a scarce resource. Production and Operations Management, 12(4), 433–442.

    Article  Google Scholar 

  46. Kelley, J. E., Jr. (1963). The critical path method: Resource planning and scheduling. In Industrial scheduling. Englewood Cliffs: Prentice Hall.

    Google Scholar 

  47. Kelley, J. E., & Walker, M. R. (1959). Critical-path planning and scheduling. In 1959 Proceedings of the eastern joint computer conference, Boston (pp. 160–173).

    Google Scholar 

  48. Kemppainen, K. (2005). Priority scheduling revisited – Dominant rules, open protocols and integrated order management. PhD thesis, Helsinki School of Economics.

    Google Scholar 

  49. Kleywegt, A. J., & Papastavrou, J. D. (1998). The dynamic and stochastic knapsack problem. Operations Research, 1, 17–35.

    Article  Google Scholar 

  50. Kleywegt, A. J., & Papastavrou, J. D. (2001). The dynamic and stochastic knapsack problem with random sized items. Operations Research, 49(1), 26–41.

    Article  Google Scholar 

  51. Klimov, G. P. (1974). Time-sharing service systems I. Theory of Probability and its Applications, 19(3), 532–551.

    Article  Google Scholar 

  52. Knudsen, N. C. (1972). Individual and social optimization in a mutliserver queue with a general cost-benefit. Econometrica, 40(3), 515–528.

    Article  Google Scholar 

  53. Kolisch, R. (1996). Efficient priority rules for the resource-constrained project scheduling problem. Journal of Operations Management, 14, 179–102.

    Article  Google Scholar 

  54. Koole, G., & Pot, A. (2005). Approximate dynamic programming in multi-skill call centers. In M. E. Kuhl, N. M. Steiger, F. B. Armstrong, & J. A. Joines (Eds.), Proceedings of the 2005 winter simulation conference, Orlando.

    Google Scholar 

  55. Kulkarni, V. G., & Adlakha, V. G. (1986). Markov and Markov-regenerative PERT networks. Operations Research, 34, 769–781.

    Article  Google Scholar 

  56. Kumar, P. R., & Seidman, T. I. (1990). Dynamic instabilities and stabilization methods in distributed real-time scheduling of manufacturing systems. IEEE Transactions on Automatic Control, 35(3), 289–298.

    Article  Google Scholar 

  57. Kurtulus, I. S., & Davis, E. W. (1982). Multi-project scheduling: Categorization of heuristic rule performance. Management Science, 28(2), 161–172.

    Article  Google Scholar 

  58. Kurtulus, I. S., & Narula, S. C. (1985). Multi-project scheduling: Analysis of project performance. IIE Transactions, 17(1), 58–66.

    Article  Google Scholar 

  59. Kutanoglu, E., & Sabuncuoglu, I. (1999). An analysis of heuristics in a dynamic job shop with weighted tardiness objectives. International Journal of Production Research, 37(1), 165–187.

    Article  Google Scholar 

  60. Lawrence, S. R., & Morton, T. E. (1993). Resource-constrained multi-project scheduling with tardy costs: Comparing myopic, bottleneck, and resource pricing heuristics. European Journal of Operational Research, 64, 168–187.

    Article  Google Scholar 

  61. Lee, H., & Suh, H.-W. (2008). Estimating the duration of stochastic workflow for product development process. International Journal of Production Economics, 111, 105–117.

    Article  Google Scholar 

  62. Levy, N., & Globerson, S. (1997). Improving multiproject management by using a queueing theory approach. Project Management Journal, 28(4), 40–46.

    Google Scholar 

  63. Lippman, S. A. (1975). Applying a new device in the optimization of exponential queueing systems. Operations Research, 23, 687–710.

    Article  Google Scholar 

  64. Loch, C. H., & Kavadias, S. (2002). Dynamic portfolio selection of NPD programs using marginal returns. Management Science, 48(10), 1227–1241.

    Article  Google Scholar 

  65. Loch, C. H., Pich, M. T., Urbschat, M., & Terwiesch, C. (2001). Selecting R&D projects at BMW: A case study of adopting mathematical programming methods. IEEE Transactions Engineering Management, 48(1), 70–80.

    Article  Google Scholar 

  66. Meyn, S. (2008). Control techniques for complex networks. Cambridge: Cambridge University Press.

    Google Scholar 

  67. Möhring, R. H., Radermacher, F. J., & Weiss, G. (1984). Stochastic scheduling problems I. Zeitschrift für Operations Research, 28, 193–260.

    Google Scholar 

  68. Morton, T. E., & Pentico, D. W. (1993). Heuristic scheduling systems (Wiley series in engineering & technology management). New York: Wiley.

    Google Scholar 

  69. Naor, P. (1969). The regulation of queue size by levying tolls. Econometrica, 37(1), 15–24.

    Article  Google Scholar 

  70. Nguyen, V. (1993). Processing networks with parallel and sequential tasks: Heavy traffic analysis and brownian limits. The Annals of Applied Probability, 3(1), 28–55.

    Article  Google Scholar 

  71. Nguyen, V. (1994). The trouble with diversity: Fork-join networks with heterogeneous customer populations. The Annals of Applied Probability, 4(1), 1–25.

    Article  Google Scholar 

  72. Nino-Mora, J. (2005). Stochastic scheduling. In P. M. Pardalos (Ed.), Encyclopedia of optimization (Vol. V, pp. 367–372). Dordrecht: Kluwer Academic.

    Google Scholar 

  73. Perry, T. C., & Hartman, J. C. (2004). Allocating manufacturing capacity by solving a dynamic, stochastic multiknapsack problem. Technical report ISE 04T-009, Lehigh University, Pennsylvania.

    Google Scholar 

  74. Powell, W. B. (2007). Approximate dynamic programming. Hoboken: Wiley.

    Book  Google Scholar 

  75. Powell, W. B. (2011). Approximate dynamic programming (2nd ed.). Hoboken: Wiley.

    Book  Google Scholar 

  76. Pritsker, A. A. B., Watters, L. J., & Wolfe, P. M. (1969). Multiproject scheduling with limited resources: A zero-one programming approach. Management Science, 16, 93–107.

    Article  Google Scholar 

  77. Ramasesh, R. (1990). Dynamic job shop scheduling: A survey of simulation research. OMEGA International Journal of Operations and Production Management, 18(1), 43–57.

    Google Scholar 

  78. Roemer, T. A., & Ahmadi, R. (2004). Concurrent crashing and overlapping in product development. Operations Research, 52(4), 606–622.

    Article  Google Scholar 

  79. Ross, K. W., & Tsang, D. H. K. (1989). Optimal circuit access policies in an ISDN environment: A Markov decision approach. IEEE Transactions on Communications, 37(9), 934–939.

    Article  Google Scholar 

  80. Roubos, D., & Bhulai, S. (2007). Average-cost approximate dynamic programming for the control of birth-death processes. Technical report, VU University Amsterdam.

    Google Scholar 

  81. Schweitzer, P. J., & Seidman, A. (1985). Generalized polynomial approximations in Markovian decision processes. Journal of Mathematical Analysis and Applications, 110, 568–582.

    Article  Google Scholar 

  82. Sennot, L. I. (1999). Stochastic dynamic programming and the control of queueing systems (Wiley series in probability and statistics). New York: Wiley.

    Google Scholar 

  83. Slotnick, S., & Morton, T. (2007). Order acceptance with weighted tardiness. Computers and Operations Research, 34, 3029–3042.

    Article  Google Scholar 

  84. Sobel, M. J., Szmerekovsky, J. G., & Tilson, V. (2009). Scheduling projects with stochastic activity duration to maximize expected net present value. European Journal of Operational Research, 198, 697–705.

    Article  Google Scholar 

  85. Stork, F. (2001). Stochastic resource-constrained project scheduling. PhD thesis, Technische Universität Berlin.

    Google Scholar 

  86. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning. Cambridge: MIT.

    Google Scholar 

  87. Talla Nobibon, F., Herbots, J., & Leus, R. (2009). Order acceptance and scheduling in a single-machine environment: Exact and heuristic algorithms. Technical report, Faculty of Business and Economics, KU Leuven.

    Google Scholar 

  88. Talla Nobibon, F., & Leus, R. (2011). Exact algorithms for a generalization of the order acceptance and scheduling problem in a single-machine environment. Computers and Operations Research, 38(1), 367–378.

    Article  Google Scholar 

  89. Tsai, D. M., & Chiu, H. N. (1996). Two heuristics for scheduling multiple projects with resource constraints. Construction Management and Economics, 14, 325–340.

    Article  Google Scholar 

  90. van Foreest, N. D., Wijngaard, J., & van der Vaart, J. T. (2010). Scheduling and order acceptance for the customized stochastic lot scheduling problem. International Journal of Production Research, 48(12), 3561–3578.

    Article  Google Scholar 

  91. Veatch, M. H. (2009). Approximate dynamic programming for networks: Fluid models and constraint reduction. Technical report, Gordon College.

    Google Scholar 

  92. Veatch, M. H., & Walker, N. (2008). Approximate linear programming for network control: Column generation and subproblems. Technical report, Gordon College.

    Google Scholar 

  93. Vepsalainen, A. P. J., & Morton, T. E. (1987). Priority rules for job shops with weighted tardiness costs. Management Science, 33(8), 1035–1047.

    Article  Google Scholar 

  94. Vyzas, E. (1997). Approximate dynamic programming for some queueing problems. Master’s thesis, Massachusetts Institute of Technology.

    Google Scholar 

  95. Wester, F. A. W., Wijngaard, J., & Zijm, W. H. M. (1992). Order acceptance strategies in a production-to-order environment with setup time and due-dates. International Journal of Production Research, 30(6), 1313–1326.

    Article  Google Scholar 

  96. Yaghoubi, S., Noori, S., Azaron, A., & Tavakkoli-Moghaddam, R. (2011). Resource allocation in dynamic PERT networks with finite capacity. European Journal of Operational Research, 215, 670–678.

    Google Scholar 

  97. Yechiali, U. (1969). On optimal balking rules and toll charges in the GI/M/1 queueing process. Operations Research, 19, 349–370.

    Article  Google Scholar 

  98. Yechiali, U. (1972). Customers’ optimal joining rules for the GI/M/s queue. Manage, 18, 434–443.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Technische Universität München, München, Germany

    Philipp Melchiors

Authors
  1. Philipp Melchiors
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Melchiors, P. (2015). Literature Review. In: Dynamic and Stochastic Multi-Project Planning. Lecture Notes in Economics and Mathematical Systems, vol 673. Springer, Cham. https://doi.org/10.1007/978-3-319-04540-5_3

Download citation

Publish with us

Policies and ethics

Search

Navigation