Dynamic Programming Basic Concepts and Applications
Dynamic programming deals with sequential decision processes, which are models of dynamic systems under the control of a decision maker. At each point in time at which a decision can be made, the decision maker chooses an action from a set of available alternatives, which generally depends on the current state of the system. The objective is to find a sequence of actions (a so-called policy) that minimizes the total cost over the decision making horizon.
In what follows, deterministic and stochastic dynamic programming problems which are discrete in time will be considered. At first, Bellman’s equation and principle of optimality will be presented upon which the solution method of dynamic programming is based. After that, a large number of applications of dynamic programming will be discussed.
KeywordsOptimal Policy Planning Horizon Action Space Knapsack Problem Markov Decision Process
Unable to display preview. Download preview PDF.
- 2.Bertsekas, D.P.: Dynamic Programming - Deterministic and Stochastic Models; Prentice-Hall, Englewood Cliffs, 1987Google Scholar
- 3.Braun, H.: Unit Commitment and Thermal Optimization — Problem Statement; SVOR/ASRO Tutorial on Optimization in Planning and Operation of Electric Power Systems, Thun 1992, SwitzerlandGoogle Scholar
- 4.Denardo, E.V.: Dynamic Programming - Models and Applications; Prentice-Hall, Englewood Cliffs, 1982Google Scholar
- 5.Efthymoglou, P.G.: Optimal Use and the Value of Water Resources in Electricity Generation; Management Science 33 (1987) 1622–1634Google Scholar
- 8.Gallo, G., Pallottino, S.: Shortest Path Algorithms; Annals of Operations Research 13 (1988) 3–79Google Scholar
- 9.Gjelsvik, A., Rotting, T.A., Roynstrand, J.: Long-Term Scheduling of Hydro-Thermal Power Systems; in Broch, E., Lysne, D.K. (eds.): Proceedings of the Second International Conference on Hydro Power, A.A. Balkema, Rotterdam (1992) 539–546Google Scholar
- 10.Handschin, E., Slomski, H.: Unit Commitment in Thermal Power Systems with Long-Term Energy Constraints; Power Industry Computer Application Conference, Seattle (1989) 211–217Google Scholar
- 11.Heyman, D.P., Sobel, M.J.: Stochastic Models in Operations Research; Vol. II. McGraw-Hill, New York, 1984Google Scholar
- 12.Heyman, D.P., Sobel, M.J.(eds.): Stochastic Models; Handbooks in Operations Research and Management Science, Vol. 2. North-Holland, Amsterdam, 1990Google Scholar
- 13.Ikem, F.M., Reisman, A.M.: An Approach to Planning for Physician Requirements in Developing Countries Using Dynamic Programming; Operations Research 38 (1990) 607–618Google Scholar
- 14.Lawler, E.L.: Efficient Implementation of Dynamic Programming Algorithms for Sequencing Problems; Report BW 106/79, Mathematisch Centrum, Amsterdam, 1979Google Scholar
- 15.Nemhauser, G.L., Wolsey, L.A.: Integer and Combinatorial Optimization; John Wiley & Sons, New York, 1988Google Scholar
- 16.Neumann, K.: Operations Research Verfahren; Band, II. Carl Hanser, München, 1977Google Scholar
- 17.Neumann, K., Morlock, M.: Operations Research; Carl Hanser, München, 1993Google Scholar
- 18.Ozden, M.: A Dynamic Planning Technique for Continuous Activities under Multiple Resource Constraints; Management Science 33 (1987) 1333–1347Google Scholar
- 19.Sanders, 11.-H., Linke, K.: Experiences with Optimization Packages for Unit Commitment; SVOR/ASRO Tutorial on Optimization in Planning and Operation of Electric Power Systems, Thun 1992, SwitzerlandGoogle Scholar
- 20.Sherali, H.D., Hobeika, A.G., Trani, A.A., Kim, B.J.: An Integrated Simulation and Dynamic Programming Approach for Determining Optimal Runway Exit Locations; Management Science 38 (1992) 1049–1062Google Scholar
- 21.Slomski, H.: Optimale Einsatzplanung thermischer Kraftwerke unter Berücksichtigung langfristiger Energiebedingungen; Ph.D. Thesis, University of Dortmund, 1990Google Scholar
- 22.Stoecker, A.L., Seidman, A., Lloyd, G.S.: A Linear Dynamic Programming Approach to Irrigation System Management with Depleting Groundwater; Management Science 31 (1985) 422–434Google Scholar