Dynamic Programming Basic Concepts and Applications

  • K. Neumann
Conference paper


Dynamic programming deals with sequential decision processes, which are models of dynamic systems under the control of a decision maker. At each point in time at which a decision can be made, the decision maker chooses an action from a set of available alternatives, which generally depends on the current state of the system. The objective is to find a sequence of actions (a so-called policy) that minimizes the total cost over the decision making horizon.

In what follows, deterministic and stochastic dynamic programming problems which are discrete in time will be considered. At first, Bellman’s equation and principle of optimality will be presented upon which the solution method of dynamic programming is based. After that, a large number of applications of dynamic programming will be discussed.


Optimal Policy Planning Horizon Action Space Knapsack Problem Markov Decision Process 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baker, K.R., Schrage, L.E.: Finding an Optimal Sequence by Dynamic Programming: An Extension to Precedence-Constrained Tasks; Operations Research 26 (1978) 111–120CrossRefGoogle Scholar
  2. 2.
    Bertsekas, D.P.: Dynamic Programming - Deterministic and Stochastic Models; Prentice-Hall, Englewood Cliffs, 1987Google Scholar
  3. 3.
    Braun, H.: Unit Commitment and Thermal Optimization — Problem Statement; SVOR/ASRO Tutorial on Optimization in Planning and Operation of Electric Power Systems, Thun 1992, SwitzerlandGoogle Scholar
  4. 4.
    Denardo, E.V.: Dynamic Programming - Models and Applications; Prentice-Hall, Englewood Cliffs, 1982Google Scholar
  5. 5.
    Efthymoglou, P.G.: Optimal Use and the Value of Water Resources in Electricity Generation; Management Science 33 (1987) 1622–1634Google Scholar
  6. 6.
    Federgruen, A., Tzur, M.: A Simple Forward Algorithm to Solve General Dynamic Lot Size Models with n Periods in O(n log n) or 0(n) Time; Management Science 37 (1991) 909–925CrossRefGoogle Scholar
  7. 7.
    Gallo, G., Pallottino, S.: Shortest Path Methods–A Unifying Approach; Math. Programming Study 26 (1986) 38–64CrossRefGoogle Scholar
  8. 8.
    Gallo, G., Pallottino, S.: Shortest Path Algorithms; Annals of Operations Research 13 (1988) 3–79Google Scholar
  9. 9.
    Gjelsvik, A., Rotting, T.A., Roynstrand, J.: Long-Term Scheduling of Hydro-Thermal Power Systems; in Broch, E., Lysne, D.K. (eds.): Proceedings of the Second International Conference on Hydro Power, A.A. Balkema, Rotterdam (1992) 539–546Google Scholar
  10. 10.
    Handschin, E., Slomski, H.: Unit Commitment in Thermal Power Systems with Long-Term Energy Constraints; Power Industry Computer Application Conference, Seattle (1989) 211–217Google Scholar
  11. 11.
    Heyman, D.P., Sobel, M.J.: Stochastic Models in Operations Research; Vol. II. McGraw-Hill, New York, 1984Google Scholar
  12. 12.
    Heyman, D.P., Sobel, M.J.(eds.): Stochastic Models; Handbooks in Operations Research and Management Science, Vol. 2. North-Holland, Amsterdam, 1990Google Scholar
  13. 13.
    Ikem, F.M., Reisman, A.M.: An Approach to Planning for Physician Requirements in Developing Countries Using Dynamic Programming; Operations Research 38 (1990) 607–618Google Scholar
  14. 14.
    Lawler, E.L.: Efficient Implementation of Dynamic Programming Algorithms for Sequencing Problems; Report BW 106/79, Mathematisch Centrum, Amsterdam, 1979Google Scholar
  15. 15.
    Nemhauser, G.L., Wolsey, L.A.: Integer and Combinatorial Optimization; John Wiley & Sons, New York, 1988Google Scholar
  16. 16.
    Neumann, K.: Operations Research Verfahren; Band, II. Carl Hanser, München, 1977Google Scholar
  17. 17.
    Neumann, K., Morlock, M.: Operations Research; Carl Hanser, München, 1993Google Scholar
  18. 18.
    Ozden, M.: A Dynamic Planning Technique for Continuous Activities under Multiple Resource Constraints; Management Science 33 (1987) 1333–1347Google Scholar
  19. 19.
    Sanders, 11.-H., Linke, K.: Experiences with Optimization Packages for Unit Commitment; SVOR/ASRO Tutorial on Optimization in Planning and Operation of Electric Power Systems, Thun 1992, SwitzerlandGoogle Scholar
  20. 20.
    Sherali, H.D., Hobeika, A.G., Trani, A.A., Kim, B.J.: An Integrated Simulation and Dynamic Programming Approach for Determining Optimal Runway Exit Locations; Management Science 38 (1992) 1049–1062Google Scholar
  21. 21.
    Slomski, H.: Optimale Einsatzplanung thermischer Kraftwerke unter Berücksichtigung langfristiger Energiebedingungen; Ph.D. Thesis, University of Dortmund, 1990Google Scholar
  22. 22.
    Stoecker, A.L., Seidman, A., Lloyd, G.S.: A Linear Dynamic Programming Approach to Irrigation System Management with Depleting Groundwater; Management Science 31 (1985) 422–434Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1993

Authors and Affiliations

  • K. Neumann
    • 1
  1. 1.Institut für Wirtschaftstheorie und Operations ResearchUniversity of KarlsruheKarlsruheGermany

Personalised recommendations