DGOR pp 468-482 | Cite as

Linear Programming Methods for Solving Finite Markovian Decision Problems

  • A. Hordijk
  • L. C. M. Kallenberg
Conference paper
Part of the Operations Research Proceedings 1980 book series (ORP, volume 1980)


An overview is given of linear programming methods for solving (semi-) Markov decision problems.


Linear Programming Problem Extra Constraint Dual Program Linear Programming Method Unique Optimal Solution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. De Ghellinck, G.T. [ 1960 ]. Les problèmes de décisions sequentielles. Cahiers du Centre d’Etudes de Recherche Opérationelle, 2, 161–179.Google Scholar
  2. De Ghellinck, G.T. and G.D. Eppen [ 1967 ]. Linear programming solutions for separable Markovian decision problems. Management Science, 13, 371–394.CrossRefGoogle Scholar
  3. Denardo, E.V. and B.L. Fox [ 1968 ]. Multichain Markov renewal programs. Siam Journal on Applied Mathematics, 16, 468–487.CrossRefGoogle Scholar
  4. Denardo, E.V. and U.G. Rothblum [ 1979 ]. Optimal stopping, exponential utility, and linear programming. Mathematical Programming, 16, 228–244.CrossRefGoogle Scholar
  5. D’Epenoux, F. [1960]. Sur un problème de production et de stockage dans l’aléatoire. Revue Française de Recherche Opérationelle, 14, 3–16.Google Scholar
  6. Derman, C. [ 1970 ]. Finite state Markovian decision processes. Academic Press, New York.Google Scholar
  7. Derman, C. and A.F. Veinott Jr. [ 1972 ]. Constrained Markov decision chains. Management Science, 19, 389–390.CrossRefGoogle Scholar
  8. Heilmann,W.-R. [ 1977 ]. Lineare Programmierung stochastischer dynamischer Entscheidungsmodelle. Thesis, University of Hamburg.Google Scholar
  9. Heilmann,W.-R. [ 1978 ]. Solving stochastic dynamic programming problems by linear programming - An annotated bibliography. Zeitschrift für Operations Research 22, 43–53.CrossRefGoogle Scholar
  10. Hordijk, A. and L.C.M. Kallenberg [ 1979 ]. Linear programming and Markov decision chains. Management Science, 25, 352–362.CrossRefGoogle Scholar
  11. Kallenberg, L.C.M. [ 1980 ]. Linear programming and finite Markovian control problems Thesis, University of Leiden.Google Scholar
  12. Manne, A.S. [1960]. Linear programming and sequential decisions. Management Science, 6, 259–267.CrossRefGoogle Scholar
  13. Mine, H. and S. Osaki [ 1970 ]. Markovian decision processes. Elsevier, New York.Google Scholar
  14. Schweitzer, P.J. [ 1971 ]. Iterative solution of the functional equations of undiscounted Markov renewal programming. Journal of Mathematical Analysis and Applications, 34, 495–501.CrossRefGoogle Scholar
  15. Wessels, J. and J.A.E.E. van Nunen [ 1975 ]. Discounted semi-Markov decision processes: linear programming and policy iteration. Statistica Neerlandica, 29, 1–7.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1981

Authors and Affiliations

  • A. Hordijk
    • 1
  • L. C. M. Kallenberg
    • 1
  1. 1.Department of MathematicsUniversity of LeidenLeidenThe Netherlands

Personalised recommendations