Solving a Dynamic Program by Linear Programming — General State and Action Spaces

Conference paper
Part of the Proceedings in Operations Research 7 book series (ORP, volume 1977)


In the past two decades it has been shown by various authors that stochastic dynamic programming problems with finite state and action spaces and different notions of optimality can be solved by linear programming. In the present lecture it will bedemonstrated that dynamic programs with general state and action spaces can be treated by a generalized linear programming approach in the discounted reward case, in the average reward case, and in the optimal stopping case.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    C. Derman. Finite State Markovian Decision Processes. Academic Press. New York 1970.Google Scholar
  2. [2]
    U. Dieter. Optimierungsaufgaben in topologischen Vektorräumen I: Dualitätstheorie. Zeitschr.f.Wahrsch.Theorie u. verw. Geb. 5, 1966, 89–117.CrossRefGoogle Scholar
  3. [3]
    F. d’Epenoux. A Probabilistic Production and Inventory Problem. Man.Sci. 10, 1963, 98–108.CrossRefGoogle Scholar
  4. [4]
    G.T. de Ghellinck and G.D. Eppen. Linear Programming Solutions for Separable Markovian Decision Problems. Man.Sci. 13, 1967, 371–394.CrossRefGoogle Scholar
  5. [5]
    R.C. Grinold. Continuous Programming. Part One: Linear Objectives. Journ.Math.Anal.Appl.28, 1969, 32–51.CrossRefGoogle Scholar
  6. [5]
    R.C. Grinold. Continuous Programming. Part Two: Nonlinear Objectives. Journ.Math.Anal.App1.27, 1969, 639–655.CrossRefGoogle Scholar
  7. [6]
    W.-R. Heilmann. Solving a General Discounted Dynamic Program by Linear Programming, 1977. To be published.Google Scholar
  8. [7]
    W.-R. Heilmann. Generalized Linear Programming in Markovian Decision Problems, 1977. To be published in Bonner Mathematische Schriften.Google Scholar
  9. [8]
    K. Hinderer. Foundations of Non-stationary Dynamic Programming with Discrete Time Parameter. Springer-Verlag. Berlin-Heidelberg-New York 1970.CrossRefGoogle Scholar
  10. [9]
    K. Hinderer. Grundbegriffe der Wahrscheinlichkeitstheorie. Springer-Verlag. Berlin-Heidelberg-New York 1972.CrossRefGoogle Scholar
  11. [10]
    C. Kallina and A.C. Williams. Linear Programming in Reflexive Spaces. SIAM Review 13, 1971, 350–376.CrossRefGoogle Scholar
  12. [11]
    W. Krabs. Lineare Optimierung in halbgeordneten Vektorraumen. Num.Math. 11, 1968, 220–231.CrossRefGoogle Scholar
  13. [12]
    W. Krabs. Zur Dualitätstheorie bei linearen Optimierungsaufgaben in halbgeordneten Vektorräumen. Math.Zeitschr. 121, 1971, 320–328.CrossRefGoogle Scholar
  14. [13]
    W. Krabs. Optimierung und Approximation. Teubner-Verlag. Stuttgart 1975.Google Scholar
  15. [14]
    K.S. Kretschmer. Programmes in Paired Spaces. Canad.Journ. Math. 13, 1961, 221–238.CrossRefGoogle Scholar
  16. [15]
    A. S. Manne. Linear Programming and Sequential Decisions. Man. Sci. 6, 1960, 259–267.CrossRefGoogle Scholar
  17. [16]
    U. Rieder. On Stopped Decision Processes with Discrete Time Parameter. Stoch.Proc. and their Appl. 3, 1975, 365–383.CrossRefGoogle Scholar
  18. [17]
    S.M. Ross. Arbitrary State Markovian Decision Processes. Ann. Math.Stat. 39, 1968, 412–423.CrossRefGoogle Scholar
  19. [18]
    M. Schechter. Duality in Continuous Linear Programming. Journ. Math.Anal.Appl. 37, 1972, 130–141.CrossRefGoogle Scholar
  20. [19]
    M. Schechter. Linear Programs in Topological Vector Spaces. Journ.Math.Anal.Appl. 37, 1972, 492–500.CrossRefGoogle Scholar
  21. [20]
    R.M. van Slyke and R.J.-B. Wets. A Duality Theory for Abstract Mathematical Programs with Applications to Optimal Control Theory. Journ.Math.Anal.Appl. 22, 1968, 679–706.CrossRefGoogle Scholar
  22. [21]
    H.M. Wagner. On the Optimality of Pure Strategies. Man.Sci. 6, 1960, 268–269.CrossRefGoogle Scholar
  23. [22]
    K. Yamada. Duality Theorem in Markovian Decision Problems. Journ.Math.Anal.Appl. 50, 1975, 579–595.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1978

Authors and Affiliations

  1. 1.HamburgGermany

Personalised recommendations