Resolution of measurability problems in discrete — time stochastic control

  • Steven E. Shreve
Part II: Research Reports
Part of the Lecture Notes in Control and Information Sciences book series (LNCIS, volume 16)


The formulation of dynamic programming as given by Bellman did not address measurability difficulties. Blackwell later initiated a study of these problems in Polish spaces, and Blackwell, Freedman and Orkin subsequently showed that by relaxing Borel measurability requirements on policies, some known heuristic results could be rigorously obtained. This paper presents three types of measurability restrictions on policies which allow rigorous proofs of all the basic existence results and characterizations of optimal and nearly optimal policies. These types of measurability, all more general than that considered by Blackwell, Freedman and Orkin, correspond to three σ-algebras in Polish spaces. The σ-algebras are constructed by radically different methods and the relationships between them are still unclear.


Dynamic Programming Optimal Control Problem Optimal Policy Analytic Subset Polish Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    R. Bellman, Dynamic Programming, Princeton Univ. Press, Princeton, New Jersey, 1957.Google Scholar
  2. [2]
    D. P. Bertsekas and S. Shreve, Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York, 1978.Google Scholar
  3. [3]
    D. Blackwell, "Borel-programmable functions," Ann. Prob. 6 (1978), 321–324.Google Scholar
  4. [4]
    D. Blackwell, D. Freedman and M. Orkin, "The optimal reward operator in dynamic programming," Ann. Prob. 2 (1974), 926–941.Google Scholar
  5. [5]
    L. D. Brown and R. Purves, "Measurable selections of extrema," Ann. Statist. 1 (1973), 902–912.Google Scholar
  6. [6]
    F. Hausdorff, Set Theory, Chelsea, New York, 1957.Google Scholar
  7. [7]
    K. Parthasarathy, Probability Measures on Metric Spaces, Academic Press, New York, 1967.Google Scholar
  8. [8]
    S. Saks, Theory of the Integral, Stechert, New York, 1937.Google Scholar
  9. [9]
    E. Selivanovskij, "Ob odnom klasse effektivnyh mnozestv (mnozestva C)", Mat. Sb. 35 (1928), 379–413.Google Scholar
  10. [10]
    S. Shreve, "Probability measures and the C-sets of Selivanovskij", Pac. J. Math. (to appear).Google Scholar
  11. [11]
    S. Shreve, "Borel-approachable functions," Working Paper, Dept. of Math. Sci., Univ. of Delaware, 1978.Google Scholar

Copyright information

© Springer-Verlag 1979

Authors and Affiliations

  • Steven E. Shreve
    • 1
  1. 1.Department of Mathematical SciencesUniversity of DelawareNewark

Personalised recommendations