Resolution of measurability problems in discrete — time stochastic control
The formulation of dynamic programming as given by Bellman did not address measurability difficulties. Blackwell later initiated a study of these problems in Polish spaces, and Blackwell, Freedman and Orkin subsequently showed that by relaxing Borel measurability requirements on policies, some known heuristic results could be rigorously obtained. This paper presents three types of measurability restrictions on policies which allow rigorous proofs of all the basic existence results and characterizations of optimal and nearly optimal policies. These types of measurability, all more general than that considered by Blackwell, Freedman and Orkin, correspond to three σ-algebras in Polish spaces. The σ-algebras are constructed by radically different methods and the relationships between them are still unclear.
KeywordsDynamic Programming Optimal Control Problem Optimal Policy Analytic Subset Polish Space
Unable to display preview. Download preview PDF.
- R. Bellman, Dynamic Programming, Princeton Univ. Press, Princeton, New Jersey, 1957.Google Scholar
- D. P. Bertsekas and S. Shreve, Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York, 1978.Google Scholar
- D. Blackwell, "Borel-programmable functions," Ann. Prob. 6 (1978), 321–324.Google Scholar
- D. Blackwell, D. Freedman and M. Orkin, "The optimal reward operator in dynamic programming," Ann. Prob. 2 (1974), 926–941.Google Scholar
- L. D. Brown and R. Purves, "Measurable selections of extrema," Ann. Statist. 1 (1973), 902–912.Google Scholar
- F. Hausdorff, Set Theory, Chelsea, New York, 1957.Google Scholar
- K. Parthasarathy, Probability Measures on Metric Spaces, Academic Press, New York, 1967.Google Scholar
- S. Saks, Theory of the Integral, Stechert, New York, 1937.Google Scholar
- E. Selivanovskij, "Ob odnom klasse effektivnyh mnozestv (mnozestva C)", Mat. Sb. 35 (1928), 379–413.Google Scholar
- S. Shreve, "Probability measures and the C-sets of Selivanovskij", Pac. J. Math. (to appear).Google Scholar
- S. Shreve, "Borel-approachable functions," Working Paper, Dept. of Math. Sci., Univ. of Delaware, 1978.Google Scholar