Introduction and summary

Abstract

The literature on stochastic dynamic programming has grown in the last 10 years rather vigorously. After the pioneering work of Bellman, Howard and others, the papers of Blackwell (65) and of Dynkin (65) were most important for the further development of the foundations, as far as the expected cost criterion is used. Blackwell was the first who gave a rigorous treatment for non-countable state spaces, whereas Dynkin studied non-stationary models (with countable state space). Another important paper for non-stationary models is Sirjaev (64). Strauch (66) extended the results of Blackwell to other stationary models. Hinderer (67) generalized the results of Blackwell (65) and Dynkin (65) to non-stationary models with general state space. Sirjaev (67) is a very useful review paper, which treats also the case of continuous time parameter. The most general sufficient condition for the existence of optimal policies (or plans, as we shall say) is due to Maitra (68).