The Ergodic Cost Problem: Formulation and Algorithms

Abstract

In this chapter, we reformulate some of the concepts in the last chapter so that they can be used on the ergodic cost problem. Before doing that it is useful to discuss the appropriate dynamic programming equations and some additional background material. The natural state spaces for control problems that are of interest over a long time interval are often unbounded, and they must be truncated for numerical purposes. One standard way of doing this involves a reflecting boundary, and this is the case dealt with in this chapter. Thus, there are no absorbing states. The basic process is the controlled diffusion (5.3.1) or jump diffusion (5.6.1). The approximating Markov chains {ξ ^h _n , n < ∞ } will be locally consistent with these processes. As in the previous chapters, S _h denotes the state space, and ∂ G ⁺ _h  ⊂ S _h the set of reflecting states. Recall that the reflection is instantaneous and that we use Δt ^h(x,α) = 0 for x ∈ ∂ G ⁺ _h .