Extensions

Abstract

Problems TDC and DC constitute special cases of a more general class of partially observable Markov decision processes. This class of processes may be loosely characterized as follows. A DM observes a sequence oi random variables X(t):t є T, where X(t) denotes the random variable observed at time t and t є T It is assumed that X(t) has a known distribution function F _s (t), which depends on the unknown state at time t Additionally, it is assumed that S(t): єT is a Markov process and that for a given value of S(t), the process X(t) consists of a sequence of independent random variables. The DM is required to make sequential decisions to optimally control (with respect to a given loss structure, an objective criterion, and well-defined constraints) the process S(t), while observing only X(t).