Discrete-Parameter Controlled Stochastic Processes

Abstract

Let two sets X and U with σ-algebras of measurable subsets $\mathfrak{A}\;and\;\mathfrak{B}$ respectively, i.e. two measurable spaces $(X,\mathfrak{A})\;and\;(U,\mathfrak{B})$ be given. The first space is called the phase space of the basic process and the second the phase space of control. Let N be the set of non-negative integers. In this Chapter all the processes are defined on the set N. To define a controlled process it is necessary to define the probability distribution of a random process with values in X provided a sequence of controls at each instant of time is given and also to define a rule according to which these controls are selected. We shall now describe the components of a controlled process in a more precise manner.