Bounded Controls

Abstract

This is the first chapter that is concerned with optimal control. The general heavy traffic problem that leads to the controlled reflected stochastic differential equation with bounded controls is treated, with both the discounted and the ergodic cost criteria. The model is the heavy traffic analogue of the classical control problem where the controls are bounded, and the weak-sense limit system is (1.1) or (1.2). We will use the canonical model (1.3) for the physical system, and assumptions of a general nature are made. It is supposed that the dynamical equation for the application can be manipulated into this general form. The assumptions are tailored to what would be typical in applications, so that they do fit a large number of cases. Random jumps can be added to (1.1) and (1.2) and vacations (or other jump-causing phenomena) incorporated in (1.3) with little change in the development, as noted at the end of Section 1. It will be shown that the optimal costs for the physical problem converge to the optimal cost for the weak-sense limit system and that (loosely speaking) the optimal physical processes converge to the optimal weak-sense limit process. Also, “nice” nearly optimal controls for the weak-sense limit system are nearly optimal for the physical system, under heavy traffic. Such results justify using the weak-sense limit system for the design and analysis of nearly optimal physical systems under heavy traffic.