Introduction: Applications and Issues

Abstract

This is the first of three chapters describing many concrete applications of stochastic approximation. The emphasis is on the problem description. Proofs of convergence and the derivation of the rate of convergence will be given in subsequent chapters for many of the examples. Since the initial work of Robbins and Monro in 1951, there has been a steady increase in the investigations of applications in many diverse areas, and this has accelerated in recent years, with new applications arising in queueing networks and manufacturing systems and in learning problems and neural nets, among others. We present only selected samples of these applications to illustrate the great breadth. The basic stochastic approximation algorithm is nothing but a stochastic difference equation with a small step size, and the basic questions for analysis concern its qualitative behavior over a long time interval, such as convergence and rate of convergence. The wide range of applications leads to a wide variety of such equations and associated stochastic processes.