An averaging method for the analysis of adaptive systems with small adjustment rate

Abstract

An averaging method for proving weak convergence of a sequence of non-Markovian processes to a diffusion, together with an averaged Liapunov function stochastic stability techniquè, are applied to an automata model for route selection in telephone routing. The model is chosen because it is a prototype of a large class to which the methods can be applied. A useful method for applying the basic theorems to such processes is illustrated. Suitably interpolated and normalized "learning or adaptive" processes converge weakly to a diffusion, as the "learning or adaption" rate goes to zero. For small learning rate, the qualitative properties (e.g., asymptotic (large-time) variances and parametric dependence) of the processes can be determined from the properties of the limit. The general approach can be used to study adaptive routing methods in computer and other networks, as well as the asymptotic properties of stochastic difference equations.