Dynamics of On-Line Learning in Radial Basis Function Neural Networks

Abstract

We present a method for analyzing the behavior of RBFs in an on-line scenario which provides a description of the learning dynamics without invoking the thermodynamic limit. Our analysis is based on a master equation that describes the dynamics of the weight space probability density for any value of the input space dimension. Because the transition probability appearing in the master equation cannot be written in closed form, some approximate form of the dynamics is developed. We assume a arbitrary small learning rate (small noise) and we derive in this limit the dynamic evolution of the means and the variances of the net weights. The analytic results are then confirmed by simulations.