Weighted Mixture of Models For On-Line Learning

Abstract

This paper proposes a weighted mixture of locally generalizing models in an attempt to resolve the trade-off between model mismatch and measurement noise given a sparse set of training samples so that the conditional mean estimation performance of the desired response can be made adequate over the input region of interest. In this architecture, each expert model has its corresponding variance model for estimating the expert’s modeling performance. Parameters associated with individual expert models are adapted in the usual least-mean-square sense, weighted by its variance model output. Whereas the variance models are adapted in such a way that expert models of higher-resolution (or greater modeling capability) are discouraged to contribute, except when the local modeling performance becomes inadequate.