Applying a restricted mating policy to determine state space niches using immediate and delayed reinforcement

Abstract

Approaches for rule based control often rely heavily on the pre-classification of the state space for their success. In the pre-determined regions individual or groups of rules may be learned. Clearly, the success of such strategies depends on the quality of the partitioning of the state space. When no such a priori partitioning is available it is a significantly more difficult task to learn an appropriate division of the state space as well as the associated rules. Yet another layer of potential difficulty is the nature of the reinforcement applied to the rules since it is not always possible to generate an immediate reinforcement signal to supply judgement on the efficacy of activated rules. One approach to combine the joint goals of determining partitioning of the state space and discovery of associated appropriate rules is to use a genetic algorithm employing a restricted mating policy to generate rule clusters which dominate regions of the state space thereby effecting the required partioning. Such rale clusters are termed niches. A niching algorithm, which includes a ‘creche’ facility to protect ‘inexperienced’ classifiers, and the results of determining a simple 2D state space using an immediate reward scheme are presented. Details of how the algorithm may modified to incorporate a delayed reinforcement scheme on a real-world beam balancing control problem are reported.