Do We Really Need to Estimate Rule Utilities in Classifier Systems?
- 662 Downloads
Classifier systems have traditionally used explicit measures of utility (strength, predicted payoff, accuracy, etc.) to quantify the performance and fitness of classifier rules. Much of the effort in designing and implementing classifier systems has focused on getting these utilities “right”. One alternative worth exploring is the idea of using endogenous fitness; that is, reinforcing successful performance with “resources” that rules need in order to reproduce. Under this regime, the best rules are those that accumulate the most resources over their lifetime and, consequently, have the most offspring. This paper describes a classifier system designed along these lines. Rules have no associated utility measure. Instead, each rule has one or more reservoirs that can be used to store resources. When enough resources have been accumulated, a rule utilizes some of its resources to reproduce and the reservoir level is reduced accordingly. Preliminary tests of this system on the multiplexor problem show that it performs as well as utility-based classifier systems such as XCS.
KeywordsReinforcement Event Reservoir Level Input String Average Reward Rule Discovery
Unable to display preview. Download preview PDF.
- 1.Booker, L. B. Classifier systems that learn internal world models. Machine Learning 3 (1988), 161–192.Google Scholar
- 2.Booker, L. B. Triggered rule discovery in classifier systems. In Proceedings of the Third International Conference on Genetic Algorithms (ICGA89) (Fairfax, VA, 1989), J. D. Schaffer, Ed., Morgan Kaufmann, pp. 265–274.Google Scholar
- 3.Holland, J. H. Adaptation. In Progress in theoretical biology, R. Rosen and F. M. Snell, Eds., vol. 4. Academic Press, New York, 1976.Google Scholar
- 4.Holland, J. H. Escaping brittleness: The possibilities of general-purpose learning algorithms applied to parallel rule-based systems. In Machine learning: An artificial intelligence approach, R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, Eds., vol. II. Morgan Kaufmann, Los Altos, CA, 1986, ch. 20, pp. 593–623.Google Scholar
- 5.Holland, J. H. Echoing emergence: Objectives, rough definitions, and speculations for Echo-class models. In Complexity: Metaphors, Models, and Reality, G. Cowan, D. Pines, and D. Melzner, Eds., vol. XIX of Santa Fe Institute Studies in the Sciences. of Complexity. Addison-Wesley, Reading, MA, 1994, pp. 309–342.Google Scholar