Strength or Accuracy? Fitness Calculation in Learning Classifier Systems
- 708 Downloads
Wilson’s XCS is a clear departure from earlier classifier systems in terms of the way it calculates the fitness of classifiers for use in the genetic algorithm. Despite the growing body of work on XCS and the advantages claimed for it there has been no detailed comparison of XCS and traditional strength-based systems. This work takes a step towards rectifying this situation by surveying a number of issues related to the change in fitness. I distinguish different definitions of overgenerality for strength and accuracy-based fitness and analyse some implications of the use of accuracy, including an apparent advantage in addressing the explore/exploit problem. I analyse the formation of strong overgenerals, a major problem for strength-based systems, and illustrate their dependence on biased reward functions. I consider motivations for biasing reward functions in single step environments, and show that non-trivial multi step environments have biased Q-functions. I conclude that XCS’s accuracy-based fitness appears to have a number of significant advantages over traditional strength-based fitness.
Keywordsstrong overgeneral classifiers biased reward functions accuracy-based fitness XCS complete covering maps exploration
Unable to display preview. Download preview PDF.
- 3.Pier Luca Lanzi and Marco Colombetti. An Extension of XCS to Stochastic Environments. In W. Banzhaf et al., editors, GECCO-99: Proceedings of the Genetic and Evolutionary Computation Conference, pages 353–360. Morgan Kaufmann, 1999.Google Scholar
- 4.Tim Kovacs. XCS Classifier System Reliably Evolves Accurate, Complete, and Minimal Representations for Boolean Functions. In Roy, Chawdhry, and Pant, editors, Soft Computing in Engineering Design and Manufacturing, pages 59–68. Springer-Verlag, 1997. ftp://ftp.cs.bham.ac.uk/pub/authors/T.Kovacs/index.html
- 5.Stewart W. Wilson. Generalization in the XCS classifier system. In J. Koza et al., editors, Genetic Programming 1998: Proceedings of the Third Annual Conference, pages 665–674. Morgan Kaufmann, 1998. http://prediction-dynamics.com/
- 6.Tim Kovacs. Evolving Optimal Populations with XCS Classifier Systems. Technical Report CSR-96-17 and CSRP-96-17, School of Computer Science, Uni. of Birmingham, 1996. ftp://ftp.cs.bham.ac.uk/pub/tech-reports/1996/CSRP-96-17.ps.gz
- 8.David E. Goldberg. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, 1989.Google Scholar
- 9.Andrea Bonarini. Evolutionary Learning of Fuzzy rules: competition and cooperation. In W. Pedrycz, editor, Fuzzy Modelling: Paradigms and Practice, pages 265–284. Kluwer Academic Press, 1996.Google Scholar
- 10.Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press, 1998.Google Scholar
- 11.Mark Humphrys. Action Selection Methods using Reinforcement Learning. PhD thesis, Cambridge University, 1997. http://www.compapp.dcu.ie/~humphrys/
- 12.Jonas Karlsson. Learning to Solve Multiple Goals. PhD thesis, University of Rochester, 1997. http://www.cs.rochester.edu/trs/ai-trs.html
- 13.Peter W. Frey and David J. Slate. Letter Recognition Using Holland-Style Adaptive Classifiers. Machine Learning, 6:161–182, 1991.Google Scholar
- 14.Adrian Hartley. Accuracy-based fitness allows similar performance to humans in static and dynamic classification environments. In W. Banzhaf et al., editors, GECCO-99: Proceedings of the Genetic and Evolutionary Computation Conference, pages 266–273. Morgan Kaufmann, 1999.Google Scholar