Skip to main content
SpringerLink
Log in

High Classification Accuracy Does Not Imply Effective Genetic Search

  • Conference paper
Genetic and Evolutionary Computation – GECCO 2004 (GECCO 2004)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 3103))

Included in the following conference series:

Abstract

Learning classifier systems, their parameterisation, and their rule discovery systems have often been evaluated by measuring classification accuracy on small Boolean functions. We demonstrate that by restricting the rule set to the initial random population high classification accuracy can still be achieved, and that relatively small functions require few rules. We argue this demonstrates that high classification accuracy on small functions is not evidence of effective rule discovery. However, we argue that small functions can nonetheless be used to evaluate rule discovery when a certain more powerful type of metric is used.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Albus, J.S.: A new approach to manipulator control: the cerebellar model articulation controller (CMAC). Journal of dynamic systems, measurement and control 97(3) (1975)

    Google Scholar 

  2. Anderson, C.W.: Learning and Problem solving with multilayer connectionist systems. PhD thesis, University of Massachussetts, Amherst, MA, USA (1986)

    Google Scholar 

  3. Barry, A.: XCS Performance and Population Structure within Multiple-Step Environments. PhD thesis, Queens University Belfast (2000)

    Google Scholar 

  4. Barto, G., Anandan, P., Anderson, C.W.: Cooperativity in networks of pattern recognizing stochastic learning automata. In: Proceedings of the Fourth Yale Workshop on Applications of Adaptive Systems Theory, pp. 85–90 (1985)

    Google Scholar 

  5. Bonelli, P., Parodi, A., Sen, S., Wilson, S.: NEWBOOLE: A Fast GBML System. In: International Conference on Machine Learning, San Mateo, California, pp. 153–159. Morgan Kaufmann, San Francisco (1990)

    Google Scholar 

  6. Booker, L.B.: Triggered rule discovery in classifier systems. In: Schaffer, D. (ed.) Proc. 3rd International Conference on Genetic Algorithms (ICGA- 1989), George Mason University, June 1989, pp. 265–274. Morgan Kaufmann, San Francisco (1989)

    Google Scholar 

  7. Butz, M., Kovacs, T., Lanzi, P.L., Wilson, S.W.: Toward a theory of generalization and learning in xcs. To appear in the IEEE Transactions on Evolutionary Computation (2004)

    Google Scholar 

  8. Butz, M.V., Goldberg, D.E., Stolzmann, W.: Investigating Generalization in the Anticipatory Classifier System. In: Deb, K., Rudolph, G., Lutton, E., Merelo, J.J., Schoenauer, M., Schwefel, H.-P., Yao, X. (eds.) PPSN 2000. LNCS, vol. 1917, Springer, Heidelberg (2000) Also technical report 2000014 of the Illinois Genetic Algorithms Laboratory

    Google Scholar 

  9. Cribbs III, H.B., Smith, R.E.: What Can I do with a Learning Classifier System? In: Karr, C., Freeman, L.M. (eds.) Industrial Applications of Genetic Algorithms, pp. 299–320. CRC Press, Boca Raton (1998)

    Google Scholar 

  10. de Boer, B.: Classifier Systems: a useful approach to machine learning? Master’s thesis, Leiden University (1994)

    Google Scholar 

  11. De Jong, K.A., Spears, W.M.: Learning Concept Classification Rules Using Genetic Algorithms. In: Proceedings of the International Joint Conference on Artificial Intelligence, Sidney, Australia, pp. 651–656 (1991)

    Google Scholar 

  12. Dixon, P.W., Corne, D.W., Oates, M.J.: A preliminary investigation of modified XCS as a generic data mining tool. In: Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds.) IWLCS 2001. LNCS (LNAI), vol. 2321, pp. 133–150. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  13. Goldberg, D.E.: Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Reading (1989)

    MATH  Google Scholar 

  14. Greene, D., Smith, S.: COGIN: Symbolic induction using genetic algorithms. In: Proceedings 10th National Conference on Artificial Intelligence, pp. 111–116. Morgan Kaufmann, San Francisco (1992)

    Google Scholar 

  15. Greene, D., Smith, S.: Using Coverage as a Model Building Constraint in Learning Classifier Systems. Evolutionary Computation 2(1), 67–91 (1994)

    Article  Google Scholar 

  16. Holmes, J.H.: Evolution-Assisted Discovery of Sentinel Features in Epidemiologic Surveillance. PhD thesis, Drexel University (1996)

    Google Scholar 

  17. Jacobs, R.A.: Increased rates of convergence through learning rate adaptation. Neural Networks 1, 295–307 (1988)

    Article  Google Scholar 

  18. Kovacs, T.: Evolving Optimal Populations with XCS Classifier Systems. Master’s thesis, University of Birmingham (1996)

    Google Scholar 

  19. Kovacs, T.: XCS Classifier System Reliably Evolves Accurate, Complete, and Minimal Representations for Boolean Functions. In: Roy, Chawdhry, Pant (eds.) Soft Computing in Engineering Design and Manufacturing, pp. 59–68. Springer, London (1997)

    Google Scholar 

  20. Kovacs, T.: Deletion schemes for classifier systems. In: Banzhaf, W., Daida, J., Eiben, A.E., Garzon, M.H., Honavar, V., Jakiela, M., Smith, R.E. (eds.) GECCO-1999: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 329–336. Morgan Kaufmann, San Francisco (1999)

    Google Scholar 

  21. Kovacs, T.: What should a classifier system learn? In: Proc. of the 2001 Congress on Evolutionary Computation (CEC 2001), pp. 775–782. IEEE Press, Los Alamitos (2001)

    Chapter  Google Scholar 

  22. Kovacs, T.: Strength or Accuracy: Credit Assignment in Learning Classifier Systems. Springer, Heidelberg (2004)

    MATH  Google Scholar 

  23. Kovacs, T., Kerber, M.: What makes a problem hard for XCS? In: Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds.) IWLCS 2000. LNCS (LNAI), vol. 1996, pp. 80–99. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  24. Li, M., Vitányi, P.: An Introduction to Kolmogorov Complexity and Its Applications. Springer, Heidelberg (1997)

    MATH  Google Scholar 

  25. Liepins, G.E., Wang, L.A.: Classifier System Learning of Boolean Concepts. In: Belew, R.K., Booker, L.B. (eds.) Proceedings of the Fourth International Conference on Genetic Algorithms (ICGA-1991), San Mateo, CA, pp. 318–323. Morgan Kaufmann Publishers, San Francisco (1991)

    Google Scholar 

  26. Merz, C., Murphy, P.M.: UCI Repository of Machine Learning Databases, http://www.ics.uci.edu/~mlearn/MLRepository.html University of California, Department of Information and Computer Science, Irvine (1997)

  27. Mitchell, T.M.: Machine Learning. McGraw–Hill, New York (1997)

    MATH  Google Scholar 

  28. Oblow, E.M.: Implementing Valiant’s Learnability Theory using Random Sets. Technical Report ORNL/TM-11512R, Oak Ridge National Laboratory (1990)

    Google Scholar 

  29. Pagallo, G., Haussler, D.: Boolean feature discovery in empirical learning. Machine Learning 5(1), 71–100 (1990)

    Article  Google Scholar 

  30. Quinlan, J.R.: An empirical comparison of genetic and decision-tree classifiers. In: Proc. of the Fifth Int. Machine Learning Conference, pp. 135–141 (1988)

    Google Scholar 

  31. Saxon, S., Barry, A.M.: XCS and the Monk’s Problems. In: Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds.) IWLCS 1999. LNCS (LNAI), vol. 1813, pp. 223–242. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  32. Schaffer, C.: A conservation law for generalization performance. In: Hirsh, H., Cohen, W.W. (eds.) Machine Learning: Proc. of the Eleventh International Conference, pp. 259–265. Morgan Kaufmann, San Francisco (1994)

    Google Scholar 

  33. Sen, S.: Classifier system learning of multiplexer function. The University of Alabama, Tuscaloosa, Alabama. Class Project (1988)

    Google Scholar 

  34. Smith, R.E., Brown Cribbs, H.: Is a Learning Classifier System a Type of Neural Network? Evolutionary Computation 2(1), 19–36 (1994)

    Article  Google Scholar 

  35. Wang, L.A.: Classifier System Learning of the Boolean Multiplexer Function. Master’s thesis, Computer Science Dept., University of Tennessee, Knoxville (1990)

    Google Scholar 

  36. Wilson, S.W.: Classifier Systems and the Animat Problem. Machine Learning 2, 199–228 (1987)

    Google Scholar 

  37. Wilson, S.W.: Bid competition and specificity reconsidered. Complex Systems 2, 705–723 (1989)

    Google Scholar 

  38. Wilson, S.W.: Perceptron Redux: Emergence of Structure. Physica D, 249–256 (1990)

    Google Scholar 

  39. Wilson, S.W.: Classifier Fitness Based on Accuracy. Evolutionary Computation 3(2), 149–175 (1995)

    Article  Google Scholar 

  40. Wilson, S.W.: Generalization in the XCS classifier system. In: Koza, J., et al. (eds.) Genetic Programming 1998: Proceedings of the Third Annual Conference, pp. 665–674. Morgan Kaufmann, San Francisco (1998)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Bristol, Bristol, BS8 1UB, U.K

    Tim Kovacs

  2. University of Birmingham, Birmingham, B15 2TT, U.K.

    Manfred Kerber

Authors
  1. Tim Kovacs
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manfred Kerber
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Indian Institute of Technology Kanpur, Department of Mechanical Engineering, Kanpur Genetic Algorithms Laboratory (KanGAL), Kanpur, 208016, Uttar Pradesh, India

    Kalyanmoy Deb

Rights and permissions

Reprints and permissions

Copyright information

© 2004 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kovacs, T., Kerber, M. (2004). High Classification Accuracy Does Not Imply Effective Genetic Search. In: Deb, K. (eds) Genetic and Evolutionary Computation – GECCO 2004. GECCO 2004. Lecture Notes in Computer Science, vol 3103. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24855-2_93

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-24855-2_93

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-22343-6

  • Online ISBN: 978-3-540-24855-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation