Linkage Learning, Rule Representation, and the χ-Ary Extended Compact Classifier System

  • Xavier Llorà
  • Kumara Sastry
  • Cláudio F. Lima
  • Fernando G. Lobo
  • David E. Goldberg
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4998)


This paper reviews a competent Pittsburgh LCS that automatically mines important substructures of the underlying problems and takes problems that were intractable with first-generation Pittsburgh LCS and renders them tractable. Specifically, we propose a χ-ary extended compact classifier system (χeCCS) which uses (1) a competent genetic algorithm (GA) in the form of χ-ary extended compact genetic algorithm, and (2) a niching method in the form restricted tournament replacement, to evolve a set of maximally accurate and maximally general rules. Besides showing that linkage exist on the multiplexer problem, and that χeCCS scales exponentially with the number of address bits (building block size) and quadratically with the problem size, this paper also explores non-traditional rule encodings. Gene expression encodings, such as the Karva language, can also be used to build χeCCS probabilistic models. However, results show that the traditional ternary encoding 0,1,# presents a better scalability than the gene expression inspired ones for problems requiring binary conditions.


Gene Expression Programming Expression Tree Rule Representation Niching Method Multiplexer Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Goldberg, D.E.: The Design of Innovation: Lessons from and for Competent Genetic Algorithms. Kluwer Academic Publishers, Norwell (2002)CrossRefzbMATHGoogle Scholar
  2. 2.
    Butz, M.V., Lanzi, P.L., Llorà, X., Goldberg, D.E.: Knowledge extraction and problem structure identification in XCS. Parallel Problem Solving from Nature - PPSN VIII 3242, 1051–1060 (2004)Google Scholar
  3. 3.
    Butz, M.V., Pelikan, M., Llorà, X., Goldberg, D.E.: Extracted global structure makes local building block processing effective in XCS. In: Proceedings of the 2005 Conference on Genetic and Evolutionary Computation, vol. 1, pp. 655–662 (2005)Google Scholar
  4. 4.
    Pelikan, M., Lobo, F., Goldberg, D.E.: A survey of optimization by building and using probabilistic models. Computational Optimization and Applications 21, 5–20 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Harik, G., Lobo, F., Goldberg, D.E.: The compact genetic algorithm. In: Proceedings of the IEEE International Conference on Evolutionary Computation, pp. 523–528 (1998) (Also IlliGAL Report No. 97006) Google Scholar
  6. 6.
    Llorà, X., Sastry, K., Goldberg, D.E.: The Compact Classifier System: Motivation, analysis, and first results. In: Proceedings of the Congress on Evolutionary Computation, vol. 1, pp. 596–603 (2005)Google Scholar
  7. 7.
    Kovacs, T.: Strength or Accuracy: Credit Assignment in Learning Classifier Systems. Springer, Heidelberg (2003)zbMATHGoogle Scholar
  8. 8.
    de la Ossa, L., Sastry, K., Lobo, F.G.: Extended compact genetic algorithm in C++: Version 1.1. IlliGAL Report No. 2006013, University of Illinois at Urbana-Champaign, Urbana, IL (March 2006)Google Scholar
  9. 9.
    Sastry, K., Goldberg, D.E.: Probabilistic model building and competent genetic programming. In: Riolo, R.L., Worzel, B. (eds.) Genetic Programming Theory and Practise, pp. 205–220. Kluwer Academic Publishers, Dordrecht (2003)CrossRefGoogle Scholar
  10. 10.
    Harik, G.R.: Finding multimodal solutions using restricted tournament selection. In: Proceedings of the Sixth International Conference on Genetic Algorithms, pp. 24–31 (1995) (Also IlliGAL Report No. 94002) Google Scholar
  11. 11.
    Wilson, S.W.: Classier Conditions Using Gene Expression Programming. IlliGAL Technical Report No. 2008001, Illinois Genetic Algorithms Laboratory, University of Illinois at Urbana-Champaign (2008)Google Scholar
  12. 12.
    Ferreira, C.: Gene expression programming: A new algorithm for solving problems. Complex Systems 13(2), 87–129 (2001)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Ferreira, C.: Gene Expression Programming: Mathematical Modeling by an Artificial Intelligence. Springer, Heidelberg (2006)zbMATHGoogle Scholar
  14. 14.
    Bernadó-Mansilla, E., Garrell-Guiu, J.M.: MOLeCS: A MultiObjective Learning Classifier System. In: Proceedings of the 2000 Conference on Genetic and Evolutionary Computation, vol. 1, p. 390 (2000)Google Scholar
  15. 15.
    Bernadó-Mansilla, E., Llorà, X., Traus, I.: MultiObjective Learning Classifier System. In: MultiObjective Machine Learning, pp. 261–288. Springer, Heidelberg (2005)Google Scholar
  16. 16.
    Harik, G.R., Lobo, F.G., Sastry, K.: Linkage learning via probabilistic modeling in the ECGA. In: Pelikan, M., Sastry, K., Cantú-Paz, E. (eds.) Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications. Springer, Heidelberg (2006) (Also IlliGAL Report No. 99010)Google Scholar
  17. 17.
    Goldberg, D.E., Korb, B., Deb, K.: Messy genetic algorithms: Motivation, analysis, and first results. Complex Systems 3(5), 493–530 (1989)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Wilson, S.: Classifier fitness based on accuracy. Evolutionary Computation 3(2), 149–175 (1995)CrossRefGoogle Scholar
  19. 19.
    Llorà, X., Sastry, K., Goldberg, D.E., Gupta, A., Lakshmi, L.: Combating user fatigue in iGAs: Partial ordering, support vector machines, and synthetic fitness. In: GECCO 2005: Proceedings of the 2005 conference on Genetic and evolutionary computation, 25-29 June 2005, vol. 2, pp. 1363–1370. ACM Press, Washington (2005)Google Scholar
  20. 20.
    Sastry, K., Goldberg, D.E.: Designing competent mutation operators via probabilistic model building of neighborhoods. In: Proceedings of the Genetic and Evolutionary Computation Conference, vol. 2, pp. 114–125 (2004) (Also IlliGAL Report No. 2004006)Google Scholar
  21. 21.
    Goldberg, D.E., Deb, K., Clark, J.H.: Genetic algorithms, noise, and the sizing of populations. Complex Systems 6, 333–362 (1992)zbMATHGoogle Scholar
  22. 22.
    Pelikan, M.: Hierarchical Bayesian Optimization Algorithm: Toward a New Generation of Evolutionary Algorithm. Springer, Berlin (2005)CrossRefzbMATHGoogle Scholar
  23. 23.
    Sastry, K., Abbass, H.A., Goldberg, D.E., Johnson, D.D.: Sub-structural niching in estimation of distribution algorithms. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 671–678 (2005)(Also IlliGAL Report No. 2005003)Google Scholar
  24. 24.
    Mahfoud, S.W.: Population size and genetic drift in fitness sharing. Foundations of Genetic Algorithms 3, 185–224 (1994)CrossRefGoogle Scholar
  25. 25.
    Llorà, X., Sastry, K., Yu, T.L., Goldberg, D.E.: Do not match, inherit: fitness surrogates for genetics-based machine learning techniques. In: GECCO 2007: Proceedings of the 9th annual conference on Genetic and evolutionary computation, pp. 1798–1805. ACM, New York (2007)Google Scholar
  26. 26.
    Lanzi, P.L.: Extending the Representation of Classifier Conditions Part I: From Binary to Messy Coding. In: Proceedings of the Genetic and Evolutinary Computation Conference (GECCO 1999), pp. 337–344. Morgan Kaufmann, San Francisco (1999)Google Scholar
  27. 27.
    Lanzi, P., Perrucci, A.: Extending the Representation of Classifier Conditions Part II: From Messy Coding to S-Expressions. In: Proceedings of the Genetic and Evolutinary Computation Conference (GECCO 1999), pp. 345–352. Morgan Kaufmann, San Francisco (1999)Google Scholar
  28. 28.
    Koza, J.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)zbMATHGoogle Scholar
  29. 29.
    Llorà, X., Alías, F., Formiga, L., Sastry, K., Goldberg, D.E.: Evaluation consistency in iGAs: User contradictions as cycles in partial-ordering graphs. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2006), vol. 1, pp. 865–868 (2006) (Also as IlliGAL TR No. 2005022)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Xavier Llorà
    • 1
  • Kumara Sastry
    • 2
  • Cláudio F. Lima
    • 3
  • Fernando G. Lobo
    • 3
  • David E. Goldberg
    • 2
  1. 1.National Center for Supercomputer ApplicationsUniversity of Illinois at Urbana-ChampaignUSA
  2. 2.Illinois Genetic Algorithms LaboratoryUniversity of Illinois at Urbana-ChampaignUSA
  3. 3.Informatics Laboratory (UALG-ILAB), Dept. of Electronics and Computer Science EngineeringUniversity of AlgarveFaroPortugal

Personalised recommendations