Abstract

Learning Classifier Systems (LCS s) essentially combine fast approximation techniques with evolutionary optimization techniques. Despite their somewhat misleading name, LCSs are not only systems suitable for classification problems, but may be rather viewed as a very general, distributed optimization technique. Essentially, LCSs have very high potential to be applied in any problem domain that is best solved or approximated by means of a distributed set of local approximations, or predictions. The evolutionary component is designed to optimize a partitioning of the problem domain for generating maximally useful predictions within each subspace of the partitioning. The predictions are generated and adapted by the approximation technique. Generally any form of spatial partitioning and prediction are possible – such as a Gaussian-based partitioning combined with linear approximations, yielding a Gaussian mixture of linear predictions. In fact, such a solution is developed and optimized by XCSF (XCS for function approximation). The LCSs XCS (X classifier system) and the function approximation version XCSF, indeed, are probably the most well-known LCS architectures to date. Their optimization technique is very-well balanced with the approximation technique: as long as the approximation technique yields reasonably good solutions and evaluations of these solutions fast, the evolutionary component will pick-up on the evaluation signal and optimize the partitioning. This chapter provides historical background on LCSs. Then XCS and XCSF are introduced in detail providing enough information to be able to implement, understand, and apply these systems. Further LCS architectures are surveyed and their potential for future research and for applications is discussed. The conclusions provide an outlook on the many possible future LCS applications and developments.

Keywords

Attenuation Assure 
ACT-R

adaptive control of thought-rational

ALCS

anticipatory learning classifier system

CS1

cognitive system

EDA

estimation of distribution algorithm

GA

genetic algorithm

LCS

learning classifier system

LWPR

locally-weighted projection regression algorithm

RL

reinforcement learning

RLS

recursive least square

SARSA

state-action-reward-state-action

UCS

supervised classifier system

XACS

x-anticipatory classifier system

XCSF

XCS for function approximation

XCS

X classifier system

ZCS

zeroth level classifier system

References

  1. [47.1]
    J.H. Holland: Adaptation in Natural and Artificial Systems (Univ. of Michigan, Ann Arbor 1975)Google Scholar
  2. [47.2]
    J.H. Holland: Adaptation. In: Progress in Theoretical Biology, Vol. 4, ed. by R. Rosen, F.M. Snell (Academic, New York 1976) pp. 263–293CrossRefGoogle Scholar
  3. [47.3]
    L.B. Booker, D.E. Goldberg, J.H. Holland: Classifier systems and genetic algorithms, Artif. Intell. 40, 235–282 (1989)CrossRefGoogle Scholar
  4. [47.4]
    J.H. Holland, J.S. Reitman: Cognitive systems based on adaptive algorithms. In: Pattern Directed Inference Systems, ed. by D.A. Waterman, F. Hayes-Roth (Academic, New York 1978) pp. 313–329Google Scholar
  5. [47.5]
    L.P. Kaelbling, M.L. Littman, A.W. Moore: Reinforcement learning: A survey, J. Artif. Intell. Res. 4, 237–285 (1996)Google Scholar
  6. [47.6]
    R.S. Sutton, A.G. Barto: Reinforcement Learning: An Introduction (MIT Press, Cambridge 1998)Google Scholar
  7. [47.7]
    J.H. Holland: Properties of the bucket brigade algorithm, Proc. Int. Conf. Genet. Algorithms Appl. (1985) pp. 1–7Google Scholar
  8. [47.8]
    D.E. Goldberg: Genetic Algorithms in Search, Optimization and Machine Learning (Addison-Wesley, Reading 1989)MATHGoogle Scholar
  9. [47.9]
    S.F. Smith: A learning system based on genetic adaptive algorithms, Ph.D. Thesis (Univ. of Pittsburgh, Pittsburgh 1980)Google Scholar
  10. [47.10]
    K.A. De Jong: An analysis of the behavior of a class of genetic adaptive systems, Ph.D. Thesis (Univ. of Michigan, Ann Arbor 1975)Google Scholar
  11. [47.11]
    L.B. Booker: Intelligent behavior as an adaptation to the task environment, Ph.D. Thesis (The Univ. of Michigan, Ann Arbor 1982)Google Scholar
  12. [47.12]
    S.W. Wilson: Knowledge growth in an artificial animal, Proc. Int. Conf. Genet. Algorit. Appl. (1985) pp. 16–23Google Scholar
  13. [47.13]
    S.W. Wilson: Classifier systems and the animat problem, Mach. Learn. 2, 199–228 (1987)Google Scholar
  14. [47.14]
    D.E. Goldberg: Computer-aided gas pipeline operation using genetic algorithms and rule learning, Diss. Abstr. Int. 44, 3174B (1983)Google Scholar
  15. [47.15]
    K.A. De Jong: Learning with genetic algorithms: An overview, Mach. Learn. 3, 121–138 (1988)Google Scholar
  16. [47.16]
    K.A. De Jong, W.M. Spears, D.F. Gordon: Using genetic algorithms for concept learning, Mach. Learn. 13, 161–188 (1993)Google Scholar
  17. [47.17]
    R.L. Riolo: Bucket brigade performance: I. Long sequences of classifiers, Proc. 2nd Int. Conf. Genet. Algorithms (ICGA87), ed. by J.J. Grefenstette (Lawrence Erlbaum Associates, Cambridge 1987) pp. 184–195Google Scholar
  18. [47.18]
    R.E. Smith, H. Brown Cribbs: Is a learning classifier system a type of neural network?, Evol. Comput. 2, 19–36 (1994)CrossRefGoogle Scholar
  19. [47.19]
    J.H. Holland, L.B. Booker, M. Colombetti, M. Dorigo, D.E. Goldberg, S. Forrest, R.L. Riolo, R.E. Smith, P.L. Lanzi, W. Stolzmann, S.W. Wilson: What is a learning classifier system?, Lect. Notes Comput. Sci. 1813, 3–6 (2000)CrossRefGoogle Scholar
  20. [47.20]
    S.W. Wilson: ZCS: A zeroth level classifier system, Evol. Comput. 2, 1–18 (1994)CrossRefGoogle Scholar
  21. [47.21]
    S.W. Wilson: Classifier fitness based on accuracy, Evol. Comput. 3, 149–175 (1995)CrossRefGoogle Scholar
  22. [47.22]
    C.J.C.H. Watkins: Learning from delayed rewards, Ph.D. Thesis (King's College, Cambridge 1989)Google Scholar
  23. [47.23]
    J. Drugowitsch: Design and Analysis of Learning Classifier Systems: A Probabilistic Approach, Studies in Computational Intelligence (Springer, Berlin, Heidelberg 2008)Google Scholar
  24. [47.24]
    J. Drugowitsch, A. Barry: A formal framework and extensions for function approximation in learning classifier systems, Mach. Learn. 70, 45–88 (2008)CrossRefGoogle Scholar
  25. [47.25]
    M.V. Butz: Rule-Based Evolutionary Online Learning Systems: A Principled Approach to LCS Analysis and Design (Springer, Berlin, Heidelberg 2006)MATHGoogle Scholar
  26. [47.26]
    B. Widrow, M. Hoff: Adaptive switching circuits, West. Electron. Show Conv. 4, 96–104 (1960)Google Scholar
  27. [47.27]
    P.-Y. Oudeyer, F. Kaplan, V.V. Hafner: Intrinsic motivation systems for autonomous mental development, IEEE Trans. Evol. Comput. 11, 265–286 (2007)CrossRefGoogle Scholar
  28. [47.28]
    G. Venturini: Adaptation in dynamic environments through a minimal probability of exploration, from animals to animats 3, Proc. 3rd Int. Conf. Simul. Adapt. Behav. (1994) pp. 371–381Google Scholar
  29. [47.29]
    A. Orriols-Puig, E. Bernadó-Mansilla, D.E. Goldberg, K. Sastry, P.L. Lanzi: Facetwise analysis of XCS for problems with class imbalances, IEEE Trans. Evol. Comput. 13, 1093–1119 (2009)CrossRefGoogle Scholar
  30. [47.30]
    M.V. Butz, K. Sastry, D.E. Goldberg: Strong, stable, and reliable fitness pressure in XCS due to tournament selection, Genet. Program. Evol. Mach. 6, 53–77 (2005)CrossRefGoogle Scholar
  31. [47.31]
    M.V. Butz, M. Pelikan, X. Llorà, D.E. Goldberg: Automated global structure extraction for effective local building block processing in XCS, Evol. Comput. 14, 345–380 (2006)CrossRefGoogle Scholar
  32. [47.32]
    S.W. Wilson: Generalization in the XCS classifier system, genetic programming 1998, Proc. 3rd Ann. Conf. (1998) pp. 665–674Google Scholar
  33. [47.33]
    M.V. Butz, T. Kovacs, P.L. Lanzi, S.W. Wilson: Toward a theory of generalization and learning in XCS, IEEE Trans. Evol. Comput. 8, 28–46 (2004)CrossRefGoogle Scholar
  34. [47.34]
    T. Kovacs: XCS classifier system reliably evolves accurate, complete, and minimal representations for Boolean functions. In: Soft Computing in Engineering Design and Manufacturing, ed. by R. Roy, P.K. Chawdhry, R.K. Pant (Springer, Berlin, Heidelberg 1997) pp. 59–68Google Scholar
  35. [47.35]
    P.O. Stalph, X. Llorà, D.E. Goldberg, M.V. Butz: Resource management and scalability of the XCSF learning classifier system, Theor. Comput. Sci. 425, 126–141 (2012)MathSciNetCrossRefMATHGoogle Scholar
  36. [47.36]
    M.V. Butz, P.L. Lanzi: Sequential problems that test generalization in learning classifier systems, Evol. Comput. 2, 141–147 (2009)Google Scholar
  37. [47.37]
    L. Bull, E. Bernadó-Mansilla, J. Holmes (Eds.): Learning Classifier Systems in Data Mining, Studies in Computational Intelligence, Vol. 125 (Springer, Berlin, Heidelberg 2008)MATHGoogle Scholar
  38. [47.38]
    B. Schökopf, A.J. Smola: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (MIT Press, Cambridge 2001)Google Scholar
  39. [47.39]
    W. Liu, J.C. Principe, S. Haykin: Kernel Adaptive Filtering: A Comprehensive Introduction, 1st edn. (Wiley, Hoboken 2010)CrossRefGoogle Scholar
  40. [47.40]
    M.V. Butz, S.W. Wilson: An algorithmic description of XCS, Soft Comput. 6, 144–153 (2002)CrossRefMATHGoogle Scholar
  41. [47.41]
    S.W. Wilson: Get real! XCS with continuous-valued inputs. In: Festschrift in honor of John H. Holland, ed. by L. Booker, S. Forrest, M. Mitchell, R.L. Riolo (Center for the Study of Complex Systems, Ann Arbor 1999) pp. 111–121Google Scholar
  42. [47.42]
    S.W. Wilson: Get real! XCS with continuous-valued inputs, Lect. Notes Comput. Sci. 1813, 209–219 (2000)CrossRefGoogle Scholar
  43. [47.43]
    S.W. Wilson: Classifiers that approximate functions, Nat. Comput. 1, 211–234 (2002)MathSciNetCrossRefMATHGoogle Scholar
  44. [47.44]
    S. Haykin: Adaptive Filter Theory, 4th edn. (Prentice Hall, Upper Saddle River 2002)MATHGoogle Scholar
  45. [47.45]
    S. Vijayakumar, A. D'Souza, S. Schaal: Incremental online learning in high dimensions, Neural Comput. 17, 2602–2634 (2005)MathSciNetCrossRefGoogle Scholar
  46. [47.46]
    P. Stalph, J. Rubinsztajn, O. Sigaud, M.V. Butz: Function approximation with LWPR and XCSF: A comparative study, Evol. Comput. 5, 103–116 (2012)Google Scholar
  47. [47.47]
    M.V. Butz: Kernel-based, ellipsoidal conditions in the real-valued XCS classifier system, Proc. Genet. Evol. Comput. Conf. (GECCO 2005) (2005) pp. 1835–1842Google Scholar
  48. [47.48]
    C. Stone, L. Bull: For real! XCS with continuous-valued inputs, Evol. Comput. 11, 299–336 (2003)CrossRefGoogle Scholar
  49. [47.49]
    M.V. Butz, P.L. Lanzi, S.W. Wilson: Function Approximation With XCS: Hyperellipsoidal Conditions, Recursive Least Squares, and Compaction, IEEE Trans. Evol. Comput. 12, 355–376 (2008)CrossRefGoogle Scholar
  50. [47.50]
    D. Loiacono, P.L. Lanzi: Recursive least squares and quadratic prediction in continuous multistep problems, Lect. Notes Comput. Sci. 6471, 70–86 (2010)CrossRefGoogle Scholar
  51. [47.51]
    P.O. Stalph, M.V. Butz: Learning local linear Jacobians for flexible and adaptive robot arm control, Genet. Program. Evol. Mach. 13, 137–157 (2012)CrossRefGoogle Scholar
  52. [47.52]
    S. Schaal, C.G. Atkeson: Constructive incremental learning from only local information, Neural Comput. 10, 2047–2084 (1998)CrossRefGoogle Scholar
  53. [47.53]
    E. Bernadó-Mansilla, J.M. Garrell-Guiu: Accuracy-based learning classifier systems: Models, analysis, and applications to classification tasks, Evol. Comput. 11, 209–238 (2003)CrossRefGoogle Scholar
  54. [47.54]
    K. Bache, M. Lichman: UCI Machine Learning Repository (Univ. of California, School of Information and Computer Sciences 2013) http://archive.ics.uci.edu/ml
  55. [47.55]
    I.H. Witten, E. Frank: Data Mining. Practical Machine Learning Tools and Techniques with Java Implementations (Morgan Kaufmann, San Francisco 2000)Google Scholar
  56. [47.56]
    X. Llorà, J.M. Garrell: Knowledge independent data mining with fine-grained parallel evolutionary algorithms, Proc. Genet. Evol. Comput. Conf. (GECCO 2001) (2001) pp. 461–468Google Scholar
  57. [47.57]
    X. Llorà, J.M. Garrell: Inducing partially-defined instances with evolutionary algorithms, Proc. 18th Int. Conf. Mach. Learn. (ICML 2001) (2001)Google Scholar
  58. [47.58]
    E. Bernadó, X. Llorà, J.M. Garrell: XCS and GALE: A comparative study of two learning classifier systems and six other learning algorithms on classification tasks, Lect. Notes Comput. Sci. 2321, 115–132 (2002)CrossRefMATHGoogle Scholar
  59. [47.59]
    J. Bacardit, J.M. Garrell: Evolving multiple discretizations with adaptive intervals for a Pittsburgh rule-based learning classifier system, Lect. Notes Comput. Sci. 2724, 1818–1831 (2003)CrossRefMATHGoogle Scholar
  60. [47.60]
    J. Bacardit, M.V. Butz: Data mining in learning classifier systems: Comparing XCS with GAssist, Lect. Notes Comput. Sci. 4399, 282–290 (2007)CrossRefGoogle Scholar
  61. [47.61]
    J. Bacardit, M.V. Butz: Data mining in learning classifier systems: Comparing XCS with GAssist (IlliGAL, Univ. of Illinois at Urbana-Champign 2004)Google Scholar
  62. [47.62]
    J. Bacardit, N. Krasnogor: Empirical evaluation of ensemble techniques for a Pittsburgh learning classifier system, Lect. Notes Comput. Sci. 4998, 255–268 (2008)CrossRefGoogle Scholar
  63. [47.63]
    J. Bacardit, N. Krasnogor: Performance and efficiency of memetic Pittsburgh learning classifier systems, Evol. Comput. 17, 307–342 (2009)CrossRefGoogle Scholar
  64. [47.64]
    K. Sastry, D.E. Goldberg, X. Llorá: Towards billion-bit optimization via a parallel estimation of distribution algorithm, Proc. Genet. Evol. Comput. Conf. (GECCO 2007) (2007) pp. 577–584Google Scholar
  65. [47.65]
    J. Bacardit, E. Burke, N. Krasnogor: Improving the scalability of rule-based evolutionary learning, Memet. Comput. 1, 55–67 (2009)CrossRefGoogle Scholar
  66. [47.66]
    X. Llorà, K. Sastry, T.-L. Yu, D.E. Goldberg: Do not match, inherit: Fitness surrogates for genetics-based machine learning techniques, Proc. Genet. Evol. Comput. Conf. (GECCO 2007) (2007) pp. 1798–1805Google Scholar
  67. [47.67]
    O. Sigaud, M.V. Butz, O. Kozlova, C. Meyer: Anticipatory Learning Classifier Systems and Factored Reinforcement Learning (Springer, Berlin, Heidelberg 2009) pp. 321–333Google Scholar
  68. [47.68]
    P.L. Lanzi: An analysis of generalization in the XCS classifier system, Evol. Comput. 7, 125–149 (1999)CrossRefGoogle Scholar
  69. [47.69]
    M.V. Butz, D.E. Goldberg, P.L. Lanzi: Gradient descent methods in learning classifier systems: Improving XCS performance in multistep problems, IEEE Trans. Evol. Comput. 9, 452–473 (2005)CrossRefGoogle Scholar
  70. [47.70]
    J. Hurst, L. Bull: Self-adaptation in classifier system controllers, Artif. Life Robot. 5, 109–119 (2001)CrossRefGoogle Scholar
  71. [47.71]
    J. Hurst, L. Bull: A neural learning classifier system with self-adaptive constructivism for mobile robot learning, Artif. Life 12, 1–28 (2006)CrossRefGoogle Scholar
  72. [47.72]
    E.C. Tolman: Cognitive maps in rats and men, Psychol. Rev. 55, 189–208 (1948)CrossRefGoogle Scholar
  73. [47.73]
    R.L. Riolo: Lookahead planning and latent learning in a classifier system, from animals to animats, Proc. 1st Int. Conf. Simul. Adapt. Behav. (1991) pp. 316–326Google Scholar
  74. [47.74]
    W. Stolzmann: Anticipatory classifier systems, Genetic Programming 1998, Proc. 3rd Ann. Conf. (1998) pp. 658–664Google Scholar
  75. [47.75]
    M.V. Butz: Anticipatory Learning Classifier Systems (Kluwer, Boston 2002)CrossRefMATHGoogle Scholar
  76. [47.76]
    M.V. Butz, D.E. Goldberg, W. Stolzmann: The anticipatory classifier system and genetic generalization, Nat. Comput. 1, 427–467 (2002)MathSciNetCrossRefMATHGoogle Scholar
  77. [47.77]
    P. Gérard, O. Sigaud: YACS: Combining dynamic programming with generalization in classifier systems, Lect. Notes Comput. Sci. 1996, 52–69 (2001)CrossRefGoogle Scholar
  78. [47.78]
    P. Gérard, J.-A. Meyer, O. Sigaud: Combining latent learning and dynamic programming in MACS, Eur. J. Oper. Res. 160, 614–637 (2005)CrossRefMATHGoogle Scholar
  79. [47.79]
    W. Stolzmann, M.V. Butz: Latent learning and action planning in robots with anticipatory classifier systems, Lect. Notes Comput. Sci. 1813, 301–317 (2000)CrossRefGoogle Scholar
  80. [47.80]
    R.S. Sutton: DYNA: an integrated architecture for learning, planning, and reacting, ACM SIGART Bull. 2(4), 160–163 (1991)CrossRefGoogle Scholar
  81. [47.81]
    W. Stolzmann, M.V. Butz, J. Hoffmann, D.E. Goldberg: First cognitive capabilities in the anticipatory classifier system, from animals to animats 6, Proc. 6th Int. Conf. Simul. Adapt. Behav. (2000) pp. 287–296Google Scholar
  82. [47.82]
    M.V. Butz, J. Hoffmann: Anticipations control behavior: Animal behavior in an anticipatory learning classifier system, Adapt. Behav. 10, 75–96 (2002)CrossRefGoogle Scholar
  83. [47.83]
    M.V. Butz: Biasing exploration in an anticipatory learning classifier system, Lect. Notes Comput. Sci. 2321, 3–22 (2002)CrossRefMATHGoogle Scholar
  84. [47.84]
    M.V. Butz, D.E. Goldberg: Generalized state values in an anticipatory learning classifier system. In: Anticipatory Behavior in Adaptive Learning Systems: Foundations, Theories, and Systems, ed. by M.V. Butz, O. Sigaud, P. Gérard (Springer, Berlin, Heidelberg 2003) pp. 282–301CrossRefGoogle Scholar
  85. [47.85]
    B. Siciliano, O. Khatib: Springer Handbook of Robotics (Springer, Berlin, Heidelberg 2007)MATHGoogle Scholar
  86. [47.86]
    M.V. Butz, O. Herbort: Context-dependent predictions and cognitive arm control with XCSF, Proc. Genet. Evol. Comput. Conf. (GECCO 2008) (2008) pp. 1357–1364Google Scholar
  87. [47.87]
    M.V. Butz, G.K.M. Pedersen, P.O. Stalph: Learning sensorimotor control structures with XCSF: Redundancy exploitation and dynamic control, Proc. Genet. Evol. Comput. Conf. (GECCO 2009) (2009) pp. 1171–1178Google Scholar
  88. [47.88]
    D.M. Wolpert, R.C. Miall, M. Kawato: Internal models in the cerebellum, Trends Cogn. Sci. 2, 338–347 (1998)CrossRefGoogle Scholar
  89. [47.89]
    J.G. Fleischer: Neural correlates of anticipation in cerebellum, basal ganglia, and hippocampus, Lect. Notes Comput. Sci. 4520, 19–34 (2007)CrossRefGoogle Scholar
  90. [47.90]
    C.T. Fernando, E. Szathmary, P. Husbands: Selectionist and evolutionary approaches to brain function: A critical appraisal, Front. Comput. Neurosci. 6, doi: 10.3389/fncom.2012.00024 (2012)Google Scholar
  91. [47.91]
    A. Tomlinson, L. Bull: A corporate XCS, Lect. Notes Comput. Sci. 1813, 195–208 (2000)CrossRefGoogle Scholar
  92. [47.92]
    X. Llorà, K. Sastry: Fast rule matching for learning classifier systems via vector instructions, Proc. Genet. Evol. Comput. Conf. (GECCO 2006) (2006) pp. 1513–1520Google Scholar
  93. [47.93]
    M.V. Butz, P.L. Lanzi, X. Llorà, D. Loiacono: An analysis of matching in learning classifier systems, Proc. Genet. Evol. Comput. Conf. (GECCO 2008) (2008) pp. 1349–1356Google Scholar
  94. [47.94]
    J.R. Anderson: Rules of the Mind (Lawrence Erlbaum Associates, Hillsdale 1993)Google Scholar
  95. [47.95]
    G.L. Drescher: Made-Up Minds: A Constructivist Approach to Artificial Intelligence (MIT Press, Cambridge 1991)MATHGoogle Scholar
  96. [47.96]
    A. Newell: Physical symbol systems, Cogn. Sci. 4, 135–183 (1980)CrossRefGoogle Scholar
  97. [47.97]
    J.R. Anderson, D. Bothell, M.D. Byrne, S. Douglass, C. Lebiere, Y. Qin: An integrated theory of the mind, Psychol. Rev. 111, 1036–1060 (2004)CrossRefGoogle Scholar
  98. [47.98]
    O. Sigaud, S. Wilson: Learning classifier systems: A survey, soft computing – a fusion of foundations, Methodol. Appl. 11, 1065–1078 (2007)MATHGoogle Scholar
  99. [47.99]
    L. Bull, T. Kovacs (Eds.): Foundations of Learning Classifier Systems, Stud. Fuzziness and Soft Comput, Vol. 183 (Springer, Berlin, Heidelberg 2005)MATHGoogle Scholar
  100. [47.100]
    L. Bull (Ed.): Applications of Learning Classifier Systems (Springer, Berlin, Heidelberg 2004)MATHGoogle Scholar
  101. [47.101]
    L. Bull: On lookahead and latent learning in Simple LCS, Learn. Classif. Syst. Int. Workshops, IWLCS 2006-2007, ed. by J. Bacardit, E. Bernad-Mansilla, M.V. Butz (Springer, Berlin, Heidelberg 2008) pp. 154–168Google Scholar
  102. [47.102]
    P:L. Lanzi: xcslib - The XCS Library. http://xcslib.sourceforge.net/
  103. [47.103]
    P. O. Stalph, M. V. Butz: Documentation of JavaXCSF (COBOSLAB, University of Würzburg, Germany, Y2009N001 2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Computer Science, Cognitive ModelingUniversity of TübingenTübingenGermany

Personalised recommendations