A Learning Classifier System with Mutual-Information-Based Fitness

  • Robert Elliott Smith
  • Max Kun Jiang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4998)


This paper introduces a new variety of learning classifier system (LCS), called MILCS, which utilizes mutual information as fitness feedback. Unlike most LCSs, MILCS is specifically designed for supervised learning. We present preliminary results, and contrast them to results from XCS. We discuss the explanatory power of the resulting rule sets and introduce a new technique for visualizing explanatory power. Final comments include future directions of this research, including investigations in neural networks and other systems.


Evolutionary computation learning classifier systems machine learning information theory mutual information supervised learning protein structure prediction explanatory power 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bacardit, J.: Pittsburgh Genetics-Based Machine Learning in the Data Mining era: representations, generalization, and run-time. PhD thesis, Ramon Llull University, Barcelona, Catalonia, Spain (2004) Google Scholar
  2. 2.
    Bernadó-Mansilla, E.: Accuracy-Based Learning Classifier Systems: Models. Analysis and Applications to Classification Tasks. Evolutionary Computation 11(3) (2003)Google Scholar
  3. 3.
    Butz, M. V.: Documentation of XCS+TS C-Code 1.2. IlliGAL report 2003023, University of Illinois at Urbana-Champaign(Source code) (2003),
  4. 4.
    (+ tournament selection) classifier system implementation in C, version 1.2 (for IlliGAL Report 2003023, University of Illinois Genetic Algorithms Laboratory) (2003),,
  5. 5.
    Fahlman, S.E., Lebiere, C.: The Cascade-Correlation learning algorithm. In: Advances in Neural Information Processing Systems 2. Morgan Kaufmann, San Francisco (1990)Google Scholar
  6. 6.
    Goldberg, D.E.: Genetic algorithms in search, optimization and machine learning. Addison-Wesley, Reading (1989)zbMATHGoogle Scholar
  7. 7.
    Heckerman, H.: A Tutorial on Learning with Bayesian Networks, Technical Report, MSR-TR-95-06 (1996)Google Scholar
  8. 8.
    Holland, J.H.: Adaptation in Natural and Artificial Systems, 2nd edn. MIT Press, Cambridge (1992)Google Scholar
  9. 9.
    Holland, J., Holyoak, K.J., Nisbett, R.E., Thagard, P.: Induction: Processes of Inference Learning and Discovery. MIT Press, Cambridge (1986)Google Scholar
  10. 10.
    Lanzi, P.L.: xcslib: source code,
  11. 11.
    Lanzi, P.L., Loiacono, D., Wilson, S.W., Goldberg, D.E.: XCS with Computed Prediction for the Learning of Boolean Functions. Evolutionary Computation 1, 588–595 (2005)Google Scholar
  12. 12.
    Shannon, C.E.: A Mathematical Theory of Communication. Bell System Technical Journal 27, 379–423,623–656 (1948)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Shannon, C.E.: Communication in the presence of noise. Proc. Institute of Radio Engineers 37(1), 10–21 (1949)MathSciNetGoogle Scholar
  14. 14.
    Smith, R.E., Cribbs, H.B.: Is a classifier system a type of neural network? Evolutionary Computation 2(1), 19–36 (1994)CrossRefGoogle Scholar
  15. 15.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  16. 16.
    Stout, M., Bacardit, J., Hirst, J., Krasogor, N., Blazewicz, J.: From HP lattice models to real proteins: coordination number prediction using learning classifier systems. In: 4th European Workshop on Evolutionary Computation and Machine Learning in Bioinformatics (2006)Google Scholar
  17. 17.
    Wilson, S.W.: Classifier Fitness based on Accuracy. Evolutionary Computation 3(2), 149–175 (1994)CrossRefGoogle Scholar
  18. 18.
    Wilson, S.W.: ZCS: A Zeroth-Level Classifier System. Evolutionary Computation 2(1), 1–18 (1994)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Wilson, S.W.: Generalization in the XCS classifier system. In: Koza, J.R., Banzhaf, W., Chellapilla, K., Deb, K., Dorigo, M., Fogel, D.B., Garzon, M.H., Goldberg, D.E., Iba, H., Riolo, R. (eds.) Genetic Programming 1998: Proceedings of the Third Annual Conference, pp. 665–674. Morgan Kaufmann, San Francisco (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Robert Elliott Smith
    • 1
  • Max Kun Jiang
    • 1
  1. 1.Department of Computer ScienceUniversity College LondonLondonUnited Kingdom

Personalised recommendations