Empirical Evaluation of Ensemble Techniques for a Pittsburgh Learning Classifier System

  • Jaume Bacardit
  • Natalio Krasnogor
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4998)


Ensemble techniques have proved to be very successful in boosting the performance of several types of machine learning methods. In this paper, we illustrate its usefulness in combination with GAssist, a Pittsburgh-style Learning Classifier System. Two types of ensembles are tested. First we evaluate an ensemble for consensus prediction. In this case several rule sets learnt using GAssist with different initial random seeds are combined using a flat voting scheme in a fashion similar to bagging. The second type of ensemble is intended to deal more efficiently with ordinal classification problems. That is, problems where the classes have some intrinsic order between them and, in case of misclassification, it is preferred to predict a class that is close to the correct one within the class intrinsic order. The ensemble for consensus prediction is evaluated using 25 datasets from the UCI repository. The hierarchical ensemble is evaluated using a Bioinformatics dataset. Both methods significantly improve the performance and behaviour of GAssist in all the tested domains.


Empirical Evaluation Minimum Description Length Ensemble Technique Consensus Prediction Ordinal Classis 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Various authors: Special issue on integrating multiple learned models. Machine Learning 36 (1999)Google Scholar
  2. 2.
    Breiman, L.: Bagging predictors. Machine Learning 24, 123–140 (1996)zbMATHGoogle Scholar
  3. 3.
    Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: International Conference on Machine Learning, pp. 148–156 (1996)Google Scholar
  4. 4.
    Bacardit, J.: Pittsburgh Genetics-Based Machine Learning in the Data Mining era: Representations, generalization, and run-time. PhD thesis, Ramon Lull University, Barcelona, Catalonia, Spain (2004)Google Scholar
  5. 5.
    Bacardit, J., Butz, M.V.: Data mining in learning classifier systems: Comparing XCS with GAssist. In: Advances at the frontier of Learning Classifier Systems, pp. 282–290. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  6. 6.
    Bacardit, J., Stout, M., Krasnogor, N., Hirst, J.D., Blazewicz, J.: Coordination number prediction using learning classifier systems: performance and interpretability. In: GECCO 2006: Proceedings of the 8th annual conference on Genetic and evolutionary computation, pp. 247–254. ACM Press, New York (2006)Google Scholar
  7. 7.
    Stout, M., Bacardit, J., Hirst, J.D., Krasnogor, N., Blazewicz, J.: From hp lattice models to real proteins: Coordination number prediction using learning classifier systems. In: Rothlauf, F., Branke, J., Cagnoni, S., Costa, E., Cotta, C., Drechsler, R., Lutton, E., Machado, P., Moore, J.H., Romero, J., Smith, G.D., Squillero, G., Takagi, H. (eds.) EvoWorkshops 2006. LNCS, vol. 3907, pp. 208–220. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  8. 8.
    Stout, M., Bacardit, J., Hirst, J.D., Krasnogor, N.: Prediction of recursive convex hull class assignments for protein residues. Bioinformatics (in press, 2008)Google Scholar
  9. 9.
    Frank, E., Hall, M.: A simple approach to ordinal classification. In: Proc 12th European Conference on Machine Learning, pp. 145–156. Springer, Heidelberg (2001)Google Scholar
  10. 10.
    Bacardit, J., Stout, M., Hirst, J.D., Sastry, K., Llora, X., Krasnogor, N.: Automated alphabet reduction method with evolutionary algorithms for protein structure prediction. In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation (GECCO2007), London, England, pp. 346–353. ACM Press, New York (2007)CrossRefGoogle Scholar
  11. 11.
    Stout, M., Bacardit, J., Hirst, J.D., Blazewicz, J., Krasnogor, N.: Prediction of residue exposure and contact number for simplified hp lattice model proteins using learning classifier systems. In: Applied Artificial Intelligence, Genova, Italy, pp. 601–608. World Scientific, Singapore (2006)CrossRefGoogle Scholar
  12. 12.
    Stout, M., Bacardit, J., Hirst, J.D., Smith, R.E., Krasnogor, N.: Prediction of topological contacts in proteins using learning classifier systems. Soft Computing, Special Issue on Evolutionary and Metaheuristic-based Data Mining (EMBDM) (in press, 2008)Google Scholar
  13. 13.
    Llorà, X., Bacardit, J., Bernadó, E., Traus, I.: Where to go once you have evolved a bunch of promising hypotheses? In: Advances at the frontier of Learning Classifier Systems (2006)Google Scholar
  14. 14.
    Bull, L., Studley, M., Whittley, A.J.B., I.: On the use of rule sharing in learning classifier system ensembles. In: Proceedings of the 2005 Congress on Evolutionary Computation (2005)Google Scholar
  15. 15.
    Kramer, S., Widmer, G., Pfahringer, B., de Groeve, M.: Prediction of ordinal classes using regression trees. Fundam. Inform. 47, 1–13 (2001)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Kramer, S.: Structural regression trees. In: Proceedings of the Thirteenth National Conference on Artificial Intelligence (AAAI 1996), pp. 812–819. AAAI Press/MIT Press (1996)Google Scholar
  17. 17.
    DeJong, K.A., Spears, W.M., Gordon, D.F.: Using genetic algorithms for concept learning. Machine Learning 13, 161–188 (1993)Google Scholar
  18. 18.
    Bacardit, J., Goldberg, D.E., Butz, M.V.: Improving the performance of a pittsburgh learning classifier system using a default rule. In: Kovacs, T., Llorà, X., Takadama, K., Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds.) IWLCS 2003. LNCS (LNAI), vol. 4399, pp. 291–307. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  19. 19.
    Bacardit, J.: Analysis of the initialization stage of a pittsburgh approach learning classifier system. In: GECCO 2005: Proceedings of the Genetic and Evolutionary Computation Conference, vol. 2, pp. 1843–1850. ACM Press, New York (2005)CrossRefGoogle Scholar
  20. 20.
    Rissanen, J.: Modeling by shortest data description. Automatica 14, 465–471 (1978)CrossRefzbMATHGoogle Scholar
  21. 21.
    Bacardit, J., Goldberg, D.E., Butz, M.V., Llorà, X., Garrell, J.M.: Speeding-up pittsburgh learning classifier systems: Modeling time and accuracy. In: Yao, X., Burke, E.K., Lozano, J.A., Smith, J., Merelo-Guervós, J.J., Bullinaria, J.A., Rowe, J.E., Tiňo, P., Kabán, A., Schwefel, H.-P. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 1021–1031. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  22. 22.
    Blake, C., Keogh, E., Merz, C.: UCI repository of machine learning databases (1998),
  23. 23.
    Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Rost, B., Sander, C.: Conservation and prediction of solvent accessibility in protein families. Proteins 20, 216–226 (1994)CrossRefGoogle Scholar
  25. 25.
    Richardson, C., Barlow, D.: The bottom line for prediction of residue solvent accessibility. Protein Eng. 12, 1051–1054 (1999)CrossRefGoogle Scholar
  26. 26.
    Liu, H., Hussain, F., Tam, C.L., Dash, M.: Discretization: An enabling technique. Data Mining and Knowledge Discovery 6, 393–423 (2002)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Jaume Bacardit
    • 1
    • 2
  • Natalio Krasnogor
    • 1
  1. 1.Automated Scheduling, Optimization and Planning research group, School of Computer ScienceUniversity of NottinghamNottinghamUK
  2. 2.Multidisciplinary Centre for Integrative Biology, School of BiosciencesUniversity of NottinghamSutton BoningtonUK

Personalised recommendations