An Empirical Comparison of Rule Induction Using Feature Selection with the LEM2 Algorithm

  • Jerzy W. Grzymala-Busse
Part of the Communications in Computer and Information Science book series (CCIS, volume 297)


The main objective of this paper is to compare a strategy to rule induction based on feature selection with another strategy, not using feature selection, exemplified by the LEM2 algorithm. It is shown that LEM2 significantly outperforms the strategy or rule induction based on feature selection in terms of an error rate (5% significance level, two-tailed test). At the same time, the LEM2 algorithm induces smaller rule sets with the smaller total number of conditions as well.


Feature Selection Rule Induction Matching Rule Minimal Complex Data Mining System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Blum, A., Langley, P.: Selection of relevant features and examples in machine learning. Artificial Intelligence 97, 245–271 (1997)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Kohavi, R., John, G.: Wrappers for feature selection. Artificial Intelligence 97, 273–324 (1997)zbMATHCrossRefGoogle Scholar
  3. 3.
    Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. Journal of Machine Learning Research 3, 1157–1182 (2003)zbMATHGoogle Scholar
  4. 4.
    Guyon, I., Gunn, S., Nikravesh, M., Zadeh, L.A.: Feauture Extraction. Foundations and Applications. Springer, Heidelberg (2006)Google Scholar
  5. 5.
    Liu, H., Motoda, H.: Computational Methods of Feature Selection. Chapman and Hall/CRC, Boca Raton, FL (2007)zbMATHGoogle Scholar
  6. 6.
    Pawlak, Z.: Rough sets. International Journal of Computer and Information Sciences 11, 341–356 (1982)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Pawlak, Z.: Rough Sets. Theoretical Aspects of Reasoning about Data. Kluwer Academic Publishers, Dordrecht (1991)zbMATHGoogle Scholar
  8. 8.
    Pawlak, Z., Grzymala-Busse, J.W., Slowinski, R., Ziarko, W.: Rough sets. Communications of the ACM 38, 89–95 (1995)CrossRefGoogle Scholar
  9. 9.
    Grzymala-Busse, J.W.: Rule induction. In: Maimon, O., Rokach, L. (eds.) Data Mining and Knowledge Discovery Handbook, 2nd edn., pp. 249–265. Springer, Heidelberg (2010)Google Scholar
  10. 10.
    Chan, C.C., Grzymala-Busse, J.W.: On the attribute redundancy and the learning programs ID3, PRISM, and LEM2. Technical report, Department of Computer Science, University of Kansas (1991)Google Scholar
  11. 11.
    Grzymala-Busse, J.W.: LERS—a system for learning from examples based on rough sets. In: Slowinski, R. (ed.) Intelligent Decision Support. Handbook of Applications and Advances of the Rough Set Theory, pp. 3–18. Kluwer Academic Publishers, Dordrecht (1992)Google Scholar
  12. 12.
    Grzymala-Busse, J.W.: A new version of the rule induction system LERS. Fundamenta Informaticae 31, 27–39 (1997)zbMATHGoogle Scholar
  13. 13.
    Booker, L.B., Holland, D.E., Goldberg, J.F.: Classifier systems and genetic algorithms. In: Carbonell, J.G. (ed.) Machine Learning. Paradigms and Methods, pp. 235–282. MIT Press, Boston (1990)Google Scholar
  14. 14.
    Holland, J.H., Holyoak, K.J., Nisbett, R.E.: Induction. Processes of Inference, Learning, and Discovery. MIT Press, Boston (1986)Google Scholar
  15. 15.
    Stefanowski, J.: Algorithms of Decision Rule Induction in Data Mining. Poznan University of Technology Press, Poznan (2001)Google Scholar
  16. 16.
    Grzymala-Busse, J.W., Grzymala-Busse, W.J.: Handling missing attribute values. In: Maimon, O., Rokach, L. (eds.) Data Mining and Knowledge Discovery Handbook, 2nd edn., pp. 33–51. Springer, Heidelberg (2010)Google Scholar
  17. 17.
    Chmielewski, M.R., Grzymala-Busse, J.W.: Global discretization of continuous attributes as preprocessing for machine learning. International Journal of Approximate Reasoning 15(4), 319–331 (1996)zbMATHCrossRefGoogle Scholar
  18. 18.
    Fang, J., Grzymala-Busse, J.: Leukemia prediction from gene expression data—a rough set approach. In: Proceedings of the Eighth International Conference on Artifical Intelligence and Soft Computing, pp. 899–908 (2006)Google Scholar
  19. 19.
    Fang, J., Grzymala-Busse, J.W.: Mining of MicroRNA Expression Data—A Rough Set Approach. In: Wang, G.-Y., Peters, J.F., Skowron, A., Yao, Y. (eds.) RSKT 2006. LNCS (LNAI), vol. 4062, pp. 758–765. Springer, Heidelberg (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jerzy W. Grzymala-Busse
    • 1
    • 2
  1. 1.Department of Electrical Engineering and Computer ScienceUniversity of KansasLawrenceUSA
  2. 2.Institute of Computer SciencePolish Academy of SciencesWarsawPoland

Personalised recommendations