Consistency measures for feature selection
The use of feature selection can improve accuracy, efficiency, applicability and understandability of a learning process. For this reason, many methods of automatic feature selection have been developed. Some of these methods are based on the search of the features that allows the data set to be considered consistent. In a search problem we usually evaluate the search states, in the case of feature selection we measure the possible feature sets. This paper reviews the state of the art of consistency based feature selection methods, identifying the measures used for feature sets. An in-deep study of these measures is conducted, including the definition of a new measure necessary for completeness. After that, we perform an empirical evaluation of the measures comparing them with the highly reputed wrapper approach. Consistency measures achieve similar results to those of the wrapper approach with much better efficiency.
KeywordsFeature selection Attribute evaluation Consistency Measures
Unable to display preview. Download preview PDF.
- Almuallim, H., & Dietterich, T. G. (1991). Learning with many irrelevant features. In Proceedings of the ninth national conference on artificial intelligence ( AAAI-91) , Anaheim, CA, vol. 2 (pp. 547–552). Menlo Park, CA: AAAI Press.Google Scholar
- Arauzo Azofra, A., Benitez, J. M., & Castro, J. L. (2003a). C-FOCUS: A continuous extension of FOCUS. In Proceedings of the 7th online world conference on soft computing in industrial applications (pp. 225–232).Google Scholar
- Arauzo Azofra, A., Benitez-Sanchez, J. M., & Castro-Peña, J. L. (2003b). A feature selection algorithm with fuzzy information. In Proceedings of the 10th IFSA world congress (pp. 220–223).Google Scholar
- Boros, E., Hammer, P. L., Ibaraki, T., Kogan, A., Mayoraz, E., & Muchnik, I. (2000). An implementation of logical analysis of data. IEEE Transactions on Knowledge Discovery and Data Engineering, 12(2), 292–306.Google Scholar
- Dash, M. (1997). Feature selection via set cover. In IEEE Knowledge and Data Engineering Exchange Workshop.Google Scholar
- Demsar, J., & Zupan, B. (2004). Orange: From experimental machine learning to interactive data mining. (White paper) http://www.ailab.si/orange.
- Hettich, S., & Bay, S. D. (1999). The uci kdd archive. http://kdd.ics.uci.edu/.
- John, G. H., Kohavi, R., & Pfleger, K. (1994). Irrelevant features and the subset selection problem. In International conference on machine learning, (pp. 121–129). Journal version in AIJ, available at http://citeseer.nj.nec.com/13663.html.
- Kira, K., & Rendell, L. A. (1992). A practical approach to feature selection. In Proceedings of the ninth international workshop on machine learning (pp. 249–256). San MAteo, CA, Morgan Kaufmann.Google Scholar
- Kohavi, R. (1994). Feature subset selection as search with probabilistic estimates. In AAAI fall symposium on relevance (pp. 122–126).Google Scholar
- Komorowski, J., Pawlak, Z., Polkowski, L., & Skowron, A. (1998). Rough sets: A tutorial. In S. K. Paland, & A. Skowron (Eds.) Rough-fuzzy hybridization: A new trend in decision-making (pp. 3–98). Singapore: Springer.Google Scholar
- Langley, P. (1994). Selection of relevant features in machine learning. In Procedings of the AAAI fall symposium on relevance, New Orleans, LA. Menlo Park, CA: AAAI Press.Google Scholar
- Liu, H., Motoda, H., & Dash, M. (1998). A monotonic measure for optimal feature selection. In European conference on machine learning (pp. 101–106).Google Scholar
- Modrzejewski, M. (1993). Feature selection using rough sets theory. In Proceedings of the European conference on machine learning (pp. 213–216).Google Scholar
- Oliveira, A., & Sangiovanni-Vicentelli, A. (1992). Constructive induction using a non-greedy strategy for feature selection. In Proceedings of ninth international conference on machine learning, Aberdeen, Scotland (pp. 355–360). San Mateo, CA: Morgan Kaufmann.Google Scholar
- Polkowski, L., & Skowron, A., (Eds.) (1998). Rough sets in knowledge discovery. Heidelberg: Physica Verlag.Google Scholar
- Schlimmer, J. (1993). Efficiently inducing determinations: A complete and systematic search algorithm that uses optimal pruning. In Proceedings of tenth international conference on machine learning (pp. 289–290).Google Scholar