Automatic Filtering of Valuable Features for Text Categorization

Abstract

Feature selection (FS) is aimed at reducing the size of the feature space. The advantage of FS is, in general, two-fold: improved accuracy of the learned classifiers, and efficiency of the learning process. Behind the use of a FS method there is the implicit assumption that only the selected terms are representative of the category being learned, while the rest are redundant. Thus, predicting an appropriate value of the feature space dimensionality is a crucial task, as a too aggressive feature selection might discard terms that carry essential information, while redundant features might deceive the learning algorithm. In “real life”, this task is usually accomplished “manually”, that is, the learning process is rerun over several vocabularies of different dimensions and the best results are eventually taken. Unfortunately, this may take very long training times.

In this paper we propose a FS technique that automatically detects an appropriate number of features for text categorization (TC) that are sufficient to learn accurate classifiers and make efficient the training process. One peculiarity of the proposed approach is that of combining both positive and negative features, the latter being considered relevant for the purpose of effective TC. The proposed approach has been tested by running three well known classifiers, notably, Ripper, C4.5 and linear SVM (the SMO implementation), over 7 real-world data sets, with varying characteristics.