Barebones Particle Swarm Optimization with a Neighborhood Search Strategy for Feature Selection

Abstract

Feature selection is a vital step in many machine learning and data mining tasks. Feature selection can reduce the dimensionality, speed up the learning process, and improve the performance of the learning models. Most of the existing feature selection methods try to find the best feature subset according to a pre-defined feature evaluation criterion. However, in many real-world datasets, there may exist many global or local optimal feature subsets, especially in the high-dimensional datasets. Classical feature selection methods can only obtain one optimal feature subset in a run of the algorithm and they cannot locate multiple optimal solutions. Therefore, this paper considers feature selection as a multimodal optimization problem and proposes a novel feature selection method which integrates the barebones particle swarm optimization (BBPSO) and a neighborhood search strategy. BBPSO is a simple but powerful variant of PSO. The neighborhood search strategy can form several steady sub-swarms in the population and each sub-swarm aims at finding one optimal feature subset. The proposed approach is compared with four PSO based feature selection methods on eight UCI datasets. Experimental results show that the proposed approach can produce superior feature subsets over the comparative methods.