Can the Relevance Index be Used to Evolve Relevant Feature Sets?

Abstract

The Relevance Index (RI) is an information theory-based measure that was originally defined to detect groups of functionally similar neurons, based on their dynamic behavior. More in general, considering the dynamical analysis of a generic complex system, the larger the RI value associated with a subset of variables, the more those variables are strongly correlated with one another and independent from the other variables describing the system status. We describe some early experiments to evaluate whether such an index can be used to extract relevant feature subsets in binary pattern classification problems. In particular, we used a PSO variant to efficiently explore the RI search space, whose size equals the number of possible variable subsets (in this case $2^{104}$) and find the most relevant and discriminating feature subsets with respect to pattern representation. We then turned such relevant subsets into a new smaller set of richer features, whose values depend on the values of the binary features they include. The paper reports some exploratory results we obtained in a simple character recognition task, comparing the performance of RI-based feature extraction and selection with other classical feature selection/extraction approaches.