A Blocking Strategy for Ranking Features According to Probabilistic Relevance
The paper presents an algorithm to rank features in “small number of samples, large dimensionality” problems according to probabilistic feature relevance, a novel definition of feature relevance. Probabilistic feature relevance, intended as expected weak relevance, is introduced in order to address the problem of estimating conventional feature relevance in data settings where the number of samples is much smaller than the number of features. The resulting ranking algorithm relies on a blocking approach for estimation and consists in creating a large number of identical configurations to measure the conditional information of each feature in a paired manner. Its implementation can be made embarrassingly parallel in the case of very large n. A number of experiments on simulated and real data confirms the interest of the approach.
KeywordsProposal Distribution Ranking Algorithm Feature Selection Technique Probabilistic Relevance Conditional Mutual Information
The author acknowledges the support of the “BruFence: Scalable machine learning for automating defense system” project (RBC/14 PFS-ICT 5), funded by the Institute for the encouragement of Scientific Research and Innovation of Brussels (INNOVIRIS, Brussels Region, Belgium).
- 2.Bontempi, G., Meyer, P.E.: Causal filter selection in microarray data. In: Proceeding of the ICML 2010 Conference (2010)Google Scholar
- 7.Liaw, A., Wiener, M.: Classification and regression by randomforest. R News 2(3), 18–22 (2002)Google Scholar
- 8.Meyer, P.E., Bontempi, G.: Information-theoretic gene selection in expression data. In: Biological Knowledge Discovery Handbook. IEEE Computer Society (2014)Google Scholar
- 9.Montgomery, D.C.: Design and Analysis of Experiments. Wiley, Hoboken (2001)Google Scholar
- 11.Ramey, J.A.: Datamicroarray: Collection of Data Sets for Classification (2013). R package version 0.2.2Google Scholar
- 13.Tsamardinos, I., Aliferis, C.: Towards principled feature selection: relevancy. In: Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics (2003)Google Scholar
- 14.Tsamardinos, I., Aliferis, C.F., Statnikov, A.: Algorithms for large scale Markov blanket discovery. In: Proceedings of the 16th International FLAIRS Conference (FLAIRS 2003) (2003)Google Scholar