Feature selection based on mutual information and redundancy-synergy coefficient
Mutual information is an important information measure for feature subset. In this paper, a hashing mechanism is proposed to calculate the mutual information on the feature subset. Redundancy-synergy coefficient, a novel redundancy and synergy measure of features to express the class feature, is defined by mutual information. The information maximization rule was applied to derive the heuristic feature subset selection method based on mutual information and redundancy-synergy coefficient. Our experiment results showed the good performance of the new feature selection method.
Key wordsMutual information Feature selection Machine learning Data mining
Unable to display preview. Download preview PDF.
- Almuallim, H., Dietterich, T.G., 1991. Learning with Many Irrelevant Features. Proceedings of the Ninth National Conference on Artifical Intelligence (AAAI-91), Anaheim, California, p.547–552.Google Scholar
- Fano, R., 1961. Tranmission of Information: A Statistical Theory of Communications. Wiley, New York.Google Scholar
- Liu, H., Motoda, H., Dash, M., 1998. A Monotonic Measure for Optimal Feature Selection. Proceedings of ECML-98, P.101–106.Google Scholar
- Liu, H., Setiono, R., 1996. A Probabilistic Approach to Feature Selection—A Filter Solution.In: ICML-96. Morgan Kaufmann Publishers, p.319–327.Google Scholar
- Yaglom, A.M., Yaglom, I.M., 1983. Probability and Information. D. Reidel Publishing Company.Google Scholar