Finding the Optimal Number of Features Based on Mutual Information
For high dimensional data analytics, feature selection is an indispensable preprocessing step to reduce dimensionality and keep the simplicity and interpretability of models. This is particularly important for fuzzy modeling since fuzzy models are widely recognized for their transparency and interpretability. Despite the substantial work on feature selection, there is little research on determining the optimal number of features for a task. In this paper, we propose a method to help find the optimal number of feature effectively based on mutual information.
KeywordsFeature selection Mutual information Number of features Fuzzy models
This work is partially supported by Philips Research within the scope of the BrainBridge Program.
- 1.Alonso, J.M., Castiello, C., Mencar, C.: Interpretability of fuzzy systems: current research trends and prospects. In: Springer Handbook of Computational Intelligence, pp. 219–237. Springer, Berlin (2015)Google Scholar
- 2.Alpaydin, E.: Introduction to Machine Learning. MIT press, Cambridge (2014)Google Scholar
- 6.Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (2012)Google Scholar
- 8.Gaspar-Cunha, A., Recio, G., Costa, L., Estébanez, C.: Self-adaptive MOEA feature selection for classification of bankruptcy prediction data. Sci. World J. 2014, 20 (2014)Google Scholar
- 9.Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3(Mar), 1157–1182 (2003)Google Scholar
- 10.Guyon, I., Gunn, S., Nikravesh, M., Zadeh, L.A.: Feature Extraction: Foundations and Applications, vol. 207. Springer, Heidelberg (2008)Google Scholar
- 12.Hall, M.A.: Correlation-based feature selection for machine learning. Ph.D. thesis, The University of Waikato (1999)Google Scholar
- 13.Hall, M.A., Smith, L.A.: Practical feature subset selection for machine learning (1998)Google Scholar
- 16.Jang, J.S.R., Sun, C.T., Mizutani, E.: Neuro-Fuzzy and Soft Computing, a Computational Approach to Learning and Machine Intelligence. Prentice Hall, Upper Saddle River (1997)Google Scholar
- 22.Wilbik, A., van Loon, S., Boer, A.K., Kaymak, U., Scharnhorst, V.: Fuzzy modeling for vitamin b12 deficiency. In: International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems, pp. 462–471. Springer (2016)Google Scholar
- 23.Xue, B., Fu, W., Zhang, M.: Multi-objective feature selection in classification: a differential evolution approach. In: Asia-Pacific Conference on Simulated Evolution and Learning, pp. 516–528. Springer (2014)Google Scholar