Feature Selection Based on Fuzzy Mutual Information

  • Michela Antonelli
  • Pietro Ducange
  • Francesco Marcelloni
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8256)


In the framework of fuzzy rule-based models for regression problems, we propose a novel approach to feature selection based on the minimal-redundancy-maximal-relevance criterion. The relevance of a feature is measured in terms of a novel definition of fuzzy mutual information between the feature and the output variable. The redundancy is computed as the average fuzzy mutual information between the feature and the just selected features. The approach results to be particularly suitable for selecting features before designing fuzzy rule-based systems (FRBSs). We tested our approach on twelve regression problems using Mamdani FRBSs built by applying the Wang and Mendel algorithm. We show that our approach is particularly effective in selecting features by comparing the mean square errors achieved by the Mamdani FRBSs generated using the features selected by a state of the art feature selection algorithm and by our approach.


Feature Selection Fuzzy Mutual Information Regression Problems High Dimensional Datasets 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alcala, R., Ducange, P., Herrera, F., Lazzerini, B., Marcelloni, F.: A multiobjective evolutionary approach to concurrently learn rule and data bases of linguistic fuzzy-rule-based systems. IEEE Transactions on Fuzzy Systems 17(5), 1106–1122 (2009)CrossRefGoogle Scholar
  2. 2.
    Battiti, R.: Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on Neural Networks 5, 537–550 (1994)CrossRefGoogle Scholar
  3. 3.
    Dash, M., Liu, H.: Feature selection for classification. Intelligent Data Analysis 1, 131–156 (1997)CrossRefGoogle Scholar
  4. 4.
    Estévez, P., Tesmer, M., Perez, C., Zurada, J.: Normalized mutual information feature selection. IEEE Transaction on Neural Networks 20(2), 189–201 (2009)CrossRefGoogle Scholar
  5. 5.
    Gerontidis, I., Petasakis, I.E.: Lumpability of absorbing markov chains and replacement chains on fuzzy partitions. In: FUZZ-IEEE, pp. 1–8 (2010)Google Scholar
  6. 6.
    Hall, M.A., Smith, L.A.: Practical Feature Subset Selection for Machine Learning (1998)Google Scholar
  7. 7.
    Huawen, L., Jigui, S., Lei, L., Huijie, Z.: Feature selection with dynamic mutual information. Pattern Recogn. 42(7), 1330–1339 (2009)CrossRefzbMATHGoogle Scholar
  8. 8.
    Kojadinovic, I.: Relevance measures for subset variable selection in regression problems based on k-additive mutual information. Comput. Stat. Data Anal. 49(4), 1205–1227 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Kudo, M., Sklansky, J.: Comparison of algorithms that select features for pattern classifiers. Pattern Recognition 33(1), 25–41 (2000)CrossRefGoogle Scholar
  10. 10.
    Kuriyama, K.: Entropy of a finite partition of fuzzy sets. Journal of Mathematical Analysis and Applications 94, 38–43 (1983)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Rossi, F., Lendasse, A., Franc̨ois, D., Wertz, V., Verleysen, M.: Mutual information for the selection of relevant variables in spectrometric nonlinear modelling. Chemometrics and Intelligent Laboratory Systems 80(2), 215–226 (2006)CrossRefGoogle Scholar
  12. 12.
    Sánchez, L., Suárez, M.R., Villar, J.R., Couso, I.: Mutual information-based feature selection and partition design in fuzzy rule-based classifiers from vague data. Int. J. Approx. Reasoning 49(3), 607–622 (2008)CrossRefGoogle Scholar
  13. 13.
    Sheskin, D.J.: Handbook of Parametric and Nonparametric Statistical Procedures, 4th edn. Chapman & Hall/CRC (2007)Google Scholar
  14. 14.
    Wang, L., Mendel, J.: Generating fuzzy rules by learning from examples. IEEE Transactions on Systems Man and Cybernetics 22(6), 1414–1427 (1992)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  • Michela Antonelli
    • 1
  • Pietro Ducange
    • 1
  • Francesco Marcelloni
    • 1
  1. 1.Dipartimento di Ingegneria dell’InformazioneUniversity of PisaPisaItaly

Personalised recommendations