Information and Rough Set Theory Based Feature Selection Techniques

  • Liam Cervante
  • Xiaoying Gao
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8210)


Feature selection is a well known and studied technique that aims to solve “the curse of dimensionality” and improve performance by removing irrelevant and redundant features. This paper highlights some well known approaches to filter feature selection, information theory and rough set theory, and compares a recent fitness function with some traditional methods. The contributions of this paper are two-fold. First, new results confirm previous research and show that the recent fitness function can also perform favorably when compared to rough set theory. Secondly, the measure of redundancy that is used in traditional information theory is shown to damage the performance when a similar approach is applied to the recent fitness function.


Feature Selection Mutual Information Feature Subset Feature Selection Algorithm Joint Entropy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)zbMATHGoogle Scholar
  2. 2.
    Kohavi, R., John, G.H.: Wrappers for feature subset selection. Artif. Intell. 97(1-2), 273–324 (1997)CrossRefzbMATHGoogle Scholar
  3. 3.
    Peng, H.P.H., Long, F.L.F., Ding, C.: Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(8), 1226–1238 (2005)CrossRefGoogle Scholar
  4. 4.
    Zhong, N., Dong, J., Ohsuga, S.: Using rough sets with heuristics for feature selection. J. Intell. Inf. Syst. 16(3), 199–214 (2001)CrossRefzbMATHGoogle Scholar
  5. 5.
    Cervante, L., Bing, X., Zhang, M.: Binary particle swarm optimisation for feature selection: A filter based approach. In: Proceedings of 2012 IEEE Congress on Evolutionary Computation, pp. 881–888. IEEE Press (2012)Google Scholar
  6. 6.
    Shannon, C.E., Weaver, W.: A Mathematical Theory of Communication. University of Illinois Press, Champaign (1963)Google Scholar
  7. 7.
    Pawlak, Z.: Rough sets. International Journal of Parallel Programming 11, 341–356 (1982), 10.1007/BF01001956Google Scholar
  8. 8.
    Yao, Y.: Probabilistic rough set approximations. Int. J. Approx. Reasoning 49(2), 255–271 (2008)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  • Liam Cervante
    • 1
  • Xiaoying Gao
    • 1
  1. 1.School of Engineering and Computer ScienceVictoria University of WellingtonWellingtonNew Zealand

Personalised recommendations