Abstract
Rough set theory has been recognized to be one of the powerful tools for feature selection. The essence of rough set approach to feature selection is to find a minimal subset (or called reduct) of original features, which can predict the decision concepts as well as the original features. Since finding a minimal feature subset is a NP-hard problem, many heuristic algorithms have been proposed, which usually employ the feature significance in rough sets as heuristic information. Shannon’s information theory provides us a feasible way to measure the information of data sets with entropy. Hence, many researchers have used information entropy to measure the feature significance in rough sets, and proposed different information entropy models in rough sets. In this paper, we present a new information entropy model, in which relative decision entropy is defined. Based on the relative decision entropy, we propose a new rough set algorithm to feature selection. To verify the efficiency of our algorithm, experiments are carried out on some UCI data sets. The results demonstrate that our algorithm can provide competitive solutions efficiently.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Liu, H., Motoda, H.: Feature Selection for Knowledge Discovery and Data Mining. Kluwer Academic Publishers, Dordrecht (1998)
Guyon, I., Elisseeff, A.: An Introduction to Variable and Feature Selection. Journal of Machine Learning Research 3, 1157–1182 (2003)
Hu, X.H.: Knowledge Discovery in Databases: An Attribute-oriented Rough Set Approach. Ph.D. thesis, Regina University (1995)
Hu, X.H., Cereone, N.: Learning in Relational Databases: A Rough Set Approach. Computational Intelligence 11(2), 323–337 (1995)
Hu, K.Y., Lu, Y.C., Shi, C.Y.: Feature Ranking in Rough Sets. AI Commun. 16(1), 41–50 (2003)
Wang, G.Y., Yu, H., Yang, D.C.: Decision Table Reduction Based on Conditional Information Entropy. Chinese Journal of Computers 25(7), 759–766 (2002)
Wang, G.Y.: Rough Set Theory and Knowledge Acquisition. Xian Jiaotong University Press, Xian (2001)
Liu, S.H., Sheng, Q.J., Wu, B., Shi, Z.Z.: Research on Efficient Algorithms for Rough Set Methods. Chinese Journal of Computers 26(5), 525–529 (2003)
Miao, D.Q., Hu, G.R.: An Heuristic Algorithm of Knowledge Reduction. Computer Research and Development 36(6), 681–684 (1999)
Düntsch, I., Gediga, G.: Uncertainty Measures of Rough Set Prediction. Artificial Intelligence 106, 109–137 (1998)
Liang, J.Y., Shi, Z.Z.: The Information Entropy, Rough Entropy and Knowledge Granulation in Rough Set Theory. International Journal of Uncertainty, Fuzziness and Knowledge Based Systems 12(1), 37–46 (2004)
Shannon, C.E.: The Mathematical Theory of Communication. Bell System Technical Journal 27(3-4), 373–423 (1948)
Sui, Y.F., Xia, Y.M., Wang, J.: The Information Entropy of Rough Relational Databases. In: Wang, G., Liu, Q., Yao, Y., Skowron, A. (eds.) RSFDGrC 2003. LNCS (LNAI), vol. 2639, pp. 320–324. Springer, Heidelberg (2003)
Xu, Z.Y., Liu, Z.P., Yang, B.R., Song, W.: A Quick Attribute Reduction Algorithm with Complexity of max(O(|C| |U|),O(|C| 2 |U/C|)). Chinese Journal of Computers 29(3), 391–399 (2006)
Jensen, R., Shen, Q.: Finding Rough Set Reducts with Ant Colony Optimization. In: Proc. of the 2003 UK Workshop on Computational Intelligence, pp. 15–22 (2003)
Wroblewski, J.: Finding Minimal Reducts Using Genetic Algorithms. In: Proc. of the Second Annual Join Conf. on Information Sciences, NC, pp. 186–189 (1995)
Bazan, J.: A Comparison of Dynamic and Non-dynamic Rough Set Methods for Extracting Laws from Decision Table. In: Rough Sets in Knowledge Discovery, pp. 321–365. Physica-Verlag, Heidelberg (1998)
Wang, X.Y., Yang, J., Teng, X.L., Xia, W.J., Jensen, R.: Feature Selection Based on Rough Sets and Particle Swarm Optimization. Pattern Recognition Letters 28(4), 459–471 (2007)
Nguyen, S.H., Nguyen, H.S.: Some Efficient Algorithms for Rough Set Methods. In: Proceedings of IPMU 1996, Granada, Spain, pp. 1451–1456 (1996)
Skowron, A., Bazan, J., Son, N.H., Wroblewski, J., et al.: RSES 2.2 User’s Guide. Institute of Mathematics, Warsaw University, Warsaw, Poland (2005)
Stefanowski, J.: On Rough Set Based Approaches to Induction of Decision Rules. In: Rough Sets in Knowledge Discovery, pp. 500–529. Physica-Verlag, Heidelberg (1998)
Bay, S.D.: The UCI KDD Repository (1999), http://kdd.ics.uci.edu
Pawlak, Z.: Rough Sets. Int. J. Comput. Informat. Sci. 11(5), 341–356 (1982)
Pawlak, Z.: Rough Sets: Theoretical Aspects of Reasoning about Data. Kluwer Academic Publishing, Dordrecht (1991)
Zhong, N., Dong, J., Ohsuga, S.: Using Rough Sets with Heuristics for Feature Selection. Journal of Intelligent Information Systems 16(3), 199–214 (2001)
Yao, J.T., Zhang, M.: Feature Selection with Adjustable Criteria. In: Ślęzak, D., Wang, G., Szczuka, M.S., Düntsch, I., Yao, Y. (eds.) RSFDGrC 2005. LNCS (LNAI), vol. 3641, pp. 204–213. Springer, Heidelberg (2005)
Pal, S.K., Shankar, B.U., Mitra, P.: Granular Computing, Rough Entropy and Object Extraction. Pattern Recognition Letters 26(16), 2509–2517 (2005)
Sen, D., Pal, S.K.: Generalized Rough Sets, Entropy and Image Ambiguity Measures. IEEE Trans. Syst., Man and Cyberns. Part B 39(1), 117–128 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhou, L., Jiang, F. (2011). A Rough Set Approach to Feature Selection Based on Relative Decision Entropy. In: Yao, J., Ramanna, S., Wang, G., Suraj, Z. (eds) Rough Sets and Knowledge Technology. RSKT 2011. Lecture Notes in Computer Science(), vol 6954. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24425-4_17
Download citation
DOI: https://doi.org/10.1007/978-3-642-24425-4_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24424-7
Online ISBN: 978-3-642-24425-4
eBook Packages: Computer ScienceComputer Science (R0)