Skip to main content
SpringerLink
Log in
Book cover

International Conference on Rough Sets and Knowledge Technology

RSKT 2011: Rough Sets and Knowledge Technology pp 110–119Cite as

A Rough Set Approach to Feature Selection Based on Relative Decision Entropy

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 6954))

Abstract

Rough set theory has been recognized to be one of the powerful tools for feature selection. The essence of rough set approach to feature selection is to find a minimal subset (or called reduct) of original features, which can predict the decision concepts as well as the original features. Since finding a minimal feature subset is a NP-hard problem, many heuristic algorithms have been proposed, which usually employ the feature significance in rough sets as heuristic information. Shannon’s information theory provides us a feasible way to measure the information of data sets with entropy. Hence, many researchers have used information entropy to measure the feature significance in rough sets, and proposed different information entropy models in rough sets. In this paper, we present a new information entropy model, in which relative decision entropy is defined. Based on the relative decision entropy, we propose a new rough set algorithm to feature selection. To verify the efficiency of our algorithm, experiments are carried out on some UCI data sets. The results demonstrate that our algorithm can provide competitive solutions efficiently.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Liu, H., Motoda, H.: Feature Selection for Knowledge Discovery and Data Mining. Kluwer Academic Publishers, Dordrecht (1998)

    Book  MATH  Google Scholar 

  2. Guyon, I., Elisseeff, A.: An Introduction to Variable and Feature Selection. Journal of Machine Learning Research 3, 1157–1182 (2003)

    MATH  Google Scholar 

  3. Hu, X.H.: Knowledge Discovery in Databases: An Attribute-oriented Rough Set Approach. Ph.D. thesis, Regina University (1995)

    Google Scholar 

  4. Hu, X.H., Cereone, N.: Learning in Relational Databases: A Rough Set Approach. Computational Intelligence 11(2), 323–337 (1995)

    Article  Google Scholar 

  5. Hu, K.Y., Lu, Y.C., Shi, C.Y.: Feature Ranking in Rough Sets. AI Commun. 16(1), 41–50 (2003)

    MATH  Google Scholar 

  6. Wang, G.Y., Yu, H., Yang, D.C.: Decision Table Reduction Based on Conditional Information Entropy. Chinese Journal of Computers 25(7), 759–766 (2002)

    MathSciNet  Google Scholar 

  7. Wang, G.Y.: Rough Set Theory and Knowledge Acquisition. Xian Jiaotong University Press, Xian (2001)

    Google Scholar 

  8. Liu, S.H., Sheng, Q.J., Wu, B., Shi, Z.Z.: Research on Efficient Algorithms for Rough Set Methods. Chinese Journal of Computers 26(5), 525–529 (2003)

    Google Scholar 

  9. Miao, D.Q., Hu, G.R.: An Heuristic Algorithm of Knowledge Reduction. Computer Research and Development 36(6), 681–684 (1999)

    Google Scholar 

  10. Düntsch, I., Gediga, G.: Uncertainty Measures of Rough Set Prediction. Artificial Intelligence 106, 109–137 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  11. Liang, J.Y., Shi, Z.Z.: The Information Entropy, Rough Entropy and Knowledge Granulation in Rough Set Theory. International Journal of Uncertainty, Fuzziness and Knowledge Based Systems 12(1), 37–46 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  12. Shannon, C.E.: The Mathematical Theory of Communication. Bell System Technical Journal 27(3-4), 373–423 (1948)

    MathSciNet  Google Scholar 

  13. Sui, Y.F., Xia, Y.M., Wang, J.: The Information Entropy of Rough Relational Databases. In: Wang, G., Liu, Q., Yao, Y., Skowron, A. (eds.) RSFDGrC 2003. LNCS (LNAI), vol. 2639, pp. 320–324. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  14. Xu, Z.Y., Liu, Z.P., Yang, B.R., Song, W.: A Quick Attribute Reduction Algorithm with Complexity of max(O(|C| |U|),O(|C| 2 |U/C|)). Chinese Journal of Computers 29(3), 391–399 (2006)

    Google Scholar 

  15. Jensen, R., Shen, Q.: Finding Rough Set Reducts with Ant Colony Optimization. In: Proc. of the 2003 UK Workshop on Computational Intelligence, pp. 15–22 (2003)

    Google Scholar 

  16. Wroblewski, J.: Finding Minimal Reducts Using Genetic Algorithms. In: Proc. of the Second Annual Join Conf. on Information Sciences, NC, pp. 186–189 (1995)

    Google Scholar 

  17. Bazan, J.: A Comparison of Dynamic and Non-dynamic Rough Set Methods for Extracting Laws from Decision Table. In: Rough Sets in Knowledge Discovery, pp. 321–365. Physica-Verlag, Heidelberg (1998)

    Google Scholar 

  18. Wang, X.Y., Yang, J., Teng, X.L., Xia, W.J., Jensen, R.: Feature Selection Based on Rough Sets and Particle Swarm Optimization. Pattern Recognition Letters 28(4), 459–471 (2007)

    Article  Google Scholar 

  19. Nguyen, S.H., Nguyen, H.S.: Some Efficient Algorithms for Rough Set Methods. In: Proceedings of IPMU 1996, Granada, Spain, pp. 1451–1456 (1996)

    Google Scholar 

  20. Skowron, A., Bazan, J., Son, N.H., Wroblewski, J., et al.: RSES 2.2 User’s Guide. Institute of Mathematics, Warsaw University, Warsaw, Poland (2005)

    Google Scholar 

  21. Stefanowski, J.: On Rough Set Based Approaches to Induction of Decision Rules. In: Rough Sets in Knowledge Discovery, pp. 500–529. Physica-Verlag, Heidelberg (1998)

    Google Scholar 

  22. Bay, S.D.: The UCI KDD Repository (1999), http://kdd.ics.uci.edu

  23. Pawlak, Z.: Rough Sets. Int. J. Comput. Informat. Sci. 11(5), 341–356 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  24. Pawlak, Z.: Rough Sets: Theoretical Aspects of Reasoning about Data. Kluwer Academic Publishing, Dordrecht (1991)

    Book  MATH  Google Scholar 

  25. Zhong, N., Dong, J., Ohsuga, S.: Using Rough Sets with Heuristics for Feature Selection. Journal of Intelligent Information Systems 16(3), 199–214 (2001)

    Article  MATH  Google Scholar 

  26. Yao, J.T., Zhang, M.: Feature Selection with Adjustable Criteria. In: Ślęzak, D., Wang, G., Szczuka, M.S., Düntsch, I., Yao, Y. (eds.) RSFDGrC 2005. LNCS (LNAI), vol. 3641, pp. 204–213. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  27. Pal, S.K., Shankar, B.U., Mitra, P.: Granular Computing, Rough Entropy and Object Extraction. Pattern Recognition Letters 26(16), 2509–2517 (2005)

    Article  Google Scholar 

  28. Sen, D., Pal, S.K.: Generalized Rough Sets, Entropy and Image Ambiguity Measures. IEEE Trans. Syst., Man and Cyberns. Part B 39(1), 117–128 (2009)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Information Science and Technology, Qingdao University of Science and Technology, Qingdao, 266061, P.R. China

    Lin Zhou & Feng Jiang

Authors
  1. Lin Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Feng Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of Regina, Regina, S4S 0A2, Saskatchewan, Canada

    JingTao Yao

  2. Department of Applied Computer Science, University of Winnipeg, R3B 2E9, Winnipeg, Canada

    Sheela Ramanna

  3. Institute of Computer Science & Technology, Chongqing University of Posts and Telecommunications, 400065, Chongqing, P.R. China

    Guoyin Wang

  4. Chair of Computer Science, University of Rzeszów, Poland

    Zbigniew Suraj

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Zhou, L., Jiang, F. (2011). A Rough Set Approach to Feature Selection Based on Relative Decision Entropy. In: Yao, J., Ramanna, S., Wang, G., Suraj, Z. (eds) Rough Sets and Knowledge Technology. RSKT 2011. Lecture Notes in Computer Science(), vol 6954. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24425-4_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-24425-4_17

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-24424-7

  • Online ISBN: 978-3-642-24425-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation