Feature Subset Selection Based on Relative Dependency between Attributes

  • Jianchao Han
  • Xiaohua Hu
  • Tsao Young Lin
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3066)


Feature subset selection is an importent component of knowledge discovery and data mining systems to help reduce the data dimensionality. Rough sets theory provides a mechanism of selecting feature subsets. In the rough set community, most feature subset selection algorithms are attributes reduct-oriented; that is, finding minimum reducts of the conditional attributes of a decision table. Two main approaches to finding attribute reducts are categorized as discernibility functions-based and attribute dependency-based. These algorithms, however, suffer from intensive computations of either discernibility functions for the former or positive regions for the latter. In this paper, we propose a new concept, called relative attribute dependency, and present a sufficient and necessary condition of the minimum conditional attributes reduct of a decision table represented with the relative attribute dependency. The relative attribute dependency can be calculated by counting the distinct rows of the sub-decision table, instead of generating discernibility functions or positive regions. Thus the computation efficiency of minimum reducts are highly improved. We develop two algorithms for finding minimum reducts of the conditional attributes, one brute-force algorithm and the other heuristic algorithm using attribute entropy as the heuristic function. We also show the results of these algorithms by an illustrative example.


Feature subset selection Data reduction Rough set theory 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bazan, J., Nguyen, H., Nguyen, S., Synak, P., Wroblewski, J.: Rough set algorithms in classification problems. In: Polkowski, L., Lin, T.Y., Tsumoto, S. (eds.) Rough Set Methods and Applications: New Developments in Knowledge Discovery in Information Systems, pp. 49–88. Physica-Verlag, Heidelberg (2000)Google Scholar
  2. 2.
    Cercone, N., Ziarko, W., Hu, X.: Rule Discovery from Databases: A Decision Matrix Approach. In: Michalewicz, M., Raś, Z.W. (eds.) ISMIS 1996. LNCS, vol. 1079, pp. 653–662. Springer, Heidelberg (1996)Google Scholar
  3. 3.
    John, G., Kohavi, R., Pfleger, K.: Irrelevant Features and the Subset Selection Problem. In: Proc. ICML, pp. 121–129 (1994)Google Scholar
  4. 4.
    Kira, K., Rendell, L.A.: The feature Selection Problem: Traditional Methods and a New Algorithm. In: Proc. AAAI, pp. 129–134. MIT Press, Cambridge (1992)Google Scholar
  5. 5.
    Kumar, A.: New Techniques for Data Reduction in Database Systems for Knowledge Discovery Applications. Journal of Intelligent Information Systems 10(1), 31–48 (1998)CrossRefGoogle Scholar
  6. 6.
    Liu, H., Motoda, H. (eds.): Feature Extraction Construction and Selection: A Data Mining Perspective. Kluwer Academic Publishers, Dordrecht (1998)zbMATHGoogle Scholar
  7. 7.
    Modrzejewski, M.: Feature Selection Using Rough Sets Theory. In: Brazdil, P.B. (ed.) ECML 1993. LNCS, vol. 667, pp. 213–226. Springer, Heidelberg (1993)Google Scholar
  8. 8.
    Nguyen, H., Nguyen, S.: Some efficient algorithms for rough set methods. In: Proc. IPMU, Granada, Spain, pp. 1451–1456 (1996)Google Scholar
  9. 9.
    Pawlak, Z.: Rough Sets: Theoretical Aspects of Reasoning about Data. Kluwer Academic Publishers, Dordrecht (1992)Google Scholar
  10. 10.
    Polkowski, L., Skowron, A.: Rough mereology: A new paradigm for approximate reasoning. J. of Approximate Reasoning 15(4), 333–365 (1996)zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)Google Scholar
  12. 12.
    Shen, Q., Chouchoulas, A.: A modular approach to generalizing fuzzy rules with reduced attributes for the monitoring of complex systems. Enginnering Applications of Artificial Intelligence 13(3), 263–278 (2000)CrossRefGoogle Scholar
  13. 13.
    Shen, Q., Chouchoulas, A.: A rough-fuzzy approach for generating classification rules. Pattern Recognition 35, 2425–2438 (2002)zbMATHCrossRefGoogle Scholar
  14. 14.
    Skowron, A., Rauszer, C.: The Discernibility Matrices and Functions in Information Systems. In: Slowinski, K. (ed.) Intelligent Decision Support - Handbook of Applications and Advances of the Rough Sets Theory, pp. 331–362. Kluwer, Dordrecht (1992)Google Scholar
  15. 15.
    Ziarko, W.: The Discovery, analysis and representation of data dependencys in databases. In: Patesky-Shapiro, G., Frawley, W.J. (eds.) Knoweldge Discovery in Databases, MIT Press, Cambridge (1991)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Jianchao Han
    • 1
  • Xiaohua Hu
    • 2
  • Tsao Young Lin
    • 3
  1. 1.Dept. of Computer ScienceCalifornia State University Dominguez HillsCarsonUSA
  2. 2.College of Information Science and TechnologyDrexel UniversityPhiladelphiaUSA
  3. 3.Dept. of Computer ScienceSan Jose State UniversitySan JoseUSA

Personalised recommendations