Greedy Algorithm for the Construction of Approximate Decision Rules for Decision Tables with Many-Valued Decisions

  • Mohammad AzadEmail author
  • Mikhail Moshkov
  • Beata Zielosko
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10020)


The paper is devoted to the study of a greedy algorithm for construction of approximate decision rules. This algorithm is applicable to decision tables with many-valued decisions where each row is labeled with a set of decisions. For a given row, we should find a decision from the set attached to this row. We consider bounds on the precision of this algorithm relative to the length of rules. To illustrate proposed approach we study a problem of recognition of labels of points in the plain. This paper contains also results of experiments with modified decision tables from UCI Machine Learning Repository.



Research reported in this publication was supported by the King Abdullah University of Science and Technology (KAUST).

The authors wish to express their gratitude to anonymous reviewers for useful comments.


  1. 1.
    Alkhalid, A., Amin, T., Chikalov, I., Hussain, S., Moshkov, M., Zielosko, B.: Dagger: a tool for analysis and optimization of decision trees andrules. Comput. Inf. Soc. Factors New Inf. Technol. Hypermedia Perspect. Avant-Garde Experiences Eraof Communicability Expansion, 29–39 (2011)Google Scholar
  2. 2.
    Azad, M., Chikalov, I., Moshkov, M., Zielosko, B.: Greedy algorithms for construction of approximate tests for decision tables with many-valued decisions. Fundamenta Informaticae 120(3–4), 231–242 (2012)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Blockeel, H., Schietgat, L., Struyf, J., Džeroski, S., Clare, A.: Decision trees for hierarchical multilabel classification: a case study in functional genomics. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) PKDD 2006. LNCS (LNAI), vol. 4213, pp. 18–29. Springer, Heidelberg (2006). doi: 10.1007/11871637_7 CrossRefGoogle Scholar
  4. 4.
    Boutell, M.R., Luo, J., Shen, X., Brown, C.M.: Learning multi-label scene classification. Pattern Recogn. 37(9), 1757–1771 (2004)CrossRefGoogle Scholar
  5. 5.
    Cheriyan, J., Ravi, R.: Lecture notes on approximation algorithms for network problems (1998).
  6. 6.
    Chikalov, I., Zielosko, B.: Decision rules for decision tables with many-valued decisions. In: Yao, J.T., Ramanna, S., Wang, G., Suraj, Z. (eds.) RSKT 2011. LNCS (LNAI), vol. 6954, pp. 763–768. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-24425-4_95 CrossRefGoogle Scholar
  7. 7.
    Clare, A., King, R.D.: Knowledge discovery in multi-label phenotype data. In: Raedt, L., Siebes, A. (eds.) PKDD 2001. LNCS (LNAI), vol. 2168, pp. 42–53. Springer, Heidelberg (2001). doi: 10.1007/3-540-44794-6_4 CrossRefGoogle Scholar
  8. 8.
    Comité, F., Gilleron, R., Tommasi, M.: Learning multi-label alternating decision trees from texts and data. In: Perner, P., Rosenfeld, A. (eds.) MLDM 2003. LNCS, vol. 2734, pp. 35–49. Springer, Heidelberg (2003). doi: 10.1007/3-540-45065-3_4 CrossRefGoogle Scholar
  9. 9.
    Feige, U.: A threshold of ln n for approximating set cover. J. ACM (JACM) 45(4), 634–652 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Greco, S., Matarazzo, B., Słowiński, R.: Rough sets theory for multicriteria decision analysis. Eur. J. Oper. Res. 129(1), 1–47 (2001)CrossRefzbMATHGoogle Scholar
  11. 11.
    Kryszkiewicz, M.: Rules in incomplete information systems. Inf. Sci. 113(34), 271–292 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Lichman, M.: UCI Machine Learning Repository (2013)Google Scholar
  13. 13.
    Lipski, W.: On databases with incomplete information. J. ACM (JACM) 28(1), 41–70 (1981)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Lipski Jr., W.: On semantic issues connected with incomplete information databases. ACM Trans. Database Syst. 4(3), 262–296 (1979)CrossRefGoogle Scholar
  15. 15.
    Mencia, E.L., Furnkranz, J.: Pairwise learning of multilabel classifications with perceptrons. In: IEEE International Joint Conference on Neural Networks, 2008, IJCNN 2008 (IEEE World Congress on Computational Intelligence), pp. 2899–2906 (2008)Google Scholar
  16. 16.
    Moshkov, M.J., Piliszczuk, M., Zielosko, B.: Partial Covers, Reducts and Decision Rules in Rough Sets–Theory and Applications. SCI, vol. 145. Springer, Heidelberg (2008)zbMATHGoogle Scholar
  17. 17.
    Moshkov, M., Zielosko, B.: Combinatorial Machine Learning–A Rough Set Approach. SCI, vol. 360. Springer, Heidelberg (2011)CrossRefzbMATHGoogle Scholar
  18. 18.
    Moshkov, M., Zielosko, B.: Construction of \(\alpha \)-decision trees for tables with many-valued decisions. In: Yao, J.T., Ramanna, S., Wang, G., Suraj, Z. (eds.) RSKT 2011. LNCS (LNAI), vol. 6954, pp. 486–494. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-24425-4_63 CrossRefGoogle Scholar
  19. 19.
    Moshkov, M.J.: Greedy algorithm for decision tree construction in context of knowledge discovery problems. In: Tsumoto, S., Słowiński, R., Komorowski, J., Grzymała-Busse, J.W. (eds.) RSCTC 2004. LNCS (LNAI), vol. 3066, pp. 192–197. Springer, Heidelberg (2004). doi: 10.1007/978-3-540-25929-9_22 CrossRefGoogle Scholar
  20. 20.
    Nguyen, H.S., Slezak, D.: Approximate reducts and association rules - correspondence and complexity results. In: Proceedings of the 7th International Workshop on New Directions in Rough Sets, Data Mining, and Granular-Soft Computing. RSFDGrC 1999, pp. 137–145. Springer, London (1999)Google Scholar
  21. 21.
    Orowska, E., Pawlak, Z.: Representation of nondeterministic information. Theoret. Comput. Sci. 29(12), 27–39 (1984)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Pawlak, Z.: Rough Sets-Theoretical Aspects of Reasoning about Data. Kluwer Academic Publishers, Dordrecht (1991)zbMATHGoogle Scholar
  23. 23.
    Pawlak, Z., Skowron, A.: Rough sets and boolean reasoning. Inf. Sci. 177(1), 41–73 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Pawlak, Z., Skowron, A.: Rough sets: some extensions. Inf. Sci. 177(1), 28–40 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Pawlak, Z., Skowron, A.: Rudiments of rough sets. Inf. Sci. 177(1), 3–27 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Rissanen, J.: Modeling by shortest data description. Automatica 14(5), 465–471 (1978)CrossRefzbMATHGoogle Scholar
  27. 27.
    Sakai, H., Ishibashi, R., Koba, K., Nakata, M.: Rules and apriori algorithm in non-deterministic information systems. In: Peters, J.F., Skowron, A., Rybiński, H. (eds.) Transactions on Rough Sets IX. LNCS, vol. 5390, pp. 328–350. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-89876-4_18 CrossRefGoogle Scholar
  28. 28.
    Sakai, H., Nakata, M., Ślȩzak, D.: Rule generation in lipski’s incomplete information databases. In: Szczuka, M., Kryszkiewicz, M., Ramanna, S., Jensen, R., Hu, Q. (eds.) RSCTC 2010. LNCS (LNAI), vol. 6086, pp. 376–385. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-13529-3_40 CrossRefGoogle Scholar
  29. 29.
    Sakai, H., Nakata, M., Ślęzak, D.: A prototype system for rule generation in lipski’s incomplete information databases. In: Kuznetsov, S.O., Ślęzak, D., Hepting, D.H., Mirkin, B.G. (eds.) RSFDGrC 2011. LNCS (LNAI), vol. 6743, pp. 175–182. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-21881-1_29 CrossRefGoogle Scholar
  30. 30.
    Skowron, A., Rauszer, C.: The discernibility matrices and functions in information systems. In: Intelligent Decision Support. Handbook of Applications and Advances of the Rough Set Theory, pp. 331–362. Kluwer Academic Publishers (1992)Google Scholar
  31. 31.
    Ślȩzak, D.: Normalized decision functions and measures for inconsistent decision tables analysis. Fundamenta Informaticae 44(3), 291–319 (2000)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Ślȩzak, D.: Approximate entropy reducts. Fundamenta Informaticae 53(3–4), 365–390 (2002)MathSciNetzbMATHGoogle Scholar
  33. 33.
    Tsoumakas, G., Katakis, I.: Multi-label classification: an overview. Int. J. Data Warehouse. Min. 3(3), 1–13 (2007)CrossRefGoogle Scholar
  34. 34.
    Tsoumakas, G., Katakis, I., Vlahavas, I.: Mining multi-label data. In: Data Mining and Knowledge Discovery Handbook, pp. 667–685. Springer, US (2010)Google Scholar
  35. 35.
    Wieczorkowska, A., Synak, P., Lewis, R., Raś, Z.W.: Extracting emotions from music data. In: Hacid, M.-S., Murray, N.V., Raś, Z.W., Tsumoto, S. (eds.) ISMIS 2005. LNCS (LNAI), vol. 3488, pp. 456–465. Springer, Heidelberg (2005). doi: 10.1007/11425274_47 CrossRefGoogle Scholar
  36. 36.
    Zhou, Z.H., Jiang, K., Li, M.: Multi-instance learning based web mining. Appl. Intell. 22(2), 135–147 (2005)CrossRefGoogle Scholar
  37. 37.
    Zhou, Z.H., Zhang, M.L., Huang, S.J., Li, Y.F.: Multi-instance multi-label learning. Artif. Intell. 176(1), 2291–2320 (2012)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany 2016

Authors and Affiliations

  • Mohammad Azad
    • 1
    Email author
  • Mikhail Moshkov
    • 1
  • Beata Zielosko
    • 2
  1. 1.Computer, Electrical and Mathematical Sciences and Engineering DivisionKing Abdullah University of Science and TechnologyThuwalSaudi Arabia
  2. 2.Institute of Computer ScienceUniversity of SilesiaSosnowiecPoland

Personalised recommendations