Advertisement

Rough Set Algorithms in Classification Problem

  • Jan G. Bazan
  • Hung Son Nguyen
  • Sinh Hoa Nguyen
  • Piotr Synak
  • Jakub Wróblewski
Part of the Studies in Fuzziness and Soft Computing book series (STUDFUZZ, volume 56)

Abstract

We we present some algorithms, based on rough set theory, that can be used for the problem of new cases classification. Most of the algorithms were implemented and included in Rosetta system [43]. We present several methods for computation of decision rules based on reducts. We discuss the problem of real value attribute discretization for increasing the performance of algorithms and quality of decision rules. Finally we deal with a problem of resolving conflicts between decision rules classifying a new case to different categories (classes). Keywords: knowledge discovery, rough sets, classification algorithms, reducts, decision rules, real value attribute discretization

Keywords

Decision Rule Decision Table Stability Coefficient Dynamic Rule Optimal Decision Rule 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Agrawal, R., Mannila, H., Srikant, R., Toivonen, H., Verkamo, I.: Fast Discovery of Association Rules, Proceedings of the Advances in Knowledge Discovery and Data Mining. AAAI Press/The MIT Press, CA (1996) 307–328.Google Scholar
  2. 2.
    Almuallim, H., Dietterich, T. G.: Learning with many irrelevant features, Proceedings of the Ninth National Conference on Artificial Intelligence (1991) 547–552.Google Scholar
  3. 3.
    Bazan, J., Skowron, A., Synak, P.: Discovery of Decision Rules from Experimental Data, Proceedings of the Third International Workshop on Rough Sets and Soft Computing. San Jose, California (1994) 526–533.Google Scholar
  4. 4.
    Bazan, J., Skowron, A., Synak, P.: Dynamic reducts as a tool for extracting laws from decision tables, Proceedings of the Eighth International Symposium on Methodologies for Intelligent Systems (ISMIS’94), Lecture Notes in Artificial Intelligence 869. Berlin: Springer-Verlag (1994) 346–355,.Google Scholar
  5. 5.
    Bazan, J., Skowron, A., Synak, P.: Market data analysis: A rough set approach. ICS Research Report 6/94, Warsaw University of Technology (1994).Google Scholar
  6. 6.
    Bazan, J.: Dynamic reducts and statistical inference, Proceedings of Information Processing and Management of Uncertainty on Knowledge Based Systems (IPMU96), July 1–5, Granada, Spain, Universidad de Granada, vol. III, (1996) 1147–1152.MATHGoogle Scholar
  7. 7.
    Bazan, J., Nguyen, H. S., Nguyen, T. T., Skowron, A., Stepaniuk, J.: Synthesis of Decision Rules for Object Classification, Orowska E. (ed.): Incomplete Information: Rough Set Analysis. Heidelberg: Physica-Verlag (1998) 23–57.Google Scholar
  8. 8.
    Bazan, J.: Discovery of Decision Rules by Matching New Objects Against Data Tables. Proceedings of the First International Conference on Rough Sets and Current Trends in Computing (RSCTC-98), Warsaw, June 22–26 (1998), Lecture Notes in Artificial Intelligence 1424. Berlin: Springer-Verlag (1998) 521–528.Google Scholar
  9. 9.
    Bazan, J.: A Comparison of Dynamic and non-Dynamic Rough Set Methods for Extracting Laws from Decision Table, Polkowski L., Skowron A. (eds.): Rough Sets in Knowledge Discovery. Heidelberg: Physica-Verlag (1998) 321–365.Google Scholar
  10. 10.
    Bloedorn, E., Michalski, R. S.: Data Driven Constructive Induction in AQ17-PRE: A Method and Experiments, Proceedings of the Third International Conference on Tools for AI. San Jose, CA (1991)Google Scholar
  11. 11.
    Brown, E. M.: Boolean reasoning. Dordrecht: Kluwer (1990).MATHGoogle Scholar
  12. 12.
    Brownlee, K. A.: Statistical theory and methodology in science and engineering. New York: John Wiley&Sons (1965).MATHGoogle Scholar
  13. 13.
    Cestnik, B., Kononenko, I., Bratko, I.: ASSISTANT 86: A Knowledge Elicitation Tool for Sophisticated Users, Proceedings of EWSL-87. Bled, Yugoslavia (1987) 31–47.Google Scholar
  14. 14.
    Clark, P., Niblett, T.: The CN2 Induction Algorithm, Machine Learning 3. Kluwer Academic, Boston, MA (1989) 261–284.Google Scholar
  15. 15.
    Cykier, A.: Prime Implicants of Boolean Functions — Applications and Methods of Computations (in Polish), MSc Thesis, University of Warsaw, Warsaw, Poland (1997).Google Scholar
  16. 16.
    Downton, A. C., Tregidgo, R. W. S., Leedham, C. G.: Recognition of handwritten British postal addresses. From Pixels to Features III. Frontiers in Handwriting Recognition North-Holland (1992) 129–144.Google Scholar
  17. 17.
    Dzeroski, S.: Handling Noise in Inductive Logic Programming. MS Thesis, Dept. of EE and CS, University of Ljubljana, Slovenia (1991).Google Scholar
  18. 18.
    Fisz, M.: Probability theory and mathematical statistics, New York (1961).Google Scholar
  19. 19.
    Fahlman, S. E., Lebiere, C.: The Cascade-Correlation Learning Architecture, in Advances in Neural Information Processing Systems, vol. II. Morgan Kaufmann, San Mateo, CA (1990).Google Scholar
  20. 20.
    Friedman, J.: Smart user’s guide. Technical Report 1. Laboratory of Computational Statistics, Department of Statistics, Stanford University (1984).Google Scholar
  21. 21.
    Goldberg, D. E.: GA in Search, Optimisation, and Machine Learning. AddisonWesley (1989).Google Scholar
  22. 22.
    Grzymala-Busse, J. W.: LERS — a system for learning from examples based on rough sets. In R. Slowiński, (ed.) Intelligent Decision Support, Kluwer Academic Publishers, Dordrecht, Boston, London (1992) 3–18.Google Scholar
  23. 23.
    Grzymala-Busse, J. W.: A new version of the rule induction system LERS. Fundamenta Informaticae 31 (1997) 27–39.MATHGoogle Scholar
  24. 24.
    Holland, J. H.: Adaptation in Natural and Artificial Systems. The MIT Press, Cambridge (1992).Google Scholar
  25. 25.
    Keeping, E. S.: Introduction to statistical inference. Prinston, New Jersey: D.Van Nostrand Company, Inc. (1962).Google Scholar
  26. 26.
    Kira, K., Rendell, L. A.: A practical approach to feature selection, In D. Sleeman (ed.), Proceedings of the Ninth International Workshop on Machine Learning (ML92), Morgan Kaufmann (1992) 249–256.Google Scholar
  27. 27.
    Kittler, J.: Feature selection and extraction, In Young and Fu (ed.), Handbook of pattern recognition and image processing. New York: Academic Press (1996).Google Scholar
  28. 28.
    Kodratoff, Y., Michalski, M. (ed.): Machine Learning vol. III. Morgan Kaufmann, San Mateo, CA (1990).Google Scholar
  29. 29.
    Michalski, R., Carbonell, J. G. and Mitchel, T. M. (ed): Machine Learning vol. I. Tioga/Morgan Kaufmann, Los Altos, CA (1983).Google Scholar
  30. 30.
    Michalski, R., Carbonell, J. G. and Mitchel, T. M. (ed): Machine Learning vol. II. Morgan Kaufmann, Los Altos, CA (1986).Google Scholar
  31. 31.
    Michalski, R. S., Mozetic, I., Hong, J. and Lavrac, N.: The Multi-Purpose Incremental Learning System AQ15 and its Testing to Three Medical Domains, Proceedings of AAAI-86. Morgan Kaufmann, San Mateo, CA (1986) 1041–1045.Google Scholar
  32. 32.
    Michalski, R., Wnęk, J.: Constructive Induction: An Automated Improvement of Knowledge Representation Spaces for Machine Learning, Proceedings of a Workshop on Intelligent Information Systems, Practical Aspect of AI II, Augustów (Poland) (1993) 188–236.Google Scholar
  33. 33.
    Michie, D., Spiegelhalter, D. J., Taylor, C. C.: Machine learning, neural and statistical classification. England: Ellis Horwood Limited (1994).MATHGoogle Scholar
  34. 34.
    Mienko, R., Slowiński, R., Stefanowski, J., Susmaga, R.: Rough Family — software implementation of rough set based data analysis and rule discovery techniques, Tsumoto S. (ed.), Proceedings of the Fourth International Workshop on Rough Sets, Fuzzy Sets and Machine Discovery, Tokyo, November 6–8 (1996), 437–440.Google Scholar
  35. 35.
    Mollestad, T.: A rough set approach to default rules data mining. PhD Thesis, supervisor J. Komorowski, Norvegian Institute of Technology, Trondheim, Norway (1996)Google Scholar
  36. 36.
    Muggelton, S. (ed.): Inductive logic programming. Academic Press (1992).Google Scholar
  37. 37.
    Nguyen, H. S., Nguyen, S. H., Skowron, A.: Searching for features defined by hyperplanes, Z.W. Ras, M. Michalewicz (ed.), Proceedings of Ninth International Symposium on Methodologies for Intelligent Systems (ISMIS-96), Zakopane, Poland, June 10–13, (1996). Lecture Notes in Artificial Intelligence vol. 1079, Springer, Berlin (1996) 366–375; see also: ICS Research Report 16/95, Warsaw University of Technology.Google Scholar
  38. 38.
    Nguyen, S. H., Nguyen, H. S.: Some efficient algorithms for rough set methods, Proceedings Information Processing and Management of Uncertainty on Knowledge Based Systems (IPMU-96), July 1–5, Granada, Spain, Universidad de Granada, vol. III, (1996) 1451–1456.Google Scholar
  39. 39.
    Nguyen, H. S., Skowron, A.: Quantization of real value attributes, Proceedings of Second Joint Annual Conf. on Information Sciences, Wrightsville Beach, North Carolina, September 28 — October 1, USA (1995) 34–37.Google Scholar
  40. 40.
    Nguyen, S. H., Skowron, A., Synak, P., Wróblewski, J.: Knowledge Discovery in Databases: Rough Set Approach. Proc. of The Seventh International Fuzzy Systems Association World Congress, vol. II, pp. 204–209, IFSA97, Prague, Czech Republic (1997).Google Scholar
  41. 41.
    Nguyen, H. S.: Discretization of Real Value Attributes: Boolean reasoning Approach. Ph.D. Thesis, Warsaw University, Warsaw, Poland (1997).Google Scholar
  42. 42.
    Nguyen, T., Winiarski, R., Skowron, A., Bazan, J., Thyagarajan, K.: Application of Rough Sets, Neural Networks and Maximum Likelihood for Texture Classification Based on Singular Value Decomposition, Proceedings of the Third International Workshop on Rough Sets and Soft Computing. San Jose, California (1994) 332–339.Google Scholar
  43. 43.
    Oehrn, A., Komorowski, J.: ROSETTA — A rough set tool kit for analysis of data. Proceedings of the Fifth International Workshop on Rough Sets and Soft Computing (RSSC’97) at the Third Joint Conference on Information Sciences (JCIS’97), Research Triangle Park, NC, March 2–5 (1997) 403–407.Google Scholar
  44. 44.
    Pawlak, Z.: Rough sets: Theoretical aspects of reasoning about data. Dordrecht: Kluwer 1991.Google Scholar
  45. 45.
    Pawlak, Z., Skowron, A.: A rough set approach for decision rules generation, ICS Research Report 23/93, Warsaw University of Technology, Proceedings of the IJCAI’93 Workshop W12: The Management of Uncertainty in AI, France 1993.Google Scholar
  46. 46.
    Piasta, Z., Lenarcik, A., Tsumoto S.: Machine discovery in databases with probabilistic rough classifiers. In: S. Tsumoto, S. Kobayashi, T. Yokomori, H. Tanaka and A. Nakamura (eds.), Proceedings of The fourth International Workshop on Rough Sets, Fuzzy Sets, and Machine Discovery (RS96FD), November 6–8, The University of Tokyo (1996) 353–359Google Scholar
  47. 47.
    Polkowski, L., Skowron, A.: Synthesis of decision systems from data tables. In: T. Y. Lin and N. Cecerone (eds.), Rough Sets and Data Mining. Analysis for Imprecise Data, Kluwer Academic Publishers, Boston, London, Dordrecht (1997) 259–299.Google Scholar
  48. 48.
    Quinlan, J. R.: Induction of Decision Trees, Machine Learning 1. Kluwer Academic, Boston, MA (1986) 81–106.Google Scholar
  49. 49.
    Quinlan, J. R.: C4.5:Programs for MachineLearning. San Mateo, California: Morgan Kaufmann (1993).Google Scholar
  50. 50.
    De Raedt, L.: Interactive Theory Revision. An Inductive Logic Programming. Academic Press (1992).Google Scholar
  51. 51.
    Skowron, A., Rauszer, C.: The Discernibility Matrices and Functions in Information Systems. In R. Slowiński (ed.), Intelligent Decision Support. Handbook of Applications and Advances of the Rough Sets Theory. Dordrecht: Kluwer (1992) 331–362.Google Scholar
  52. 52.
    Skowron, A.: Boolean reasoning for decision rules generation, Proceedings of the 7th International Symposium ISMIS’93, Trondheim, Norway 1993, In Komorowski J. and Ras Z. (ed.), Lecture Notes in Artificial Intelligence, vol. 689. Springer-Verlag (1993) 295–305.Google Scholar
  53. 53.
    Skowron, A.: A synthesis of decision rules: Applications of discernibility matrix properties, Proceedings of the Workshop Intelligent Information Systems, Augustów (Poland), 7–11 June, 1993.Google Scholar
  54. 54.
    Slowiński, R. (ed.): Intelligent Decision Support. Handbook of Applications and Advances of the Rough Sets Theory. Dordrecht: Kluwer (1992).Google Scholar
  55. 55.
    Slowiński, R., Stefanowski, J.:’Rough DAS’ and ‘Rough Class’ software implementations of the rough set approach, Słowiński R. (ed.): Intelligent Decision Support. Handbook of Applications and Advances of the Rough Sets Theory. Dordrecht: Kluwer (1992) 445–456.Google Scholar
  56. 56.
    Thrun, S. B, Bala, J., Bloedorn, E., Bratko, I., Cestnink, B., Cheng, J., De Jong, K. A., Dzeroski, S., Fahlman, S. E., Hamann, R., Kaufman, K., Keller, S., Kononenko, I., Kreuziger, J., Michalski, R.S., Mitchell, T., Pachowicz, P., Vafaie, H., Van de Velde, W., Wenzel, W., Wnçk, J., and Zhang, J.: The MONK’s Problems: A Performance Comparison of Different Learning Algorithms. Technical Report, Carnegie Mellon University (1991).Google Scholar
  57. 57.
    Tsumoto, S., Tanaka H.: Incremental learning of probabilistic rules from clinical databases. Proceedings Information Processing and Management of Uncertainty on Knowledge Based Systems (IPMU-96), July 1–5, Granada, Spain, Universidad de Granada, vol. II, (1996) 1457–1462.Google Scholar
  58. 58.
    Utgoff, P. E.: Incremental Learning of Decision Trees, Machine Learning, vol. IV. Kluwer Academic, Boston, MA (1990) 161–186.Google Scholar
  59. 59.
    Vafaie, L. G., De Jong, K.: Improving the Performance of a Rule Induction System Using Genetic Algorithm, Proceedings of the First International Workshop on Multistrategy Learning. Harpers Ferry WV, George Mason University, Center for Artificial Intelligence (1991) 305–315.Google Scholar
  60. 60.
    Van De Velde, W.: IDL, or Taming the Multiplexer, Proceedings of the 4th European Working Session on Learning. Pitman, London (1989).Google Scholar
  61. 61.
    Wnęk, J., Michalski, R. S.: Hypothesis-driven Constructive Induction in AQ17: A Method and Experiments, Proceedings of the IJCAI-91 Workshop on Evaluating and Changing Representation, K. Mork, F. Bergadano and W. Buntine (ed.). Sydney, Australia (1991) 13–22.Google Scholar
  62. 62.
    Wróblewski, J.: Finding minimal reducts using genetic algorithm (extended version), Proceedings of Second Joint Annual Conference on Information Sciences, Wrightsville Beach, North Carolina, 28 September — 1 October, USA, (1995) 186– 189; see also: ICS Research Report 16/95, Warsaw University of Technology.Google Scholar
  63. 63.
    Wróblewski, J.: Theoretical Foundations of Order-Based Genetic Algorithms. Fundamenta Informaticae, vol. 28 (3, 4), pp: 423–430. IOS Press, (1996).MathSciNetMATHGoogle Scholar
  64. 64.
    Wróblewski, J.: Genetic algorithms in decomposition and classification problem. In: L. Polkowski, A. Skowron (eds.). Rough Sets in Knowledge Discovery. Physica Verlag, (1998).Google Scholar
  65. 65.
    Wróblewski, J.: Covering with reducts — a fast algorithm for rule generation, Proceedings of RSCTC’98, Springer-Verlag (LNAI 1424), Berlin Heidelberg (1998) 402 – 407.Google Scholar
  66. 66.
    Ziarko, W., Shan, N.: An incremental learning algorithm for constructing decision rules, Proceedings of the International Workshop on Rough Sets and Knowledge Discovery. Banff. (1993) 335–346.Google Scholar
  67. 67.
    Ziarko, W.: Variable Precision Rough Set Model. Journal of Computer and System Sciences 40 (1993) 39–59.MathSciNetCrossRefGoogle Scholar

Copyright information

© Physica-Verlag Heidelberg 2000

Authors and Affiliations

  • Jan G. Bazan
    • 1
  • Hung Son Nguyen
    • 2
    • 3
  • Sinh Hoa Nguyen
    • 2
    • 3
  • Piotr Synak
    • 3
  • Jakub Wróblewski
    • 2
    • 3
  1. 1.Institute of MathematicsPedagogical University of RzeszówRzeszówPoland
  2. 2.Institute of MathematicsWarsaw UniversityWarsawPoland
  3. 3.Polish-Japanese Institute of Information TechnologyWarsawPoland

Personalised recommendations