Skip to main content
SpringerLink
Log in

Evolutionary Multi-objective Rule Selection for Classification Rule Mining

  • Chapter
Multi-Objective Evolutionary Algorithms for Knowledge Discovery from Databases

Part of the book series: Studies in Computational Intelligence ((SCI,volume 98))

This chapter discusses the application of evolutionary multi-objective optimization (EMO) to classification rule mining. In the field of classification rule mining, classifiers are designed through the following two phases: rule discovery and rule selection. In the rule discovery phase, a large number of classification rules are extracted from training data. This phase is based on two rule evaluation criteria: support and confidence. An association rule mining technique such as Apriori is usually used to extract classification rules satisfying pre-specified threshold values of the minimum support and confidence. In some studies, EMO algorithms were used to search for Pareto-optimal rules with respect to support and confidence. On the other hand, a small number of rules are selected from the extracted rules to design an accurate and compact classifier in the rule selection phase. A heuristic rule sorting criterion is usually used for rule selection. In some studies, EMO algorithms were used for multi-objective rule selection to maximize the accuracy of rule sets and minimize their complexity. In this chapter, first we explain the above-mentioned two phases in classification rule mining. Next we explain the search for Pareto-optimal rules and the search for Pareto-optimal rule sets. Then we explain evolutionary multi-objective rule selection as a post processing procedure in the second phase of classification rule mining. A number of Pareto-optimal rule sets are found from a large number of candidate rules, which are extracted from training data in the first phase. Finally we show experimental results on some data sets from the UCI machine learning repository. Through computational experiments, we demonstrate that evolutionary rule selection can drastically decrease the number of extracted rules without severely degrading their classification accuracy.We also examine the relation between Paretooptimal rules and Pareto-optimal rule sets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Cherkassky V, Mulier F (1998) Learning from data: concept, theory, and methods. John Wiley & Sons, New York

    Google Scholar 

  2. Deb K (2001) Multi-objective optimization using evolutionary algorithms. John Wiley & Sons, Chichester

    MATH  Google Scholar 

  3. Freitas A A (2002) Data mining and knowledge discovery with evolutionary algorithms. Springer, Berlin

    MATH  Google Scholar 

  4. Quinlan J R (1993) C4.5: Programs for machine learning. Morgan Kaufmann, San Mateo

    Google Scholar 

  5. Agrawal R, Mannila H, Srikant R, Toivonen H, Verkamo A I (1996) Fast discovery of association rules. In Fayyad U M, Piatetsky-Shapiro G, Smyth P, Uthurusamy R (eds) Advances in Knowledge Discovery and Data Mining. AAAI Press, 307–328

    Google Scholar 

  6. Casillas J, Cordon O, Herrera F, Magdalena L (2003) (eds) Interpretability issues in fuzzy modeling. Springer, Berlin

    MATH  Google Scholar 

  7. Casillas J, Cordon O, Herrera F, Magdalena L (2003) (eds) Accuracy improvements in linguistic fuzzy modeling. Springer, Berlin

    MATH  Google Scholar 

  8. Jin Y (2006) (eds) Multi-objective machine learning. Springer, Berlin

    MATH  Google Scholar 

  9. Bayardo Jr R J, Agrawal R (1999) Mining the most interesting rules. Proc. of 5th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 145–153

    Google Scholar 

  10. Cano J R, Herrera F, Lozano M (2005) Stratification for scaling up evolutionary prototype selection. Pattern Recognition Letters 26:953–963

    Article  Google Scholar 

  11. Cano J R, Herrera F, Lozano M (2006) On the combination of evolutionary algorithms and stratified strategies for training set selection in data mining. Applied Soft Computing 6: 323–332

    Article  Google Scholar 

  12. Chiu C C, Hsu P L (2005) A Constraint-based genetic algorithm approach for mining classification rules. IEEE Transactions on Systems, Man, and Cybernetics: Part C - Applications and Reviews 35: 205–220

    Article  Google Scholar 

  13. Coenen F, Leng P (2005) Obtaining best parameter values for accurate classification. Proc. of 5th IEEE International Conference on Data Mining, 549–552

    Google Scholar 

  14. Coenen F, Leng P, Zhang L (2005) Threshold tuning for improved classification association rule mining. Lecture Notes in Artificial Intelligence, Vol. 3518: Advances in Knowledge Discovery and Data Mining - PAKDD 2005. Springer, Berlin, 216–225

    Google Scholar 

  15. Curry R, Heywood M I (2004) Towards efficient training on large datasets for genetic programming. Lecture Notes in Artificial Intelligence, Vol. 3060: Advances in Artificial Intelligence - Canadian AI 2004. Springer, Berlin, 161–174

    Google Scholar 

  16. de la Iglesia B, Philpott M S, Bagnall A J, Rayward-Smith V J (2003) Data mining rules using multi-objective evolutionary algorithms. Proc. of 2003 Congress on Evolutionary Computation, 1552–1559

    Google Scholar 

  17. de la Iglesia B, Reynolds A, Rayward-Smith V J (2005) Developments on a multi-objective metaheuristic (MOMH) Algorithm for finding interesting sets of classification rules. Lecture Notes in Computer Science, Vol. 3410: Evolutionary Multi-Criterion Optimization - EMO 2005. Springer, Berlin, 826–840

    Google Scholar 

  18. de la Iglesia B, Richards G, Philpott M S, Rayward-Smith V J (2006) The application and effectiveness of a multi-objective metaheuristic algorithm for partial classification. European Journal of Operational Research 169: 898–917

    Article  MATH  MathSciNet  Google Scholar 

  19. Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6:182–197

    Article  Google Scholar 

  20. Elomaa T, Rousu J (1999) General and efficient multisplitting of numerical attributes. Machine Learning 36:201–244

    Article  MATH  Google Scholar 

  21. Ghosh A, Nath B T (2004) Multi-objective rule mining using genetic algorithms. Information Sciences 163:123–133

    Article  MathSciNet  Google Scholar 

  22. Ishibuchi H, Murata T, Turksen I B (1997) Single-objective and Two-objective genetic algorithms for selecting linguistic rules for pattern classification problems. Fuzzy Sets and Systems 89:135–150

    Article  Google Scholar 

  23. Ishibuchi H, Nakashima T, Murata T (2001) Three-objective genetics-based machine learning for linguistic rule extraction. Information Sciences 136:109–133

    Article  MATH  Google Scholar 

  24. Ishibuchi H, Nakashima T, Nii M (2004) Classification and modeling with linguistic information granules: Advanced approaches to linguistic data mining. Springer, Berlin

    Google Scholar 

  25. Ishibuchi H, Namba S (2004) Evolutionary multiobjective knowledge extraction for high-dimensional pattern classification problems. Lecture Notes in Computer Science, Vol. 3242: Parallel Problem Solving from Nature - PPSN VIII. Springer, Berlin, 1123–1132

    Google Scholar 

  26. Ishibuchi H, Nojima Y (2005) Accuracy-complexity tradeoff analysis by multiobjective rule selection. Proc. of ICDM 2005 Workshop on Computational Intelligence in Data Mining, 39–48

    Google Scholar 

  27. Ishibuchi H, Nojima Y (2006) Analysis of interpretability-accuracy tradeoff by multiobjective fuzzy genetics-based machine learning. International Journal of Approximate Reasoning (in press)

    Google Scholar 

  28. Ishibuchi H, Nozaki K, Yamamoto N, Tanaka H (1995) Selecting fuzzy if-then rules for classification problems using genetic algorithms. IEEE Transactions on Fuzzy Systems 3:260–270

    Article  Google Scholar 

  29. Ishibuchi H, Yamamoto T (2004) Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining. Fuzzy Sets and Systems 141:59–88

    Article  MATH  MathSciNet  Google Scholar 

  30. Kaya M (2006) Multi-objective genetic algorithm based approaches for mining optimized fuzzy association rules. Soft Computing 10:578–586

    Article  MATH  MathSciNet  Google Scholar 

  31. Kupinski M A, Anastasio M A (1999) Multi-objective genetic optimization of diagnostic classifiers with implications for generating receiver operating characteristic curve. IEEE Transactions on Medical Imaging 18:675–685

    Article  Google Scholar 

  32. Li W, Han J, Pei J (2001) CMAR: accurate and efficient classification based on multiple class-association rules. Proc. of 1st IEEE International Conference on Data Mining, 369–376

    Google Scholar 

  33. Liu B, Hsu W, Ma Y (1998) Integrating classification and association rule mining. Proc. of 4th International Conference on Knowledge Discovery and Data Mining, 80–86

    Google Scholar 

  34. Llora X, Goldberg D E (2003) Bounding the effect of noise in multi-objective learning classifier systems. Evolutionary Computation 11:278–297

    Article  Google Scholar 

  35. Mitra S, Pal S K, Mitra P (2002) Data mining in soft computing framework: A survey. IEEE Transactions on Neural Networks 13:3–14

    Article  Google Scholar 

  36. Mutter S, Hall M, Frank E (2004) Using classification to evaluate the output of confidence-based association rule mining. Lecture Notes in Artificial Intelligence, Vol. 3339: Advances in Artificial Intelligence - AI 2004. Springer, Berlin, 538–549

    Google Scholar 

  37. Pal S K, Talwar V, Mitra P (2002) Web mining in soft computing framework: Relevance, state of the art and future directions. IEEE Transactions on Neural Networks 13:1163–1177

    Article  Google Scholar 

  38. Reynolds A, de la Iglesia B (2006) Rule induction using multi-objective metaheuristics: Encouraging rule diversity. Proc. of 2006 International Joint Conference on Neural Networks, 6375–6382

    Google Scholar 

  39. Tan K C, Yu Q, Lee T H (2005) A distributed evolutionary classifier for knowledge discovery in data mining. IEEE Transactions on Systems, Man, and Cybernetics: Part C - Applications and Reviews 35:131–142

    Article  Google Scholar 

  40. Thabtah F, Cowling P, Hammoud S (2006) Improving rule sorting, predictive accuracy and training time in associative classification. Expert Systems with Applications 31:414–426

    Article  Google Scholar 

  41. Wang H, Kwong S, Jin Y, Wei W, Man K F (2005) Multi-objective hierarchical genetic algorithm for interpretable fuzzy rule-based knowledge extraction. Fuzzy Sets and Systems 149:149–186

    Article  MATH  MathSciNet  Google Scholar 

  42. Wang H, Kwong S, Jin Y, Wei W, Man K F (2005) Agent-based evolutionary approach for interpretable rule-based knowledge extraction. IEEE Transactions on Systems, Man, and Cybernetics: Part C - Applications and Reviews 35:143–155

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Intelligent Systems, Osaka Prefecture University, 599-8531, Osaka, Japan

    Professor Hisao Ishibuchi, Isao Kuwajima & Yusuke Nojima

Authors
  1. Professor Hisao Ishibuchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Isao Kuwajima
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yusuke Nojima
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Indian Statistical Institute, 203 B. T. Road, 700 108, Kolkata, India

    Ashish Ghosh

  2. F. M. University, 756 019, Balasore, India

    Satchidananda Dehuri

  3. Jadavpur University, 700 032, Kolkata, India

    Susmita Ghosh

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Ishibuchi, H., Kuwajima, I., Nojima, Y. (2008). Evolutionary Multi-objective Rule Selection for Classification Rule Mining. In: Ghosh, A., Dehuri, S., Ghosh, S. (eds) Multi-Objective Evolutionary Algorithms for Knowledge Discovery from Databases. Studies in Computational Intelligence, vol 98. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77467-9_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-77467-9_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-77466-2

  • Online ISBN: 978-3-540-77467-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation