Evaluating Learning Algorithms with Meta-learning Schemes for a Rule Evaluation Support Method Based on Objective Indices

  • Hidenao Abe
  • Shusaku Tsumoto
  • Miho Ohsaki
  • Hideto Yokoi
  • Takahira Yamaguchi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4303)


In this paper, we present evaluations of learning algorithms for a novel rule evaluation support method in data mining post-processing, which is one of the key processes in a data mining process. It is difficult for human experts to evaluate many thousands of rules from a large dataset with noises completely. To reduce the costs of rule evaluation task, we have developed the rule evaluation support method with rule evaluation models, which are learned from a dataset consisted of objective indices and evaluations of a human expert for each rule. To enhance adaptability of rule evaluation models, we introduced a constructive meta-learning system to choose proper learning algorithms for constructing them. Then, we have done a case study on the meningitis data mining result, the hepatitis data mining results and rule sets from the eight UCI datasets.


Training Dataset Human Expert Weight Vote Rule Evaluation Objective Index 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abe, H., Yamaguchi, T.: Constructive Meta-learning with Machine Learning Method Repositories. In: Orchard, B., Yang, C., Ali, M. (eds.) IEA/AIE 2004. LNCS (LNAI), vol. 3029, pp. 502–511. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Abe, H., Ohsaki, M., Yokoi, H., Yamaguchi, T.: Implementing an Integrated Time-Series Data Mining Environment Based on Temporal Pattern Extraction Methods: A Case Study of an Interferon Therapy Risk Mining for Chronic Hepatitis. In: Washio, T., Sakurai, A., Nakajima, K., Takeda, H., Tojo, S., Yokoo, M. (eds.) JSAI Workshop 2006. LNCS (LNAI), vol. 4012, pp. 425–435. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  3. 3.
    Ali, K., Manganaris, S., Srikant, R.: Partial Classification Using Association Rules. In: Proc. of Int. Conf. on Knowledge Discovery and Data Mining KDD 1997, pp. 115–118 (1997)Google Scholar
  4. 4.
    Booker, L.B., Holland, J.H., Goldberg, D.E.: Classifier Systems and Genetic Algorithms. Artificail Inteligence 40, 235–282 (1989)CrossRefGoogle Scholar
  5. 5.
    Breiman, L.: Bagging Predictors. Machine Learning 24(2), 123–140 (1996)zbMATHMathSciNetGoogle Scholar
  6. 6.
    Brin, S., Motwani, R., Ullman, J., Tsur, S.: Dynamic itemset counting and implication rules for market basket data. In: Proc. of ACM SIGMOD Int. Conf. on Management of Data, pp. 255–264 (1997)Google Scholar
  7. 7.
    Frank, E., Wang, Y., Inglis, S., Holmes, G., Witten, I.H.: Using model trees for classification. Machine Learning 32(1), 63–76 (1998)zbMATHCrossRefGoogle Scholar
  8. 8.
    Frank, E., Witten, I.H.: Generating accurate rule sets without global optimization. In: Proc. of the Fifteenth International Conference on Machine Learning, pp. 144–151 (1998)Google Scholar
  9. 9.
    Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Proc. of Thirteenth International Conference on Machine Learning, pp. 148–156 (1996)Google Scholar
  10. 10.
    Gago, P., Bento, C.: A Metric for Selection of the Most Promising Rules. In: Proc. of Euro. Conf. on the Principles of Data Mining and Knowledge Discovery PKDD 1998, pp. 19–27 (1998)Google Scholar
  11. 11.
    Goodman, L.A., Kruskal, W.H.: Measures of association for cross classifications. Springer Series in Statistics, vol. 1. Springer, Heidelberg (1979)zbMATHGoogle Scholar
  12. 12.
    Gray, B., Orlowska, M.E.: CCAIIA: Clustering Categorical Attributes into Interesting Association Rules. In: Proc. of Pacific-Asia Conf. on Knowledge Discovery and Data Mining PAKDD 1998, pp. 132–143 (1998)Google Scholar
  13. 13.
    Hamilton, H.J., Shan, N., Ziarko, W.: Machine Learning of Credible Classifications. In: Proc. of Australian Conf. on Artificial Intelligence AI 1997, pp. 330–339 (1997)Google Scholar
  14. 14.
    Hatazawa, H., Negishi, N., Suyama, A., Tsumoto, S., Yamaguchi, T.: Knowledge Discovery Support from a Meningoencephalitis Database Using an Automatic Composition Tool for Inductive Applications. In: Terano, T., Chen, A.L.P. (eds.) PAKDD 2000. LNCS, vol. 1805, pp. 28–33. Springer, Heidelberg (2000)Google Scholar
  15. 15.
    Hettich, S., Blake, C.L., Merz, C.J.: UCI Repository of machine learning databases, Irvine, CA: University of California, Department of Information and Computer Science (1998),
  16. 16.
    Hilderman, R.J., Hamilton, H.J.: Knowledge Discovery and Measure of Interest. Kluwer Academic Publishers, Dordrecht (2001)Google Scholar
  17. 17.
    Hinton, G.E.: Learning distributed representations of concepts. In: Proc. of 8th Annual Conference of the Cognitive Science Society, Amherest, MA (1986) REprinted in R.G.M.Morris (ed.)Google Scholar
  18. 18.
    Holte, R.C.: Very simple classification rules perform well on most commonly used datasets. Machine Learning 11, 63–91 (1993)zbMATHCrossRefGoogle Scholar
  19. 19.
    Klösgen, W.: Explora: A Multipattern and Multistrategy Discovery Assistant. In: Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R. (eds.) Advances in Knowledge Discovery and Data Mining, pp. 249–271. AAAI/MIT Press, California (1996)Google Scholar
  20. 20.
    Michalski, R., Mozetic, I., Hong, J., Lavrac, N.: The AQ15 Inductive Learning System: An Over View and Experiments, Reports of Machine Learning and Inference Laboratory, No.MLI-86-6, George Mason University (1986)Google Scholar
  21. 21.
    Mitchell, T.M.: Generalization as Search. Artificial Intelligence 18(2), 203–226 (1982)CrossRefMathSciNetGoogle Scholar
  22. 22.
    Ohsaki, M., Sato, Y., Kitaguchi, S., Yokoi, H., Yamaguchi, T.: Comparison between Objective Interestingness Measures and Real Human Interest in Medical Data Mining. In: Orchard, B., Yang, C., Ali, M. (eds.) IEA/AIE 2004. LNCS (LNAI), vol. 3029, pp. 1072–1081. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  23. 23.
    Ohsaki, M., Kitaguchi, S., Kume, S., Yokoi, H., Yamaguchi, T.: Evaluation of Rule Interestingness Measures with a Clinical Dataset on Hepatitis. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) PKDD 2004. LNCS (LNAI), vol. 3202, pp. 362–373. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  24. 24.
    Piatetsky-Shapiro, G.: Discovery, Analysis and Presentation of Strong Rules. In: Piatetsky-Shapiro, G., Frawley, W.J. (eds.) Knowledge Discovery in Databases, pp. 229–248. AAAI/MIT Press (1991)Google Scholar
  25. 25.
    Platt, J.: Fast Training of Support Vector Machines using Sequential Minimal Optimization. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods - Support Vector Learning, pp. 185–208. MIT Press, Cambridge (1999)Google Scholar
  26. 26.
    Quinlan, J.R.: Induction of Decision Tree. Machine Learning 1, 81–106 (1986)Google Scholar
  27. 27.
    Quinlan, R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, San Francisco (1993)Google Scholar
  28. 28.
    Rijsbergen, C.: Information Retrieval, ch. 7. Butterworths, London (1979), Google Scholar
  29. 29.
    Smyth, P., Goodman, R.M.: Rule Induction using Information Theory. In: Piatetsky-Shapiro, G., Frawley, W.J. (eds.) Knowledge Discovery in Databases, pp. 159–176. AAAI/MIT Press (1991)Google Scholar
  30. 30.
    Tan, P.N., Kumar, V., Srivastava, J.: Selecting the Right Interestingness Measure for Association Patterns. In: Proc. of Int. Conf. on Knowledge Discovery and Data Mining KDD 2002, pp. 32–41 (2002)Google Scholar
  31. 31.
    Witten, I.H., Frank, E.: DataMining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Francisco (2000)Google Scholar
  32. 32.
    Wolpert, D.: Stacked Generalization. Neural Network 5(2), 241–260 (1992)CrossRefGoogle Scholar
  33. 33.
    Yao, Y.Y., Zhong, N.: An Analysis of Quantitative Measures Associated with Rules. In: Proc. of Pacific-Asia Conf. on Knowledge Discovery and Data Mining PAKDD 1999, pp. 479–488 (1999)Google Scholar
  34. 34.
    Zhong, N., Yao, Y.Y., Ohshima, M.: Peculiarity Oriented Multi-Database Mining. IEEE Trans. on Knowledge and Data Engineering 15(4), 952–960 (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Hidenao Abe
    • 1
  • Shusaku Tsumoto
    • 1
  • Miho Ohsaki
    • 2
  • Hideto Yokoi
    • 3
  • Takahira Yamaguchi
    • 4
  1. 1.Department of Medical InformaticsShimane University, School of MedicineShimaneJapan
  2. 2.Faculty of EngineeringDoshisha University 
  3. 3.Department of Medical InformaticsKagawa University Hospital 
  4. 4.Faculty of Science and TechnologyKeio University 

Personalised recommendations