Hybrid Inductive Machine Learning: An Overview of CLIP Algorithms

  • Krzysztof J. Cios
  • Łukasz A. Kurgan
Part of the Studies in Fuzziness and Soft Computing book series (STUDFUZZ, volume 84)


The chapter describes inductive machine learning methods for generating hypotheses about given training data. It focuses on hybrid algorithms that generate hypotheses in the form of production if... then... rules, which constitute the model of the data.


Information Gain Machine Learning Algorithm Information Function Integer Program Model Decision Tree Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bellmore, M and Ratliff, H.D., 1971, Set Covering and Involuntary Bases, Management Science, vol. 18, no 3Google Scholar
  2. Breiman L., Friedman J., Olse R., and Stone C., 1984, Classification and Regresion Trees, Chapman and Hall, New York, NYGoogle Scholar
  3. Brodley C.E., Utgoff P.E., 1995, Multivariate decision trees, Machine Learning, 19: 45–77MATHGoogle Scholar
  4. Buntine W., 1992, Learning classification trees, Statistics and Computing, 2: 63–73CrossRefGoogle Scholar
  5. Chvatal, V, 1979, A Greedy-Heuristic for The Set-Covering Problem, Math. Operations Research 4, no 3Google Scholar
  6. Cios, K.J. and Liu, N., 1992, Machine learning in generation of a neural network architecture: a Continuous ID3 approach, IEEE Trans. on Neural Networks, 3 (2): 280–291CrossRefGoogle Scholar
  7. Cios, K.J., Liu, N., Goodenday, L.S., 1993, Generation of Diagnostic Rules via Inductive Machine learning, Kybernetes, vol. 22, no 5, pp 44–56CrossRefGoogle Scholar
  8. Cios, K.J. and Liu, N., 1995, An algorithm which learns multiple covers via integer linear programming. Part I–The CLILP2 Algorithm, Kybernetes, vol. 24, no 2, pp. 29–50CrossRefGoogle Scholar
  9. Cios, K.J. and Liu, N., 1995, An algorithm which learns multiple covers via integer linear programming. Part II — experimental results and conclusions, Kybernetes, vol. 24, no 3, pp. 24–36CrossRefGoogle Scholar
  10. Cios, K.J., Wedding, D.K. and Liu, N, 1997, CLIP3: cover learning using integer programming. Kybernetes,26(4–5): 513–536 (The Norbert Wiener 1997 Outstanding Paper Award) Google Scholar
  11. Cios, K.J., Pedrycz, W., Swiniarski, R., 1998, Data Mining Methods for Knowledge Discovery, Kluwer, http://www.wkap.nl /book. htm/ (1)– 7923–8252–8
  12. Cios, K.J., Teresinska, A., Konieczna, S., Potocka, J., Sharma, S., 2000, Diagnosing Myocardial Perfusion from SPECT Bull’s-eye Maps - A Knowledge Discovery Approach, IEEE Engineering in Medicine and Biology Magazine, Special issue on Medical Data Mining and Knowledge DiscoveryGoogle Scholar
  13. Cios, K.J. and Kurgan, L., 2001, CLIP4 - a hybrid algorithm generating production rules, submittedGoogle Scholar
  14. Clark, P. and Niblett, T., 1989, The CN2 algorithm, Machine learning, 3: 261–283Google Scholar
  15. Clark, P. and Boswell, R., 1991, Rule Induction with CN2: Some Recent Improvements, Lecture Notes in Artificial Intelligence, Proc. of European Working Session on Learning, Springer-VerlagGoogle Scholar
  16. Clark L.A., Pregibon D., 1993, Tree-based models, in Chambers J.M., Hastie T.J. (ed.), Statisticsl Models in S, Chapman and Hall, New York, NY, p. 377–419Google Scholar
  17. Fisher, D.H. and Langley, P., 1986, Conceptual clustering and its relation to numerical taxonomy, Artificial Intelligence and Statistics, Gale W.A. (ed ), Addison-WesleyGoogle Scholar
  18. Fisher, D.H., 1987, Conceptual clustering, learning from examples and inference, Proc. of the 4th Int. Workshop on Machine learning, Morgan-Kaufmann, pp. 38–49Google Scholar
  19. Grafinkel, R.S. and Nembauser, G.L., 1972, Integer Programming, John Wiley and Sons, New YorkGoogle Scholar
  20. Hochbaum, D.S., 1982, Approximation Algorithm for The Weighted Set-Covering and Vertex Cover Problems, Siam J. Comput., vol. 11, no 3Google Scholar
  21. Holte, R.C., 1993, Very simple classification rules perform well on most commonly used data sets, Machine learning, 11: 63–90MATHCrossRefGoogle Scholar
  22. Hunt, E.B., Marin, J. and Stone, P.J., 1966, Experiments in Induction,Academic PressGoogle Scholar
  23. Kaufman, K.A. and Michalski, R.S., 1999, Learning from Inconsistent and Noisy Data: The AQ18 Approach, Proc. of the Eleventh International Symposium on Methodologies for Intelligent Systems, Warsaw, June 8–11Google Scholar
  24. Kononenko, I., Bratko, L., Roskar, E., 1984, Experiments in automatic learning of medical diagnostic rules, Technical Report, E. Kardelj University, Faculty of Electrical Engineering, Ljubljana, YugoslaviaGoogle Scholar
  25. Kurgan, L.A., Cios, K.J., Tadeusiewicz, R., Ogiela, M., Goodenday, L.S., 2001, Knowledge Discovery Approach to Automated Cardiac SPECT Diagnosis, Artificial Intelligence in Medicine, to appear in November 2001Google Scholar
  26. Langley, P., 1996, Elements of Machine learning,Morgan-KaufmannGoogle Scholar
  27. Loh W.Y., Shih Y.S., 1997, Split selection methods for classification trees, Statistical Sinica, 7: 815–840MathSciNetMATHGoogle Scholar
  28. Loh, W.Y., Vanichsetakul, 1988, Tree structured classification via generalized discriminant analysis (with discussion), Journal of the American Statistical Association, 83: 715–728MathSciNetMATHGoogle Scholar
  29. Mangasarian, O. L. and Wolberg, W. H., 1990, Cancer Diagnosis Via Linear Programming, Siam News, vol. 23, no 5, pp. 1–18Google Scholar
  30. Michalski, R.S., 1974, Variable-valued logic: System VL1, Proc. 1974 Int. Symposium on Multiple-Valued Logic and Pattern Recognition, West Virginia University, Morgantown, pp. 323–346Google Scholar
  31. Michalski, R.S., 1980, Knowledge acquisition through conceptual clustering: a theoretical framework and algorithm for partitioning data into conjunctive concepts, Int. Journal of Policy Analysis and Information Systems, 4: 219–243MathSciNetGoogle Scholar
  32. Michalski, R.S., Mozetic, I., Hong, J., Lavrac, N., 1986, The multipurpose incremental learning system AQ15 and its testing application to three medical domains, Proc. 5th National Conf. on Artificial Intelligence, Morgan-Kaufmann, pp. 1041–1045Google Scholar
  33. Michalski R.S., 1990, Learning flexible concepts: fundamental ideas and a method based on two-tiered representation, Kodratoff, Y. and Michalski, R.S., (eds), Machine learning: An Artificial Intelligence Approach, Morgan-Kaufmann, vol. III, pp. 63–102Google Scholar
  34. Murthy S.K, Kasif S., Salzberg S., 1994, A system for induction of oblique decision trees, Journal of Artificial Intelligence Research, 2: 1–33MATHGoogle Scholar
  35. Ravindran, A., Phillips, D.T., Solberg, J.J, 1987, Operations Research, Principles and Practice, Second addition, pp. 1–69, John Wiley Sons.Google Scholar
  36. Quinlan, J.R., 1993, C4. 5 Programs for Machine learning, Morgan-KaufmannGoogle Scholar
  37. Schapire, R. E., (1999), A brief introduction to boosting. Proceedings of theSixteenth International Joint Conference on Artificial Intelligence Google Scholar
  38. Shank R.C., Where is AI, (1991), AI Magazine, winter 1991, p. 38–49Google Scholar
  39. Thrun, S.B., et al., 1991, The MONK’s problems: A Performance Comparison of Different Learning Algorithms, School of Computer Science, Carnegie Mellon UniversityGoogle Scholar
  40. Torkzadeh, G., Cios, K.J., Pfughoeft, K.A, 1996, Inductive machine learning for instrument development, Information and Management, vol. 31, pp 47–55Google Scholar
  41. University of California, Irvine: UCI Machine learning Repository Content Summary. http://www.ics.uci.edu/-mlearn/MLSummary.html
  42. Ching, J.Y., Wong, A.K.C. and Chan, K.C.C., 1995, Class-dependent discretization for inductive learning from continuous and mixed-mode data, IEEE Trans. on PAMI, 17: 641–651CrossRefGoogle Scholar
  43. Cios, K.J., Pedrycz, W., Swiniarski, R., 1998, Data Mining Methods for Knowledge Discovery, Kluwer, http://www.wkap.nl /book. htm/ 0–7923–8252–8
  44. Cios, K.J. and Kurgan, L., 2001, CLIP4 - a hybrid algorithm generating production rules, submittedGoogle Scholar
  45. Paterson, A. and Niblett, T.B., 1982, ACLS Manual, Edinburgh: Intelligent Terminals, LtdGoogle Scholar
  46. Wong, A.K.C. and Liu, T.S., 1975, Typicality, diversity and feature pattern of an ensemble, IEEE Trans. on Computers, 24: 158–181MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Krzysztof J. Cios
    • 1
  • Łukasz A. Kurgan
    • 1
  1. 1.Computer Science and Engineering DepartmentUniversity of Colorado at DenverDenverUSA

Personalised recommendations