Skip to main content
SpringerLink
Log in

How to Upgrade Propositional Learners to First Order Logic: A Case Study

  • Chapter
Relational Data Mining

Abstract

We describe a methodology for upgrading existing attribute-value learners towards first-order logic. This method has several advantages: one can profit from existing research on propositional learners (and inherit its efficiency and effectiveness), relational learners (and inherit its expressiveness) and PAC-learning (and inherit its theoretical basis). Moreover there is a clear relationship between the new relational system and its propositional counterpart. This makes the ILP system easy to use and understand by users familiar with the propositional counterpart. We demonstrate the methodology on the ICL system which is an upgrade of the propositional learner CN2.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. R. Agrawal, H. Mannila, R. Srikant, H. Toivonen, and A. Verkamo. Fast discovery of association rules. In U. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, editors, Advances in Knowledge Discovery and Data Mining, pages 307–328. MIT Press, Cambridge, MA, 1996.

    Google Scholar 

  2. R. Agrawal, T. Imielinski, and A. Swami. Mining association rules between sets of items in large databases. In Proceedings of the ACM SIGMOD Conference on Management of Data, pages 207–216. ACM Press, New York, 1993.

    Google Scholar 

  3. D. Angluin, M. Frazier, and L. Pitt. Learning conjunctions of Horn clauses. Machine Learning, 9: 147–162, 1992.

    MATH  Google Scholar 

  4. F. Bergadano and D. Gunetti. An interactive system to learn functional logic programs. In Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence, pages 1044–1049. Morgan Kaufmann, San Mateo, CA, 1993.

    Google Scholar 

  5. A. Berger, V. Delia Pietra, and S. Delia Pietra. A maximum entropy approach to natural language processing. Computational Linguistics, 22(1): 39–71, 1996.

    Google Scholar 

  6. G. Bisson. Conceptual clustering in a first order logic representation. In Proceedings of the Tenth European Conference on Artificial Intelligence, pages 458–462. John Wiley and Sons, 1992.

    Google Scholar 

  7. H. Blockeel. Top-down induction of first order logical decision trees. PhD thesis, Department of Computer Science, Katholieke Universiteit Leuven, 1998. http://www.cs.kuleuven.ac.be/~ml/PS/blockeel98:phd.ps.gz.

    Google Scholar 

  8. H. Blockeel and L. De Raedt. Experiments with top-down induction of logical decision trees. Technical Report CW 247, Department of Computer Science, Katholieke Universiteit Leuven, Belgium, 1997. http://www.cs.kuleuven.ac.be/publicaties/rapporten/CW1997.html.

    Google Scholar 

  9. H. Blockeel and L. De Raedt. Top-down induction of first order logical decision trees. Artificial Intelligence, 101(1–2): 285–297, 1998.

    Article  MathSciNet  MATH  Google Scholar 

  10. M. Bongard. Pattern Recognition. Spartan Books, 1970.

    MATH  Google Scholar 

  11. I. Bratko and S. Muggleton. Applications of inductive logic programming. Communications of the ACM, 38(11): 65–70, 1995.

    Article  Google Scholar 

  12. L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. Classification and Regression Trees. Wadsworth, Belmont, 1984.

    MATH  Google Scholar 

  13. P. Clark and R. Boswell. Rule induction with CN2: Some recent improvements. In Proceedings of the Fifth European Working Session on Learning, pages 151–163. Springer, Berlin, 1991.

    Google Scholar 

  14. P. Clark and T. Niblett. The CN2 algorithm. Machine Learning, 3(4): 261–284, 1989.

    Google Scholar 

  15. W. W. Cohen. Fast effective rule induction. In Proceedings of the Twelfth International Conference on Machine Learning, pages 115–123. Morgan Kaufmann, San Mateo, CA, 1995.

    Google Scholar 

  16. W. Cohen. Grammatically biased learning: learning logic programs using an explicit antecedent description language. Artificial Intelligence, 68: 303–366, 1994.

    Article  MATH  Google Scholar 

  17. L. De Raedt, editor. Advances in Inductive Logic Programming. IOS Press, Amsterdam, 1996.

    MATH  Google Scholar 

  18. L. De Raedt. Logical settings for concept learning. Artificial Intelligence, 95: 187–201, 1997.

    Article  MathSciNet  MATH  Google Scholar 

  19. L. De Raedt. Attribute-value learning versus inductive logic programming: The missing links (extended abstract). In Proceedings of the Eighth International Conference on Inductive Logic Programming, pages 1–8. Springer, Berlin, 1998.

    Chapter  Google Scholar 

  20. L. De Raedt and L. Dehaspe. Clausal discovery. Machine Learning, 26: 99–146, 1997.

    Article  MATH  Google Scholar 

  21. L. De Raedt and S. Džeroski. First order jk-clausal theories are PAC-learnable. Artificial Intelligence, 70: 375–392, 1994.

    Article  MathSciNet  MATH  Google Scholar 

  22. L. De Raedt, N. Lavrac, and S. Dzeroski. Multiple predicate learning. In Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence, pages 1037–1042. Morgan Kaufmann, San Mateo, CA, 1993.

    Google Scholar 

  23. L. De Raedt and W. Van Laer. Inductive constraint logic. In Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, pages 80–94. Springer, Berlin, 1995.

    Chapter  Google Scholar 

  24. L. Dehaspe. Maximum entropy modeling with clausal constraints. In Proceedings of the Seventh International Workshop on Inductive Logic Programming, pages 109–124. Springer, Berlin, 1997.

    Chapter  Google Scholar 

  25. L. Dehaspe. Frequent Pattern Discovery in First-Order Logic. PhD thesis, Department of Computer Science, Katholieke Universiteit Leuven, Belgium, 1998. http://www.cs.kuleuven.ac.be/~ldh/.

    Google Scholar 

  26. L. Dehaspe and L. De Raedt. Mining association rules in multiple relations. In Proceedings of the Seventh International Workshop on Inductive Logic Programming, pages 125–132. Springer, Berlin, 1997.

    Chapter  Google Scholar 

  27. B. Dolšak, I. Bratko, and A. Jezernik. Finite element mesh design: An engineering domain for ilp application. In Proceedings of the Fourth International Workshop on Inductive Logic Programming, pages 305–320. GMD, Sankt August in, Germany, 1994.

    Google Scholar 

  28. P. Domingos. A process-oriented heuristic for model selection. In Proceedings of the Fifteenth International Conference on Machine Learning, pages 127–135. Morgan Kaufmann, San Francisco, CA, 1998.

    Google Scholar 

  29. J. Dougherty, R. Kohavi, and M. Sahami. Supervised and unsupervised discretization of continuous features. In Proceedings of the Twelfth International Conference on Machine Learning. Morgan Kaufmann, San Mateo, CA, 1995.

    Google Scholar 

  30. S. Džeroski, S. Schulze-Kremer, et al. Diterpene structure elucidation from 13C NMR spectra with inductive logic programming. Applied Artificial Intelligence, 12(5): 363–384, 1998.

    Article  Google Scholar 

  31. S. Džeroski and I. Bratko. Applications of inductive logic programming. In L. De Raedt, editor, Advances in inductive logic programming, pages 65–81. IOS Press, Amsterdam, 1996.

    Google Scholar 

  32. S. Džeroski, B. Cestnik, and I. Petrovski. Using the m-estimate in rule induction. Journal of Computing and Information Technology, 1(1): 37–46, 1993.

    Google Scholar 

  33. S. Džeroski, N. Jacobs, M. Molina, and C. Moure. ILP experiments in detecting traffic problems. In Proceedings of the Tenth European Conference on Machine Learning, pages 61–66. Springer, Berlin, 1998.

    Google Scholar 

  34. S. Dzeroski, H. Blocked, B. Kompare, S. Kramer, B. Pfahringer, and W. Van Laer. Experiments in Predicting Biodegradability. In Proceedings of the Ninth International Workshop on Inductive Logic Programming, pages 80–91. Springer, Berlin, 1999.

    Chapter  Google Scholar 

  35. W. Emde and D. Wettschereck. Relational instance-based learning. In Proceedings of the Thirteenth International Conference on Machine Learning, pages 122–130. Morgan Kaufmann, San Mateo, CA, 1996.

    Google Scholar 

  36. U. Fayyad and K. Irani. Multi-interval discretization of continuous-valued attributes for classification learning. In Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence, pages 1022–1027. Morgan Kaufmann, San Mateo, CA, 1993.

    Google Scholar 

  37. P. Flach. Strongly typed inductive concept learning. In Proceedings of the Eighth International Conference on Inductive Logic Programming, pages 185–194. Springer, Berlin, 1998.

    Chapter  Google Scholar 

  38. J. Ganascia and Y. Kodratoff. Improving the generalization step in learning. In R. Michalski, J. Carbonell, and T. Mitchell, editors, Machine Learning: An Artificial Intelligence Approach, pages 215–241. Morgan Kaufmann, San Mateo, CA, 1986.

    Google Scholar 

  39. F. Hayes-Roth and J. McDermott. An interference matching technique for inducing abstractions. Communications of the ACM, 21: 401–410, 1978.

    Article  MATH  Google Scholar 

  40. J.-U. Kietz and S. Wrobel. Controlling the complexity of learning in logic through syntactic and task-oriented models. In S. Muggleton, editor, Inductive logic programming, pages 335–359. Academic Press, London, 1992.

    Google Scholar 

  41. R. D. King, M. J. E. Sternberg, A. Srinivasan, and S. Muggleton. Relating chemical activity to structure: an examination of ILP successes. New Generation Computing, 13(3–4): 411–434, 1995.

    Article  Google Scholar 

  42. M. Kirsten and S. Wrobel. Relational distance-based clustering. In Proceedings of the Eighth International Conference on Inductive Logic Programming, pages 261–270. Springer, Berlin, 1998.

    Chapter  Google Scholar 

  43. D. Koller. Probabilistic relational models. In Proceedings of the Ninth International Workshop on Inductive Logic Programming, pages 3–13. Springer, Berlin, 1999.

    Chapter  Google Scholar 

  44. S. Kramer. Structural regression trees. In Proceedings of the Thirteenth National Conference on Artificial Intelligence, pages 812–819. AAAI Press, Menlo Park, CA, 1996.

    Google Scholar 

  45. R. Michalski, I. Mozetič, J. Hong, and N. Lavrač. The multipurpose incremental learning system AQ15 and its testing application on three medical domains. In Proceedings of the Fifth National Conference on Artificial Inteligence, pages 1041–1045. Morgan Kaufmann, San Mateo, CA, 1986.

    Google Scholar 

  46. R. S. Michalski. A theory and methodology of inductive learning. In R. S. Michalski, J. Carbonell, and T. Mitchell, editors, Machine Learning: An Artificial Intelligence Approach, pages 83–134. Morgan Kaufmann, San Mateo, CA, 1983.

    Google Scholar 

  47. S. Muggleton. Inverse entailment and Progol. New Generation Computing, 13(3–4): 245–286, 1995.

    Article  Google Scholar 

  48. S. Muggleton and L. De Raedt. Inductive logic programming : Theory and methods. Journal of Logic Programming, 19, 20: 629–679, 1994.

    Article  MathSciNet  Google Scholar 

  49. S. Muggleton and C. Feng. Efficient induction of logic programs. In Proceedings of the First Conference on Algorithmic Learning Theory, pages 368–381. Ohmsma, Tokyo, Japan, 1990.

    Google Scholar 

  50. C. Nedellec, H. Adé, F. Bergadano, and B. Tausend. Declarative bias in ILP. In L. De Raedt, editor, Advances in Inductive Logic Programming, pages 82–103. IOS Press, 1996.

    Google Scholar 

  51. S.-H. Nienhuys-Cheng and R. Wolf. Foundations of inductive logic programming. Springer, Berlin, 1997.

    Book  Google Scholar 

  52. G. Plotkin. A note on inductive generalization. In B. Meltzer and D. Michie, editors, Machine Intelligence, pages 153–163. Edinburgh University Press, Edinburgh, 1970.

    Google Scholar 

  53. U. Pompe and I. Kononenko. Probabilistic first-order classification. In Proceedings of the Seventh International Workshop on Inductive Logic Programming. Springer, Berlin, 1997.

    Google Scholar 

  54. J. R. Quinlan. C4–5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA, 1993.

    Google Scholar 

  55. J. R. Quinlan. Induction of decision trees. Machine Learning, 1: 81–106, 1986.

    Google Scholar 

  56. J. R. Quinlan. Learning logical definitions from relations. Machine Learning, 5: 239–266, 1990.

    Google Scholar 

  57. J. R. Quinlan. Determinate Literals in Inductive Logic Programming. In Proceedings of the Eighth International Workshop on Machine Learning, pages 442–446. Morgan Kaufmann, San Mateo, CA 1991.

    Google Scholar 

  58. J. R. Quinlan. Learning first-order definitions of functions. Journal of Artificial Intelligence Research, 5: 139–161, 1996.

    MATH  Google Scholar 

  59. C. Reddy and P. Tadepalli. Learning first-order acyclic Horn programs from entailment. In Proceedings of the Eighth International Conference on Inductive Logic Programming, pages 23–37. Springer, Berlin, 1998.

    Chapter  Google Scholar 

  60. E. Shapiro. An algorithm that infers theories from facts. In Proceedings of the Seventh International Joint Conference on Artificial Intelligence, pages 446–452. Morgan Kaufmann, San Mateo, CA, 1981.

    Google Scholar 

  61. A. Srinivasan, R. D. King, S. Muggleton, and M. J. E. Sternberg. The predictive toxicology evaluation challenge. In Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence, pages 1–6. Morgan Kaufmann, San Mateo, CA, 1997.

    Google Scholar 

  62. A. Srinivasan, R. King, and D. Bristol. An assessment of ILP-assisted models for toxicology and the PTE-3 experiment. In Proceedings of the Ninth International Workshop on Inductive Logic Programming, pages 291–302. Springer, Berlin, 1999.

    Chapter  Google Scholar 

  63. A. Srinivasan, S. Muggleton, M. J. E. Sternberg, and R. D. King. Theories for mutagenicity: A study in first-order and feature-based induction. Artificial Intelligence, 85(1,2): 277–299, 1996.

    Article  Google Scholar 

  64. H. Toivonen, M. Klemettinen, P. Ronkainen, K. Hatonen, and H. Mannila. Pruning and grouping discovered association rules. In Proceedings of the ML-net Familiarization Workshop on Statistics, Machine Learning and Knowledge Discovery in Databases, pages 47–52. FORTH, Heraklion, Greece, 1995.

    Google Scholar 

  65. L. Valiant. A theory of the learnable. Communications of the ACM, 27: 1134–1142, 1984.

    Article  MATH  Google Scholar 

  66. P. R. J. van der Laag and S.-H. Nienhuys-Cheng. Completeness and properness of refinement operators in inductive logic programming. Journal of Logic Programming, 34(3): 201–225, 1998.

    Article  MathSciNet  MATH  Google Scholar 

  67. W. Van Laer, L. De Raedt, and S. Dzeroski. On multi-class problems and discretization in inductive logic programming. In Proceedings of the Tenth International Symposium on Methodologies for Intelligent Systems, pages 277–286. Springer, Berlin, 1997.

    Google Scholar 

  68. W. Van Laer, S. Dzeroski, and L. De Raedt. Multi-class problems and discretization in ICL. In Proceedings of the MLnet Familiarization Workshop on Data Mining with Inductive Logic Programming, pages 53–60. FORTH, Heraklion, Greece, 1995.

    Google Scholar 

  69. S. Vere. Induction of concepts in the predicate calculus. In Proceedings of the Fourth International Joint Conference on Artificial Intelligence, pages 282–287. Morgan Kaufmann, San Mateo, CA, 1975.

    Google Scholar 

  70. C. Vrain. Ogust: A system that learns using domain properties expressed as theorems. In Y. Kodratoff and R. S. Michalski, editors, Machine Learning: An Artificial Intelligence Approach, pages 360–381. Morgan Kaufmann, San Mateo, CA, 1990.

    Google Scholar 

  71. P. Winston. Learning structural descriptions from examples. In P. Winston, editor, Psychology of Computer Vision. MIT Press, Cambridge, MA, 1975.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Katholieke Universiteit Leuven, Celestijnenlaan 200A, B-3001, Leuven, Belgium

    Wim Van Laer

  2. Institut für Informatik, Albert-Ludwigs-Universität Freiburg, Am Flughafen 17, D-79110, Freiburg, Germany

    Luc De Raedt

Authors
  1. Wim Van Laer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luc De Raedt
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Jožef Stefan Institute, Jamova 39, 1000, Ljubljana, Slovenia

    Sašo Džeroski  & Nada Lavrač  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2001 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Van Laer, W., De Raedt, L. (2001). How to Upgrade Propositional Learners to First Order Logic: A Case Study. In: Džeroski, S., Lavrač, N. (eds) Relational Data Mining. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-04599-2_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-04599-2_10

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-07604-6

  • Online ISBN: 978-3-662-04599-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation