Machine Learning

, Volume 94, Issue 1, pp 25–49 | Cite as

Meta-interpretive learning: application to grammatical inference

  • Stephen H. Muggleton
  • Dianhuan Lin
  • Niels Pahlavi
  • Alireza Tamaddoni-Nezhad


Despite early interest Predicate Invention has lately been under-explored within ILP. We develop a framework in which predicate invention and recursive generalisations are implemented using abduction with respect to a meta-interpreter. The approach is based on a previously unexplored case of Inverse Entailment for Grammatical Inference of Regular languages. Every abduced grammar H is represented by a conjunction of existentially quantified atomic formulae. Thus ¬H is a universally quantified clause representing a denial. The hypothesis space of solutions for ¬H can be ordered by θ-subsumption. We show that the representation can be mapped to a fragment of Higher-Order Datalog in which atomic formulae in H are projections of first-order definite clause grammar rules and the existentially quantified variables are projections of first-order predicate symbols. This allows predicate invention to be effected by the introduction of first-order variables. Previous work by Inoue and Furukawa used abduction and meta-level reasoning to invent predicates representing propositions. By contrast, the present paper uses abduction with a meta-interpretive framework to invent relations. We describe the implementations of Meta-interpretive Learning (MIL) using two different declarative representations: Prolog and Answer Set Programming (ASP). We compare these implementations against a state-of-the-art ILP system MC-TopLog using the dataset of learning Regular and Context-Free grammars as well learning a simplified natural language grammar and a grammatical description of a staircase. Experiments indicate that on randomly chosen grammars, the two implementations have significantly higher accuracies than MC-TopLog. In terms of running time, Metagol is overall fastest in these tasks. Experiments indicate that the Prolog implementation is competitive with the ASP one due to its ability to encode a strong procedural bias. We demonstrate that MIL can be applied to learning natural grammars. In this case experiments indicate that increasing the available background knowledge, reduces the running time. Additionally ASPM (ASP using a meta-interpreter) is shown to have a speed advantage over Metagol when background knowledge is sparse. We also demonstrate that by combining Metagol R (Metagol with a Regular grammar meta-interpreter) and Metagol CF (Context-Free meta-interpreter) we can formulate a system, Metagol RCF , which can change representation by firstly assuming the target to be Regular, and then failing this, switch to assuming it to be Context-Free. Metagol RCF runs up to 100 times faster than Metagol CF on grammars chosen randomly from Regular and non-Regular Context-Free grammars.


Inductive logic programming Meta-interpretative learning Predicate invention Recursion Grammatical inference 


  1. Andres, B., Kaufmann, B., Matheis, O., & Schaub, T. (2012). Unsatisfiability-based optimization in clasp. In Proceedings of the 28th International Conference on Logic Programming. Google Scholar
  2. Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. K. (1989). Learnability and the Vapnik-Chervonenkis dimension. Journal of the ACM, 36(4), 929–965. MathSciNetCrossRefMATHGoogle Scholar
  3. Boström, H. (1998). Predicate invention and learning from positive examples only. In 10th European Conference on Machine Learning (ECML-98) (pp. 226–237). Berlin: Springer. CrossRefGoogle Scholar
  4. Cussens, J., & Pulman, S. (2000). Experiments in inductive chart parsing. In J. Cussens & S. Dzeroski (Eds.), LNAI: Vol. 1925. Proceedings of Learning Language in Logic (LLL2000) (pp. 143–156). Berlin: Springer. CrossRefGoogle Scholar
  5. de la Higuera, C. (2005). A bibliographical study of grammatical inference. Pattern Recognition, 38, 1332–1348. CrossRefGoogle Scholar
  6. Denis, F. (2001). Learning regular languages from simple positive examples. Machine Learning, 44(1), 37–66. CrossRefMATHGoogle Scholar
  7. Farid, R., & Sammut, C. (2012, to appear). Plane-based object categorization using relational learning. ILP2012 MLJ special issue. Google Scholar
  8. Flach, P. A. & Kakas, A. C. (Eds.) (2000). Abductive and Inductive Reasoning. Pure and Applied Logic. Amsterdam: Kluwer. MATHGoogle Scholar
  9. Florêncio, C. (2002). Consistent identification in the limit of rigid grammars from strings is np-hard. In Grammatical Inference: Algorithms and Applications (pp. 729–733). Google Scholar
  10. Gebser, M., Kaminski, R., Kaufmann, B., & Schaub, T. (2012). Answer Set Solving in Practice. Synthesis Lectures on Artificial Intelligence and Machine Learning. San Mateo: Morgan and Claypool. MATHGoogle Scholar
  11. Gebser, M., Kaufmann, B., Neumann, A., & Schaub, T. (2007). clasp: A conflict-driven answer set solver. In C. Baral, G. Brewka, & J. Schlipf (Eds.), Lecture Notes in Computer Science: Vol. 4483. Logic Programming and Nonmonotonic Reasoning (pp. 260–265). Berlin: Springer. CrossRefGoogle Scholar
  12. Hopcroft, J. E., & Ullman, J. D. (1979). Introduction to Automata and Formal Languages. Reading: Addison-Wesley. MATHGoogle Scholar
  13. Inoue, K., Furukawa, K., Kobayashiand, I., & Nabeshima, H. (2010). Discovering rules by meta-level abduction. In L. De Raedt (Ed.), LNAI: Vol. 5989. Proceedings of the Nineteenth International Conference on Inductive Logic Programming (ILP09) (pp. 49–64). Berlin: Springer. CrossRefGoogle Scholar
  14. Kakas, A. C., Van Nuffelen, B., & Denecker, M. (2001). A-system: Problem solving through abduction. In IJCAI (pp. 591–596). Google Scholar
  15. Langley, P., & Stromsten, S. (2000). Learning context-free grammars with a simplicity bias. In R. López de Mántaras & E. Plaza (Eds.), Lecture Notes in Computer Science: Vol. 1810. Machine Learning: ECML 2000 (pp. 220–228). Berlin: Springer. CrossRefGoogle Scholar
  16. Moore, E. F. (1956). Gedanken-experiments on sequential machines. In C. E. Shannon & J. McCarthy (Eds.), Automata Studies (pp. 129–153). Princeton: Princeton University Press. Google Scholar
  17. Muggleton, S. H. (1990). Inductive Acquisition of Expert Knowledge. Wokingham: Addison-Wesley. Google Scholar
  18. Muggleton, S. H. (1995). Inverse entailment and Progol. New Generation Computing, 13, 245–286. CrossRefGoogle Scholar
  19. Muggleton, S. H. (1996). Stochastic logic programs. In L. de Raedt (Ed.), Advances in Inductive Logic Programming (pp. 254–264). Amsterdam: IOS Press. Google Scholar
  20. Muggleton, S. H., & Bryant, C. H. (2000). Theory completion using inverse entailment. In Proc. of the 10th International Workshop on Inductive Logic Programming (ILP-00) (pp. 130–146). Berlin: Springer. CrossRefGoogle Scholar
  21. Muggleton, S. H., & Buntine, W. (1988). Machine invention of first-order predicates by inverting resolution. In Proceedings of the 5th International Conference on Machine Learning (pp. 339–352). Los Altos: Kaufmann. Google Scholar
  22. Muggleton, S. H., Lin, D., & Tamaddoni-Nezhad, A. (2012). MC-Toplog: Complete multi-clause learning guided by a top theory. In LNAI: Vol. 7207. Proceedings of the 21st International Conference on Inductive Logic Programming (pp. 238–254). CrossRefGoogle Scholar
  23. Muggleton, S. H., De Raedt, L., Poole, D., Bratko, I., Flach, P., & Inoue, K. (2011). ILP turns 20: biography and future challenges. Machine Learning, 86(1), 3–23. CrossRefMathSciNetMATHGoogle Scholar
  24. Muggleton, S. H., Santos, J., & Tamaddoni-Nezhad, A. (2010). TopLog: ILP using a logic program declarative bias. In LNCS: Vol. 5366. Proceedings of the International Conference on Logic Programming 2008 (pp. 687–692). Berlin: Springer. Google Scholar
  25. Muggleton, S. H., & Pahlavi, N. (2012, in press). Towards efficient higher-order logic learning in a first-order datalog framework. In Latest Advances in Inductive Logic Programming. London: Imperial College Press. CrossRefGoogle Scholar
  26. Nabeshima, H., Iwanuma, K., Inoue, K., & Ray, O. (2010). Solar: An automated deduction system for consequence finding. AI Commun., 23(2–3), 183–203. MathSciNetMATHGoogle Scholar
  27. Nienhuys-Cheng, S.-H., & de Wolf, R. (1997). LNAI: Vol. 1228. Foundations of Inductive Logic Programming. Berlin: Springer. MATHGoogle Scholar
  28. Plotkin, G. D. (1969). A note on inductive generalisation. In B. Meltzer & D. Michie (Eds.), Machine Intelligence (Vol. 5, pp. 153–163). Edinburgh: Edinburgh University Press. Google Scholar
  29. De Raedt, L. (2012). Declarative modeling for machine learning and data mining. In Proceedings of the International Conference on Algorithmic Learning Theory (p. 12). CrossRefGoogle Scholar
  30. Sakakibara, Y. (1992). Efficient learning of context-freegrammars from positive structural examples. Information and Computation, 97(1), 23–60. MathSciNetCrossRefMATHGoogle Scholar
  31. Salvador, I., & Benedi, J. M. (2002). Rna modeling by combining stochastic context-free grammars and n-gram models. International Journal of Pattern Recognition and Artificial Intelligence, 16(3), 309–316. CrossRefGoogle Scholar
  32. Stolcke, A. (1995). An efficient probabilistic context-free parsing algorithm that computes prefix probabilities. Computational Linguistics, 21(2), 165–201. MathSciNetGoogle Scholar
  33. Vanlehn, K., & Ball, W. (1987). A version space approach to learning context-free grammars. Machine Learning, 2, 39–74. Google Scholar

Copyright information

© The Author(s) 2013

Authors and Affiliations

  • Stephen H. Muggleton
    • 1
  • Dianhuan Lin
    • 1
  • Niels Pahlavi
    • 1
  • Alireza Tamaddoni-Nezhad
    • 1
  1. 1.Department of ComputingImperial College LondonLondonUK

Personalised recommendations