Learning Theories Using Estimation Distribution Algorithms and (Reduced) Bottom Clauses

  • Cristiano Grijó Pitangui
  • Gerson Zaverucha
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7207)


Genetic Algorithms (GAs) are known for their capacity to explore large search spaces and due to this ability they were applied (to some extent) to Inductive Logic Programming (ILP).  Although Estimation of Distribution Algorithms (EDAs) generally perform better than standard GAs, they have not been applied to ILP.  This work presents EDA-ILP, an ILP system based on EDA and inverse entailment, and also its extension, the REDA-ILP, which employs the Reduce algorithm in bottom clauses to considerably reduce the search space. Experiments in real-world datasets showed that both systems were successfully compared to Aleph and GA-ILP (another variant of EDA-ILP created replacing the EDA by a standard GA). EDA-ILP was also successfully compared to Progol-QG/GA (and its other variants) in phase transition benchmarks. Additionally, we found that REDA-ILP usually obtains simpler theories than EDA-ILP, more efficiently and with equivalent accuracies. These results show that EDAs provide a good base for stochastic search in ILP.


Inductive Logic Programming Estimation Distribution Algorithm Reduce Algorithm 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Pitangui, C., Zaverucha, G.: Inductive Logic Programming Through Estimation Distribution Algorithm. In: Proceedings of IEEE Congress of Evolutionary Computation (CEC 2011), New Orleans, LA, EUA, pp. 54–61 (2011) 978-1-4244-7834-7Google Scholar
  2. 2.
    Muggleton, S., De Raedt, L.: Inductive Logic Programming: Theory and Methods. Journal of Logic Programming 19(20) (1994)Google Scholar
  3. 3.
    Mühlenbein, H., Paaß, G.: From Recombination of Genes to the Estimation of Distributions I. Binary Parameters. In: Ebeling, W., Rechenberg, I., Voigt, H.-M., Schwefel, H.-P. (eds.) PPSN 1996. LNCS, vol. 1141, pp. 178–187. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  4. 4.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers Inc., San Francisco (1988)Google Scholar
  5. 5.
    Baluja, S.: Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning. Carnegie Mellon University, Pittsburgh (1994); Technical Report: CMU-CS-94-163Google Scholar
  6. 6.
    Holland, J.: Adaptation in natural and artificial systems. MIT Press, Cambridge (1975)Google Scholar
  7. 7.
    Muggleton, S.H., Feng, C.: Efficient induction of logic programs. In: Proceedings of the First Conference on Algorithmic Learning Theory, pp. 368–381. Ohmsha, Tokyo (1990)Google Scholar
  8. 8.
    Srinivasan, A.: The Aleph Manual, (last access September 29, 2011)
  9. 9.
    Alphonse, E., Rouveirol, C.: Lazy propositionalisation for Relational Learning. In: 14th European Conference on Artificial Intelligence (ECAI 2000), pp. 256–260. IOS Press (2000)Google Scholar
  10. 10.
    Muggleton, S., Tamaddoni-Nezhad, A.: QG/GA: A stochastic search approach for Progol. Machine Learning 70(2-3), 123–133 (2007), doi:10.1007/s10994-007-5029-3Google Scholar
  11. 11.
    Muggleton, S.: Inverse entailment and Progol. New Generation Computing, Special issue on Inductive Logic Programming 13(3-4), 245–286 (1995)Google Scholar
  12. 12.
    Oliphant, L., Shavlik, J.: Using Bayesian Networks to Direct Stochastic Search in Inductive Logic Programming. In: Blockeel, H., Ramon, J., Shavlik, J., Tadepalli, P. (eds.) ILP 2007. LNCS (LNAI), vol. 4894, pp. 191–199. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  13. 13.
    Srinivasan, A., King, R.D.S.H., Muggleton, S., Sternberg, M.: Carcinogenesis Predictions using ILP. In: Džeroski, S., Lavrač, N. (eds.) ILP 1997. LNCS (LNAI), vol. 1297, pp. 273–287. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  14. 14.
    King, R.D., Srinivasan, A., Sternberg, M.J.E.: Relating chemical activity to structure: an examination of ILP successes. New Gen. Comp. 13, 411–433 (1995)CrossRefGoogle Scholar
  15. 15.
    Nadeau, C., Bengio, Y.: Inference for the Generalization Error. Machine Learning 52(3), 239–281 (2003)zbMATHCrossRefGoogle Scholar
  16. 16.
    Huynh, T., Mooney, R.: Discriminative Structure and Parameter Learning for Markov Logic Networks. In: Proceedings of the 25th International Conference on Machine Learning (ICML 2008), Helsinki, Finland, pp. 416–423 (2008)Google Scholar
  17. 17.
    Muggleton, S.H., Santos, J.C.A., Tamaddoni-Nezhad, A.: TopLog: ILP Using a Logic Program Declarative Bias. In: Garcia de la Banda, M., Pontelli, E. (eds.) ICLP 2008. LNCS, vol. 5366, pp. 687–692. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  18. 18.
    Bratko, I.: Refining Complete Hypotheses in ILP. In: Džeroski, S., Flach, P.A. (eds.) ILP 1999. LNCS (LNAI), vol. 1634, pp. 44–55. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  19. 19.
    Paes, A., Zaverucha, G., Santos Costa, V.: Revising First-Order Logic Theories from Examples Through Stochastic Local Search. In: Blockeel, H., Ramon, J., Shavlik, J., Tadepalli, P. (eds.) ILP 2007. LNCS (LNAI), vol. 4894, pp. 200–210. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  20. 20.
    Srinivasan, A.: A study of two probabilistic methods for searching large spaces with ILP(Technical Report PRG-TR-16-00). Oxford University Computing Laboratory, Oxford (2000)Google Scholar
  21. 21.
    Paes, A., Železný, F., Zaverucha, G., Page, D.L., Srinivasan, A.: ILP Through Propositionalization and Stochastic k-Term DNF Learning. In: Muggleton, S.H., Otero, R., Tamaddoni-Nezhad, A. (eds.) ILP 2006. LNCS (LNAI), vol. 4455, pp. 379–393. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  22. 22.
    Tamaddoni-Nezhad, A., Muggleton, S.H.: Searching the Subsumption Lattice by a Genetic Algorithm. In: Cussens, J., Frisch, A.M. (eds.) ILP 2000. LNCS (LNAI), vol. 1866, pp. 243–252. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  23. 23.
    Pelikan, M.: Hierarchical Bayesian Optimization Algorithm Toward a New Generation of Evolutionary Algorithms, 1st edn. STUDFUZZ, vol. 170. Springer (2005)Google Scholar
  24. 24.
    Železný, F., Srinivasan, A., Page, D.: Lattice-Search Runtime Distributions May Be Heavy-Tailed. In: Matwin, S., Sammut, C. (eds.) ILP 2002. LNCS (LNAI), vol. 2583, pp. 333–345. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  25. 25.
    Goadrich, M., Oliphant, L., Shavlik, J.: Gleaner: Creating Ensembles of First-Order Clauses to Improve Recall-Precision Curves. Machine Learning 64(1-3), 231–261 (2006)zbMATHCrossRefGoogle Scholar
  26. 26.
    Botta, M., Giordana, A., Saitta, L., Sebag, M.: Relational learning as search in a critical region. Journal of Machine Learning Research 4, 431–463 (2003)MathSciNetGoogle Scholar
  27. 27.
    Alphonse, E., Osmani, A.: On the connection between the phase transition of the covering test and the learning success rate in ILP. Machine Learning Journal 70(2-3), 135–150 (2008)CrossRefGoogle Scholar
  28. 28.
    Henrion, M.: Propagating Uncertainty in Bayesian Networks by Probabilistic Logic Sampling. In: Lemmer, J.F., Kanal, L.N. (eds.) Uncertainty in Artificial Intelligence, vol. 2, pp. 149–163. North Holland (1988)Google Scholar
  29. 29.
    Pitangui, C., Zaverucha, G.: Genetic local search for rule learning. In: Genetic And Evolutionary Computation Conference (GECCO) Atlanta, GA, USA, pp. 1427–1428 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Cristiano Grijó Pitangui
    • 1
  • Gerson Zaverucha
    • 1
  1. 1.PESC - COPPEUniversidade Federal do Rio de JaneiroRio de JaneiroBrazil

Personalised recommendations