Learning Probabilistic Logic Programs over Continuous Data

  • Stefanie SpeichertEmail author
  • Vaishak Belle
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11770)


The field of statistical relational learning aims at unifying logic and probability to reason and learn from data. Perhaps the most successful paradigm in the field is probabilistic logic programming (PLP): the enabling of stochastic primitives in logic programming. While many systems offer inference capabilities, the more significant challenge is that of learning meaningful and interpretable symbolic representations from data. In that regard, inductive logic programming and related techniques have paved much of the way for the last few decades, but a major limitation of this exciting landscape is that only discrete features and distributions are handled. Many disciplines express phenomena in terms of continuous models.

In this paper, we propose a new computational framework for inducing probabilistic logic programs over continuous and mixed discrete-continuous data. Most significantly, we show how to learn these programs while making no assumption about the true underlying density. Our experiments show the promise of the proposed framework.


  1. 1.
    Alberti, M., Bellodi, E., Cota, G., Riguzzi, F., Zese, R.: cplint on SWISH: probabilistic logical inference with a web browser. Intell. Arti. 11(1), 47–64 (2017)Google Scholar
  2. 2.
    Antanas, L., Frasconi, P., Costa, F., Tuytelaars, T., De Raedt, L.: A relational kernel-based framework for hierarchical image understanding. In: Gimel’farb, G., et al. (eds.) SSPR /SPR 2012. LNCS, vol. 7626, pp. 171–180. Springer, Heidelberg (2012). Scholar
  3. 3.
    Baldoni, V., Berline, N., De Loera, J., Köppe, M., Vergne, M.: How to integrate a polynomial over a simplex. Math. Comput. 80(273), 297–325 (2011)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Baral, C., Gelfond, M., Rushton, J.N.: Probabilistic reasoning with answer sets. TPLP 9(1), 57–144 (2009)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Belle, V., Van den Broeck, G., Passerini, A.: Hashing-based approximate probabilistic inference in hybrid domains. In: UAI (2015)Google Scholar
  6. 6.
    Belle, V., Passerini, A., Van den Broeck, G.: Probabilistic inference in hybrid domains by weighted model integration. In: Proceedings of 24th International Joint Conference on Artificial Intelligence (IJCAI), pp. 2770–2776 (2015)Google Scholar
  7. 7.
    Bellodi, E., Riguzzi, F.: Learning the structure of probabilistic logic programs. In: Muggleton, S.H., Tamaddoni-Nezhad, A., Lisi, F.A. (eds.) ILP 2011. LNCS (LNAI), vol. 7207, pp. 61–75. Springer, Heidelberg (2012). Scholar
  8. 8.
    Bellodi, E., Riguzzi, F.: Structure learning of probabilistic logic programs by searching the clause space. Theory Pract. Logic Program. 15(2), 169–212 (2015)CrossRefGoogle Scholar
  9. 9.
    Chavira, M., Darwiche, A.: On probabilistic inference by weighted model counting. Artif. Intell. 172(6–7), 772–799 (2008)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Darwiche, A., Marquis, P.: A knowledge compilation map. J. Artif. Intell. Res. 17, 229–264 (2002)MathSciNetCrossRefGoogle Scholar
  11. 11.
    De Boor, C., De Boor, C., Mathématicien, E.U., De Boor, C., De Boor, C.: A practical guide to splines, vol. 27. Springer, New York (1978)CrossRefGoogle Scholar
  12. 12.
    De Maeyer, D., Renkens, J., Cloots, L., De Raedt, L., Marchal, K.: Phenetic: network-based interpretation of unstructured gene lists in E. coli. Mol. BioSyst. 9(7), 1594–1603 (2013)CrossRefGoogle Scholar
  13. 13.
    De Raedt, L., Dries, A., Thon, I., Van den Broeck, G., Verbeke, M.: Inducing probabilistic relational rules from probabilistic examples. In: Proceedings of 24th International Joint Conference on Artificial Intelligence (IJCAI), pp. 1835–1842 (2015)Google Scholar
  14. 14.
    De Raedt, L., Kimmig, A.: Probabilistic (logic) programming concepts. Mach. Learn. 100(1), 5–47 (2015). Scholar
  15. 15.
    Dheeru, D., Karra Taniskidou, E.: UCI machine learning repository (2017).
  16. 16.
    Dougherty, J., Kohavi, R., Sahami, M., et al.: Supervised and unsupervised discretization of continuous features. In: Machine Learning: Proceedings of the Twelfth International Conference, vol. 12, pp. 194–202 (1995)Google Scholar
  17. 17.
    Džeroski, S., Cestnik, B., Petrovski, I.: Using the m-estimate in rule induction. J. Comput. Inf. Technol. 1(1), 37–46 (1993)Google Scholar
  18. 18.
    Fierens, D., Van den Broeck, G., Thon, I., Gutmann, B., Raedt, L.D.: Inference in probabilistic logic programs using weighted CNF’s. In: UAI, pp. 211–220 (2011)Google Scholar
  19. 19.
    Getoor, L., Friedman, N., Koller, D., Pfeffer, A.: Learning probabilistic relational models. In: Džeroski, S., Lavrač, N. (eds.) Relational data Mining, pp. 307–335. Springer, Heidelberg (2001). Scholar
  20. 20.
    Goodman, N.D., Mansinghka, V.K., Roy, D.M., Bonawitz, K., Tenenbaum, J.B.: Church: a language for generative models. In: Proceedings of UAI, pp. 220–229 (2008)Google Scholar
  21. 21.
    Gutmann, B., Jaeger, M., De Raedt, L.: Extending ProbLog with continuous distributions. In: Frasconi, P., Lisi, F.A. (eds.) ILP 2010. LNCS (LNAI), vol. 6489, pp. 76–91. Springer, Heidelberg (2011). Scholar
  22. 22.
    Islam, M.A., Ramakrishnan, C., Ramakrishnan, I.: Parameter learning in prism programs with continuous random variables. arXiv preprint arXiv:1203.4287 (2012)
  23. 23.
    Kimmig, A., Van den Broeck, G., De Raedt, L.: An algebraic prolog for reasoning about possible worlds. In: Proceedings of the AAAI (2011).
  24. 24.
    Kok, S., Domingos, P.: Learning the structure of Markov logic networks. In: Proceedings of the International Conference on Machine Learning, pp. 441–448 (2005)Google Scholar
  25. 25.
    Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT Press, Cambridge (2009)zbMATHGoogle Scholar
  26. 26.
    Landwehr, N., Kersting, K., De Raedt, L.: nFOIL: integrating Naïve Bayes and FOIL. In: AAAI 2005, pp. 795–800 (2005)Google Scholar
  27. 27.
    Landwehr, N., Passerini, A., De Raedt, L., Frasconi, P., et al.: kFOIL: learning simple relational kernels. AAAI 6, 389–394 (2006)Google Scholar
  28. 28.
    Lauritzen, S.L., Jensen, F.: Stable local computation with conditional gaussian distributions. Stat. Comput. 11(2), 191–203 (2001)MathSciNetCrossRefGoogle Scholar
  29. 29.
    López-Cruz, P.L., Bielza, C., Larrañaga, P.: Learning mixtures of polynomials of multidimensional probability densities from data using b-spline interpolation. Int. J. Approximate Reasoning 55(4), 989–1010 (2014)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Martires, P.Z.D., Dries, A., Raedt, L.D.: Knowledge compilation with continuous random variables and its application in hybrid probabilistic logic programming (2018).
  31. 31.
    Milch, B., Marthi, B., Russell, S.J., Sontag, D., Ong, D.L., Kolobov, A.: BLOG: probabilistic models with unknown objects. In: Proceedings of the IJCAI, pp. 1352–1359 (2005)Google Scholar
  32. 32.
    Muggleton, S.: Inverse entailment and progol. New Gener. Comput. 13(3), 245–286 (1995)CrossRefGoogle Scholar
  33. 33.
    Murphy, K.P.: A variational approximation for Bayesian networks with discrete and continuous latent variables. In: UAI, pp. 457–466 (1999)Google Scholar
  34. 34.
    Nitti, D., Laet, T.D., Raedt, L.D.: A particle filter for hybrid relational domains. In: IROS, pp. 2764–2771 (2013)Google Scholar
  35. 35.
    Nitti, D., Ravkic, I., Davis, J., De Raedt, L.: Learning the structure of dynamic hybrid relational models. In: ECAI 2016, vol. 285, pp. 1283–1290 (2016)Google Scholar
  36. 36.
    Pasula, H., Marthi, B., Milch, B., Russell, S.J., Shpitser, I.: Identity uncertainty and citation matching. In: NIPS, pp. 1401–1408 (2002).
  37. 37.
    Poole, D., Bacchus, F., Kisyński, J.: Towards completely lifted search-based probabilistic inference. CoRR abs/1107.4035 (2011)Google Scholar
  38. 38.
    Quinlan, J.R.: Learning logical definitions from relations. Mach. Learn. 5(3), 239–266 (1990)Google Scholar
  39. 39.
    Raedt, L.D., Kersting, K., Natarajan, S., Poole, D.: Statistical Relational Artificial Intelligence: Logic, Probability, and Computation. Synthesis Lectures on Artificial Intelligence and Machine Learning, vol. 10, no. 2, pp. 1–189 (2016)Google Scholar
  40. 40.
    Raghavan, S., Mooney, R.J., Ku, H.: Learning to read between the lines using Bayesian logic programs. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pp. 349–358. Association for Computational Linguistics (2012)Google Scholar
  41. 41.
    Ravkic, I., Ramon, J., Davis, J.: Learning relational dependency networks in hybrid domains. Mach. Learn. 100(2–3), 217–254 (2015)MathSciNetCrossRefGoogle Scholar
  42. 42.
    Richardson, M., Domingos, P.: Markov logic networks. Mach. Learn. 62(1), 107–136 (2006)CrossRefGoogle Scholar
  43. 43.
    Russell, S.: Unifying logic and probability. Commun. ACM 58(7), 88–97 (2015)CrossRefGoogle Scholar
  44. 44.
    Schoenmackers, S., Etzioni, O., Weld, D.S., Davis, J.: Learning first-order horn clauses from web text. In: Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pp. 1088–1098. Association for Computational Linguistics (2010)Google Scholar
  45. 45.
    Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978)MathSciNetCrossRefGoogle Scholar
  46. 46.
    Zong, Z.: Information-Theoretic Methods for Estimating of Complicated Probability Distributions, vol. 207. Elsevier (2006) Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.University of EdinburghEdinburghUK
  2. 2.Alan Turing InstituteLondonUK

Personalised recommendations