Complementing Logical Reasoning with Sub-symbolic Commonsense

  • Federico BianchiEmail author
  • Matteo Palmonari
  • Pascal Hitzler
  • Luciano Serafini
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11784)


Neuro-symbolic integration is a current field of investigation in which symbolic approaches are combined with deep learning ones. In this work we start from simple non-relational knowledge that can be extracted from text by considering the co-occurrence of entities inside textual corpora; we show that we can easily integrate this knowledge with Logic Tensor Networks (LTNs), a neuro-symbolic model. Using LTNs it is possible to integrate axioms and facts with commonsense knowledge represented in a sub-symbolic form in one single model performing well in reasoning tasks. In spite of some current limitations, we show that results are promising.



We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research.


  1. 1.
    Bach, S.H., Broecheler, M., Huang, B., Getoor, L.: Hinge-loss markov random fields and probabilistic soft logic. J. Mach. Learn. Res. 18, 1–67 (2017)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Bianchi, F., Hitzler, P.: On the capabilities of logic tensor networks for deductive reasoning. In: AAAI Spring Symposium: Combining Machine Learning with Knowledge Engineering (2019)Google Scholar
  3. 3.
    Bianchi, F., Palmonari, M., Nozza, D.: Towards encoding time in text-based entity embeddings. In: Vrandečić, D., et al. (eds.) ISWC 2018. LNCS, vol. 11136, pp. 56–71. Springer, Cham (2018). Scholar
  4. 4.
    Boleda, G., Herbelot, A.: Formal distributional semantics: introduction to the special issue. Comput. Linguist. 42(4), 619–635 (2016)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: NIPS, pp. 2787–2795 (2013)Google Scholar
  6. 6.
    Bouchard, G., Singh, S., Trouillon, T.: On approximate reasoning capabilities of low-rank vector spaces. In: Integrating Symbolic and Neural Approaches, AAAI Spring Syposium on Knowledge Representation and Reasoning (KRR) (2015)Google Scholar
  7. 7.
    Chudnoff, E.: Intuitive knowledge. Philos. Stud. 162(2), 359–378 (2013)CrossRefGoogle Scholar
  8. 8.
    De Raedt, L., Kimmig, A.: Probabilistic (logic) programming concepts. Mach. Learn. 100(1), 5–47 (2015)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Donadello, I., Serafini, L., d’Avila Garcez, A.: Logic tensor networks for semantic image interpretation. In: IJCAI, pp. 1596–1602 (2017)Google Scholar
  10. 10.
    Ebrahimi, M., Sarker, M.K., Bianchi, F., Xie, N., Doran, D., Hitzler, P.: Reasoning over RDF knowledge bases using deep learning. arXiv preprint arXiv:1811.04132 (2018)
  11. 11.
    Garcez, A.d., Gori, M., Lamb, L.C., Serafini, L., Spranger, M., Tran, S.N.: Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning. arXiv preprint arXiv:1905.06088 (2019)
  12. 12.
    Garcez, A.S., Lamb, L.C., Gabbay, D.M.: Neural-symbolic cognitive reasoning. Springer, Heidelberg (2008). Scholar
  13. 13.
    Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)zbMATHGoogle Scholar
  14. 14.
    Guo, S., Wang, Q., Wang, L., Wang, B., Guo, L.: Jointly embedding knowledge graphs and logical rules. In: EMNLP, pp. 192–202 (2016)Google Scholar
  15. 15.
    Harris, Z.S.: Distributional structure. Word 10(2–3), 146–162 (1954)CrossRefGoogle Scholar
  16. 16.
    Hohenecker, P., Lukasiewicz, T.: Deep learning for ontology reasoning. arXiv preprint arXiv:1705.10342 (2017)
  17. 17.
    Kuipers, B.: On representing commonsense knowledge. In: Findler, N.V. (ed.) Associative Networks, pp. 393–408. Elsevier, New York (1979)CrossRefGoogle Scholar
  18. 18.
    Lenci, A.: Distributional semantics in linguistic and cognitive research. Ital. J. Linguist. 20(1), 1–31 (2008)Google Scholar
  19. 19.
    Manhaeve, R., Dumančić, S., Kimmig, A., Demeester, T., De Raedt, L.: Deepproblog: neural probabilistic logic programming. arXiv preprint arXiv:1805.10872 (2018)
  20. 20.
    Meza-Ruiz, I., Riedel, S.: Jointly identifying predicates, arguments and senses using markov logic. In: NAACL, pp. 155–163. ACL (2009)Google Scholar
  21. 21.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: NIPS, pp. 3111–3119 (2013)Google Scholar
  22. 22.
    Rizzo, G., Troncy, R.: NERD: a framework for unifying named entity recognition and disambiguation extraction tools. In: EACL, pp. 73–76. ACL (2012)Google Scholar
  23. 23.
    Rocktäschel, T., Riedel, S.: End-to-end differentiable proving. In: NIPS, pp. 3788–3800 (2017)Google Scholar
  24. 24.
    Serafini, L., d’Avila Garcez, A.S.: Learning and reasoning with logic tensor networks. In: Adorni, G., Cagnoni, S., Gori, M., Maratea, M. (eds.) AI*IA 2016. LNCS (LNAI), vol. 10037, pp. 334–348. Springer, Cham (2016). Scholar
  25. 25.
    Socher, R., Chen, D., Manning, C.D., Ng, A.: Reasoning with neural tensor networks for knowledge base completion. In: Advances in neural information processing systems, pp. 926–934 (2013)Google Scholar
  26. 26.
    Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: ICML, pp. 2071–2080 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Federico Bianchi
    • 1
    • 2
    Email author
  • Matteo Palmonari
    • 1
  • Pascal Hitzler
    • 2
    • 3
  • Luciano Serafini
    • 4
  1. 1.University of Milan-BicoccaMilanItaly
  2. 2.Wright State UniversityDaytonUSA
  3. 3.Kansas State UniversityManhattanUSA
  4. 4.Fondazione Bruno KesslerTrentoItaly

Personalised recommendations