Advertisement

Ontology Completion Using Graph Convolutional Networks

  • Na LiEmail author
  • Zied BouraouiEmail author
  • Steven SchockaertEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11778)

Abstract

Many methods have been proposed to automatically extend knowledge bases, but the vast majority of these methods focus on finding plausible missing facts, and knowledge graph triples in particular. In this paper, we instead focus on automatically extending ontologies that are encoded as a set of existential rules. In particular, our aim is to find rules that are plausible, but which cannot be deduced from the given ontology. To this end, we propose a graph-based representation of rule bases. Nodes of the considered graphs correspond to predicates, and they are annotated with vectors encoding our prior knowledge about the meaning of these predicates. The vectors may be obtained from external resources such as word embeddings or they could be estimated from the rule base itself. Edges connect predicates that co-occur in the same rule and their annotations reflect the types of rules in which the predicates co-occur. We then use a neural network model based on Graph Convolutional Networks (GCNs) to refine the initial vector representation of the predicates, to obtain a representation which is predictive of which rules are plausible. We present experimental results that demonstrate the strong performance of this method.

Keywords

Knowledge base completion Rule induction Graph Convolutional Networks Commonsense reasoning 

Notes

Acknowledgements

Steven Schockaert was supported by ERC Starting Grant 637277. Zied Bouraoui was supported by CNRS PEPS INS2I MODERN.

References

  1. 1.
    Alfarone, D., Davis, J.: Unsupervised learning of an IS-A taxonomy from a limited domain-specific corpus. In: Proceedings IJCAI, pp. 1434–1441 (2015)Google Scholar
  2. 2.
    Baader, F., Calvanese, D., McGuinness, D.L., Nardi, D., Patel-Schneider, P.F. (eds.): The Description Logic Handbook: Theory, Implementation, and Applications. Cambridge University Press, New York (2003)zbMATHGoogle Scholar
  3. 3.
    Baader, F., Ganter, B., Sertkaya, B., Sattler, U.: Completing description logic knowledge bases using formal concept analysis. In: Proceedings IJCAI, vol. 7, pp. 230–235 (2007)Google Scholar
  4. 4.
    Baget, J., Leclère, M., Mugnier, M., Salvat, E.: On rules with existential variables: walking the decidability line. Artif. Intell. 175(9–10), 1620–1654 (2011).  https://doi.org/10.1016/j.artint.2011.03.002MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Beltagy, I., Chau, C., Boleda, G., Garrette, D., Erk, K., Mooney, R.: Montague meets Markov: deep semantics with probabilistic logical form. In: Proceedings of *SEM13, pp. 11–21 (2013)Google Scholar
  6. 6.
    Bloehdorn, S., Sure, Y.: Kernel methods for mining instance data in ontologies. In: Aberer, K., et al. (eds.) ASWC/ISWC -2007. LNCS, vol. 4825, pp. 58–71. Springer, Heidelberg (2007).  https://doi.org/10.1007/978-3-540-76298-0_5CrossRefGoogle Scholar
  7. 7.
    Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Proceedings NIPS, pp. 2787–2795 (2013)Google Scholar
  8. 8.
    Bouraoui, Z., Jameel, S., Schockaert, S.: Inductive reasoning about ontologies using conceptual spaces. In: Proceedings AAAI, pp. 4364–4370 (2017)Google Scholar
  9. 9.
    Bouraoui, Z., Schockaert, S.: Automated rule base completion as Bayesian concept induction. In: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, Honolulu, Hawaii, USA, 27 January–1 February (2019)Google Scholar
  10. 10.
    Bühmann, L., Lehmann, J., Westphal, P.: Dl-learner–a framework for inductive learning on the semantic web. J. Web Semant. 39, 15–24 (2016)CrossRefGoogle Scholar
  11. 11.
    Camacho-Collados, J., Pilehvar, M.T., Navigli, R.: Nasari: integrating explicit knowledge and corpus statistics for a multilingual representation of concepts and entities. Artif. Intell. 240, 36–64 (2016)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Duvenaud, D.K., et al.: Convolutional networks on graphs for learning molecular fingerprints. In: Advances in Neural Information Processing Systems, pp. 2224–2232 (2015)Google Scholar
  13. 13.
    Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 1263–1272. JMLR. org (2017)Google Scholar
  14. 14.
    Grover, A., Zweig, A., Ermon, S.: Graphite: iterative generative modeling of graphs. arXiv preprint arXiv:1803.10459 (2018)
  15. 15.
    Guo, S., Wang, Q., Wang, L., Wang, B., Guo, L.: Knowledge graph embedding with iterative guidance from soft rules. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)Google Scholar
  16. 16.
    Hamaguchi, T., Oiwa, H., Shimbo, M., Matsumoto, Y.: Knowledge transfer for out-of-knowledge-base entities: a graph neural network approach. arXiv preprint arXiv:1706.05674 (2017)
  17. 17.
    Hill, F., Cho, K., Korhonen, A.: Learning distributed representations of sentences from unlabelled data. In: Proceedings NAACL-HLT, pp. 1367–1377 (2016)Google Scholar
  18. 18.
    Horrocks, I.: Ontologies and the semantic web. Commun. ACM 51(12), 58–67 (2008).  https://doi.org/10.1145/1409360.1409377CrossRefGoogle Scholar
  19. 19.
    Jameel, S., Bouraoui, Z., Schockaert, S.: MEmbER: max-margin based embeddings for entity retrieval. In: Proceedings SIGIR, pp. 783–792 (2017)Google Scholar
  20. 20.
    Jameel, S., Schockaert, S.: Entity embeddings with conceptual subspaces as a basis for plausible reasoning. In: ECAI, pp. 1353–1361 (2016)Google Scholar
  21. 21.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  22. 22.
    Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
  23. 23.
    Kok, S., Domingos, P.: Statistical predicate invention. In: Proceedings ICML, pp. 433–440 (2007)Google Scholar
  24. 24.
    Kozareva, Z., Hovy, E.: A semi-supervised method to learn and construct taxonomies using the web. In: Proceedings EMNLP, pp. 1110–1118 (2010)Google Scholar
  25. 25.
    Lao, N., Mitchell, T., Cohen, W.W.: Random walk inference and learning in a large scale knowledge base. In: Proceedings EMNLP, pp. 529–539 (2011)Google Scholar
  26. 26.
    Lin, Y., Liu, Z., Luan, H., Sun, M., Rao, S., Liu, S.: Modeling relation paths for representation learning of knowledge bases. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 705–714 (2015)Google Scholar
  27. 27.
    Medina, J., Ojeda-Aciego, M., Vojtáš, P.: Similarity-based unification: a multi-adjoint approach. Fuzzy Sets Syst. 146, 43–62 (2004)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Proceedings of the 27th Annual Conference on Neural Information Processing Systems, pp. 3111–3119 (2013)Google Scholar
  29. 29.
    Mintz, M., Bills, S., Snow, R., Jurafsky, D.: Distant supervision for relation extraction without labeled data. In: Proceedings ACL, pp. 1003–1011 (2009)Google Scholar
  30. 30.
    Muggleton, S.H., Lin, D., Tamaddoni-Nezhad, A.: Meta-interpretive learning of higher-order dyadic datalog: predicate invention revisited. Mach. Learn. 100(1), 49–73 (2015)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Neelakantan, A., Chang, M.: Inferring missing entity type instances for knowledge base completion: new dataset and methods. In: Proceedings NAACL, pp. 515–525 (2015)Google Scholar
  32. 32.
    Qian, W., Fu, C., Zhu, Y., Cai, D., He, X.: Translating embeddings for knowledge graph completion with relation attention mechanism. In: IJCAI, pp. 4286–4292 (2018)Google Scholar
  33. 33.
    Riedel, S., Yao, L., McCallum, A.: Modeling relations and their mentions without labeled text. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds.) ECML PKDD 2010. LNCS, vol. 6323, pp. 148–163. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-15939-8_10CrossRefGoogle Scholar
  34. 34.
    Riedel, S., Yao, L., McCallum, A., Marlin, B.M.: Relation extraction with matrix factorization and universal schemas. In: Proceedings HLT-NAACL, pp. 74–84 (2013)Google Scholar
  35. 35.
    Rocktäschel, T., Riedel, S.: Learning knowledge base inference with neural theorem provers. In: Proceedings of the 5th Workshop on Automated Knowledge Base Construction, pp. 45–50 (2016)Google Scholar
  36. 36.
    Rocktäschel, T., Riedel, S.: End-to-end differentiable proving. In: Proceedings NIPS, pp. 3791–3803 (2017)Google Scholar
  37. 37.
    Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-93417-4_38CrossRefGoogle Scholar
  38. 38.
    Schockaert, S., Prade, H.: Interpolative and extrapolative reasoning in propositional theories using qualitative knowledge about conceptual spaces. Artif. Intell. 202, 86–131 (2013)MathSciNetCrossRefGoogle Scholar
  39. 39.
    Šourek, G., Manandhar, S., Železný, F., Schockaert, S., Kuželka, O.: Learning predictive categories using lifted relational neural networks. In: Cussens, J., Russo, A. (eds.) ILP 2016. LNCS, vol. 10326, pp. 108–119. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-63342-8_9CrossRefGoogle Scholar
  40. 40.
    Speer, R., Havasi, C., Lieberman, H.: AnalogySpace: reducing the dimensionality of common sense knowledge. In: Proceedings AAAI, pp. 548–553 (2008)Google Scholar
  41. 41.
    Toutanova, K., Chen, D., Pantel, P., Poon, H., Choudhury, P., Gamon, M.: Representing text for joint embedding of text and knowledge bases. In: Proceedings of EMNLP-15, pp. 1499–1509 (2015)Google Scholar
  42. 42.
    Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: Proceedings ICML, pp. 2071–2080 (2016)Google Scholar
  43. 43.
    Völker, J., Niepert, M.: Statistical schema induction. In: Antoniou, G., et al. (eds.) ESWC 2011. LNCS, vol. 6643, pp. 124–138. Springer, Heidelberg (2011).  https://doi.org/10.1007/978-3-642-21034-1_9CrossRefGoogle Scholar
  44. 44.
    West, R., Gabrilovich, E., Murphy, K., Sun, S., Gupta, R., Lin, D.: Knowledge base completion via search-based question answering. In: Proceedings WWW, pp. 515–526 (2014)Google Scholar
  45. 45.
    Xiao, H., Huang, M., Meng, L., Zhu, X.: SSP: semantic space projection for knowledge graph embedding with text descriptions. In: Proceedings AAAI, vol. 17, pp. 3104–3110 (2017)Google Scholar
  46. 46.
    Xie, R., Liu, Z., Jia, J., Luan, H., Sun, M.: Representation learning of knowledge graphs with entity descriptions. In: Proceedings of AAAI, pp. 2659–2665 (2016)Google Scholar
  47. 47.
    Yang, B., Yih, W., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. In: Proceedings of ICLR-15 (2015)Google Scholar
  48. 48.
    Zhong, H., Zhang, J., Wang, Z., Wan, H., Chen, Z.: Aligning knowledge and text embeddings by entity descriptions. In: EMNLP, pp. 267–272 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.State Key Laboratory for Novel Software TechnologyNanjing UniversityNanjingChina
  2. 2.CRIL - CNRS & University of ArtoisLensFrance
  3. 3.Cardiff UniversityCardiffUK

Personalised recommendations