Learning to Compose Relational Embeddings in Knowledge Graphs

  • Wenye Chen
  • Huda HakamiEmail author
  • Danushka Bollegala
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1215)


Knowledge Graph Embedding methods learn low-dimensional representations for entities and relations in knowledge graphs, which can be used to infer previously unknown relations between pairs of entities in the knowledge graph. This is particularly useful for expanding otherwise sparse knowledge graphs. However, the relation types that can be predicted using knowledge graph embeddings are confined to the set of relations that already exists in the KG. Often the set of relations that exist between two entities are not independent, and it is possible to predict what other relations are likely to exist between two entities by composing the embeddings of the relations in which each entity participates. We introduce relation composition as the task of inferring embeddings for unseen relations by combining existing relations in a knowledge graph. Specifically, we propose a supervised method to compose relational embeddings for novel relations using pre-trained relation embeddings for existing relations. Our experimental results on a previously proposed benchmark dataset for relation composition ranking and triple classification show that the proposed supervised relation composition method outperforms several unsupervised relation composition methods.


Knowledge graphs Knowledge graph embeddings Relation composition Novel relation types 



We would like to thank Ran Tian for sharing the relation composition benchmark dataset.


  1. 1.
    Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of SIGMOD, pp. 1247–1250 (2008)Google Scholar
  2. 2.
    Bollegala, D., Hakami, H., Yoshida, Y., Kawarabayashi, K.i.: Relwalk - a latent variable model approach to knowledge graph embedding (2019).
  3. 3.
    Bordes, A., Usunier, N., Garcia-Durán, A., Weston, J., Yakhenko, O.: Translating embeddings for modeling multi-relational data. In: Proceedings of NIPS (2013)Google Scholar
  4. 4.
    Bordes, A., Weston, J., Collobert, R., Bengio, Y.: Learning structured embeddings of knowledge bases. In: Proceedings of AAAI (2011)Google Scholar
  5. 5.
    Ding, B., Wang, Q., Wang, B., Guo, L.: Improving knowledge graph embedding using simple constraints. In: Proceedings of ACL, pp. 110–121 (2018)Google Scholar
  6. 6.
    Guu, K., Miller, J., Liang, P.: Traversing knowledge graphs in vector space. In: Proceedings of EMNLP, pp. 318–327 (2015)Google Scholar
  7. 7.
    Hornik, K.: Multilayer feedforward networks are universal approximators. Neural Netw. 2, 359–366 (1989)CrossRefGoogle Scholar
  8. 8.
    Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of ACL, pp. 687–696 (2015)Google Scholar
  9. 9.
    Ji, G., Liu, K., He, S., Zhao, J.: Knowledge graph completion with adaptive sparse transfer matrix. In: Proceedings of AAAI, pp. 985–991 (2016)Google Scholar
  10. 10.
    Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In: Proceedings of ICLR (2015)Google Scholar
  11. 11.
    Lao, N., Cohen, W.W.: Relational retrieval using a combination of path-constrained random walks. Mach. Learn. 81(1), 53–67 (2010). Scholar
  12. 12.
    Lao, N., Mitchell, T., Cohen, W.W.: Random walk inference and learning in a large scale knowledge base. In: Proceedings of EMNLP, pp. 529–539 (2011)Google Scholar
  13. 13.
    Larochelle, H., Erhan, D., Bengio, Y.: Zero-data learning of new tasks. In: Proceedings of AAAI, pp. 646–651 (2008)Google Scholar
  14. 14.
    Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of AAAI, pp. 2181–2187 (2015)Google Scholar
  15. 15.
    Neelakantan, A., Roth, B., McCallum, A.: Compositional vector space models for knowledge base completion. In: Proceedings of ACL, pp. 156–166 (2015)Google Scholar
  16. 16.
    Nguyen, D.Q., Sirts, K., Qu, L., Johnson, M.: Stranse: a novel embedding model of entities and relationships in knowledge bases. In: Proceedings of NAACL-HLT, pp. 460–466 (2016)Google Scholar
  17. 17.
    Nickel, M., Rosasco, L., Poggio, T.: Holographic embeddings of knowledge graphs. In: Proceedings of AAAI (2016)Google Scholar
  18. 18.
    Nickel, M., Tresp, V., Kriegel, H.P.: A three-way model for collective learning on multi-relational data. In: Proceedings of ICML, pp. 809–816 (2011)Google Scholar
  19. 19.
    Riedel, S., Yao, L., McCallum, A., Marlin, B.M.: Relation extraction with matrix factorization and universal schemas. In: Proceedings of NAACL, pp. 74–84 (2013)Google Scholar
  20. 20.
    Socher, R., Chen, D., Manning, C.D., Ng, A.: Reasoning with neural tensor networks for knowledge base completion. In: Advances in Neural Information Processing Systems, pp. 926–934 (2013)Google Scholar
  21. 21.
    Socher, R., Chen, D., Manning, C.D., Ng, A.Y.: Reasoning with neural tensor networks for knowledge base completion. In: Proceedings of NIPS (2013)Google Scholar
  22. 22.
    Takahashi, R., Tian, R., Inui, K.: Interpretable and compositional relation learning by joint training with an autoencoder. In: Proceedings of ACL, pp. 2148–2159 (2018)Google Scholar
  23. 23.
    Toutanova, K., Chen, D.: Observed versus latent features for knowledge base and text inference. In: Proceedings of 3rd Workshop on Continuous Vector Space Models and their Compositionality, pp. 57–66 (2015)Google Scholar
  24. 24.
    Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: Proceedings of ICML (2016)Google Scholar
  25. 25.
    Wang, Q., Mao, Z., Wang, B., Guo, L.: Knowledge graph embedding: a survey of approaches and applications. IEEE Trans. Knowl. Data Eng. 29(12), 2724–2743 (2017). Scholar
  26. 26.
    Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Proceedings of AAAI, pp. 1112–1119 (2014)Google Scholar
  27. 27.
    Xiao, H., Huang, M., Zhu, X.: TransG : a generative model for knowledge graph embedding. In: Proceedings of ACL, pp. 2316–2325 (2016)Google Scholar
  28. 28.
    Yang, B., Yih, W.T., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. In: ICLR (2015)Google Scholar
  29. 29.
    Yoon, H.G., Song, H.J., Park, S.B., Park, S.Y.: A translation-based knowledge graph embedding preserving logical property of relations. In: Proceedings of NAACL, pp. 907–916 (2016)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of LiverpoolLiverpoolUK
  2. 2.Department of Computer ScienceTaif UniveristyTaifSaudi Arabia

Personalised recommendations