Path-based reasoning with constrained type attention for knowledge graph completion

  • Kai Lei
  • Jin Zhang
  • Yuexiang Xie
  • Desi Wen
  • Daoyuan Chen
  • Min Yang
  • Ying ShenEmail author
Original Article


Multi-hop reasoning over paths in knowledge graphs has attracted rising research interest in the field of knowledge graph completion. Entity types and relation types both contain various kinds of information content though only a subset of them are helpful in the specific triples. Although significant progress has been made by existing models, they have two major shortcomings. First, these models seldom learn an explicit representation of entities and relations with semantic information. Second, they reason without discriminating distinct role types that the same entity with multiple types plays in different triples. To address these issues, we develop a novel path-based reasoning with constrained type attention model, which tries to identify entity types by leveraging relation type constraints in the corresponding triples. Our experimental evaluation shows that the proposed model outperforms the state of the art on a real-world dataset. Further analyses also confirm that both word-level and triple-level attention mechanisms of our model are effective.


Neural network Knowledge graph completion Multi-hop reasoning Attention mechanism 



This work was financially supported by the National Natural Science Foundation of China (No. 61602013), Natural Science Foundation of Guangdong (No. 2018A030313017) and the Shenzhen Fundamental Research Project (No. JCYJ20170818091546869).

Compliance with ethical standards

Conflict of interest

The authors declare that there is no actual or potential conflict of interest in relation to this article.


  1. 1.
    Suchanek FM, Kasneci G, Weikum G (2007) Yago: a core of semantic knowledge. In: Proceedings of the 16th international conference on World Wide Web. ACM, pp 697–706Google Scholar
  2. 2.
    Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD international conference on Management of data. ACM, pp 1247–1250Google Scholar
  3. 3.
    Carlson A, Betteridge J, Kisiel B, Settles B, Hruschka Jr ER, Mitchell TM (2010) Toward an architecture for never-ending language learning. In: AAAI, vol 5, p 3Google Scholar
  4. 4.
    Lehmann J, Isele R, Jakob M, Jentzsch A, Kontokostas D, Mendes PN, Bizer C (2015) DBpediaa large-scale, multilingual knowledge base extracted from Wikipedia. Semantic Web 6(2):167–195Google Scholar
  5. 5.
    Bordes A, Weston J, Usunier N (2014) Open question answering with weakly supervised embedding models. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, Berlin, Heidelberg, pp 165–180Google Scholar
  6. 6.
    Bordes A, Chopra S, Weston J (2014) Question answering with subgraph embeddings. arXiv preprint arXiv:1406.3676
  7. 7.
    Zhang F, Yuan NJ, Lian D, Xie X, Ma WY (2016) Collaborative knowledge base embedding for recommender systems. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 353–362Google Scholar
  8. 8.
    Catherine R, Cohen W (2016) Personalized recommendations using knowledge graphs: a probabilistic logic programming approach. In: Proceedings of the 10th ACM conference on recommender systems. ACM, pp 325–332Google Scholar
  9. 9.
    Lao N, Mitchell T, Cohen WW (2011) Random walk inference and learning in a large scale knowledge base. In: Proceedings of the conference on empirical methods in natural language processing. Association for Computational Linguistics, pp 529–539Google Scholar
  10. 10.
    Wen D, Liu Y, Yuan K, Si S, Shen Y (2017) Attention-aware path-based relation extraction for medical knowledge graph. In: International conference on smart computing and communication. Springer, Cham, pp 321–331Google Scholar
  11. 11.
    Neelakantan A, Roth B, Mc-Callum A (2015) Compositional vector space models for knowledge base inference. In: 2015 AAAI spring symposium seriesGoogle Scholar
  12. 12.
    Das R, Neelakantan A, Belanger D, McCallum A (2017) Chains of reasoning over entities, relations, and text using recurrent neural networks. In: Proceedings of the 15th conference of the European Chapter of the Association for Computational Linguistics: volume 1, Long Papers, p 141Google Scholar
  13. 13.
    Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Advances in neural information processing systems, pp 2787–2795Google Scholar
  14. 14.
    Ji G, He S, Xu L, Liu K, Zhao J (2015) Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 1: long papers), pp 687–696Google Scholar
  15. 15.
    Ji G, Liu K, He S, Zhao J (2016) Knowledge graph completion with adaptive sparse transfer matrix. In: AAAI, pp 985–991Google Scholar
  16. 16.
    Zelenko D, Aone C, Richardella A (2003) Kernel methods for relation extraction. J Mach Learn Res 3:1083–1106MathSciNetzbMATHGoogle Scholar
  17. 17.
    Gormley MR, Yu M, Dredze M (2015) Improved relation extraction with feature-rich compositional embedding models. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 1774–1784Google Scholar
  18. 18.
    Socher R, Huval B, Manning C D, Ng A Y (2012, July) Semantic compositionality through recursive matrix-vector spaces. In: Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning. Association for Computational Linguistics, pp 1201–1211Google Scholar
  19. 19.
    Angeli G, Manning C (2013) Philosophers are mortal: Inferring the truth of unseen facts. In: Proceedings of the seventeenth conference on computational natural language learning, pp 133–142Google Scholar
  20. 20.
    Lao N, Cohen WW (2010) Relational retrieval using a combination of path-constrained random walks. Mach Learn 81(1):53–67MathSciNetCrossRefGoogle Scholar
  21. 21.
    Socher R, Chen D, Manning C D, Ng A (2013) Reasoning with neural tensor networks for knowledge base completion. In: Advances in neural information processing systems, pp 926–934Google Scholar
  22. 22.
    Gardner M, Talukdar P, Krishnamurthy J, Mitchell T (2014) Incorporating vector space similarity in random walk inference over knowledge bases. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 397–406Google Scholar
  23. 23.
    Gardner M, Talukdar P P, Kisiel B, Mitchell T (2013) Improving learning and inference in a large knowledge-base using latent syntactic cues. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp 833-838Google Scholar
  24. 24.
    Toutanova K, Chen D, Pantel P, Poon H, Choudhury P, Gamon M (2015) Representing text for joint embedding of text and knowledge bases. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 1499–1509Google Scholar
  25. 25.
    Jiang X, Wang Q, Qi B, Qiu Y, Li P, Wang B (2017) Attentive path combination for knowledge graph completion. In: Asian conference on machine learning, pp 590–605Google Scholar
  26. 26.
    Chen W, Xiong W, Yan X, Wang WY (2018) Variational knowledge graph reasoning. In: Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (Long Papers), pp 1823–1832Google Scholar
  27. 27.
    Shen Y, Deng Y, Yang M, Li Y, Du N, Fan W, Lei K (2018) Knowledge-aware attentive neural network for ranking question answer pairs. In: The 41st international ACM SIGIR conference on research development in information retrieval. ACM, pp 901–904Google Scholar
  28. 28.
    Bahdanau D, Cho K, Bengio Y (2015) Neural machine translation by jointly learning to align and translate. In: Proceedings of international conference on learning representationsGoogle Scholar
  29. 29.
    Zhou P, Shi W, Tian J, Qi Z, Li B, Hao H, Xu B (2016) Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th annual meeting of the association for computational linguistics (volume 2: short papers), pp 207–212Google Scholar
  30. 30.
    Lin Y, Shen S, Liu Z, Luan H, Sun M (2016) Neural relation extraction with selective attention over instances. In: Proceedings of the 54th annual meeting of the association for computational linguistics (volume 1: long papers), pp 2124–2133Google Scholar
  31. 31.
    Yang Z, Yang D, Dyer C, He X, Smola A, Hovy E (2016) Hierarchical attention networks for document classification. In: Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 1480–1489Google Scholar
  32. 32.
    Zhang H, Li J, Ji Y, Yue H (2017) Understanding subtitles by character-level sequence-to-sequence learning. IEEE Trans Ind Inf 13(2):616–624CrossRefGoogle Scholar
  33. 33.
    Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780CrossRefGoogle Scholar
  34. 34.
    Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp 249–256Google Scholar
  35. 35.
    Kingma D P, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.Shenzhen Key Lab for Information Centric Networking and Blockchain Technology (ICNLAB), School of Electronics and Computer Engineering (SECE)Peking UniversityShenzhenPeople’s Republic of China
  2. 2.Shenzhen Institutes of Advanced Technology (SIAT)Chinese Academy of SciencesShenzhenPeople’s Republic of China
  3. 3.PCL Research Center of Networks and Communications, Peng Cheng LaboratoryShenzhenPeople’s Republic of China

Personalised recommendations