Retrofitting Soft Rules for Knowledge Representation Learning

  • Bo AnEmail author
  • Xianpei Han
  • Le Sun
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12032)


Recently, a significant number of studies have focused on knowledge graph completion using rule-enhanced learning techniques, supported by the mined soft rules in addition to the hard logic rules. However, due to the difficulty in determining the confidences of the soft rules without the global semantics of knowledge graph such as the semantic relatedness between relations, the knowledge representation may not be optimal, leading to degraded effectiveness in its application to knowledge graph completion tasks. To address this challenge, this paper proposes a retrofit framework that iteratively enhances the knowledge representation and confidences of soft rules. Specifically, the soft rules guide the learning of knowledge representation, and the representation, in turn, provides global semantic of the knowledge graph to optimize the confidences of soft rules. Extensive evaluation shows that our method achieves new state-of-the-art results on link prediction and triple classification tasks, brought by the fine-tuned confidences of soft rules.


Knowledge representation Soft rules Link prediction 



This work is supported by the National Natural Science Foundation of China under Grants no. 61433015, 61572477 and 61772505.


  1. 1.
    Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase:a collaboratively created graph database for structuring human knowledge. In: ACM SIGMOD International Conference on Management of Data, SIGMOD 2008, Vancouver, Bc, Canada, June, pp. 1247–1250 (2008)Google Scholar
  2. 2.
    Suchanek, F.M., Kasneci, G., Weikum, G.: Yago: a core of semantic knowledge. In: International Conference on World Wide Web, WWW 2007, Banff, Alberta, Canada, May, pp. 697–706 (2007)Google Scholar
  3. 3.
    Dong, X., et al.: Knowledge vault: a web-scale approach to probabilistic knowledge fusion. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 601–610 (2014)Google Scholar
  4. 4.
    Socher, R., Chen, D., Manning, C.D., Ng, A.Y.: Reasoning with neural tensor networks for knowledge base completion. In: International Conference on Intelligent Control and Information Processing, pp. 464–469 (2013)Google Scholar
  5. 5.
    Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: NIPS, pp. 2787–2795 (2013)Google Scholar
  6. 6.
    Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: AAAI, pp. 1112–1119 (2014)Google Scholar
  7. 7.
    Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, pp. 687–696 (2015)Google Scholar
  8. 8.
    Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: AAAI, pp. 2181–2187 (2015)Google Scholar
  9. 9.
    Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: International Conference on Machine Learning, pp. 2071–2080 (2016)Google Scholar
  10. 10.
    Rocktäschel, T., Singh, S., Riedel, S.: Injecting logical background knowledge into embeddings for relation extraction. In: HLT-NAACL, pp. 1119–1129 (2015)Google Scholar
  11. 11.
    Guo, S., Wang, Q., Wang, L., Wang, B., Guo, L.: Jointly embedding knowledge graphs and logical rules. In: EMNLP, pp. 192–202 (2016)Google Scholar
  12. 12.
    Guo, S., Wang, Q., Wang, L., Wang, B., Guo, L.: Knowledge graph embedding with iterative guidance from soft rules. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)Google Scholar
  13. 13.
    Galárraga, L., Teflioudi, C., Hose, K., Suchanek, F.M.: Fast rule mining in ontological knowledge bases with AMIE+. VLDB J. 24(6), 707–730 (2015)CrossRefGoogle Scholar
  14. 14.
    Zhong, H., Zhang, J., Wang, Z., Wan, H., Chen, Z.: Aligning knowledge and text embeddings by entity descriptions. In: EMNLP, pp. 267–272 (2015)Google Scholar
  15. 15.
    Xie, R., Liu, Z., Jia, J., Luan, H., Sun, M.: Representation learning of knowledge graphs with entity descriptions. In: AAAI, pp. 2659–2665 (2016)Google Scholar
  16. 16.
    Xu, J., Chen, K., Qiu, X., Huang, X.: Knowledge graph representation with jointly structural and textual encoding. arXiv preprint arXiv:1611.08661 (2016)
  17. 17.
    An, B., Chen, B., Han, X., Sun, L.: Accurate text-enhanced knowledge graph representation learning. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 745–755 (2018)Google Scholar
  18. 18.
    Riedel, S., Yao, L., McCallum, A., Marlin, B.M.: Relation extraction with matrix factorization and universal schemas. In: HLT-NAACL. pp. 74–84 (2013)Google Scholar
  19. 19.
    Toutanova, K., Chen, D., Pantel, P., Poon, H., Choudhury, P., Gamon, M.: Representing text for joint embedding of text and knowledge bases. EMNLP 15, 1499–1509 (2015)Google Scholar
  20. 20.
    Xiao, H., Huang, M., Zhu, X.: Transg: a generative model for knowledge graph embedding. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 2316–2325 (2016)Google Scholar
  21. 21.
    Wang, Z., Li, J., Liu, Z., Tang, J.: Text-enhanced representation learning for knowledge graph. In: To appear in IJCAI 2016, pp. 04–17 (2016)Google Scholar
  22. 22.
    Lin, Y., Liu, Z., Luan, H., Sun, M., Rao, S., Liu, S.: Modeling relation paths for representation learning of knowledge bases. arXiv preprint arXiv:1506.00379 (2015)
  23. 23.
    Toutanova, K., Lin, X.V., Yih, W.T., Poon, H., Quirk, C.: Compositional learning of embeddings for relation paths in knowledge bases and text. In: ACL2016, vol. 1, pp. 1434–1444 (2016)Google Scholar
  24. 24.
    Xiong, W., Hoang, T., Wang, W.Y.: Deeppath: a reinforcement learning method for knowledge graph reasoning. arXiv preprint arXiv:1707.0669 (2017)
  25. 25.
    Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graph embeddings. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)Google Scholar
  26. 26.
    Wang, Q., Wang, B., Guo, L.: Knowledge base completion using embeddings and rules. In: International Conference on Artificial Intelligence, pp. 1859–1865 (2015)Google Scholar
  27. 27.
    Guo, S., Ding, B., Wang, Q., Wang, L., Wang, B.: Knowledge base completion via rule-enhanced relational learning. In: Chen, H., Ji, H., Sun, L., Wang, H., Qian, T., Ruan, T. (eds.) CCKS 2016. CCIS, vol. 650, pp. 219–227. Springer, Singapore (2016). Scholar
  28. 28.
    Minervini, P.: Adversarial sets for regularising neural link predictors. In: Conference on Uncertainty in Artificial Intelligence (2017)Google Scholar
  29. 29.
    Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)
  30. 30.
    Hu, Z., Ma, X., Liu, Z., Hovy, E., Xing, E.: Harnessing deep neural networks with logic rules. arXiv preprint arXiv:1603.06318 (2016)
  31. 31.
    Hájek, P.: Metamathematics of Fuzzy Logic, vol. 4. Springer Science & Business Media, Dordrecht (1998)CrossRefGoogle Scholar
  32. 32.
    Ganchev, K., Gillenwater, J., Taskar, B., et al.: Posterior regularization for structured latent variable models. J.f Mach. Learn. Res. 11, 2001–2049 (2010)MathSciNetzbMATHGoogle Scholar
  33. 33.
    Bordes, A., Weston, J., Collobert, R., Bengio, Y.: Learning structured embeddings of knowledge bases. In: AAAI Conference on Artificial Intelligence, AAAI 2011, San Francisco, California, USA, August (2011)Google Scholar
  34. 34.
    Yang, B., Yih, W.T., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. Eprint Arxiv (2014)Google Scholar
  35. 35.
    Nickel, M., Rosasco, L., Poggio, T.A., et al.: Holographic embeddings of knowledge graphs. In: AAAI, pp. 1955–1961 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.State Key Laboratory of Computer Science, Institute of SoftwareChinese Academy of SciencesBeijingChina

Personalised recommendations