Adaptive-Skip-TransE Model: Breaking Relation Ambiguities for Knowledge Graph Embedding

  • Shoukang Han
  • Xiaobo GuoEmail author
  • Lei Wang
  • Zeyi Liu
  • Nan Mu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11775)


Knowledge graph embedding aims to encode entities and relations into a low-dimensional vector space, obtaining its distributed vector representation for further knowledge learning and reasoning. Most existing methods assume that each relation owns one unique vector. However, in the real world, many relations are multi-semantic. We note that a reasonable adaptive learning method for the number of semantics for a given relation is lacking in knowledge graph embedding. In this paper, we propose a probabilistic model Skip-TransE, which comprehensively considers the two-way prediction ability and global loss intensity of the golden triplets. Then based on Skip-TransE, its non-parametric Bayesian extended model Adaptive-Skip-TransE is presented to automatically learn the number of semantics for each relation. Extensive experiments show that the proposed models can achieve some substantial improvements above the state-of-the-art baselines.


Knowledge embedding Multi-semantic relations Adaptive learning 


  1. 1.
    Bartunov, S., Kondrashkin, D., Osokin, A., Vetrov, D.: Breaking sticks and ambiguities with adaptive skip-gram. In: Artificial Intelligence and Statistics, pp. 130–138 (2016)Google Scholar
  2. 2.
    Blei, D.M., Jordan, M.I., et al.: Variational inference for dirichlet process mixtures. Bayesian Anal. 1(1), 121–143 (2006)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 1247–1250. ACM (2008)Google Scholar
  4. 4.
    Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, pp. 2787–2795 (2013)Google Scholar
  5. 5.
    Bordes, A., Weston, J., Collobert, R., Bengio, Y.: Learning structured embeddings of knowledge bases. In: Conference on Artificial Intelligence, No. EPFL-CONF-192344 (2011)Google Scholar
  6. 6.
    Fabian, M., Gjergji, K., Gerhard, W., et al.: Yago: a core of semantic knowledge unifying wordnet and wikipedia. In: 16th International World Wide Web Conference, WWW, pp. 697–706 (2007)Google Scholar
  7. 7.
    Ferguson, T.S.: A bayesian analysis of some nonparametric problems. Ann. Stat. 1, 209–230 (1973)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Hoffman, M.D., Blei, D.M., Wang, C., Paisley, J.W.: Stochastic variational inference. J. Mach. Learn. Res. 14(1), 1303–1347 (2013)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: ACL, vol. 1, pp. 687–696 (2015)Google Scholar
  10. 10.
    Ji, G., Liu, K., He, S., Zhao, J.: Knowledge graph completion with adaptive sparse transfer matrix. In: Thirtieth AAAI Conference on Artificial Intelligence (2016)Google Scholar
  11. 11.
    Jordan, M.I., Ghahramani, Z., Jaakkola, T.S., Saul, L.K.: An introduction to variational methods for graphical models. Mach. Learn. 37(2), 183–233 (1999)CrossRefGoogle Scholar
  12. 12.
    Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: AAAI, pp. 2181–2187 (2015)Google Scholar
  13. 13.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
  14. 14.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)Google Scholar
  15. 15.
    Miller, G.A.: Wordnet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)CrossRefGoogle Scholar
  16. 16.
    Sethuraman, J.: A constructive definition of dirichlet priors. Statistica Sinica 4, 639–650 (1994)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Socher, R., Chen, D., Manning, C.D., Ng, A.: Reasoning with neural tensor networks for knowledge base completion. In: Advances in Neural Information Processing Systems, pp. 926–934 (2013)Google Scholar
  18. 18.
    Chandrahas, S.R., Talukdar, P.P.: Revisiting simple neural networks for learning representations of knowledge graphs (2017)Google Scholar
  19. 19.
    Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: AAAI, pp. 1112–1119. Citeseer (2014)Google Scholar
  20. 20.
    Xiao, H., Huang, M., Zhu, X.: Transg : A generative model for knowledge graph embedding. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Long Papers, vol. 1, pp. 2316–2325. Association for Computational Linguistics (2016).,

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Shoukang Han
    • 1
    • 2
    • 3
  • Xiaobo Guo
    • 1
    • 2
    • 3
    Email author
  • Lei Wang
    • 2
    • 3
  • Zeyi Liu
    • 3
  • Nan Mu
    • 1
    • 2
    • 3
  1. 1.School of Cyber SecurityUniversity of Chinese Academy of SciencesBeijingChina
  2. 2.State Key Laboratory of Information SecurityChinese Academy of SciencesBeijingChina
  3. 3.Institute of Information EngineeringChinese Academy of SciencesBeijingChina

Personalised recommendations