Advertisement

JECI: A Joint Knowledge Graph Embedding Model for Concepts and Instances

  • Jing Zhou
  • Peng WangEmail author
  • Zhe Pan
  • Zhongkai Xu
Conference paper
  • 52 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12032)

Abstract

Concepts and instances are important parts in knowledge graphs, but most knowledge graph embedding models treat them as entities equally, that leads to inaccurate embeddings of concepts and instances. Aiming to address this problem, we propose a novel knowledge graph embedding model called JECI to jointly embed concepts and instances. First, JECI organizes concepts in the knowledge graph as a hierarchical tree, which maps concepts to a tree. Meanwhile, for an instance, JECI generates a context vector to represent the neighbor context in the knowledge graph. Then, based on the context vector and supervision information generated from the hierarchical tree, an embedding learner is designed to precisely locate an instance in embedding space from the coarse-grained to the fine-grained. A prediction function, as the form of convolution, is designed to predict concepts of different granularities that an instance belongs to. In this way, concepts and instances are jointly embedded, and hierarchical structure is preserved in embedds. Especially, JECI can handle the complex relation by incorporating neighbor information of instances. JECI is evaluated by link prediction and triple classification on real world data. Experimental results demonstrate that it outperforms state-of-the-art models in most cases.

Keywords

Knowledge graph Embedding Hierarchical tree Context vector 

References

  1. 1.
    Wang, Q., Mao, Z., Wang, B., Guo, L.: Knowledge graph embedding: a survey of approaches and applications. IEEE Trans. Knowl. Data Eng. 29(12), 2724–2743 (2017)CrossRefGoogle Scholar
  2. 2.
    Turian, J., Ratinov, L., Bengio, Y.: Word representations: a simple and general method for semi-supervised learning. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pp. 384–394 (2010)Google Scholar
  3. 3.
    Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, pp. 2787–2795 (2013)Google Scholar
  4. 4.
    Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence, pp. 1112–1119 (2014)Google Scholar
  5. 5.
    Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the 29th AAAI Conference on Artificial Intelligence, pp. 2181–2187 (2015)Google Scholar
  6. 6.
    Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 687–696 (2015)Google Scholar
  7. 7.
    Yang, B., Yih, W., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. In: 3rd International Conference on Learning Representations (2015)Google Scholar
  8. 8.
    Nickel, M., Rosasco, L., Poggio, T.A., et al.: Holographic embeddings of knowledge graphs. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence, pp. 1955–1961 (2016)Google Scholar
  9. 9.
    Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: International Conference on Machine Learning, pp. 2071–2080 (2016)Google Scholar
  10. 10.
    Socher, R., Chen, D., Manning, C.D., Ng, A.: Reasoning with neural tensor networks for knowledge base completion. In: Advances in Neural Information Processing Systems, pp. 926–934 (2013)Google Scholar
  11. 11.
    Bordes, A., Glorot, X., Weston, J., Bengio, Y.: A semantic matching energy function for learning with multi-relational data. Mach. Learn. 94(2), 233–259 (2014)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graph embeddings. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence, pp. 1811–1818 (2018)Google Scholar
  13. 13.
    Guo, S., Wang, Q., Wang, B., Wang, L., Guo, L.: Semantically smooth knowledge graph embedding. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics, pp. 84–94 (2015)Google Scholar
  14. 14.
    Xie, R., Liu, Z., Sun, M.: Representation learning of knowledge graphs with hierarchical types. In: Proceedings of the 25th International Joint Conference on Artificial Intelligence, pp. 2965–2971 (2016)Google Scholar
  15. 15.
    Lin, Y., Liu, Z., Luan, H., Sun, M., Rao, S., Liu, S.: Modeling relation paths for representation learning of knowledge bases. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 705–714 (2015)Google Scholar
  16. 16.
    Zhong, H., Zhang, J., Wang, Z., Wan, H., Chen, Z.: Aligning knowledge and text embeddings by entity descriptions. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 267–272 (2015)Google Scholar
  17. 17.
    Guo, S., Wang, Q., Wang, L., Wang, B., Guo, L.: Jointly embedding knowledge graphs and logical rules. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 192–202 (2016)Google Scholar
  18. 18.
    Ding, B., Wang, Q., Wang, B., Guo, L.: Improving knowledge graph embedding using simple constraints. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 110–121 (2018)Google Scholar
  19. 19.
    Asprino, L., Basile, V., Ciancarini, P., Presutti, V.: Empirical analysis of foundational distinctions in linked open data. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, pp. 3962–3969 (2018)Google Scholar
  20. 20.
    Miller, G.: WordNet: an on-line lexical database. special issue of the international. J. Lexicogr. 3(4) (1990) Google Scholar
  21. 21.
    Lv, X., Hou, L., Li, J., Liu, Z.: Differentiating concepts and instances for knowledge graph embedding. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1971–1979 (2018)Google Scholar
  22. 22.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: 1st International Conference on Learning Representations (2013)Google Scholar
  23. 23.
    Bordes, A., Weston, J., Collobert, R., Bengio, Y.: Learning structured embeddings of knowledge bases. In: Proceedings of the 25th AAAI Conference on Artificial Intelligence, pp. 301–306 (2011)Google Scholar
  24. 24.
    Ling, X., Weld, D.S.: Fine-grained entity recognition. In: Proceedings of the 26th AAAI Conference on Artificial Intelligence, pp. 94–100 (2012)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.School of Computer Science and EngineeringSoutheast UniversityNanjingChina
  2. 2.School of Cyber Science and EngineeringSoutheast UniversityNanjingChina

Personalised recommendations