Abstract
Knowledge graph embedding aims at embedding entities and relations in a knowledge graph into a continuous, dense, low-dimensional and real-valued vector space. Among various embedding models appeared in recent years, translation-based models such as TransE, TransH and TransR achieve state-of-the-art performance. However, in these models, negative triples used for training phase are generated by replacing each positive entity in positive triples with negative entities from the entity set with the same probability; as a result, a large number of invalid negative triples will be generated and used in the training process. In this paper, a method named adaptive negative sampling (ANS) is proposed to generate valid negative triples. In this method, it first divided all the entities into a number of groups which consist of similar entities by some clustering algorithms such as K-Means. Then, corresponding to each positive triple, the head entity was replaced by a negative entity from the cluster in which the head entity was located and the tail entity was replaced in a similar approach. As a result, it generated a set of high-quality negative triples which benefit for improving the effectiveness of embedding models. The ANS method was combined with the TransE model and the resulted model was named as TransE-ANS. Experimental results show that TransE-ANS achieves significant improvement in the link prediction task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bordes, A., Usunier, N., GarcÃa-Durán, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Proceedings of NIPS, Lake Tahoe, NV, USA, pp. 2787–2795 (2013)
Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Proceedings of AAAI, Quebéc city, QC, Canada, pp. 1112–1119 (2014)
Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of AAAI, Austin, TX, USA, pp. 2181–2187 (2015)
Ji, G., Liu, K., He, S., Zhao, J.: Knowledge graph completion with adaptive sparse transfer matrix. In: Proceedings of AAAI, Phienix, AZ, USA, pp. 985–991 (2016)
Feng, J., Huang, M., Wang, M., Zhou, M., Hao, Y., Zhu, X.: Knowledge graph embedding by flexible translation. In: Proceedings of KR, Cape Town, South Africa, pp. 557–560 (2016)
Chang, L., Zhu, M., Gu, T., Bin, C., Qian, J., Zhang, J.: Knowledge graph embedding by dynamic translation. IEEE Access 5, 20898–20907 (2017)
Bordes, A., Weston, J., Collobert, R., Bengio, Y.: Learning structured embeddings of knowledge bases. In: Proceedings of AAAI, San Francisco, CA, USA, pp. 301–306 (2011)
Jenatton, R., Roux, N.L., Bordes, A., Obozinski, G.: A latent factor model for highly multi-relational data. In: Proceedings of NIPS, Lake Tahoe, NV, USA, pp. 3167–3175 (2012)
Sutskever, I., Salakhutdinov, R., Tenenbaum, J.B.: Modelling relational data using Bayesian clustered tensor factorization. In: Proceedings of NIPS, Vancouver, BC, Canada, pp. 1821–1828 (2009)
Bordes, A., Glorot, X., Weston, J., Bengio, Y.: A semantic matching energy function for learning with multi-relational data. Mach. Learn. 94(2), 233–259 (2014)
Bordes, A., Glorot, X., Weston, J., Bengio, Y.: Joint learning of words and meaning representations for open-text semantic parsing. In: Proceedings of AISTATS, La Palma, Canary Islands, pp. 127–135 (2012)
Socher, R., Chen, D., Manning, C.D., Ng, A.Y.: Reasoning with neural tensor networks for knowledge base completion. In: Proceedings of NIPS, Lake Tahoe, NV, USA, pp. 926–934 (2013)
Hartigan, J.A., Wong, M.A.: Algorithm AS 136: a K-Means clustering algorithm. J. R. Stat. Soc. 28(1), 100–108 (1979)
Hamerly, G., Elkan, C.: Algorithm AS 136: a K-Means clustering algorithm. In: Proceedings of CIKM, McLean, VA, USA, pp. 600–607 (2002)
Celebi, M.E., Kingravi, H.A., Vela, P.A.: A comparative study of efficient initialization methods for the k-means clustering algorithm. Expert Syst. Appl. 40(1), 200–210 (2013)
Duchi, J.C., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. JMLR 12, 2121–2159 (2011)
Acknowledgements
This work was partially supported by the National Natural Science Foundation of China (Nos. U1501252, 61572146 and U1711263), the Natural Science Foundation of Guangxi Province (No. 2016GXNSFDA380006), the Guangxi Innovation-Driven Development Project (No. AA17202024) and the Guangxi Universities Young and Middle-aged Teacher Basic Ability Enhancement Project (No. 2018KY0203).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Qin, S., Rao, G., Bin, C., Chang, L., Gu, T., Xuan, W. (2019). Knowledge Graph Embedding Based on Adaptive Negative Sampling. In: Cheng, X., Jing, W., Song, X., Lu, Z. (eds) Data Science. ICPCSEE 2019. Communications in Computer and Information Science, vol 1058. Springer, Singapore. https://doi.org/10.1007/978-981-15-0118-0_42
Download citation
DOI: https://doi.org/10.1007/978-981-15-0118-0_42
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-0117-3
Online ISBN: 978-981-15-0118-0
eBook Packages: Computer ScienceComputer Science (R0)