Knowledge Fusion via Joint Tensor and Matrix Factorization
- 32 Downloads
We consider the task of knowledge fusion, an important aspect of cognitive intelligence, with the goal of combining part-of knowledge drawn from different sources. For this, entities and relations are cast into matrix-based representations. Unlike previous work on relation prediction, we consider the challenging setting of graphs with large amounts of completely separate connected components and no overlap between the training and test set entities. In order to address these challenges, we propose a novel cognitively inspired factorization method that jointly factorizes a subject–predicate–object tensor via RESCAL and a similarity matrix via matrix factorization. Our experimental results show that our method significantly outperforms several strong baseline models, including RESCAL and several TransE-style models. The proposed joint factorization of a subject–predicate–object tensor while applying matrix factorization to a similarity matrix obtains substantially higher average accuracy rates than previous approaches. This shows that it can successfully address the challenge of knowledge fusion of disconnected data.
KeywordsKnowledge fusion Connected components Entity overlap Tensor factorization Word similarities
The authors would like to acknowledge the financial support provided by the National Natural Science Foundation of China (no. 61503217)
Compliance with Ethical Standards
Conflict of Interest
The authors declare that they have no conflict of interest.
This article does not describe any studies with human participants or animals performed by any of the authors.
Informed consent was not required as no humans or animals were involved.
- 1.Acar E, Rasmussen M, Savorani F, Næs T., Bro R. 2013. Understanding data fusion within the framework of coupled matrix and tensor factorizations 129, 53-63.Google Scholar
- 2.Bordes A, Glorot X, Weston J, Bengio Y. 2013. A semantic matching energy function for learning with multi-relational data. Machine Learning. To appear.Google Scholar
- 3.Bordes A, Usunier N, García-Durán A, Weston J, Yakhnenko O. Translating embeddings for modeling multi-relational data. NIPS; 2013. p. 2787–2795.Google Scholar
- 5.Chew PA, Bader BW, Kolda TG, Abdelali A. Cross-language information retrieval using PARAFAC2. SIGKDD; 2007. p. 143–152.Google Scholar
- 7.Dong X, Gabrilovich E, Heitz G, Horn W, Lao N, Murphy K, Strohmann T, Sun S, Zhang W. Knowledge vault: a web-scale approach to probabilistic knowledge fusion. SIGKDD; 2014. p. 601–610.Google Scholar
- 8.Dong XL, Gabrilovich E, Heitz G, Horn W, Murphy K, Sun S, Zhang W. From data fusion to knowledge fusion. PVLDB 2014;7(10):881–892.Google Scholar
- 9.Dong XL, Srivastava D. Knowledge curation and knowledge fusion: challenges, models and applications. Proceedings of the 2015 ACM SIGMOD international conference on management of data, Melbourne, Victoria, Australia, May 31 - June 4, 2015; 2015. p. 2063–2066.Google Scholar
- 11.Fellbaum C, (ed). 1998. WordNet: an electronic lexical database. http://www.amazon.ca/exec/obidos/redirect?tag=citeulike04-20%&path=ASIN/026206197X. Cambridge: The MIT Press.
- 13.He S, Liu K, Ji G, Zhao J. Learning to represent knowledge graphs with gaussian embedding. CIKM. ACM; 2015. p. 623–632.Google Scholar
- 14.Ji G, Liu K, He S, Zhao J. Knowledge graph completion with adaptive sparse transfer matrix. AAAI; 2016. p. 985–991.Google Scholar
- 15.Jiang JJ, Conrath DW. 1997. Semantic similarity based on corpus statistics and lexical taxonomy. arXiv:cmp-lg/9709008.
- 18.Krishna R, Zhu Y, Groth O, Johnson J, Hata K, Kravitz J, Chen S, Kalantidis Y, Li LJ, Shamma DA, Bernstein M, Fei-Fei L. 2016. Visual genome: connecting language and vision using crowdsourced dense image annotations. arXiv:1602.07332 .
- 19.Leacock C, Chodorow M. Combining local context and wordnet similarity for word sense identification. WordNet: an electronic lexical database 1998;49(2):265–283.Google Scholar
- 20.Lin D. An information-theoretic definition of similarity. ICML; 1998.Google Scholar
- 21.Lin Y, Liu Z, Luan H, Sun M, Rao S, Liu S. 2015. Modeling relation paths for representation learning of knowledge bases. Computer Science.Google Scholar
- 22.Lin Y, Liu Z, Sun M, Liu Y, Zhu X. Learning entity and relation embeddings for knowledge graph completion. AAAI; 2015. p. 2181–2187.Google Scholar
- 23.Lin Y, Liu Z, Zhu X, Zhu X, Zhu X. Learning entity and relation embeddings for knowledge graph completion. AAAI; 2015. p. 2181–2187.Google Scholar
- 25.Mikolov T, Chen K, Corrado G, Dean J. 2013. Efficient estimation of word representations in vector space. arXiv:1301.3781.
- 26.Nengfu X, Wensheng W, Xiaorong Y, Lihua J. Rule-based agricultural knowledge fusion in web information integration. NJAS - Wageningen Journal of Life Sciences 2012;10(1):635–638(4).Google Scholar
- 27.Nickel M, Tresp V, Kriegel H. A three-way model for collective learning on multi-relational data. ICML; 2011. p. 809–816.Google Scholar
- 28.Nickel M, Tresp V, Kriegel H. Factorizing YAGO: scalable machine learning for linked data. WWW; 2012. p. 271–280.Google Scholar
- 29.Pennington J, Socher R, Manning CD. Glove: Global vectors for word representation. EMNLP; 2014. p. 1532–1543.Google Scholar
- 30.Pilehvar MT, Jurgens D, Navigli R. Align, disambiguate and walk: a unified approach for measuring semantic similarity. ACL; 2013. p. 1341–1351.Google Scholar
- 33.Resnik P. 1995. Using information content to evaluate semantic similarity in a taxonomy. arXiv:cmp-lg/9511007.
- 36.Tandon N, Hariman C, Urbani J, Rohrbach A, Rohrbach M, Weikum G. Commonsense in parts: mining part-whole relations from the web and image tags. Proceedings of the thirtieth AAAI conference on artificial intelligence, February 12–17, 2016, Phoenix, Arizona, USA; 2016. p. 243–250.Google Scholar
- 37.Thoma S, Rettinger A, Both F. 2017. Knowledge fusion via embeddings from text, knowledge graphs, and images. arXiv:1704.06084.
- 38.Wang Y, Widrow B, Zadeh LA, Howard N, Wood S, Bhavsar VC, Budin G, Chan CW, Fiorini RA, Gavrilova ML, Shell DF. Cognitive intelligence: deep learning, thinking, and reasoning by brain-inspired systems. IJCINI 2016;10(4):1–20.Google Scholar
- 39.Wang Z, Zhang J, Feng J, Chen Z. Knowledge graph embedding by translating on hyperplanes. AAAI; 2014. p. 1112–1119.Google Scholar
- 40.Wu Z, Palmer MS. Verb semantics and lexical selection. ACL; 1994. p. 133–138.Google Scholar
- 41.Yu Xl, Qiao L. 2017. Knowledge fusion methods: a survey. DEStech Transactions on Computer Science and Engineering (smce).Google Scholar