Dimensions Effect in Word Embeddings of Knowledge Graph

Abstract

Word embedding is one of the basic of knowledge graph. It is designed to represent the entities and relations with vectors or matrix to make knowledge graph model. Recently many related models and methods were proposed, such as translational methods, deep learning based methods, multiplicative approaches. The TransE models take a relation as transition from head entity to tail entity in principle. The further researches noticed that relations and entities might be able to have different representation to be casted into real world relations. Thus it could improve the embedding accuracy of embeddings in some scenarios. To improve model accuracy, the variant algorithms based on TransE adopt strategies like adjusting the loss function, freeing word embedding dimension freedom limitations or increasing other parameters size etc. After carefully investigate these algorithms, we motivated by researches on the effect of embedding dimension size factor. In this paper, we carefully analyzed the factor impact of dimensions on the accuracy and algorithm complexity of word embedding. By comparing some typical word embedding algorithms and methods, we found there are tradeoff problem to deal with between algorithm’s simplicity and expressiveness. We carefully designed an experiment to test such kind of effect and give some description and possible measure to adopt.