Advertisement

Journal of Central South University

, Volume 26, Issue 10, pp 2746–2758 | Cite as

Tri-party deep network representation learning using inductive matrix completion

  • Zhong-lin Ye (冶忠林)
  • Hai-xing Zhao (赵海兴)Email author
  • Ke Zhang (张科)
  • Yu Zhu (朱宇)
  • Yu-zhi Xiao (肖玉芝)
Article
  • 11 Downloads

Abstract

Most existing network representation learning algorithms focus on network structures for learning. However, network structure is only one kind of view and feature for various networks, and it cannot fully reflect all characteristics of networks. In fact, network vertices usually contain rich text information, which can be well utilized to learn text-enhanced network representations. Meanwhile, Matrix-Forest Index (MFI) has shown its high effectiveness and stability in link prediction tasks compared with other algorithms of link prediction. Both MFI and Inductive Matrix Completion (IMC) are not well applied with algorithmic frameworks of typical representation learning methods. Therefore, we proposed a novel semi-supervised algorithm, tri-party deep network representation learning using inductive matrix completion (TDNR). Based on inductive matrix completion algorithm, TDNR incorporates text features, the link certainty degrees of existing edges and the future link probabilities of non-existing edges into network representations. The experimental results demonstrated that TFNR outperforms other baselines on three real-world datasets. The visualizations of TDNR show that proposed algorithm is more discriminative than other unsupervised approaches.

Key words

network representation network embedding representation learning matrix-forestindex inductive matrix completion 

基于诱导矩阵补全的三元深度网络表示学习算法

摘要

现有的网络表示学习算法多侧重于学习的网络结构特征。然而,网络结构只是网络的一种视图 和特征,其不能充分反映出网络的所有特性。事实上,网络节点通常包含丰富的文本信息,这些信息 可以被很好地用来学习文本增强的网络表示向量。同时,矩阵森林指数(MFI)与其他链接预测指标相 比,其具有较高的预测效率和稳定性。但是目前,矩阵森林指数和诱导矩阵补全算法并没有很好地应 用于典型的表示学习框架中。因此,本文提出一种基于诱导矩阵补全的三元深度网络表示学习算法 (TDNR),该算法是一种半监督学习算法。基于诱导矩阵补全算法,TDNR 将网络结构、网络节点的 文本特征、网络中已存在边的连接确定度和非存在边的未来连接概率等特征融入到网络的表示学习过 程中,使得学习得到的网络表示向量中含有网络的更多属性因子。实验结果表明,在三个真实的网络 数据集上,TDNR 优于其他对比算法。可视化实验表明了TDNR 算法能够产生具有较强聚类能力的网 络表示向量。

关键词

网络表示 网络嵌入 表示学习 矩阵森林指标 诱导矩阵补全 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    TSOUMAKAS G, KATAKIS I. Multi-label classification: an overview [J]. International Journal of Data Warehousing and Mining, 2007, 3(3): 1–13. DOI: 10.4018/ jdwm.2007070101.CrossRefGoogle Scholar
  2. [2]
    LIBEN-NOWELL D, KLEINBERG J. The link-prediction problem for social networks [J]. Journal of the American Society for Information Science and Technology, 2007, 58(7): 1019–1031. DOI: 10.4018/jdwm.2007070101.CrossRefGoogle Scholar
  3. [3]
    TU C, LIU Z, SUN M. Inferring correspondences from multiple sources for microblog user tags [C]// The Chinese National Conference on Social Media Processing. Heidelberg: Springer, 2014: 1–12.Google Scholar
  4. [4]
    YU H F, JAIN P, KAR P, et al. Large-scale multi-label learning with missing labels [C]// Proceedings of the 31st International Conference on Machine Learning. Heidelberg: Springer, 2014: 593–601.Google Scholar
  5. [5]
    PEROZZI B, AL-RFOU R, SKIENA S. DeepWalk: online learning of social representations [C]// ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2014: 701–710.Google Scholar
  6. [6]
    MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality [C]// Proceedings of the 27th Annual Conference on Neural Information Processing Systems. Massachusetts: MIT, 2013: 3111–3119.Google Scholar
  7. [7]
    TANG Jian, QU Meng, WANG Ming-zhe, et al. LINE: large-scale information network embedding [C]// Proceedings of the 24th International World Wide Web Conferences Steering Committee. Heidelberg: Springer, 2015: 1067–1077.Google Scholar
  8. [8]
    CAO Shao-sheng, LU Wei, XU Qiong-kai. GraRep: learning graph representations with global structural information [C]// Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. New York: ACM, 2015: 891–900.Google Scholar
  9. [9]
    WANG Dai-xin, CUI Peng, ZHU Wen-wu. Structural deep network embedding [C]// ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2016: 1225–1234.Google Scholar
  10. [10]
    GROVER A, LESKOVEC J. Node2vec: scalable feature learning for networks [C]// ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2016: 855–864.Google Scholar
  11. [11]
    TANG Jian, QU Meng, MEI Qiao-zhu. PTE: predictive text embedding through large-scale heterogeneous text networks [C]// ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2015: 1165–1174.Google Scholar
  12. [12]
    SUN X, GUO J, DING X, et al. A general framework for content-enhanced network representation learning [EB/OL]. [2018-01-05]. http://pdfs.semanticscholar.org/fad9/08515d149bce1fe4bad84728657b8b83009a.pdf.Google Scholar
  13. [13]
    TU C C, WANG H, ZENG X K, LIU Z Y, SUN M S. Community-enhanced network representation learning for network analysis [EB/OL]. [2017-12-03]. http://pdfs.semanticscholar.org/6199/79db74a6d5896e4f21798614e80f9ce6d107.pdf.Google Scholar
  14. [14]
    PAN Shi-rui, WU Jia, ZHU Xing-quan, et al. Tri-party deep network representation [C]// International Joint Conference on Artificial Intelligence. San Francisco: Morgan Kaufmann, 2016: 1895–1901.Google Scholar
  15. [15]
    GARCIADURAN A, NIEPERT M. Learning graph representations with embedding propagation [EB/OL]. [2017-12-05]. http://in.arxiv.org/abs/1710.03059v1.Google Scholar
  16. [16]
    WANG X, CUI P, WANG J, et al. Community preserving network embedding[EB/OL]. [2017-12-05]. https://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14589.Google Scholar
  17. [17]
    ZHANG Dao-kun, YIN Jie, ZHU Xing-quan, et al. User profile preserving social network embedding [C]// 26th International Joint Conference on Artificial Intelligence. San Francisco: Morgan Kaufmann, 2017: 3378–3384.Google Scholar
  18. [18]
    LI C, WANG S, YANG D, et al. PPNE: property preserving network embedding [J]. Database Systems for Advanced Applications, 2017:163–179.CrossRefGoogle Scholar
  19. [19]
    HUANG X, LI J, HU X. Accelerated attributed network embedding [EB/OL]. [2018-01-13]. http://www.public.asu.edu/~jundongl/paper/SDM17_AANE.pdf.Google Scholar
  20. [20]
    HUANGZ P, MAMOULIS N. Heterogeneous information network embedding for meta path-based proximity [EB/OL]. [2018-01-13]. http://pdfs.semanticscholar.org/52a1/50d6a098ef142bece099dadaa613fddbae50.pdf.Google Scholar
  21. [21]
    TU K, CUI P, WANG X, et al. Structural deep embedding for hyper-networks [EB/OL]. [2018-01-09]. http://media.cs.tsinghua.edu.cn/~multimedia/cuipeng/papers/DHNE.pdf.Google Scholar
  22. [22]
    LEVY O, GOLDBERY Y. Neural word embedding as implicit matrix factorization [C]// Conference on Neural Information Processing Systems. Massachusetts: MIT, 2014: 2177–2185.Google Scholar
  23. [23]
    YU H F, JAIN P, KAR P, et al. Large-scale multi-label learning with missing labels [EB/OL]. [2018-01-10]. https://www.cse.iitk.ac.in/users/purushot/papers/leml.pdf.Google Scholar
  24. [24]
    YANG C, LIU Z. Comprehend deepWalk as matrix factorization [EB/OL]. [2018-01-13]. https://www.researchgate.net/publication/270454626_Comprehend_DeepWalk_as_Matrix_Factorization.Google Scholar
  25. [25]
    YANG Cheng, LIU Zhi-yuan, ZHAO De-li, et al. Network representation learning with rich text information [C]// Proceedings of of the 24th International Joint Conference on Artificial Intelligence. San Francisco: Morgan Kaufmann, 2015: 2111–2117.Google Scholar
  26. [26]
    TU Cun-chao, ZHANG Wei-cheng, LIU Zhi-yuan, et al. Max-margin deepwalk: Discriminative learning of network representation [C]// International Joint Conference on Artificial Intelligence. San Francisco: Morgan Kaufmann, 2016: 3889–3895.Google Scholar
  27. [27]
    HEARST M A, DUMAIS S T, OSMAN E, et al. Support vector machines [J]. IEEE Intelligent Systems & Their Applications, 2002, 13(4): 18–28. DOI: 10.1109/5254. 708428.CrossRefGoogle Scholar
  28. [28]
    ZHU J, AHMED A, XING E P. MedLDA: maximum margin supervised topic models [J]. Journal of Machine Learning Research, 2012, 13: 2237–2278.MathSciNetzbMATHGoogle Scholar
  29. [29]
    NATARAJAN N, DHILLON I S. Inductive matrix completion for predicting gene-disease associations [J]. Bioinformatics, 2014, 30(12): 60–68. DOI: 10.1093/bioinformatics/btu269.CrossRefGoogle Scholar
  30. [30]
    AOUAY S, JAMOUSSI S, GARGOURI F. Feature based link prediction [C]// IEEE/ACS International Conference on Computer Systems and Applications. New York: USA, 2014: 523–527.Google Scholar
  31. [31]
    LI D, XU Z, LI S, SUN X. Link prediction in social networks based on hypergraph [C]// International Conference on World Wide Web. New York, USA: ACM Press, 2013: 41–42.Google Scholar
  32. [32]
    DONG E, LI J, XIE Z. Link prediction via convex nonnegative matrix factorization on multiscale blocks [J]. Journal of Applied Mathematics, 2014, 15(3): 1–9. DOI: 10.1155/2014/786156.MathSciNetGoogle Scholar
  33. [33]
    FARASAT A, NIKOLAEV A, SRIHARI S N, et al. Probabilistic graphical models in modern social network analysis [J]. Social Network Analysis and Mining, 2015, 5(1): 1–29. DOI: 10.1007/s13278-015-0289-6.CrossRefGoogle Scholar
  34. [34]
    MEI Y, TAN G. An improved brain emotional learning algorithm for accurate and efficient data analysis [J]. Journal of Central South University, 2018, 25(5): 1084–1098. DOI: 10.1007/s11771-018-3808-6.CrossRefGoogle Scholar
  35. [35]
    JHA B N, LI H. Structural reliability analysis using a hybrid HDMR-ANN method [J]. Journal of Central South University, 2017, 24(11): 2532–2541. DOI: 10.1007/s11771-017-3666-7.CrossRefGoogle Scholar
  36. [36]
    FOUSS F, YEN L, PIROTTE A, et al. An experimental investigation of graph kernels on a collaborative recommendation task [C]// International Conference on Data Mining. Piscataway, NJ: IEEE, 2006: 863–868.Google Scholar
  37. [37]
    MORIN F, BENGION Y. Hierarchical probabilistic neural network language model [C]// 10th International Workshop on Artificial Intelligence and Statistics. Piscataway, NJ: IEEE, 2005: 246–252.Google Scholar
  38. [38]
    JAIN P, DHILLON I S. Provable inductive matrix completion [EB/OL]. [2017-12-05]. https://arxiv.org/pdf/1306.0626.pdf.Google Scholar
  39. [39]
    LIU P, ZHAO H, TENG J, YANG Y, LIU Y. Parallel naive Bayes algorithm for large-scale Chinese text classification based on spark [J]. Journal of Central South University, 2019, 26(1): 1–12. DOI: 10.1007/s11771-019-3978-x.CrossRefGoogle Scholar
  40. [40]
    FAN R E, CHANG K W, HSIEH C J, et al. LIBLINEAR: A library for large linear classification [J]. Journal of Machine Learning Research, 2008, 9(9): 1871–1874. DOI: 10.1145/1390681.1442794.zbMATHGoogle Scholar

Copyright information

© Central South University Press and Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.College of ComputerQinghai Normal UniversityXiningChina
  2. 2.Tibetan Information Processing and Machine Translation Key Laboratory of Qinghai ProvinceXiningChina
  3. 3.Key Laboratory of Tibetan Information Processing of Ministry of EducationXiningChina
  4. 4.College of Computer ScienceShaanxi Normal UniversityXi’anChina

Personalised recommendations