Advertisement

PLinkSHRINK: a parallel overlapping community detection algorithm with Link-Graph for large networks

  • Yunlei Zhang
  • Dingyi YinEmail author
  • Bin WuEmail author
  • Feiyu Long
  • Yinchang Cui
  • Xun Bian
Original Article
  • 15 Downloads

Abstract

Overlapping communities are pervasive in real-world networks. Therefore, overlapping community detection is an important task in network analysis. Recently, many overlapping community detection methods are proposed to achieve different goals. However, how to detect communities effectively and efficiently is still an open problem. In this paper, we use our previously proposed method LinkSHRINK to detect overlapping community detection, which is based on density structure and modularity optimization. It successfully solves the excessive overlapping problem. Moreover, it can detect both overlapping communities of multi-granularity and outliers. To deal with very large networks, we choose to sample on the large graph and then parallelize LinkSHRINK by distributed computing frameworks. Experiments are conducted on benchmark networks and some real-world networks with known ground-truth communities. The experimental results demonstrate that LinkSHRINK outperforms most of the baseline methods and its parallel versions PLinkSHRINK and MLinkSHRINK can process large networks efficiently.

Keywords

Overlapping community Community detection Link graph Multi-granularity SHRINK Parallelization 

Notes

Acknowledgements

This work is supported by the National Key R&D Program of China under Grant 2018YFC0831500. We are grateful to the anonymous reviewers for their careful reading and valuable suggestions.

References

  1. Ahn YY, Bagrow JP, Lehmann S (2010) Link communities reveal multiscale complexity in networks. Nature 466(7307):761CrossRefGoogle Scholar
  2. Bai X, Yang P, Shi X (2017) An overlapping community detection algorithm based on density peaks. Neurocomputing 226:7–15CrossRefGoogle Scholar
  3. Ball B, Karrer B, Newman ME (2011) Efficient and principled method for detecting communities in networks. Phys Rev E 84(3):036103CrossRefGoogle Scholar
  4. Bezdek JC, Ehrlich R, Full W (1984) Fcm: the fuzzy c-means clustering algorithm. Comput Geosci 10(2–3):191–203CrossRefGoogle Scholar
  5. Cheong CY, Huynh HP, Lo D, Goh RSM (2013) Hierarchical parallel algorithm for modularity-based community detection using gpus. In: European conference on parallel processing, Springer, pp 775–787Google Scholar
  6. Ester M, Kriegel H, Sander J, Xu X (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In: International conference on knowledge discovery and data mining, AAAI Press, pp 226–231Google Scholar
  7. Evans T, Lambiotte R (2009) Line graphs, link partitions, and overlapping communities. Phys Rev E 80(1):016105CrossRefGoogle Scholar
  8. Feng Z, Xu X, Yuruk N, Schweiger TA (2007) A novel similarity-based modularity function for graph partitioning. In: International conference on data warehousing and knowledge discovery, Springer, pp 385–396Google Scholar
  9. Fortunato S, Barthelemy M (2007) Resolution limit in community detection. Proc Natl Acad Sci 104(1):36–41CrossRefGoogle Scholar
  10. Gopalan PK, Blei DM (2013) Efficient discovery of overlapping communities in massive networks. Proc Natl Acad Sci 110(36):14534–14539MathSciNetCrossRefGoogle Scholar
  11. Gregory S (2010) Finding overlapping communities in networks by label propagation. New J Phys 12(10):103018CrossRefGoogle Scholar
  12. Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. In: Advances in neural information processing systems, pp 1024–1034Google Scholar
  13. Huang J, Sun H, Han J, Deng H, Sun Y, Liu Y (2010) Shrink: a structural clustering algorithm for detecting hierarchical communities in networks. In: ACM international conference on information and knowledge management, ACM, pp 219–228Google Scholar
  14. Huang J, Sun H, Han J, Feng B (2011) Density-based shrinkage for revealing hierarchical and overlapping community structure in networks. Phys A Stat Mech Appl 390(11):2160–2171CrossRefGoogle Scholar
  15. Jin S, Yu PS, Li S, Yang S (2015) A parallel community structure mining method in big social networks. Math Probl Eng 2015:934301Google Scholar
  16. Kuzmin K, Shah SY, Szymanski BK (2013) Parallel overlapping community detection with slpa. In: 2013 international conference on social computing, IEEE, pp 204–212Google Scholar
  17. Lancichinetti A, Fortunato S (2009) Benchmarks for testing community detection algorithms on directed and weighted graphs with overlapping communities. Phys Rev E 80(1):016118CrossRefGoogle Scholar
  18. Lancichinetti A, Fortunato S (2011) Limits of modularity maximization in community detection. Phys Rev E 84(6):066122CrossRefGoogle Scholar
  19. Lancichinetti A, Fortunato S, Kertész J (2009) Detecting the overlapping and hierarchical community structure in complex networks. New J Phys 11(3):033015CrossRefGoogle Scholar
  20. Li R, Guo W, Guo K, Qiu Q (2015) Parallel multi-label propagation for overlapping community detection in large-scale networks. In: International workshop on multi-disciplinary trends in artificial intelligence, Springer, pp 351–362Google Scholar
  21. Li Y, He K, Kloster K, Bindel D, Hopcroft J (2018) Local spectral clustering for overlapping community detecion. ACM Trans Knowl Discov Data 12(2):17CrossRefGoogle Scholar
  22. Lim S, Ryu S, Kwon S, Jung K, Lee JG (2014) Linkscan*: overlapping community detection using the link-space transformation. In: IEEE international conference on data engineering, IEEE, pp 292–303Google Scholar
  23. Moon S, Lee JG, Kang M, Choy M, Jw Lee (2016) Parallel community detection on large graphs with mapreduce and graphchi. Data Knowl Eng 104:17–31CrossRefGoogle Scholar
  24. Newman ME, Girvan M (2004) Finding and evaluating community structure in networks. Phys Rev E 69(2):026113CrossRefGoogle Scholar
  25. Palla G, Derényi I, Farkas I, Vicsek T (2005) Uncovering the overlapping community structure of complex networks in nature and society. Nature 435(7043):814CrossRefGoogle Scholar
  26. Qiao S, Guo J, Han N, Zhang X, Yuan C, Tang C (2017) Parallel algorithm for discovering communities in large-scale complex networks. Chin J ComputGoogle Scholar
  27. Sarswat A, Jami V, Guddeti RMR (2017) A novel two-step approach for overlapping community detection in social networks. Soc Netw Anal Min 7(1):47CrossRefGoogle Scholar
  28. Shen H, Cheng X, Guo J (2009) Quantifying and identifying the overlapping community structure in networks. J Stat Mech Theory Exp 2009(07):P07042CrossRefGoogle Scholar
  29. Sun B, Shen H, Cheng X (2014) Detecting overlapping communities in massive networks. Europhys Lett 108(6):68001CrossRefGoogle Scholar
  30. Thang ND (2017) Community detection in large-scale networks. Master’s thesis, ThangLong UniversityGoogle Scholar
  31. Wang S, Dong Y, Li Z, Chen H, Qian J (2015) The identification of overlapping communities in large-scale complex networks. Acta Electron Sin 43(8):1575–1581Google Scholar
  32. Wickramaarachchi C, Frincu M, Small P, Prasanna VK (2014) Fast parallel algorithm for unfolding of communities in large graphs. In: 2014 IEEE high performance extreme computing conference, IEEE, pp 1–6Google Scholar
  33. Xie J, Szymanski BK (2012) Towards linear time overlapping community detection in social networks. In: Pacific-Asia conference on knowledge discovery and data mining, Springer, pp 25–36Google Scholar
  34. Xu X, Yuruk N, Feng Z, Schweiger TA (2007) Scan: a structural clustering algorithm for networks. In: ACM SIGKDD international conference on Knowledge discovery and data mining, ACM, pp 824–833Google Scholar
  35. Yang J, Leskovec J (2013) Overlapping community detection at scale: a nonnegative matrix factorization approach. In: Proceedings of the sixth ACM international conference on Web search and data mining, ACM, pp 587–596Google Scholar
  36. Yin D, Wu B, Zhang Y (2016) Linkshrink: overlapping community detection with link-graph. In: IEEE international conference on data science in cyberspace, IEEE, pp 44–53Google Scholar
  37. Zeng J, Yu H (2015) Parallel modularity-based community detection on large-scale graphs. In: 2015 IEEE international conference on cluster computing, IEEE, pp 1–10Google Scholar
  38. Zhang H, Niu X, King I, Lyu MR (2018) Overlapping community detection with preference and locality information: a non-negative matrix factorization approach. Soc Netw Anal Min 8(1):43CrossRefGoogle Scholar
  39. Zhang Q, Qiu Q, Guo W, Guo K, Xiong N (2016) A social community detection algorithm based on parallel grey label propagation. Comput Netw 107:133–143CrossRefGoogle Scholar
  40. Zhu M, Meng F, Zhou Y (2013) Density-based link clustering algorithm for overlapping community detection. J Comput Res Dev 12(006)Google Scholar

Copyright information

© Springer-Verlag GmbH Austria, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Beijing Key Laboratory of Intelligence Telecommunications Software and Multimedia, School of Computer ScienceBeijing University of Posts and TelecommunicationsBeijingChina
  2. 2.User Behavior Big Data Research CenterChina Telecom Beijing Research InstituteBeijingChina
  3. 3.Router and VRP Technology Development DepartmentHuawei Technologies Co., Ltd.BeijingChina
  4. 4.Shenzhen Branch Information Technology DepartmentChina Merchants BankShenzhenChina

Personalised recommendations