The Journal of Supercomputing

, Volume 74, Issue 11, pp 6135–6155 | Cite as

An adaptive breadth-first search algorithm on integrated architectures

  • Feng Zhang
  • Heng Lin
  • Jidong ZhaiEmail author
  • Jie Cheng
  • Dingyi Xiang
  • Jizhong Li
  • Yunpeng Chai
  • Xiaoyong Du


In the big data era, graph applications are becoming increasingly important for data analysis. Breadth-first search (BFS) is one of the most representative algorithms; therefore, accelerating BFS using graphics processing units (GPUs) is a hot research topic. However, due to their random data access pattern, it is difficult to take full advantage of the power of GPUs. Recently, hardware designers have integrated CPUs and GPUs on the same chip, allowing both devices to share physical memory, which provides the convenience of switching between CPUs and GPUs with little cost. BFS processing can be divided into several levels, and various traversal orders can be used at each level. Using different traversal orders on different devices (CPUs or GPUs) results in diverse performances. Thus, the challenge in using BFS on integrated architectures is how to select the traversal order and the device for each level. Previous works have failed to address this problem effectively. In this study, we propose an adaptive performance model that automatically finds a suitable traversal order and device for each level. We evaluated our method on Graph500, where it not only shows the best energy efficiency but also achieves a giga-traversed edges per second (GTEPS) performance of approximately 2.1 GTEPS, which is a \(2.3\,\times \) speed improvement over the state-of-the-art BFS on integrated architectures.


Breadth-first search Integrated architectures Performance Energy efficiency Modeling 



The authors sincerely thank the anonymous reviewers for their valuable comments and suggestions. This work is partially supported by the National Key R&D Program of China (Grant No. 2016YFB0200100), National Natural Science Foundation of China (Grant Nos. 61732014, 61722208, 61472201, and 61472427). This work is also supported by Huawei Technologies Co. Ltd, Beijing Natural Science Foundation (No. 4172031), China Postdoctoral Science Foundation (2017M620992), and the Fundamental Research Funds for the Central Universities and the Research Funds of Renmin University of China (Nos. 16XNLQ02, 18XNLG07). Jidong Zhai is the corresponding author of this paper.


  1. 1.
    Agarwal V, Petrini F, Pasetto D, Bader DA (2010) Scalable graph exploration on multicore processors. In: Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE Computer Society, pp 1–11Google Scholar
  2. 2.
    Ashari A, Sedaghati N, Eisenlohr J, Parthasarath S, Sadayappan P (2014) Fast sparse matrix–vector multiplication on GPUs for graph applications. In: International Conference for High Performance Computing, Networking, Storage and Analysis, SC14. IEEE, pp 781–792Google Scholar
  3. 3.
    AMD (2018) AMD Ryzen 5 2400G with Radeon RX Vega 11 Graphics.
  4. 4.
    Beamer S, Asanović K, Patterson D (2013) Direction-optimizing breadth-first search. Sci Program 21(3–4):137–148Google Scholar
  5. 5.
    Bouvier D, Sander B (2014) Applying AMDs Kaveri APU for heterogeneous computing. In: Hot Chips: A Symposium on High Performance Chips (HC26)Google Scholar
  6. 6.
    Brandes U (2001) A faster algorithm for betweenness centrality. J Math Sociol 25(2):163–177CrossRefGoogle Scholar
  7. 7.
    Branover A, Foley D, Steinman M (2012) AMD fusion APU: Llano. IEEE Micro 32(2):28–37CrossRefGoogle Scholar
  8. 8.
    Broder A, Kumar R, Maghoul F, Raghavan P, Rajagopalan S, Stata R, Tomkins A, Wiener J (2000) Graph structure in the web. Comput Netw 33(1):309–320CrossRefGoogle Scholar
  9. 9.
    Chakrabarti D, Zhan Y, Faloutsos C (2004) R-MAT: a recursive model for graph mining. In: SDM, vol 4. SIAM, pp 442–446Google Scholar
  10. 10.
    Chhugani J, Satish N, Kim C, Sewall J, Dubey P (2012) Fast and efficient graph traversal algorithm for CPUs: maximizing single-node efficiency. In: 2012 IEEE 26th International Parallel and Distributed Processing Symposium (IPDPS). IEEE, pp 378–389Google Scholar
  11. 11.
    Cormen TH (2009) Introduction to algorithms. MIT Press, CambridgezbMATHGoogle Scholar
  12. 12.
    Daga M, Nutter M, Meswani M (2014) Efficient breadth-first search on a heterogeneous processor. In: 2014 IEEE International Conference on Big Data (Big Data). IEEE, pp 373–382Google Scholar
  13. 13.
    Dongarra JJ, Meuer HW, Strohmaier E et al (1997) Top500 supercomputer sites. Supercomputer 13:89–111Google Scholar
  14. 14.
    Erdös Rényi (1959) On random graphs I. Publ Math Debr 6:290–297zbMATHGoogle Scholar
  15. 15.
    Hong S, Kim SK, Oguntebi T, Olukotun K (2011) Accelerating CUDA graph algorithms at maximum warp. In: ACM SIGPLAN Notices, vol 46. ACM, pp 267–276Google Scholar
  16. 16.
    Hong S, Oguntebi T, Olukotun K (2011) Efficient parallel graph exploration on multi-core CPU and GPU. In: 2011 International Conference on Parallel Architectures and Compilation Techniques (PACT). IEEE, pp 78–88Google Scholar
  17. 17.
    Intel Corporation (2014) The compute architecture of Intel processor graphics Gen7.5.
  18. 18.
    Jensen TR, Toft B (2011) Graph coloring problems, vol 39. Wiley, LondonzbMATHGoogle Scholar
  19. 19.
    Kepner J, Gilbert J (2011) Graph algorithms in the language of linear algebra. SIAM, PhiladelphiaCrossRefGoogle Scholar
  20. 20.
    Korf RE (1985) Depth-first iterative-deepening: an optimal admissible tree search. Artif Intell 27(1):97–109MathSciNetCrossRefGoogle Scholar
  21. 21.
    Korf RE, Schultze P (2005) Large-scale parallel breadth-first search. In: Association for the Advancement of Artificial Intelligence (AAAI), vol 5, pp 1380–1385Google Scholar
  22. 22.
    Kumar P, Huang HH (2016) G-store: high-performance graph store for trillion-edge processing. In: International Conference for High Performance Computing, Networking, Storage and Analysis, SC16. IEEE, pp 830–841Google Scholar
  23. 23.
    Li J, Tan G, Chen M, Sun N (2013) SMAT: an input adaptive auto-tuner for sparse matrix–vector multiplication. In: ACM SIGPLAN Notices, vol 48. ACM, pp 117–126Google Scholar
  24. 24.
    Liu H, Huang HH (2015) Enterprise: breadth-first graph traversal on GPUs. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. ACM, p 68Google Scholar
  25. 25.
    Liu H, Huang HH (2017) Graphene: fine-grained IO management for graph computing. In: USENIX Conference on File and Storage Technologies (FAST), pp 285–300Google Scholar
  26. 26.
    Liu H, Huang HH, Hu Y (2016) iBFS: concurrent breadth-first search on GPUs. In: Proceedings of the 2016 International Conference on Management of Data. ACM, pp 403–416Google Scholar
  27. 27.
    Liu T, Chen CC, Kim W, Milor L (2015) Comprehensive reliability and aging analysis on SRAMs within microprocessor systems. Microelectron Reliab 55(9):1290–1296CrossRefGoogle Scholar
  28. 28.
    Liu T, Chen CC, Wu J, Milor L (2016) Sram stability analysis for different cache configurations due to bias temperature instability and hot carrier injection. In: 2016 IEEE 34th International Conference on Computer Design (ICCD). IEEE, pp 225–232Google Scholar
  29. 29.
    Liu W, Vinter B (2015) A framework for general sparse matrix–matrix multiplication on GPUs and heterogeneous processors. J Parallel Distrib Comput 85:47–61CrossRefGoogle Scholar
  30. 30.
    Liu W, Vinter B (2015) CSR5: an efficient storage format for cross-platform sparse matrix–vector multiplication. In: Proceedings of the 29th ACM on International Conference on Supercomputing. ACM, pp 339–350Google Scholar
  31. 31.
    Liu W, Vinter B (2015) Speculative segmented sum for sparse matrix–vector multiplication on heterogeneous processors. Parallel Comput 49:179–193MathSciNetCrossRefGoogle Scholar
  32. 32.
    Luo L, Wong M, Hwu W (2010) An effective GPU implementation of breadth-first search. In: Proceedings of the 47th Design Automation Conference. ACM, pp 52–55Google Scholar
  33. 33.
    Merrill D, Garland M, Grimshaw A (2012) Scalable GPU graph traversal. In: ACM SIGPLAN Notices, vol 47. ACM, pp 117–128Google Scholar
  34. 34.
    Murphy RC, Wheeler KB, Barrett BW, Ang JA (2010) Introducing the Graph 500. In: Cray Users Group (CUG) ProceedingsGoogle Scholar
  35. 35.
    YOKOGAWA (2017) WT210/WT230 digital power meters.
  36. 36.
    Nikolskiy VP, Stegailov VV, Vecher VS (2016) Efficiency of the Tegra K1 and X1 systems-on-chip for classical molecular dynamics. In: 2016 International Conference on High Performance Computing and Simulation (HPCS). IEEE, pp 682–689Google Scholar
  37. 37.
    Pearce R, Gokhale M, Amato NM (2013) Scaling techniques for massive scale-free graphs in distributed (external) memory. In: 2013 IEEE 27th International Symposium on Parallel and Distributed Processing (IPDPS). IEEE, pp 825–836Google Scholar
  38. 38.
    Saad Y (1990) SPARSKIT: a basic tool kit for sparse matrix computations. NASA technical report, NASA, pp 1–30Google Scholar
  39. 39.
    Satish N, Sundaram N, Patwary MMA, Seo J, Park J, Hassaan MA, Sengupta S, Yin Z, Dubey P (2014) Navigating the maze of graph analytics frameworks using massive graph datasets. In: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data. ACM, pp 979–990Google Scholar
  40. 40.
    Scarpazza DP, Villa O, Petrini F (2008) Efficient breadth-first search on the Cell/BE processor. IEEE Trans Parallel Distrib Syst 19(10):1381–1395CrossRefGoogle Scholar
  41. 41.
    Sedaghati N, Mu T, Pouchet LN, Parthasarathy S, Sadayappan P (2015) Automatic selection of sparse matrix representation on GPUs. In: Proceedings of the 29th ACM on International Conference on Supercomputing, ICS ’15, pp 99–108Google Scholar
  42. 42.
    Shi X, Zheng Z, Zhou Y, Jin H, He L, Liu B, Hua QS (2018) Graph processing on GPUs: a survey. ACM Comput Surv 50(6):81CrossRefGoogle Scholar
  43. 43.
    Stone JE, Gohara D, Shi G (2010) OpenCL: a parallel programming standard for heterogeneous computing systems. Comput Sci Eng 12(3):66–73CrossRefGoogle Scholar
  44. 44.
    Su BY, Keutzer K (2012) clSpMV: a cross-platform OpenCL SpMV framework on GPUs. In: Proceedings of the 26th ACM International Conference on Supercomputing. ACM, pp 353–364Google Scholar
  45. 45.
    Wang X, Liu W, Xue W, Wu L (2018) swSpTRSV: a fast sparse triangular solve with sparse level tile layout on sunway architectures. In: Proceedings of the 23rd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. ACM, pp 338–353Google Scholar
  46. 46.
    Wang Y, Davidson A, Pan Y, Wu Y, Riffel A, Owens JD (2016) Gunrock: a high-performance graph processing library on the GPU. In: Proceedings of the 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. ACM, p 11Google Scholar
  47. 47.
    Yan S, Li C, Zhang Y, Zhou H (2014) yaSpMV: yet another SpMV framework on GPUs. In: ACM SIGPLAN Notices, vol 49. ACM, pp 107–118Google Scholar
  48. 48.
    Yang C, Buluc A, Owens JD (2018) Implementing push–pull efficiently in GraphBLAS. In: International Conference on Parallel Processing (ICPP)Google Scholar
  49. 49.
    Yasui Y, Fujisawa K (2015) Fast and scalable NUMA-based thread parallel breadth-first search. In: 2015 International Conference on High Performance Computing and Simulation (HPCS). IEEE, pp 377–385Google Scholar
  50. 50.
    Zhang F, Zhai J, Chen W, He B, Zhang S (2015) To co-run, or not to co-run: a performance study on integrated architectures. In: 2015 IEEE 23rd International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS). IEEE, pp 89–92Google Scholar
  51. 51.
    Zhang F, Wu B, Zhai J, He B, Chen W (2017) FinePar: irregularity-aware fine-grained workload partitioning on integrated architectures. In: International Symposium on Code Generation and Optimization (CGO). IEEE Press, pp 27–38Google Scholar
  52. 52.
    Zhang F, Zhai J, He B, Zhang S, Chen W (2017) Understanding co-running behaviors on integrated CPU/GPU architectures. IEEE Trans Parallel Distrib Syst 28(3):905–918CrossRefGoogle Scholar
  53. 53.
    Zhang R, Liu T, Yang K, Milor L (2017) Analysis of time-dependent dielectric breakdown induced aging of SRAM cache with different configurations. Microelectron Reliab 76:87–91CrossRefGoogle Scholar
  54. 54.
    Zhong J, He B (2014) Medusa: simplified graph processing on GPUs. IEEE Trans Parallel Distrib Syst 25(6):1543–1552MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Key Laboratory of Data Engineering and Knowledge Engineering (MOE)Renmin University of ChinaBeijingChina
  2. 2.School of InformationRenmin University of ChinaBeijingChina
  3. 3.Department of Computer Science and TechnologyTsinghua UniversityBeijingChina
  4. 4.Huawei Technologies Co., Ltd.ShenzhenChina

Personalised recommendations