Collaborating CPUs and MICs for Large-Scale LBM Multiphase Flow Simulations

  • Chuanfu XuEmail author
  • Xi Wang
  • Dali Li
  • Yonggang Che
  • Zhenghua Wang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11783)


This paper highlights the use of the OpenMP4.5 accelerator programming model to collaborate CPUs and Intel Many Integrated Cores (MIC) co-processors for large-scale LBM multiphase flow simulationson the Tianhe-2 supercomputer. To enhance the collaborative efficiency among intra-node CPUs and co-processors, we propose a flexible load balance model with heterogeneous domain decomposition for CPU-MIC task allocation, as well as asynchronous offloading to overlap operations of CPUs and multiple MICs. Tests for 3D multi-phase (liquid and gases) problem (about 100 Billion lattices) simulating drop impact with gravity effect using D3Q19 Lattice Boltzmann discretization and Shan-Chen BGK single relaxation time collision model are presented, achieving a weak parallel efficiency of above 80% in going from 128 to 2048 compute nodes.


Heterogeneous parallel computing Lattice Boltzmann methods Many-core processor OpenMP4.5 accelerator programming model 


  1. 1.
    Succi, S., Benzi, R., et al.: The lattice Boltzmann equation: a new tool for computational fluid-dynamics. Phys. D Nonlinear Phenom. 47, 219–230 (1991)CrossRefGoogle Scholar
  2. 2.
    Martineau, M., Price, J., McIntosh-Smith, S., Gaudin, W.: Pragmatic performance portability with OpenMP 4.x. In: Maruyama, N., de Supinski, B.R., Wahib, M. (eds.) IWOMP 2016. LNCS, vol. 9903, pp. 253–267. Springer, Cham (2016). Scholar
  3. 3.
    Xiangke, L., Liquan, X., Canqun, Y.: MilkyWay-2 supercomputer: system and application. Front. Comput. Sci. 8(3), 345–356 (2014)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Crimi, G., Mantovani, F., Pivanti, M., Schifano, S.F., Tripiccione, R.: Early experience on porting and running a Lattice Boltzmann code on the Xeon-Phi co-processor. Procedia Comput. Sci. 18, 551–560 (2013)CrossRefGoogle Scholar
  5. 5.
    McIntosh-Smith, S., Curran, D.: Evaluation of a performance portable lattice Boltzmann code using OpenCL. In: International Workshop on OpenCL, pp. 1–12 (2014)Google Scholar
  6. 6.
    Dali, L., Chuanfu, X., Yongxian, W., Zhifang, S., et al.: Parallelizing and optimizing large-scale 3D multi-phase flow simulations on the Tianhe-2 supercomputer. Concurr. Comput. Pract. Exp. 28, 1678–1692 (2015)Google Scholar
  7. 7.
    Chuanfu, X., Xiaogang, D., Lilun, Z., et al.: Collaborating CPU and GPU for large-scale high-order CFD simulations with complex grids on the TianHe-1A supercomputer. J. Comput. Phys. 278, 275–C297 (2014)CrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2019

Authors and Affiliations

  • Chuanfu Xu
    • 1
    Email author
  • Xi Wang
    • 1
  • Dali Li
    • 1
  • Yonggang Che
    • 1
  • Zhenghua Wang
    • 1
  1. 1.College of Computer ScienceNational University of Defense TechnologyChangshaPeople’s Republic of China

Personalised recommendations