Abstract
In this study, we present our parallel implementation for large-scale cosmological simulations of 3D supersonic fluids based on CPU and GPU clusters. Our developments are based on an OpenMP parallelized CPU code named WIGEON. It is shown that a speedup of 13~31 (depending on the specific GPU card) can be achieved compared to the sequential Fortran code by using the GPU as the accelerator. Further more, our results show that the pure MPI parallelization scales very well up to ten thousand CPU cores. In addition, a hybrid CPU/GPU parallelization scheme is introduced and a detailed analysis of the speedup and the scaling on the different number of CPU and GPU cards are presented (up to 256 GPU cards due to computing resource limitation). The efficiency of our scaling and high speedup relies on domain decomposition approach, optimization of the WENO algorithm and a series of techniques to optimize the CUDA implementation, especially in the memory access pattern. We believe this hybrid MPI+CUDA code can be an excellent candidate for 10 Peta-scale computing and beyond.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Feng, L.-L., Shu, C.-W., Zhang, M.: A hybrid cosmological hydrodynamic/N-body code based on a weighted essentially non-oscillatory scheme. The Astrophysical Journal (September 2004)
Anderson Jr., J.D.: Fundamentals of Aerodynamics, 3rd edn. (January 2001)
Robert, W.F., Alan, T.M.: Introduction To Fluid Mechanics, 4th edn.
Juan-Chen, H., Herng, L., Tsang-Jen, H., Tse-Yang, H.: Parallel preconditioned WENO scheme for three-dimensional flow simulation of NREL Phase VI Rotor. Computers & Fluids, 276-282 (2011)
Laurent, T., Andres, E.T., Thomas, B.G., Gilmar, M.: A massively parallel hybrid scheme for direct numerical simulation of turbulent viscoelastic channel flow. Computers & Fluids, 134–142 (2011)
Kestener, P., Château, F., Teyssier, R.: Accelerating euler equations numerical solver on graphics processing units. In: Hsu, C.-H., Yang, L.T., Park, J.H., Yeo, S.-S. (eds.) ICA3PP 2010, Part I. LNCS, vol. 6082, pp. 281–288. Springer, Heidelberg (2010)
Tölke, J., Krafczyk, M.: TeraFLOP computing on a desktop PC with GPUs for 3D CFD. International Journal of Computational Fluid Dynamics, 443–456 (2008)
Athanasios, S.A., Konstantinos, I.K., Eleftherios, D.P., John, A.E.: Acceleration of a Finite-Difference WENO Scheme for Large-Scale Simulations on Many-Core Architectures. The American Institute of Aeronautics and Astronautics (2010)
Appleyard, J., Drikakis, D.: Higher-order CFD and interface tracking methods on highly-Parallel MPI and GPU systems. Computers & Fluids, 101–105 (2011)
Michael, G., Peter, Z.: A multi-GPU accelerated solver for the three-dimensional two-phase incompressible Navier-Stokes equations. Computer Science-Research and Development, 65–73 (2010)
Paulius, M.: 3D finite difference computation on GPUs using CUDA. Architectual Support for Programming Languages and Operating Systems, 79–84 (2009)
Jiang, G.S., Shu, C.W.: Efficient Implementation of Weighted ENO Schemes. J. Computational Physics, 202–208 (1996)
Balsara, D.S., Shu, C.W.: Monotonicity Preserving Weighted Essentially Non-oscillatory Schemes with Increasingly High Order of Accuracy. J. Computational Physics, 405–452 (2000)
Chi-Wang, S.: Total Variation Diminishing Time Discretizations. Siam Journal on Scientific and Statistical Computing (1988)
Dana, A.J., Julien, C.T., Inanc, S.: An MPI-CUDA Implementation for Massively Parallel Incompressible Flow Computaions on multi-CPU clusters. The American Institute of Aeronautics and Astronautics (2010)
John, L.H., David, A.P.: Computer Architecture: A Quantitative Approach, 5th edn.
Paulius, M.: Analysis-Driven Optimization. In: SC 2010. ACM (2010)
NVIDIA’s Next Generation CUDA Compute Architecture: Kepler GK110 (v1.0, 2012)
Compute Command Line Profiler User Guide. DU-05982-001_v03 (November 2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Meng, C., Wang, L., Cao, Z., Feng, Ll., Zhu, W. (2014). Large-Scale Parallelization Based on CPU and GPU Cluster for Cosmological Fluid Simulations . In: Li, K., Xiao, Z., Wang, Y., Du, J., Li, K. (eds) Parallel Computational Fluid Dynamics. ParCFD 2013. Communications in Computer and Information Science, vol 405. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-53962-6_18
Download citation
DOI: https://doi.org/10.1007/978-3-642-53962-6_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-53961-9
Online ISBN: 978-3-642-53962-6
eBook Packages: Computer ScienceComputer Science (R0)