Cluster Computing

, Volume 22, Supplement 1, pp 2371–2381 | Cite as

Performance optimization and evaluation for parallel processing of big data in earth system models

  • Yuzhu Wang
  • Huiqun Hao
  • Junqiang ZhangEmail author
  • Jinrong Jiang
  • Juanxiong He
  • Yan MaEmail author


Big data and high performance computing in Earth System Models (ESMs) are receiving increased attention in earth science research. When scaling to large-scale multi-core computing, efficient parallelization of an ESM, which demands fast parallel computing for long-term integration or climate simulation, becomes extremely challenging because of time-consuming internal big data communication. In this paper, an optimization algorithm for the massive data communication between the Weather Research and Forecasting model and Coupler version 7 in the Chinese Academy of Sciences-Earth System Model (CAS-ESM) is proposed. The optimization strategy is to transmit data from a small packet into a larger packet. Through experiments on a multi-core cluster, the efficiency of the algorithm is confirmed. Then, the parallel performance of the CAS-ESM is evaluated fully. Results show that the parallel efficiency of the CAS-ESM on 1024 CPU cores reaches nearly 70%, indicating that the CAS-ESM has desirable parallel performance and strong scalability. In addition, a generic performance evaluation method for ESMs from perspectives of optimal load balance and efficiency is proposed. Results show that the computing speed is the fastest and computational efficiency is the highest when the CAS-ESM runs on a certain number of cores.


Big data High performance computing Performance optimization Earth system model 



This work is supported by the National Key Research and Development Program of China (No. 2016YFB0200800), National Natural Science Foundation of China (No. 61602477, No. 41401512), China Postdoctoral Science Foundation (No. 2016M601158), Youth Innovation Promotion Association of CAS (No. Y6YR0300QM), and the Fundamental Research Funds for the Central Universities (No. 2652017113).


  1. 1.
    Wang, L., Geng, H., Liu, P., et al.: Particle swarm optimization based dictionary learning for remote sensing big data. Knowl.-Based Syst. 79, 43–50 (2015)CrossRefGoogle Scholar
  2. 2.
    Wang, L., Lu, K., Liu, P., et al.: IK-SVD: dictionary learning for spatial big data via incremental atom update. Comput. Sci. Eng. 16(4), 41–52 (2014)CrossRefGoogle Scholar
  3. 3.
    Song, W., Liu, P., Wang, L.: Sparse representation-based correlation analysis of non-stationary spatiotemporal big data. Int. J. Digit. Earth 9(9), 892–913 (2016)CrossRefGoogle Scholar
  4. 4.
    Wang, L., Song, W., Liu, P.: Link the remote sensing big data to the image features via wavelet transformation. Clust. Comput. 19(2), 793–810 (2016)CrossRefGoogle Scholar
  5. 5.
    He, Z., Wu, C., Liu, G., Zheng, Z., Tian, Y.: Decomposition tree: a spatio-temporal indexing method for movement big data. Clust. Comput. 18(4), 1481–1492 (2015)CrossRefGoogle Scholar
  6. 6.
    Wang, Y., Liu, Z., Liao, H., Li, C.: Improving the performance of GIS polygon overlay computation with MapReduce for spatial big data processing. Clust. Comput. 18(2), 507–516 (2015)CrossRefGoogle Scholar
  7. 7.
    Deng, Z., Hu, Y., Zhu, M., Huang, X., Du, B.: A scalable and fast OPTICS for clustering trajectory big data. Clust. Comput. 18(2), 549–562 (2015)CrossRefGoogle Scholar
  8. 8.
    Chen, Y., Li, F., Fan, J.: Mining association rules in big data with NGEP. Clust. Comput. 18(2), 577–585 (2015)CrossRefGoogle Scholar
  9. 9.
    Song, W., Deng, Z., Wang, L., Du, B., Liu, P., Lu, K.: G-IK-SVD: parallel IK-SVD on GPUs for sparse representation of spatial big data. J. Supercomput. 73(8), 3433–3450 (2017)CrossRefGoogle Scholar
  10. 10.
    Xue, W., Yang, C., Fu, H., Wang, X., Xu, Y., Liao, J., Gan, L., Lu, Y., Ranjan, R., Wang, L.: Ultra-scalable CPU-MIC acceleration of mesoscale atmospheric modeling on Tianhe-2. IEEE Trans. Comput. 64(8), 2382–2393 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Wang, L., Ma, Y., Yan, J., Chang, V., Zomaya, A.Y.: pipsCloud: high performance cloud computing for remote sensing big data management and processing. Future Gener. Comput. Syst. 78, 353–368 (2018)CrossRefGoogle Scholar
  12. 12.
    Hurrell, J.W., Holland, M.M., Gent, P.R., et al.: The community earth system model: a framework for collaborative research. Bull. Am. Meteorol. Soc. 94(9), 1339–1360 (2013)CrossRefGoogle Scholar
  13. 13.
    Vertenstein, M., Craig, T., Middleton, A., Feddema, D., Fischer, C.: CESM1. 0.4 Users Guide. Technical report, Community Earth System Model, NCAR, USA (2011)Google Scholar
  14. 14.
    Sun, H., Zhou, G., Zeng, Q.: Assessments of the climate system model (CAS-ESM-C) using IAP AGCM4 as its atmospheric component. Chin. J. Atmos. Sci. 36(2), 215–233 (2012). in ChineseGoogle Scholar
  15. 15.
    Dong, X., Su, T., Wang, J., Lin, R.: Decadal variation of the Aleutian low-icelandic low seesaw simulated by a climate system model (CAS-ESM-C). Atmos. Ocean. Sci. Lett. 7(2), 110–114 (2014)CrossRefGoogle Scholar
  16. 16.
    Taylor, K.E., Stouffer, R.J., Meehl, G.A.: An overview of CMIP5 and the experiment design. Bull. Am. Meteorol. Soc. 93(4), 485–498 (2012)CrossRefGoogle Scholar
  17. 17.
    Montoya, M., Griesel, A., Levermann, A., Mignot, J., Hofmann, M., Ganopolski, A., Rahmstorf, S.: The earth system model of intermediate complexity CLIMBER-3\(\alpha \). Part I: description and performance for present-day conditions. Clim. Dyn. 25(2–3), 237–263 (2005)CrossRefGoogle Scholar
  18. 18.
    Duffy, P.B., Govindasamy, B., Iorio, J.P., et al.: High-resolution simulations of global climate, part 1: present climate. Clim. Dyn. 21(5–6), 371–390 (2003)CrossRefGoogle Scholar
  19. 19.
    Khairoutdinov, M.F., Randall, D.A.: A cloud resolving model as a cloud parameterization in the NCAR community climate system model: preliminary results. Geophys. Res. Lett. 28(18), 3617–3620 (2001)CrossRefGoogle Scholar
  20. 20.
    Washington, W.M., Buja, L., Craig, A.: The computational future for climate and earth system models: on the path to petaflop and beyond. Philos. Trans. R. Soc. Lond. A 367(1890), 833–846 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Wehner, M.F., Reed, K.A., Li, F., et al.: The effect of horizontal resolution on simulation quality in the Community Atmospheric Model, CAM5.1. J. Adv. Model. Earth Syst. 6(4), 980–997 (2014)CrossRefGoogle Scholar
  22. 22.
    Nakaegawa, T., Kitoh, A., Ishizaki, Y., Kusunoki, S., Murakami, H.: Caribbean low-level jets and accompanying moisture fluxes in a global warming climate projected with CMIP3 multi-model ensemble and fine-mesh atmospheric general circulation models. Int. J. Climatol. 34(4), 964–977 (2014)CrossRefGoogle Scholar
  23. 23.
    Miyamoto, Y., Kajikawa, Y., Yoshida, R., Yamaura, T., Yashiro, H., Tomita, H.: Deep moist atmospheric convection in a subkilometer global simulation. Geophys. Res. Lett. 40(18), 4922–4926 (2013)CrossRefGoogle Scholar
  24. 24.
    Craig, A.P., Vertenstein, M., Jacob, R.: A new flexible coupler for earth system modeling developed for CCSM4 and CESM1. Int. J. High Perform. Comput. Appl. 26(1), 31–42 (2012)CrossRefGoogle Scholar
  25. 25.
    Dennis, J.M., Edwards, J., Evans, K.J., et al.: CAM-SE: a scalable spectral element dynamical core for the Community Atmosphere Model. Int. J. High Perform. Comput. Appl. 26(1), 74–89 (2012)CrossRefGoogle Scholar
  26. 26.
    Dennis, J.M., Vertenstein, M., Worley, P.H., Mirin, A.A., Craig, A.P., Jacob, R., Mickelson, S.: Computational performance of ultra-high-resolution capability in the Community Earth System Model. Int. J. High Perform. Comput. Appl. 26(1), 5–16 (2012)CrossRefGoogle Scholar
  27. 27.
    Wehner, M.F., Ambrosiano, J.J., Brown, J.C., et al.: Toward a high performance distributed memory climate model. In: High Performance Distributed Computing, 1993. IEEE Proceedings the 2nd International Symposium, pp. 102–113 (1993)Google Scholar
  28. 28.
    Mechoso, C.R., Drummond, L.A., Farrara, J.D., Spahr, J.A.: The UCLA AGCM in high performance computing environments. In: Proceedings of the 1998 ACM/IEEE Conference on Supercomputing, pp. 1–7. IEEE Computer Society (1998)Google Scholar
  29. 29.
    Drake, J., Foster, I., Michalakes, J., Toonen, B., Worley, P.: Design and performance of a scalable parallel community climate model. Parallel Comput. 21(10), 1571–1591 (1995)CrossRefzbMATHGoogle Scholar
  30. 30.
    Mirin, A.A., Sawyer, W.B.: A scalable implementation of a finite-volume dynamical core in the Community Atmosphere Model. Int. J. High Perform. Comput. Appl. 19(3), 203–212 (2005)CrossRefGoogle Scholar
  31. 31.
    Zou, Y., Xue, W., Liu, S.: A case study of large-scale parallel I/O analysis and optimization for numerical weather prediction system. Future Gener. Comput. Syst. 37, 378–389 (2014)CrossRefGoogle Scholar
  32. 32.
    Li, L., Xue, W., Ranjan, R., Jin, Z.: A scalable Helmholtz solver in GRAPES over large-scale multicore cluster. Concurr. Comput. 25(12), 1722–1737 (2013)CrossRefGoogle Scholar
  33. 33.
    Zhang, T., Sun, X., Xue, W., Qiao, N., Huang, H., Shu, J., Zheng, W.: ParSA: high-throughput scientific data analysis framework with distributed file system. Future Gener. Comput. Syst. 51, 111–119 (2015)CrossRefGoogle Scholar
  34. 34.
    Zhang, T., Li, L., Lin, Y., Xue, W., Xie, F., Xu, H., Huang, X.: An automatic and effective parameter optimization method for model tuning. Geosci. Model Dev. 8(11), 3579–3591 (2015)CrossRefGoogle Scholar
  35. 35.
    Wang, Y., Jiang, J., Ye, H., He, J.: A distributed load balancing algorithm for climate big data processing over a multi-core CPU cluster. Concurr. Comput. 28(15), 4144–4160 (2016)CrossRefGoogle Scholar
  36. 36.
    Zhang, H., Zhang, M., Zeng, Q.: Sensitivity of simulated climate to two atmospheric models: interpretation of differences between dry models and moist models. Mon. Weather Rev. 141(5), 1558–1576 (2013)CrossRefGoogle Scholar
  37. 37.
    Wang, Y., Jiang, J., Zhang, H., et al.: A scalable parallel algorithm for atmospheric general circulation models on a multi-core cluster. Future Gener. Comput. Syst. 72, 1–10 (2017)CrossRefGoogle Scholar
  38. 38.
    Skamarock, W.C., Klemp, J.B., Dudhia, J., et al.: A description of the advanced research WRF version 3. NCAR technical note, TN-475+STR (2008)Google Scholar
  39. 39.
    Johnsen, P., Straka, M., Shapiro, M., Norton, A., Galarneau, T.: Petascale WRF simulation of hurricane sandy: Deployment of NCSA’s cray XE6 blue waters. In: High Performance Computing, Networking, Storage and Analysis (SC’13), pp. 1–7. IEEE (2013)Google Scholar
  40. 40.
    Xie, S., Zhang, M., Branson, M., et al.: Simulations of midlatitude frontal clouds by single-column and cloud-resolving models during the atmospheric radiation measurement March 2000 cloud intensive operational period. J. Geophys. Res. 110(D15) (2005)Google Scholar
  41. 41.
    He, J., Zhang, M., Lin, W., Colle, B., Liu, P., Vogelmann, A.M.: The WRF nested within the CESM: simulations of a midlatitude cyclone over the Southern Great Plains. J. Adv. Model. Earth Syst. 5(3), 611–622 (2013)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2017

Authors and Affiliations

  1. 1.School of Information EngineeringChina University of GeosciencesBeijingPeople’s Republic of China
  2. 2.Computer Network Information CenterChinese Academy of SciencesBeijingPeople’s Republic of China
  3. 3.University of Chinese Academy of SciencesBeijingPeople’s Republic of China
  4. 4.School of Computer ScienceChina University of GeosciencesWuhanPeople’s Republic of China
  5. 5.Institute of Atmospheric PhysicsChinese Academy of SciencesBeijingPeople’s Republic of China
  6. 6.Institute of Remote Sensing and Digital EarthChinese Academy of SciencesBeijingPeople’s Republic of China

Personalised recommendations