Metrics for Gauging Model Performance Over the East Asian–Western Pacific Domain

  • Tianjun ZhouEmail author
  • Bo Wu
  • Yunying Li
  • Hailong Liu
  • Lijuan Li
  • Lixia Zhang
  • Fengfei Song
  • Chongbo Zhao
  • Lu Dong
  • Chao He
  • Yi Zhang
  • Weihua Yuan


A summary of the development of observational metrics for gauging model performance over the East Asian–western Pacific domain is presented. The proposed metrics focus on the multi-scale features of the East Asian summer monsoon (EASM), ranging from diurnal cycle to intraseasonal, interannual, and interdecadal variability, as well as the distribution of cloud and radiation in East Asia. We further extend these metrics from East Asia to the tropical Pacific and examine the processes responsible for the tropical bias. The performances of current state-of-the-art climate models in their simulation of the monsoon–ENSO relationship are also assessed, and some evidence of how to improve ENSO simulation is presented. In addition to the metrics, the performance of the LASG/IAP decadal prediction system is also assessed.


Observational metrics East Asian–western Pacific monsoon ENSO Tropical bias Decadal prediction 


  1. Bellenger, H., Guilyardi, E., Leloup, J., Lengaigne, M., Vialard, J.: ENSO representation in climate models: from CMIP3 to CMIP5. Climate Dyn. 42, 1999–2018 (2013). doi: 10.1007/s00382-013-1783-z
  2. Bodas-Salcedo, A., et al.: COSP: satellite simulation software for model assessment. Bull. Amer. Meteor. Soc. 92, 1023–1043 (2011)Google Scholar
  3. Boo, K.‐O., Martin, G., Sellar, A., Senior, C., Byun, Y.‐H.: Evaluating the East Asian monsoon simulation in climate models. J. Geophys. Res. 116, D01109 (2011). doi: 10.1029/2010JD014737
  4. Bony, S., Dufresne, J.-L.: Marine boundary layer clouds at the heart of tropical cloud feedback uncertainties in climate models. Geophys. Res. Lett. 32, L20806 (2005). doi: 10.1029/2005GL023851
  5. Chen, L., Yu, Y., Sun, D.-Z.: Cloud and water vapor feedbacks to the El Niño warming: Are they still biased in CMIP5 models? J. Climate 26, 2947–2961 (2013). doi: 10.1175/JCLI-D-12-00575.1
  6. Chen, H.M., Zhou, T.J., Neale, R.B., Wu, X.Q., Zhang, J.: Performance of the new NCAR CAM3.5 in East Asian summer monsoon simulations: sensitivity to modifications of the convection scheme. J. Climate 23, 3657–3675 (2010)CrossRefGoogle Scholar
  7. Comstock, J.M., Ackerman, T.P., Mace, G.G.: Ground-based lidar and radar remote sensing of tropical cirrus clouds at Nauru Island: Cloud statistics and radiative impacts. J. Geophys. Res. 107(D23), 4714 (2002). doi: 10.1029/2002JD002203
  8. Dong, L., Li, L.J., Huang, W.Y., Wang, Y., Wang, B.: Preliminary evaluation of cloud fraction simulations by GAMIL2 using COSP. Atmos. Oceanic Sci. Lett. 5, 258–263 (2012)CrossRefGoogle Scholar
  9. Gent, P.R., et al.: The community climate system model version 4. J. Climate 24, 4973–4991 (2011)Google Scholar
  10. Guinehut, S., Coatanoan, C., Dhomps, A.L., Le Traon, P.Y., Larnicol, G.: On the use of satellite altimeter data in Argo quality control. J. Atmos. Oceanic Technol. 26, 395–402 (2009)CrossRefGoogle Scholar
  11. He, C., Zhou, T.J.: The two interannual variability modes of the Western North Pacific Subtropical High simulated by 28 CMIP5-AMIP models. Climate Dyn. 43, 2455–2469 (2014). doi: 10.1007/s00382-00014-02068-x CrossRefGoogle Scholar
  12. He, C., Zhou, T.: Responses of the western North Pacific Subtropical High to global warming under RCP4.5 and RCP8.5 scenarios projected by 33 CMIP5 models: the dominance of tropical Indian Ocean—tropical western Pacific SST gradient. J Climate 28, 365–380 (2015)CrossRefGoogle Scholar
  13. Kay, J.E., Hillman, B.R., Klein, S.A., et al.: Exposing global cloud biases in the Community Atmosphere Model (CAM) using satellite observations and their corresponding instrument simulators. J. Climate (2012, in press). doi: 10.1175/JCLI-D-11-00469.1
  14. Keenlyside, N., Latif, M., Jungclaus, J., Kornblueh, L., Roeckner, E.: Advancing decadal-scale climate prediction in the North Atlantic sector. Nature 453, 84–88 (2008)CrossRefGoogle Scholar
  15. King, M.D., Menzel, W.P., Kaufman, Y.J., Tanre, D., Bo-Cai, G., Platnick, S., Ackerman, S.A., Remer, L.A., Pincus, R., Hubanks, P.A.: Cloud and aerosol properties, precipitable water, and profiles of temperature and water vapor from MODIS, IEEE Trans. Geosci. Remote Sens. 41(2), 442–458 (2003)Google Scholar
  16. Li, Y.Y., Gu, H.: Relationship between middle stratiform clouds and large scale circulation over eastern China. Geophys. Res. Lett. 33, L09706 (2006). doi: 10.1029/2005GL025615 Google Scholar
  17. Li, L.J., Wang, B.: Influences of two convective schemes on the radiative energy budget in GAMIL1.0. Acta Meteor. Sinica, 24(3), 318–327 (2010)Google Scholar
  18. Li, J., Yu, R.: A method to linearly evaluate rainfall frequency-intensity distribution. J. Appl. Meteor. Climatol. 53(4), 928–934 (2014)CrossRefGoogle Scholar
  19. Li, T., Liu, P., Fu, X., Wang, B., Meehl, G.A.: Spatiotemporal structures and mechanisms of the tropospheric biennial oscillation in the Indo-Pacific warm ocean regions. J. Climate 19, 3070–3087 (2006)CrossRefGoogle Scholar
  20. Li, L.J., Xie, X., Wang, B., Dong, L.: Evaluating the performances of GAMIL1.0 and GAMIL2.0 during TWP-ICE with CAPT. Atmos. Oceanic Sci. Lett. 5, 38–42 (2012)CrossRefGoogle Scholar
  21. Li, L.J., et al.: Evaluation of grid-point atmospheric model of IAP LASGversion 2 (GAMIL2). Adv. Atmos. Sci. 30(3), 855–867 (2013). doi: 10.1007/s00376-013-2157-5
  22. Li, L.J., Wang, B., Zhang, G.J.: The role of non-convective condensation processes in response of surface shortwave cloud radiative forcing to El Niño warming. J. Climate 27, 6721–6736 (2014). doi: 10.1175/JCLI-D-13-00632.1 CrossRefGoogle Scholar
  23. Li, J., Yu, R., Yuan, W., Chen, H., Sun, W., Zhang, Y.: Precipitation over East Asia simulated by NCAR CAM5 at different horizontal resolutions. J. Adv. Model. Earth Syst. 7(2), 774–790 (2015). doi: 10.1002/2014MS000414 CrossRefGoogle Scholar
  24. Lin, J.L.: The double-ITCZ problem in IPCC AR4 coupled GCMs: ocean–atmosphere feedback analysis. J. Climate 20, 4497–4525 (2007)CrossRefGoogle Scholar
  25. Liu, X.C., Liu, H.L.: Heat budget of South-Central equatorial Pacific in IPCC AR4 models. Atmos. Sci, Adv (2014). doi: 10.1007/s00376-013-229-5 Google Scholar
  26. Liu, H.L., Lin, W.Y., Zhang, M.H.: Heat budget of the upper ocean in the South Central Equatorial Pacific. J. Climate 23, 1779–1792 (2010)CrossRefGoogle Scholar
  27. Liu, H.L., Liu, X.C., Zhang, M.H., Liu, W.Y.: A critical evaluation of the upper ocean heat budget in the climate forecast system reanalysis data for the south central equatorial Pacific. Environ. Res. Lett. 6, 034022 (2011). doi: 10.1088/1748-9326/6/3/034022 CrossRefGoogle Scholar
  28. Liu, H.L., Zhang, M.H., Lin, W.Y.: An investigation of the initial development of the double ITCZ warm SST biases in the CCSM. J. Climate 25, 140–155 (2012)CrossRefGoogle Scholar
  29. Liu, X.C., Liu, H.L., Lin, P.F., Yu, Y.Q.: Tropical biases, Chap. 11. In: Zhou, T., et al. (eds.) Flexible Global Ocean–Atmosphere–Land System Model. Springer, Berlin (2014)Google Scholar
  30. Marchand, R., Ackerman, T.: An analysis of cloud cover in multiscale modeling framework global climate model simulations using 4 and 1 km horizontal grids. J. Geophys. Res. 115, D16207 (2010). doi: 10.1029/2009JD013423
  31. Mechoso, C.R., et al.: The seasonal cycle over the tropical Pacific in coupled ocean-atmosphere general circulation models. Mon. Wea. Rev. 123, 2825–2838 (1995)Google Scholar
  32. Meehl, G.A., et al.: Decadal prediction: can it be skillful? Bull. Amer. Meteor. Soc. 90, 1467–1485 (2009)Google Scholar
  33. Philander, S., Gu, D., Lambert, G., Li, T., Halpern, D., Lau, N.-C., Pacanowski, R.: Why the ITCZ is mostly north of the equator. J. Climate 9, 2958–2972 (1996). doi: 10.1175/1520-0442(1996)009,2958:WTIIMN.2.0.CO;2
  34. Phillips, T.J., Potter, G.L., Williamson, D.L., Cederwall, R.T., Boyle, J.S., Fiorino, M., Hnilo, J.J., Olson, J.G., Xie, S., Yio, J.J.: Evaluating Parameterizations in General Circulation Models: Climate Simulation Meets Weather Prediction. Bull. Amer. Meteor. Soc. 85, 1903–1915 (2004)Google Scholar
  35. Rossow, W.B., Schiffer, R.A.: Advances in Understanding Clouds from ISCCP. Bull. Amer. Meteor. Soc. 80, 2261–2287 (1999). doi: 10.1175/1520-0477(1999)080<2261:AIUCFI>2.0.CO;2
  36. Schumacher, C., Houze, R.A., Kraucunas, I.: The tropical dynamical response to latent heating estimates derived from the TRMM precipitation radar. J. Atmos. Sci. 61, 1341–1358 (2004). doi: 10.1175/1520-0469(2004)061,1341:TTDRTL.2.0.CO;2
  37. Song, X., Zhang, G.J.: Convection parameterization, tropical Pacific double ITCZ, and upper ocean biases in the NCAR CCSM3. Part I: Climatology and atmospheric feedback. J. Climate 22, 4299–4315 (2009)Google Scholar
  38. Song, F.F., Zhou, T.J.: Interannual variability of East Asian Summer monsoon simulated by CMIP3 and CMIP5 AGCMs: skill dependence on Indian Ocean-Western Pacific anticyclone teleconnection. J. Climate 27, 1679–1697 (2014a)CrossRefGoogle Scholar
  39. Song, F., Zhou, T.: The climatology and interannual variability of East Asian summer monsoon in CMIP5 coupled models: Does air-sea coupling improve the simulations? J. Clim. 27, 8761–8777 (2014b)CrossRefGoogle Scholar
  40. Song, F., Zhou, T.J., Qian, Y.: Responses of East Asian summer monsoon to natural and anthropogenic forcings in the 17 latest CMIP5 models. Geophys. Res. Lett. 41, 596–603 (2014). doi: 10.1002/2013GL058705 CrossRefGoogle Scholar
  41. Sperber, K.R., Annamalai, H., Kang, I.-S., Kitoh, A., Moise, A., Turner, A., Wang, B., Zhou, T.: The Asian summer monsoon: an intercomparison of CMIP5 vs. CMIP3 simulations of the late 20th century. Climate Dyn. 41, 2711–2744 (2013)CrossRefGoogle Scholar
  42. Wang, B., Ding, Q.: The global monsoon: major modes of annual variation in Tropical precipitation and circulation. Dyn. Atmos. Oceans 44, 165–183 (2008)Google Scholar
  43. Weng, F., Grody, N., Ferraro, R., Zhao, Q., Chen, C.: Global cloud water distribution derived from Special Sensor Microwave Imager/Sounder and its comparison with GCM simulation. Adv. Space Res. 19, 407–411 (1997). doi: 10.1016/S0273-1177(97)00048-3
  44. Wang, B., Liu, J., Kim, H., Webster, P.J., Yim, S.Y.: Recent change of the global monsoon precipitation (1979–2008). Climate Dyn. (2011). doi: 10.1007/s00382-011-1266-z
  45. Wang, B., Wu, R.G., Lukas, R., An, S.I.: A possible mechanism for ENSO turnabout. Dynamics of Atmospheric General circulation and Climate, IAP/Academia Sinica, Ed., China Meteorological Press, 552–578 (2001)Google Scholar
  46. Winker, D.M., Pelon, J., Coakley, J.A.Jr., Ackerman, S.A., Charlson, R.J., Colarco, P.R., Flamant, P., Fu, Q., Hoff, R.M., Kittaka, C., Kubar, T.L., Le Treut, H., McCormick, M.P., Megie, G., Poole, L., Powell, K., Trepte, C., Vaughan, M.A., Wielicki, B.A.: The CALIPSO Mission: A Global 3D View of Aerosols and Clouds. Bull. Amer. Meteor. Soc. 91, 1211–1229 (2010)Google Scholar
  47. Wu, B., Zhou, T.J.: Prediction of decadal variability of sea surface temperature by a coupled global climate model FGOALS_gl developed in LASG/IAP. Chin. Sci. Bull. 57, 2453–2459 (2012). doi: 10.1007/s11434-012-5134-y CrossRefGoogle Scholar
  48. Wu, B., Zhou, T.J.: Relationships between East Asian-western North Pacific monsoon and ENSO simulated by FGOALS-s2. Adv. Atmos. Sci. 30(3), 713–725 (2013)CrossRefGoogle Scholar
  49. Wu, B., Zhou, T.: Relationships between ENSO and the East Asian–western North Pacific monsoon: observations versus 18 CMIP5 models. Clim. Dyn. (2015). doi: 10.1007/s00382-015-2609-y Google Scholar
  50. Wu, B., Zhou, T., Li, T.: Seasonally evolving dominant interannual variability modes of East Asian Climate. J. Clim. 22, 2992–3005 (2009)CrossRefGoogle Scholar
  51. Wu, B., Li, T., Zhou, T.: Relative contributions of the Indian Ocean and local SST anomalies to the maintenance of the western North Pacific anomalous anticyclone during El Nino decaying summer. J. Clim. 23, 2974–2986 (2010)CrossRefGoogle Scholar
  52. Xie, S.P.: The shape of continents, air-sea interaction, and the rising branch of the Hadley circulation. In: Diaz, H.F., Bradley, R. S. (eds.) The Hadley Circulation: Present, Past and Future. Advances in Global Change Research Series (2005)Google Scholar
  53. Yu, R.C., Li, W., Zhang, X.C., Yu, Y.Q., Liu, H.L., Zhou, T.J.: Climatic features related to eastern China summer rainfalls in the NCAR CCM3. Adv. Atmos. Sci. 17, 503–518 (2000)CrossRefGoogle Scholar
  54. Yu, R., Yu, Y., Zhang, M.: Comparing cloud radiative properties between the Eastern China and the Indian monsoon region. Adv. Atmos. Sci. 18, 1090–1102 (2001)CrossRefGoogle Scholar
  55. Yu, R., Wang, B., Zhou, T.: Climate effects of the deep continental stratus clouds generated by the Tibetan Plateau. J. Climate 17, 2702–2713 (2004)CrossRefGoogle Scholar
  56. Yuan, W.H.: Diurnal cycles of precipitation over subtropical China in IPCC AR5 AMIP simulations. Adv. Atmos. Sci. 30(6), 1679–1694 (2013)CrossRefGoogle Scholar
  57. Zhang, Y., Li, J.: Shortwave cloud radiative forcing on major stratus cloud regions in AMIP-type simulations of CMIP3 and CMIP5 models. Adv. Atmos. Sci. 30(3), 884–907 (2013)CrossRefGoogle Scholar
  58. Zhang, Y., Yu, R., Li, J., Yuan, W., Zhang, M.: Dynamic and thermodynamic relations of distinctive stratus clouds on the lee side of the Tibetan Plateau in the cold season. J. Climate 26, 8378–8391 (2013)CrossRefGoogle Scholar
  59. Zhang, L., Zhou, T.: An Assessment of Improvements in Global Monsoon Precipitation Simulation in FGOALS-s2?, Advances in Atmospheric Sciences, 31(1), 165–178 (2014). doi: 10.1007/s00376-013-2164-6
  60. Zhao, C.B., Zhou, T.J., Song, L.C., Ren, H.L.: The boreal summer intraseasonal oscillation simulated by 4 Chinese AGCMs participated in CMIP5 project. Adv. Atm. Sci. 31, 1167–1180 (2014). doi: 10.1007/s00376-014-3211-7 CrossRefGoogle Scholar
  61. Zhou, T., Li, Z.: Simulation of the East Asian summer monsoon using a variable resolution atmospheric GCM. Climate Dyn. 19, 167–180 (2002)CrossRefGoogle Scholar
  62. Zhou, T., Zou, L.: Understanding the predictability of East Asian Summer Monsoon from the reproduction of Land-Sea thermal contrast change in AMIP-type simulation. J. Climate 23(22), 6009–6026 (2010)CrossRefGoogle Scholar
  63. Zhou, T., Wu, B., Wen, X., Li, L., Wang, B.: A fast version of LASG/IAP climate system model and its 1000-year control integration. Adv. Atmos. Sci. 25, 655–672 (2008)CrossRefGoogle Scholar
  64. Zhou, T., Gong, D., Li, J., Li, B.: Detecting and understanding the multi-decadal variability of the East Asian Summer Monsoon: recent progress and state of affairs. Meteorol. Z. 18(4), 455–467 (2009a)CrossRefGoogle Scholar
  65. Zhou, T., Wu, B., Wang, B.: How well do atmospheric general circulation models capture the leading modes of the interannual variability of the Asian-Australian Monsoon? J. Climate 22, 1159–1173 (2009b)CrossRefGoogle Scholar
  66. Zhou, T., Song, F., Lin, R., Chen, X., Chen, X.: The 2012 North China floods: explaining an extreme rainfall event in the context of a long-term drying tendency [in Explaining extreme events of 2012 from a climate perspective]. Bull. Am. Meteorol. Soc. 94(9), S49–S51 (2013)Google Scholar
  67. Zhou T., Chen X., Dong L., Wu B., Man W., Zhang L., Lin R., Yao J., Song F., Zhao C.: Chinese contribution to CMIP5: an overview of five Chinese models’ performances. J. Meteorol. Res. 28(4), 481–509 (2014). doi: 10.1007/s13351-014-4001-y

Copyright information

© Springer Science+Business Media Singapore 2016

Authors and Affiliations

  • Tianjun Zhou
    • 1
    Email author
  • Bo Wu
    • 2
  • Yunying Li
    • 2
  • Hailong Liu
    • 2
  • Lijuan Li
    • 2
  • Lixia Zhang
    • 2
  • Fengfei Song
    • 2
  • Chongbo Zhao
    • 2
  • Lu Dong
    • 2
  • Chao He
    • 2
  • Yi Zhang
    • 2
  • Weihua Yuan
    • 2
  1. 1.Institute of Atmospheric Physics (IAP)Chinese Academy of SciencesBeijingChina
  2. 2.Institute of Atmospheric PhysicsChinese Academy of SciencesBeijingChina

Personalised recommendations