Skip to main content
SpringerLink
Log in

Preliminary comparison of the CMA, ECMWF, NCEP, and JMA ensemble prediction systems

  • Published:
Acta Meteorologica Sinica Aims and scope Submit manuscript

Abstract

Based on The Observing System Research and Predictability Experiment (THORPEX) Interactive Grand Global Ensemble (TIGGE) dataset, using various verification methods, the performances of four typical ensemble prediction systems (EPSs) from the China Meteorological Administration (CMA), the European Centre for Medium-Range Weather Forecasts (ECMWF), the US National Centers for Environmental Prediction (NCEP), and the Japan Meteorological Agency (JMA) are compared preliminarily. The verification focuses on the 500-hPa geopotential height forecast fields in the mid- and high-latitude Eurasian region during July 2007 and January 2008. The results show that for the forecast of 500-hPa geopotential height, in both summer and winter, the ECMWF EPS exhibits the highest forecast skill, followed by that of NCEP, then by JMA, and the CMA EPS gets in the last. The better system behaviors benefit from the better combination of the following: data assimilation system, numerical models, initial perturbations, and stochastic model perturbations. For the medium-range forecast, the ensemble forecasting can effectively filter out the forecast errors associated with the initial uncertainty, and the reliability and resolution (the two basic attributions of the forecast system) of these EPSs are better in winter than in summer. Specifically, the CMA EPS has certain advantage on the reliability of ensemble probabilistic forecasts. The forecasts are easy to be underestimated by the JMA EPS. The deficiency of ensemble spread, which is the universal problem of EPS, also turns up in this study. Although the systems of ECMWF, NCEP, and JMA have more ensemble members, this problem cannot be ignored. This preliminary comparison helps to further recognize the prediction capability of the four EPSs over the Eurasian region, provides important references for wide applications of the TIGGE dataset, and supplies useful information for improving the CMA EPS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Atger, F., 1999: The skill of ensemble prediction systems. Mon. Wea. Rev., 127, 1941–1953.

    Article  Google Scholar 

  • Bougeault, P., Z. Toth, C. Bishop, et al., 2010: The THORPEX interactive grand global ensemble. Bull. Amer. Meteor. Soc., 91, 1059–1072.

    Article  Google Scholar 

  • Bourke, W., R. Buizza, and M. Naughton, 2004: Performance of the ECMWF and the BoM ensemble prediction systems in the Southern Hemisphere. Mon. Wea. Rev., 132, 2338–2357.

    Article  Google Scholar 

  • Buizza, R., and T. N. Palmer, 1995: The singular-vector structure of the atmospheric global circulation. J. Atmos. Sci., 52, 1434–1456.

    Article  Google Scholar 

  • —, M. Miller, and T. N. Palmer, 1999: Stochastic representation of model uncertainties in the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 125, 2887–2908.

    Article  Google Scholar 

  • —, P. L. Houtekamer, Z. Toth, et al., 2005: A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems. Mon. Wea. Rev., 133, 1076–1097.

    Article  Google Scholar 

  • Chen Jing, Xue Jishan, and Yan Hong, 2005: A new initial perturbation method of ensemble mesoscale heavy rain prediction. J. Appl. Meteor. Sci., 29, 717–726. (in Chinese)

    Google Scholar 

  • Duan Mingkeng, 2006: Ensemble forecasting performance studies on the important medium-range circulation processes affecting China. Ph. D. dissertation, Nanjing University of Information Science & Technology, China, 144 pp. (in Chinese)

    Google Scholar 

  • — and Wang Panxing, 2006: A new weighted average method on ensemble mean forecasting. J. Appl. Meteor. Sci., 17, 488–493. (in Chinese)

    Google Scholar 

  • —, —, Wu Hongbao, et al., 2009: The ensemble forecasting verification on the summer Eurasian middlehigh latitude circulation. J. Appl. Meteor. Sci., 20, 56–61. (in Chinese)

    Google Scholar 

  • Froude, L. S. R., 2010: TIGGE: Comparison of the prediction of Northern Hemisphere extratropical cyclones by different ensemble prediction systems. Wea. Forecasting, 25, 819–836.

    Article  Google Scholar 

  • —, L. Bengtsson, and K. I. Hodges, 2007: The prediction of extratropical storm tracks by the ECMWF and NCEP ensemble prediction systems. Mon. Wea. Rev., 135, 2545–2567.

    Article  Google Scholar 

  • Gong, J. D., W. J. Li, and J. F. Chou, 1999: Forming proper ensemble forecast initial members with four-dimensional variational data assimilation method. Chinese Sci. Bull., 44, 1527–1531.

    Article  Google Scholar 

  • Hamill, T. M., 1997: Reliability diagrams for multicategory probabilistic forecasts. Wea. Forecasting, 12, 736–741.

    Article  Google Scholar 

  • —, 2001: Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev., 129, 550–560.

    Article  Google Scholar 

  • Houtekamer, P. L., L. Lefaivre, J. Derome, et al., 1996: A system simulation approach to ensemble prediction. Mon. Wea. Rev., 124, 1225–1242.

    Article  Google Scholar 

  • Huangfu Xueguan, 2002: Verification of the ensemble prediction system of the National Meteorological Center. J. Appl. Meteor. Sci., 13, 29–36. (in Chinese)

    Google Scholar 

  • Jin Ronghua, Li Weijing, and Sun Zhaobo, 2002: Study on the interpretation method of ensemble prediction products. J. Appl. Meteor. Sci., 25, 442–448. (in Chinese)

    Google Scholar 

  • Kalnay, E., M. Kanamitsu, R. Kistler, et al., 1996: The NCEP/NCAR 40-year reanalysis project. Bull. Amer. Meteor. Soc., 77, 437–472.

    Article  Google Scholar 

  • Kistler, R., E. Kalnay, W. Collins, et al., 2001: The NCEP/NCAR 50-year reanalysis: Monthly means CD-ROM and documentation. Bull. Amer. Meteor. Soc., 82, 247–268.

    Article  Google Scholar 

  • Leith, C. E., 1974: Theoretical skill of Monte Carlo forecasts. Mon. Wea. Rev., 102, 409–418.

    Article  Google Scholar 

  • Liu Huanzhu and Zhang Shaoqing, 1992: Statistical veri-fication on the medium-range numerical forecasting. Meteor. Mon., 18, 50–54. (in Chinese)

    Google Scholar 

  • Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20, 130–141.

    Article  Google Scholar 

  • Ma Xulin, Xue Jishan, and Lu Weisong, 2009: Study on ETKF-based initial perturbation scheme for GRAPES global ensemble prediction. Acta Meteor. Sinica, 23, 562–574.

    Google Scholar 

  • Molteni, F., R. Buizza, T. N. Palmer, et al., 1996: The ECMWF ensemble prediction system: Methodology and validation. Quart. J. Roy. Meteor. Soc., 122, 73–120.

    Article  Google Scholar 

  • Mu, M., and Z. N. Jiang, 2008: A new approach to the generation of initial perturbations for ensemble prediction: Conditional nonlinear optimal perturbation. Chinese Sci. Bull., 53, 2062–2068.

    Article  Google Scholar 

  • Mullen, S. L., and R. Buizza, 2001: Quantitative precipitation forecasts over the United States by the ECMWF ensemble prediction system. Mon. Wea. Rev., 129, 638–663.

    Article  Google Scholar 

  • Park, Y. Y., R. Buizza, and M. Leutbecher, 2008: TIGGE: Preliminary reliminary results on comparing and combining ensembles. Quart. J. Roy. Meteor. Soc., 134, 2029–2050.

    Article  Google Scholar 

  • Richardson, D., 2000: Skill and relative economic value of the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 126, 649–667.

    Article  Google Scholar 

  • — —, R. Buizza, and R. Hagedorn, 2005: Final report of the 1st Workshop on the THORPEX Interactive Grand Global Ensemble (TIGGE), WMO/TDNo. 1273, WWRP/THORPEX, No.5, 39 pp.

  • Shapiro, M., and A. Thorpe, 2004: THORPEX International Science Plan. WMO/TD-No. 1246, WWRP/THORPEX, No. 2, 51 pp.

  • Talagrand, O., R. Vautard, and B. Strauss, 1997: Evaluation of probabilistic prediction systems. Proceedings of the ECMWF Workshop on Predictability, Reading, United Kingdom, ECMWF, 1–25.

  • Titley, H., N. Savage, R. Swinbank, et al., 2008: Comparison between Met Office and ECMWF medium-range ensemble forecast systems. Meteorology Research and Development Technical Report No. 512, Met Office, United Kingdom, 41 pp.

    Google Scholar 

  • Tracton, M. S., and E. Kalnay, 1993: Operational ensemble prediction at the National Meteorological Center: Practical aspects. Wea. Forecasting, 8, 379–398.

    Article  Google Scholar 

  • Toth, Z., and E. Kalnay, 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc., 74, 2317–2330.

    Article  Google Scholar 

  • —, O. Talagrand, and Y. Zhu, 2006: The attributes of forecast systems: A general framework for the evaluation and calibration of weather forecasts. Predictability of Weather and Climate, T. N. Palmer and R. Hagedorn, Eds., Cambridge University Press, 584–595.

  • —, Y. Zhu, and T. Marchok, 2001: The use of ensembles to identify forecasts with small and large uncertainty. Wea. Forecasting, 16, 463–477.

    Article  Google Scholar 

  • Wang Chenxi, Yao Jianqun, and Liang Xudong, 2007: The establishment and verification of the operational ensemble forecast system for Shanghai regional precipitation. J. Appl. Meteor. Sci., 18, 173–180. (in Chinese)

    Google Scholar 

  • Wei, M., and Z. Toth, 2003: A new measure of ensemble performance: Perturbation versus error correlation analysis (PECA). Mon. Wea. Rev., 131, 1549–1565.

    Article  Google Scholar 

  • Zhou Bing, Zhao Cuiguang, and Zhao Shengrong, 2006: Multi-model ensemble forecast technology with analysis and verification of the results. J. Appl. Meteor. Sci., 17, 104–109. (in Chinese)

    Google Scholar 

  • Zhu, Y., Z. Toth, R. Wobus, et al., 2002: The economic value of ensemble-based weather forecasts. Bull. Amer. Meteor. Soc., 83, 73–83.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Key Laboratory of Meteorological Disaster of Ministry of Education, Nanjing University of Information Science & Technology, Nanjing, 210044, China

    Mingkeng Duan  (段明铿), Juhui Ma  (麻巨慧) & Panxing Wang  (王盘兴)

Authors
  1. Mingkeng Duan  (段明铿)
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juhui Ma  (麻巨慧)
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Panxing Wang  (王盘兴)
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mingkeng Duan  (段明铿).

Additional information

Supported by the China Meteorological Administration PublicWelfare Research Fund (GYHY200706001 and GYHY200906007) and the Priority Academic Development Project of Jiangsu Higher Education Institutions.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Duan, M., Ma, J. & Wang, P. Preliminary comparison of the CMA, ECMWF, NCEP, and JMA ensemble prediction systems. Acta Meteorol Sin 26, 26–40 (2012). https://doi.org/10.1007/s13351-012-0103-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13351-012-0103-6

Key words

Search

Navigation