Assessing the fidelity of predictability estimates
- 267 Downloads
Predictability is an intrinsic limit of the climate system due to uncertainty in initial conditions and the chaotic nature of the atmosphere. Estimates of predictability together with calculations of current prediction skill are used to define the gaps in our prediction capabilities, inform future model developments, and indicate to stakeholders the potential for making forecasts that can inform their decisions. The true predictability of the climate system is not known and must be estimated, typically using a perfect model estimate from an ensemble prediction system. However, different prediction systems can give different estimates of predictability. Can we determine which estimate of predictability is most representative of the true predictability of the climate system? We test three metrics as potential indicators of the fidelity of predictability estimates in an idealized framework—the spread-error relationship, autocorrelation and skill. Using the North American multi-model ensemble re-forecast database, we quantify whether these metrics accurately indicate a model’s ability to properly estimate predictability. It is found that none of these metrics is a robust measure for determining whether a predictability estimate is realistic for El Nino-Southern oscillation events. For temperature and precipitation over land, errors in the spread-error ratio are related to errors in estimating predictability at the shortest lead-times, while skill is not related to predictability errors. The relationship between errors in the autocorrelation and errors in estimating predictability varies by lead-time and region.
Constructive comments from three anonymous reviewers helped to improve a previous version of this manuscript. We are also grateful to Dr. Laurie Ternary for her assistance with LaTeX. This study was supported by NOAA’s Climate Program Office’s Modeling, Analysis, Predictions, and Projections Program, Grant #NA15OAR4310072. We acknowledge the agencies that support the NMME system, and we thank the climate modeling groups (Environment Canada, NASA, NCAR, NOAA/GFDL, NOAA/NCEP, and University of Miami) for producing and making available their model output. NOAA/NCEP, NOAA/CTB, and NOAA/CPO jointly provided coordinating support and led development of the NMME system. Additional support was provided by the National Science Foundation (AGS-1338427), National Aeronautics and Space Administration (NNX14AM19G), the National Oceanic and Atmospheric Administration (NA14OAR4310160). The views expressed herein are those of the authors and do not necessarily reflect the views of these agencies.
- Chen M, Shi W, Xie P, Silva VBS, Kousky VE, Wayne Higgins R, Janowiak JE (2008) Assessing objective techniques for gauge-based analyses of global daily precipitation. J Geophys Res 113(D4):D04 (110–13) Google Scholar
- DeWitt DG (2005) Diagnosis of the tropical Atlantic near-equatorial SST bias in a directly coupled atmosphere–ocean general circulation model. Geophys Res Lett 32(L01):703Google Scholar
- Fan Y, Van Den Dool H (2008) A global monthly land surface air temperature analysis for 1948–present. J Geophys Res 113(D1):D01 (103–18) Google Scholar
- Infanti JM, Kirtman BP (2013) Southeast US rainfall prediction in the North American multi-model ensemble. J Hydrometeorol p 131031134244006Google Scholar
- Kirtman BP, Min D, Infanti JM, Kinter JL III, Paolino DA, Zhang Q, Van Den Dool H, Saha S, Mendez MP, Becker E, Peng P, Tripp P, Huang J, DeWitt DG, Tippett MK, Barnston AG, Li S, Rosati A, Schubert SD, Rienecker M, Suarez M, Li ZE, Marshak J, Lim YK, Tribbia J, Pegion K, Merryfield WJ, Denis B, Wood EF (2014) The North American multimodel ensemble: phase-1 seasonal-to-interannual prediction; phase-2 toward developing intraseasonal prediction. Bull Am Meteorol Soc 95(4):585–601CrossRefGoogle Scholar
- Palmer TN, Williams PD (2008) Introduction. Stochastic physics and climate modelling. Philos Trans R Soc A Math Phys Eng Sci 366:2421–2427Google Scholar
- Saha S, Moorthi S, Pan HL, Wu X, Wang J, Nadiga S, Tripp P, Kistler R, Woollen J, Behringer D, Liu H, Stokes D, Grumbine R, Gayno G, Wang J, Hou YT, Chuang HY, Juang HMH, Sela J, Iredell M, Treadon R, Kleist D, Van Delst P, Keyser D, Derber J, Ek M, Meng J, Wei H, Yang R, Lord S, Van Den Dool H, Kumar A, Wang W, Long C, Chelliah M, Xue Y, Huang B, Schemm JK, Ebisuzaki W, Lin R, Xie P, Chen M, Zhou S, Higgins W, Zou CZ, Liu Q, Chen Y, Han Y, Cucurull L, Reynolds RW, Rutledge G, Goldberg M (2010) The NCEP climate forecast system reanalysis. Bull Am Meteorol Soc 91(8):1015–1057CrossRefGoogle Scholar
- Tang Y, Lin H, Moore AM (2008) Measuring the potential predictability of ensemble climate predictions. J Geophys Res 113(D04):108Google Scholar
- Vernieres G, Rienecker M, Kovach CL, Robin ad Keppenne, (2013) Technical report series on global modeling and data assimilation, vol 30. Tech. rep, NASAGoogle Scholar
- Weisheimer A, Doblas-Reyes FJ, Palmer TN, Alessandri A, Arribas A, Déqué M, Keenlyside N, MacVean M, Navarra A, Rogel P (2009) ENSEMBLES: a new multi-model ensemble for seasonal-to-annual predictions–skill and progress beyond DEMETER in forecasting tropical Pacific SSTs. Geophys Res Lett 36(21):L21 (711) Google Scholar