Uncertainty component estimates in transient climate projections

Precision of estimators in a single time or time series approach
  • Benoit HingrayEmail author
  • Juliette Blanchet
  • Guillaume Evin
  • Jean-Philippe Vidal


Quantifying model uncertainty and internal variability components in climate projections has been paid a great attention in the recent years. For multiple synthetic ensembles of climate projections, we compare the precision of uncertainty component estimates obtained respectively with the two Analysis of Variance (ANOVA) approaches mostly used in recent works: the popular Single Time approach (STANOVA), based on the data available for the considered projection lead time and a time series based approach (QEANOVA), which assumes quasi-ergodicity of climate outputs over the available simulation period. We show that the precision of all uncertainty estimates is higher when more members are used, when internal variability is smaller and/or the response-to-uncertainty ratio is higher. QEANOVA estimates are much more precise than STANOVA ones: QEANOVA simulated confidence intervals are roughly 3–5 times smaller than STANOVA ones. Except for STANOVA when less than three members is available, the precision is rather high for total uncertainty and moderate for internal variability estimates. For model uncertainty or response-to-uncertainty ratio estimates, the precision is low for QEANOVA to very low for STANOVA. In the most unfavorable configurations (small number of members, large internal variability), large over- or underestimation of uncertainty components is thus very likely. In a number of cases, the uncertainty analysis should thus be preferentially carried out with a time series approach or with a local-time series approach, applied to all predictions available in the temporal neighborhood of the target prediction lead time.


Uncertainty sources Climate projections ANOVA Internal variability Model uncertainty Scenario uncertainty Precision of estimates 



We thank the three anonymous reviewers for their constructive suggestions which helped to significantly improve the content of our manuscript.

Author contributions

BH designed the analysis, developed the local-QEANOVA and the synthetic simulations. JB derived the theoretical expressions for unbiased estimators of uncertainty components and wrote the appendixes. All authors contributed to write the manuscript and discuss results.

Supplementary material

382_2019_4635_MOESM1_ESM.pdf (1 mb)
Supplementary material 1 (PDF 1038 KB)


  1. Bosshard T, Carambia M, Goergen K, Kotlarski S, Krahe P, Zappa M, Schaer C (2013) Quantifying uncertainty sources in an ensemble of hydrological climate-impact projections. Water Resour Res 49(3):1523–1536CrossRefGoogle Scholar
  2. Bracegirdle TJ, Turner J, Hosking JS, Phillips T (2014) Sources of uncertainty in projections of twenty-first century westerly wind changes over the Amundsen Sea, West Antarctica, in CMIP5 climate models. Clim Dyn 43(7–8):2093–2104CrossRefGoogle Scholar
  3. Braun M, Caya D, Frigon A, Slivitzky M (2012) Internal variability of the Canadian RCMs hydrological variables at the basin scale in Quebec and Labrador. J Hydrometeorol 13(2):443–462CrossRefGoogle Scholar
  4. Charlton-Perez AJ, Hawkins E, Eyring V, Cionni I, Bodeker GE, Kinnison DE, Akiyoshi H, Frith SM, Garcia R, Gettelman A, Lamarque JF, Nakamura T, Pawson S, Yamashita Y, Bekki S, Braesicke P, Chipperfield MP, Dhomse S, Marchand M, Mancini E, Morgenstern O, Pitari G, Plummer D, Pyle JA, Rozanov E, Scinocca J, Shibata K, Shepherd TG, Tian W, Waugh DW (2010) The potential to narrow uncertainty in projections of stratospheric ozone over the 21st century. Atmos Chem Phys 10(19):9473–9486CrossRefGoogle Scholar
  5. Cleveland SW (1979) Robust locally weighted regression and smoothing scatterplots. J Am Stat Assoc 74(368):829–836CrossRefGoogle Scholar
  6. Deser C, Phillips A, Bourdette V, Teng H (2012) Uncertainty in climate change projections: the role of internal variability. Clim Dyn 38(3–4):527–546CrossRefGoogle Scholar
  7. Evin G, Eckert N, Hingray B, Verfaillie D, Morin S, Lafaysse M, Blanchet J (2018) Traiter l’incertitude des projections climatiques: l’exemple des conditions meteorologiques et d’enneigement dans les Alpes francaises. Schweizerische Zeitschrift fur Forstwesen 169(4):203–209CrossRefGoogle Scholar
  8. Gelman A (2005) Analysis of variance—Why it is more important than ever. Ann Stat 33(1):1–53CrossRefGoogle Scholar
  9. Giuntoli J, Vidal JP, Prudhomme C, Hannah DM (2015) Future hydrological extremes: the uncertainty from multiple global climate and global hydrological models. Earth Syst Dyn 16:267–285CrossRefGoogle Scholar
  10. Hartley HO, Rao JNK (1967) Maximum-likelihood estimation for the mixed analysis of variance model. Biometrika 54(1/2):93–108CrossRefGoogle Scholar
  11. Hawkins E, Sutton R (2009) The potential to narrow uncertainty in regional climate predictions. Bull Am Meteorol Soc 90(8):1095–1107CrossRefGoogle Scholar
  12. Hawkins E, Sutton R (2011) The potential to narrow uncertainty in projections of regional precipitation change. Clim Dyn 37(1–2):407–418CrossRefGoogle Scholar
  13. Hingray B, Blanchet J (2018) Uncertainty components estimates in transient climate projections. bias of moment-based estimators in the single time and time series approaches. Tech. rep., IGE, Univ-Grenoble Alpes, Grenoble, FR.
  14. Hingray B, Saïd M (2014) Partitioning internal variability and model uncertainty components in a multimodel multimember ensemble of climate projections. J Clim 27(17):6779–6798CrossRefGoogle Scholar
  15. Hingray B, Mezghani A, Buishand TA (2007) Development of probability distributions for regional climate change from uncertain global mean warming and an uncertain scaling relationship. Hydrol Earth Syst Sci 11(3):1097–1114CrossRefGoogle Scholar
  16. Jacob D, Petersen J, Eggert B, Alias A, Christensen OB, Bouwer LM, Braun A, Colette A, Deque M, Georgievski G, Georgopoulou E, Gobiet A, Menut L, Nikulin G, Haensler A, Hempelmann N, Jones C, Keuler K, Kovats S, Kroener N, Kotlarski S, Kriegsmann A, Martin E, van Meijgaard E, Moseley C, Pfeifer S, Preuschmann S, Radermacher C, Radtke K, Rechid D, Rounsevell M, Samuelsson P, Somot S, Soussana JF, Teichmann C, Valentini R, Vautard R, Weber B, Yiou P (2014) EURO-CORDEX: new high-resolution climate change projections for European impact research. Reg Environ Change 14(2):563–578CrossRefGoogle Scholar
  17. Johns T, Royer JF, Höschel I, Huebener H, Roeckner E, Manzini E, May W, Dufresne JL, Otteå O, Vuuren D, Salas y Melia D, Giorgetta M, Denvil S, Yang S, Fogli P, Körper J, Tjiputra J, Stehfest E, Hewitt C (2011) Climate change under aggressive mitigation: the ensembles multi-model experiment. Clim Dyn 37(9–10):1975–2003CrossRefGoogle Scholar
  18. Kendon EJ, Rowell DP, Jones RG, Buonomo E (2008) Robustness of future changes in local precipitation extremes. J Clim 21(17):4280–4297CrossRefGoogle Scholar
  19. Kew SF, Selten FM, Lenderink G, Hazeleger W (2011) Robust assessment of future changes in extreme precipitation over the Rhine basin using a GCM. Hydrol Earth Syst Sci 15(4):1157–1166CrossRefGoogle Scholar
  20. Lafaysse M, Hingray B, Mezghani A, Gailhard J, Terray L (2014) Internal variability and model uncertainty components in a multireplicate multimodel ensemble of hydrometeorological projections : the Alpine Durance basin. Water Resour Res 50:3317–3341CrossRefGoogle Scholar
  21. Montgomery DC (2012) Design and analysis of experiments, 8th edn. Wiley, HobokenGoogle Scholar
  22. Northrop PJ, Chandler RE (2014) Quantifying sources of uncertainty in projections of future climate. J Clim 27(23):8793–8808CrossRefGoogle Scholar
  23. Olonscheck D, Notz D (2017) Consistently estimating internal climate variability from climate model simulations. J Clim 30(23):9555–9573. CrossRefGoogle Scholar
  24. Paeth H, Vogt G, Paxian A, Hertig E, Seubert S, Jacobeit J (2017) Quantifying the evidence of climate change in the light of uncertainty exemplified by the Mediterranean hot spot region. Global Planet Change 151(SI):144–151. CrossRefGoogle Scholar
  25. van Pelt SC, Beersma JJ, Buishand TA, van den Hurk BJJM, Schellekens J (2015) Uncertainty in the future change of extreme precipitation over the Rhine basin: the role of internal climate variability. Clim Dyn 44(7–8):1789–1800CrossRefGoogle Scholar
  26. Räisänen J (2001) CO2-induced climate change in CMIP2 experiments: quantification of agreement and role of internal variability. J Clim 14(9):2088–2104CrossRefGoogle Scholar
  27. Reintges A, Martin T, Latif M, Keenlyside NS (2017) Uncertainty in twenty-first century projections of the Atlantic Meridional Overturning Circulation in CMIP3 and CMIP5 models. Clim Dyn 49(5–6):1495–1511. CrossRefGoogle Scholar
  28. Vidal JP, Hingray B, Magand C, Sauquet E, Ducharne A (2016) Hierarchy of climate and hydrological uncertainties in transient low-flow projections. Hydrol Earth Syst Sci 20(9):3651–3672. CrossRefGoogle Scholar
  29. Yip S, Ferro CAT, Stephenson DB, Hawkins E (2011) A simple, coherent framework for partitioning uncertainty in climate predictions. J Clim 24(17):4634–4643CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Benoit Hingray
    • 1
    Email author
  • Juliette Blanchet
    • 1
  • Guillaume Evin
    • 2
  • Jean-Philippe Vidal
    • 3
  1. 1.Univ. Grenoble Alpes, CNRS, IGE UMR 5001GrenobleFrance
  2. 2.Univ. Grenoble Alpes, Irstea, UR ETNAGrenobleFrance
  3. 3.Irstea, UR RiverLy, centre de Lyon-VilleurbanneVilleurbanneFrance

Personalised recommendations