Climatic Change

, Volume 154, Issue 3–4, pp 367–386 | Cite as

Identifying credible and diverse GCMs for regional climate change studies—case study: Northeastern United States

  • Ambarish V. KarmalkarEmail author
  • Jeanne M. Thibeault
  • Alexander M. Bryan
  • Anji Seth


Climate data obtained from global climate models (GCMs) form the basis of most studies of regional climate change and its impacts. Using the northeastern U.S. as a test case, we develop a framework to systematically sub-select reliable models for use in climate change studies in the region. Model performance over the historical period is evaluated first for a wide variety of standard and process metrics including large-scale atmospheric circulation features that drive regional climate variability. The inclusion of process-based metrics allows identification of credible models in capturing key processes relevant for the climate of the northeastern U.S. Model performance is then used in conjunction with the assessment of redundancy in model projections, especially in summer precipitation, to eliminate models that have better performing counterparts. Finally, we retain some mixed-performing models to maintain the range of climate model uncertainty, required by the fact that model biases are not strongly related to their respective projections. This framework leads to the retention of 16 of 36 CMIP5 GCMs that (a) have a satisfactory historical performance for a variety of metrics and (b) provide diverse climate projections consistent with uncertainties in the multi-model ensemble (MME). Overall, the models show significant variations in their performance across metrics and seasons with none emerging as the best model in all metrics. The retained set reduces the number of models by more than one half, easing the computational burden of using the entire CMIP5 MME, while still maintaining a wide range of projections for risk assessment. The retention of some mixed-performing models to maintain ensemble uncertainty suggests a potential to narrow the ranges in temperature and precipitation. But any further refinement should be based on a more detailed analysis of models in capturing regional climate variability and extremes to avoid providing overconfident projections.



We acknowledge the WCRP Working Group on Coupled Modelling, and thank the climate modeling centers for producing and making available model output. We thank the three anonymous reviewers for their careful review and insightful comments that have helped improve the manuscript substantially.

Funding information

This research was supported by the U.S. DOI’s Northeast Climate Adaptation Science Center by Grant or Cooperative Agreement No. G12AC00001 from the United States Geological Survey (USGS) and in part by NSF CAREER Award No. 1056216. Its contents are solely the responsibility of the authors.

Supplementary material

10584_2019_2411_MOESM1_ESM.pdf (6.5 mb)
(PDF 6.47 MB)


  1. Barsugli JJ, Guentchev G, Horton RM, Wood A, Mearns LO, Liang XZ, Winkler JA, Dixon K, Hayhoe K, Rood RB et al (2013) The practitioner’s dilemma: how to assess the credibility of downscaled climate projections. Eos Trans Amer Geophys Union 94(46):424–425CrossRefGoogle Scholar
  2. Bradbury JA, Keim BD, Wake CP (2003) The influence of regional storm tracking and teleconnections on winter precipitation in the northeastern United States. Ann Assoc Am Geogr 93(3):544–556CrossRefGoogle Scholar
  3. Dee DP, Uppala S, Simmons A, Berrisford P, Poli P, Kobayashi S, Andrae U, Balmaseda M, Balsamo G, Bauer P et al (2011) The ERA-Interim reanalysis: Configuration and performance of the data assimilation system. Q J R Meteorol Soc 137(656):553–597CrossRefGoogle Scholar
  4. Deser C, Knutti R, Solomon S, Phillips AS (2012) Communication of the role of natural variability in future North American climate. Nat Clim Chang 2 (11):775–779CrossRefGoogle Scholar
  5. Deser C, Phillips AS, Alexander MA, Smoliak BV (2014) Projecting North American climate over the next 50 years: uncertainty due to internal variability*. J Clim 27(6):2271–2296CrossRefGoogle Scholar
  6. Gleckler PJ, Taylor KE, Doutriaux C (2008) Performance metrics for climate models. J Geophys Res Atmos 113(D06104).
  7. Harris I, Jones P, Osborn T, Lister D (2014) Updated high-resolution grids of monthly climatic observations–the CRU TS3. 10 dataset. Int J Climatol 34(3):623–642CrossRefGoogle Scholar
  8. Hawkins E, Sutton R (2009) The potential to narrow uncertainty in regional climate predictions. Bull Am Meteorol Soc 90(8):1095–1107CrossRefGoogle Scholar
  9. Horton R, Yohe G, Easterling W, Kates R, Ruth M, Sussman E, Whelchel A, Wolfe D, Lipschultz F (2014) Ch. 16: Northeast. Climate Change Impacts in the United States: The Third National Climate Assessment, pp 371–395Google Scholar
  10. Kalnay E, Kanamitsu M, Kistler R, Collins W, Deaven D, Gandin L, Iredell M, Saha S, White G, Woollen J et al (1996) The NCEP/NCAR 40-year reanalysis project. Bullet Amer Meteorol Soc 77(3):437–472CrossRefGoogle Scholar
  11. Karmalkar AV, Bradley RS (2017) Consequences of global warming of 1.5 C and 2 C for regional temperature and precipitation changes in the contiguous United States. PloS one 12(1):e0168697CrossRefGoogle Scholar
  12. Kay J, Deser C, Phillips A, Mai A, Hannay C, Strand G, Arblaster J, Bates S, Danabasoglu G, Edwards J et al (2015) The Community Earth System Model (CESM) large ensemble project: A community resource for studying climate change in the presence of internal climate variability. Bull Am Meteorol Soc 96(8):1333–1349CrossRefGoogle Scholar
  13. Knutti R, Furrer R, Tebaldi C, Cermak J, Meehl GA (2010) Challenges in combining projections from multiple climate models. J Clim 23(10):2739–2758CrossRefGoogle Scholar
  14. Knutti R, Masson D, Gettelman A (2013) Climate model genealogy: Generation CMIP5 and how we got there. Geophys Res Lett 40(6):1194–1199CrossRefGoogle Scholar
  15. Knutti R, Sedláček J, Sanderson BM, Lorenz R, Fischer EM, Eyring V (2017) A climate model projection weighting scheme accounting for performance and interdependence. Geophys Res Lett 44(4):1909– 1918Google Scholar
  16. Lynch C, Seth A, Thibeault J (2016) Recent and projected annual cycles of temperature and precipitation in the northeast United States from CMIP5. J Clim 29(1):347–365CrossRefGoogle Scholar
  17. Maloney ED, Camargo SJ, Chang E, Colle B, Fu R, Geil KL, Hu Q, Jiang X, Johnson N, Karnauskas KB et al (2014) North American climate in CMIP5 experiments: Part III: Assessment of twenty-first-century projections. J Clim 27(6):2230–2270CrossRefGoogle Scholar
  18. Masson D, Knutti R (2011) Climate model genealogy. Geophys Res Lett 38(L08703).
  19. Matsuura K, Willmott CJ (2012) Terrestrial precipitation: 1900-2010 gridded monthly time series (v. 3.01). Center for Climatic Research, Department of Geography, University of Delaware Newark, DE, USAGoogle Scholar
  20. McSweeney CF, Jones RG, Booth BB (2012) Selecting ensemble members to provide regional climate change information. J Clim 25(20):7100–7121CrossRefGoogle Scholar
  21. McSweeney C, Jones R, Lee R, Rowell D (2015) Selecting CMIP5 GCMs for downscaling over multiple regions. Clim Dyn 44(11-12):3237–3260CrossRefGoogle Scholar
  22. Meinshausen M, Smith SJ, Calvin K, Daniel JS, Kainuma M, Lamarque JF, Matsumoto K, Montzka S, Raper S, Riahi K et al (2011) The RCP greenhouse gas concentrations and their extensions from 1765 to 2300. Clim Change 109(1-2):213CrossRefGoogle Scholar
  23. Monerie PA, Sanchez-Gomez E, Boé J (2017) On the range of future Sahel precipitation projections and the selection of a sub-sample of CMIP5 models for impact studies. Clim Dyn 48(7-8):2751–2770CrossRefGoogle Scholar
  24. Mote P, Brekke L, Duffy PB, Maurer E (2011) Guidelines for constructing climate scenarios. Eos Trans Amer Geophys Union 92(31):257–258CrossRefGoogle Scholar
  25. Mote PW, Allen MR, Jones RG, Li S, Mera R, Rupp DE, Salahuddin A, Vickers D (2016) Superensemble regional climate modeling for the western United States. Bull Am Meteorol Soc 97(2):203–215CrossRefGoogle Scholar
  26. Ning L, Bradley RS (2015) Winter climate extremes over the northeastern United States and southeastern Canada and teleconnections with large-scale modes of climate variability. J Clim 28(6):2475–2493CrossRefGoogle Scholar
  27. Overland JE, Wang M, Bond NA, Walsh JE, Kattsov VM, Chapman WL (2011) Considerations in the selection of global climate models for regional climate projections: The Arctic as a case study. J Clim 24(6):1583–1597CrossRefGoogle Scholar
  28. Pierce DW, Barnett TP, Santer BD, Gleckler PJ (2009) Selecting global climate models for regional climate change studies. Proc Natl Acad Sci 106(21):8441–8446CrossRefGoogle Scholar
  29. Rousseeuw PJ (1987) Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 20:53–65CrossRefGoogle Scholar
  30. Rowell DP, Senior CA, Vellinga M, Graham RJ (2016) Can climate projection uncertainty be constrained over Africa using metrics of contemporary performance? Clim Chang 134(4):621–633CrossRefGoogle Scholar
  31. Rupp DE, Abatzoglou JT, Hegewisch KC, Mote PW (2013) Evaluation of CMIP5 20th century climate simulations for the Pacific Northwest USA. J Geophys Res: Atmos 118(19):10–884Google Scholar
  32. Sanderson BM, Knutti R, Caldwell P (2015) A representative democracy to reduce interdependency in a multimodel ensemble. J Clim 28(13):5171–5194CrossRefGoogle Scholar
  33. Seager R, Naik N, Vecchi GA (2010) Thermodynamic and dynamic mechanisms for large-scale changes in the hydrological cycle in response to global warming. J Clim 23(17):4651–4668CrossRefGoogle Scholar
  34. Sheffield J, Barrett AP, Colle B, Nelun Fernando D, Fu R, Geil KL, Hu Q, Kinter J, Kumar S, Langenbrunner B et al (2013) North American climate in CMIP5 experiments. Part I: evaluation of historical simulations of continental and regional climatology. J Clim 26(23):9209–9245CrossRefGoogle Scholar
  35. Snover AK, Mantua NJ, Littell JS, Alexander MA, Mcclure MM, Nye J (2013) Choosing and using climate-change scenarios for ecological-impact assessments and conservation decisions. Conserv Biol 27(6):1147–1157CrossRefGoogle Scholar
  36. Staudinger MD, Morelli TL, Bryan AM (2015) Integrating climate change into northeast and midwest state wildlife action plans. DOI Northeast Climate Science Center Report, AmherstGoogle Scholar
  37. Taylor KE (2001) Summarizing multiple aspects of model performance in a single diagram. J Geophys Res: Atmos 106(D7):7183–7192CrossRefGoogle Scholar
  38. Taylor KE, Stouffer RJ, Meehl GA (2012) An overview of CMIP5 and the experiment design. Bull Am Meteorol Soc 93(4):485–498CrossRefGoogle Scholar
  39. Tebaldi C, Knutti R (2007) The use of the multi-model ensemble in probabilistic climate projections. Philosophical Transactions of the Royal Society of London A: Mathematical. Phys Eng Sci 365(1857):2053–2075CrossRefGoogle Scholar
  40. Thibeault JM, Seth A (2014) A framework for evaluating model credibility for warm-season precipitation in northeastern North America: a case study of CMIP5 simulations and projections. J Clim 27(2):493–510CrossRefGoogle Scholar
  41. Thibeault JM, Seth A (2015) Toward the credibility of Northeast United States summer precipitation projections in CMIP5 and NARCCAP simulations. J Geophys Res Atmos 120(19):10050–10073.
  42. Vano JA, Kim JB, Rupp DE, Mote PW (2015) Selecting climate change scenarios using impact-relevant sensitivities. Geophys Res Lett 42(13):5516–5525CrossRefGoogle Scholar
  43. Wallace JM, Deser C, Smoliak BV, Phillips AS (2016) Attribution of climate change in the presence of internal variability. In: Climate change: Multidecadal and Beyond, World Scientific, pp 1–29Google Scholar
  44. Weigel AP, Knutti R, Liniger MA, Appenzeller C (2010) Risks of model weighting in multimodel climate projections. J Clim 23(15):4175–4191CrossRefGoogle Scholar
  45. Whetton P, Hennessy K, Clarke J, McInnes K, Kent D (2012) Use of representative climate futures in impact and adaptation assessment. Clim Change 115(3-4):433–442CrossRefGoogle Scholar
  46. Wuebbles D, Fahey D, Hibbard K, Dokken B, Stewart B, Maycock T (2017) Climate science special report: Fourth national climate assessment, Volume I. In: Washington, DC, pp 470Google Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Northeast Climate Adaptation Science Center and Department of GeosciencesUniversity of Massachusetts Amherst, MassachusettsAmherstUSA
  2. 2.Department of GeographyUniversity of ConnecticutStorrsUSA
  3. 3.Northeast Climate Adaptation Science Center, U.S. Geological SurveyUniversity of Massachusetts Amherst, MassachusettsAmherstUSA

Personalised recommendations