Climatic Change

, Volume 152, Issue 3–4, pp 431–448 | Cite as

A framework for testing dynamic classification of vulnerable scenarios in ensemble water supply projections

  • Bethany RobinsonEmail author
  • Jonathan D. Herman


Recent water resources planning studies have proposed climate adaptation strategies in which infrastructure and policy actions are triggered by observed thresholds or “signposts.” However, the success of such strategies depends on whether thresholds can be accurately linked to future vulnerabilities. This study presents a framework for testing the ability of adaptation thresholds to dynamically identify vulnerable scenarios within ensemble projections. Streamflow projections for 91 river sites predominantly in the western USA are used as a case study in which vulnerability is determined by the ensemble members with the lowest 10% of end-of-century mean annual flow. Illustrative planning thresholds are defined through time for each site based on the mean streamflow below which a specified fraction of scenarios is vulnerable. We perform a leave-one-out cross-validation to compute the frequency of incorrectly identifying or failing to identify a vulnerable scenario (false positives and false negatives, respectively). Results show that in general, this method of defining thresholds can identify vulnerable scenarios with low false positive rates (< 10%), but with false negative rates for many rivers remaining higher than random chance until roughly 2060. This finding highlights the tradeoff between frequently triggering unnecessary action and failing to identify potential vulnerabilities until later in the century, and suggests room for improvement in the threshold-setting technique that could be benchmarked with this approach. This testing framework could extend to thresholds defined with multivariate statistics, or to any application using thresholds and ensemble projections, such as long-term flood and drought risk, or sea level rise.



Any opinions, findings, and conclusions are those of the authors and do not necessarily reflect the views or policies of the NSF. We further acknowledge the World Climate Research Program’s Working Group on Coupled Modeling and the climate modeling groups listed in the Supplement of this paper for producing and making available their model output.

Funding information

This work was partially supported by the U.S. National Science Foundation grants CNS-1639268 and CNH-1716130.

Supplementary material

10584_2018_2347_MOESM1_ESM.pdf (822 kb)
ESM 1 (PDF 821 kb)


  1. Adger WN, Arnell NW, Tompkins EL (2005) Successful adaptation to climate change across scales. Glob Environ Chang 15(2):77–86. CrossRefGoogle Scholar
  2. Anderson J, Chung F, Anderson M, Brekke L, Easton D, Ejeta M, Peterson R, Snyder R (2008) Progress on incorporating climate change into management of California’s water resources. Clim Chang 87(1 SUPPL).
  3. Barnett TP, Pierce DW, Hidalgo HG, Bonfils C, Santer BD, Das T, Bala G, Wood AW, Nozawa T, Mirin A a, Cayan DR, Dettinger MD (2008) Human-induced changes in the hydrology of the Western United States. Science 319:1080–1083. CrossRefGoogle Scholar
  4. Borgomeo E, Farmer CL, Hall JW (2015) Numerical rivers: a synthetic streamflow generator for water resrouces vulnerability assessments. Water Resour Res 51:5382–5405. CrossRefGoogle Scholar
  5. Brekke L, Wood A, Pruitt T (2014) Downscaled CMIP3 and CMIP5 hydrology climate projections: release of hydrology projections, comparison with preceding information, and summary of user needs, US Bureau of Reclamation. Available at:
  6. Brown C, Ghile Y, Laverty M, Li K (2012) Decision scaling: linking bottom-up vulnerability analysis with climate projections in the water sector. Water Resour Res 48(9):1–12. CrossRefGoogle Scholar
  7. Bryant BP, Lempert RJ (2010) Thinking inside the box: a participatory, computer-assisted approach to scenario discovery. Technol Forecast Soc Chang 77(1):34–49. CrossRefGoogle Scholar
  8. Buurman J, Babovic V (2016) Adaptation pathways and real options analysis: an approach to deep uncertainty in climate change adaptation policies. Policy Soc 35(2):137–150. CrossRefGoogle Scholar
  9. Cayan DR, Kammerdiener SA, Dettinger MD, Caprio JM, Peterson DH (2001) Changes in the onset of spring in the Western United States. Bull Am Meteorol Soc 82(3):399–416CrossRefGoogle Scholar
  10. Dessai S, Hulme M (2004) Does climate adaptation policy need probabilities? Clim Pol 4(2):107–128. CrossRefGoogle Scholar
  11. DiFrancesco KN, Tullos DD (2014) Flexibility in water resources management: review of concepts and development of assessment measures for flood management systems. J Am Water Resour Assoc 50(6):1527–1539. CrossRefGoogle Scholar
  12. Donat MG et al (2013) Updated analyses of temperature and precipitation extreme indices since the beginning of the twentieth century: the HadEX2 dataset. J Geophys Res Atmos 118(5):2098–2118. CrossRefGoogle Scholar
  13. Dottori F, Szewczyk W, Ciscar J-C, Zhao F, Alfieri L, Hirabayashi Y, Bianchi A, Mongelli I, Frieler K, Betts R, Feyen L (2018) Increased human and economic losses from river flooding with anthropogenic warming. Nat Clim Chang.
  14. Fletcher SM, Miotti M, Swaminathan J, Klemun MM, Strzepek K, Siddiqi A (2017) Water supply infrastructure planning: decision-making framework to classify multiple uncertainties and evaluate flexible design. J Water Resour Plan Manag 143(10):4017061. CrossRefGoogle Scholar
  15. Frigg R, Smith LA, Stainforth DA (2013) The Myopia of Imperfect Climate Models : The Case of UKCP09, 80(December):886–897Google Scholar
  16. Giuliani M, Castelletti A (2016) Is robustness really robust? How different definitions of robustness impact decision-making under climate change. Clim Chang 135(3–4):409–424. CrossRefGoogle Scholar
  17. Haasnoot M, Kwakkel JH, Walker WE, ter Maat J (2013) Dynamic adaptive policy pathways: a method for crafting robust decisions for a deeply uncertain world. Glob Environ Chang 23(2):485–498. CrossRefGoogle Scholar
  18. Hallegatte S (2009) Strategies to adapt to an uncertain climate change. Glob Environ Chang 19(2):240–247. CrossRefGoogle Scholar
  19. Hallegatte S, Shah A, Lempert R, Brown C, Gill S (2012) Investment decision making under deep uncertainty: application to climate change, Policy Research Working Paper, (6193), p 41.
  20. Herman JD, Giuliani M (2018) Policy tree optimization for threshold-based water resources management over multiple timescales. Environ Model Softw 99:39–51. CrossRefGoogle Scholar
  21. Herman JD, Reed PM, Zeff HB, Characklis GW (2015) How should robustness be defined for water systems planning under change? J Water Resour Plan Manag 141(10):4015012. CrossRefGoogle Scholar
  22. Herman JD, Zeff HB, Lamontagne JR, Reed PM, Characklis GW (2016) Synthetic drought scenario generation to support bottom-up water supply vulnerability assessments. J Water Resour Plan Manag 142(11):4016050. CrossRefGoogle Scholar
  23. Hirabayashi Y, Mahendran R, Koirala S, Konoshima L, Yamazaki D, Watanabe S, Kim H, Kanae S (2013) Global flood risk under climate change. Nat Clim Chang 3(9):816–821. CrossRefGoogle Scholar
  24. Hui R, Herman J, Lund J, Madani K (2018) Adaptive water infrastructure planning for nonstationary hydrology. Adv Water Resour 118(May):83–94. CrossRefGoogle Scholar
  25. Jeuland M, Whittington D (2014) Water resources planning under climate change: assessing the robustness of real options for the Blue Nile. Water Resour Res:2086–2107.
  26. Katz RW (2002) Techniques for estimating uncertainty in climate change scenarios and impact studies. Clim Res 20:167–185CrossRefGoogle Scholar
  27. Knowles N, Dettinger MD, Cayan DR (2006) Trends in snowfall versus rainfall in the Western United States. J Clim 19(18):4545–4559. CrossRefGoogle Scholar
  28. Kundzewicz ZW, Krysanova V, Benestad RE, Hov O, Piniewski M, Otto IM (2018) Uncertainty in climate change impacts on water resources. Environ Sci Policy 79:1–8. CrossRefGoogle Scholar
  29. Kwadijk JCJ, Haasnoot M, Mulder JPM, Hoogvliet MMC, Jeuken ABM, van der Krogt RAA, van Oostrom NGC, Schelfhout HA, van Velzen EH, van Waveren H, de Wit MJM (2010) Using adaptation tipping points to prepare for climate change and sea level rise: a case study in the Netherlands. Wiley Interdiscip Rev Clim Chang 1(5):729–740. CrossRefGoogle Scholar
  30. Kwakkel JH, Pruyt E (2013) Exploratory modeling and analysis, an approach for model-based foresight under deep uncertainty. Technol Forecast Soc Chang 80(3):419–431. CrossRefGoogle Scholar
  31. Kwakkel JH, Haasnoot M, Walker WE (2015) Developing dynamic adaptive policy pathways: a computer-assisted approach for developing adaptive strategies for a deeply uncertain world. Clim Chang 132(3):373–386. CrossRefGoogle Scholar
  32. Lempert RJ (2002) A new decision sciences for complex systems. Proc Natl Acad Sci 99(Supplement 3):7309–7313. CrossRefGoogle Scholar
  33. Lempert RJ, Collins MT (2007) Managing the risk of uncertain threshold responses: comparison of robust, optimum, and precautionary approaches. Risk Anal 27(4):1009–1026. CrossRefGoogle Scholar
  34. Leung LR, Qian Y, Bian X, Washington WM, Han J, Roads JO (2004) Mid-century ensemble regional climate change scenarios for the western United States. Clim Chang 62(1–3):75–113. CrossRefGoogle Scholar
  35. Liang X, Lettenmaier DP, Wood EF, Burges SJ (1994) A simple hydrologically based model of land surface water and energy fluxes for general circulation models. J Geophys Res Atmos 99(D7):14415–14428. CrossRefGoogle Scholar
  36. Luckman BH (1998) Landscape and climate change in the Central Canadian Rockies during the 20th century. Can Geogr 42(4):319–336. CrossRefGoogle Scholar
  37. Mote PW, Hamlet AF, Clark MP, Lettenmaier DP (2005) Declining mountain snowpack in Western North America, (January), pp. 39–49.
  38. Pederson GT, Gray ST, Woodhouse CA, Betancourt JL, Fagre DB, Littell JS, Watson E, Luckman BH, Graumlich LJ (2011) The unusual nature of recent snowpack declines in the North American Cordillera 543(July): 332–336Google Scholar
  39. Ranger N, Reeder T, Lowe J (2013) Addressing “deep” uncertainty over long-term climate in major infrastructure projects: four innovations of the Thames estuary 2100 project. EURO J Decis Process 1(3–4):233–262. CrossRefGoogle Scholar
  40. Ray PA, Bonzanigo L, Wi S, Yang YCE, Karki P, García LE, Rodriguez DJ, Brown CM (2018) Multidimensional stress test for hydropower investments facing climate, geophysical and financial uncertainty. Glob Environ Chang 48(January 2017):168–181. CrossRefGoogle Scholar
  41. Seaber PR, Kapinos FP, Knapp GL (1987) Hydrologic unit maps: US Geological Survey Water Supply Paper 2294.Google Scholar
  42. Stainforth DA, Allen MR, Tredger ER, Smith LA (2007) Confidence, uncertainty and decision-support relevance in climate predictions. Philos Trans R Soc A Math Phys Eng Sci 365(1857):2145–2161. CrossRefGoogle Scholar
  43. Steinschneider S, McCrary R, Mearns LO, Brown C (2015) The effects of climate model similarity on probabilistic climate projections and the implications for local, risk-based adaptation planning. Geophys Res Lett 42(12):5014–5022. CrossRefGoogle Scholar
  44. Stewart IT, Cayan DR, Dettinger MD (2005) Changes toward earlier streamflow timing across Western North America. J Clim 18(8):1136–1155. CrossRefGoogle Scholar
  45. Taner MÜ, Ray P, Brown C (2017) Robustness-based evaluation of hydropower infrastructure design under climate change. Climat Risk Manag 18(July):34–50. CrossRefGoogle Scholar
  46. US Global Change Research Program (2009) Global climate change impacts in the United States. Cambridge University PressGoogle Scholar
  47. Walker WE, Haasnoot M, Kwakkel JH (2013) Adapt or perish: a review of planning approaches for adaptation under deep uncertainty. Sustainability (Switzerland) 5(3):955–979. CrossRefGoogle Scholar
  48. Wilby RL, Dessai S (2010) Robust adaptation to climate change. Weather 65(7):180–185. CrossRefGoogle Scholar
  49. Zeff HB, Kasprzyk JR, Herman JD, Reed PM, Characklis GW (2014) Navigating financial and supply reliability tradeoffs in regional drought management portfolios. Water Resour Res:4906–4923.
  50. Zeff HB, Herman JD, Reed PM, Characklis GW (2016) Cooperative drought adaptation: integrating infrastructure development, conservation, and water transfers into adaptive policy pathways. Water Resour Res 52:7327–7346. CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2018

Authors and Affiliations

  1. 1.Department of Civil and Environmental EngineeringUniversity of CaliforniaDavisUSA

Personalised recommendations