Climatic Change

, Volume 153, Issue 1–2, pp 267–283 | Cite as

How to evaluate a monitoring system for adaptive policies: criteria for signposts selection and their model-based evaluation

  • Luciano RasoEmail author
  • Jan Kwakkel
  • Jos Timmermans
  • Geremy Panthou


Adaptive policies have emerged as a valuable strategy for dealing with uncertainties by recognising the capacity of systems to adapt over time to new circumstances and surprises. The efficacy of adaptive policies hinges on detecting on-going change and ensuring that actions are indeed taken if and when necessary. This is operationalised by including a monitoring system composed of signposts and triggers in the design of the plan. A well-designed monitoring system is indispensable for the effective implementation of adaptive policies. Despite the importance of monitoring for adaptive policies, the present literature has not considered criteria enabling the a-priori evaluation of the efficacy of signposts. In this paper, we introduce criteria for the evaluation of individual signposts and the monitoring system as a whole. These criteria are relevance, observability, completeness, and parsimony. These criteria are intended to enhance the capacity to detect the need for adaptation in the presence of noisy and ambiguous observations of the real system. The criteria are identified from an analysis of the information chain, from system observations to policy success, focusing on how data becomes information. We illustrate how models, in particular, the combined use of stochastic and exploratory modelling can be used to assess individual signposts, and the whole monitoring system according to these criteria. This analysis provides significant insight into critical factors that may hinder learning from data. The proposed criteria are demonstrated using a hypothetical case, in which a monitoring system for a flood protection policy in the Niger River is designed and tested.


Monitoring Climate change Adaptive policies Dynamic adaptive policy pathways Signposts Evidence based Monitoring Information Flood management Extremes Deep uncertainty Niger River 



We are grateful to the anonymous reviewers whose precious comments helped improved the manuscript.

Funding information

This work is partially supported by the Netherlands Organisation for Scientific Research (


  1. Aich V, Koné B., Hattermann FF, Paton EN (2016) Time series analysis of floods across the Niger river basin. Water 8(4):165CrossRefGoogle Scholar
  2. Bankes S (1993) Exploratory modeling for policy analysis. Oper Res 41(3):435–449CrossRefGoogle Scholar
  3. Blyth W, Bradley R, Bunn D, Clarke C, Wilson T, Yang M (2007) Investment risks under uncertain climate change policy. Energy Policy 35(11):5766–5773CrossRefGoogle Scholar
  4. Brown C, Ghile Y, Laverty M, Li K (2012) Decision scaling: linking bottom-up vulnerability analysis with climate projections in the water sector. Water Resour Res 48:W09537.
  5. Bryant BP, Lempert R (2010) Thinking inside the box: a participatory, computer-assisted approach to scenario discovery. Technol Forecast Soc Chang 77 (1):34–49CrossRefGoogle Scholar
  6. Ceres RL, Forest CE, Keller K (2017) Understanding the detectability of potential changes to the 100-year peak storm surge. Clim Change 145:221.
  7. Coles S, Bawa J, Trenner L, Dorazio P (2001) An introduction to statistical modeling of extreme values, vol 208. Springer, BerlinCrossRefGoogle Scholar
  8. Cover TM, Thomas JA (2006) Elements of information theory, 2nd edn., Wiley-interscience, New YorkGoogle Scholar
  9. Cover TM, Thomas JA (2012) Elements of information theory. Wiley, New YorkGoogle Scholar
  10. Dekking M (2005) A modern introduction to probability and statistics: understanding why and how. Springer, BerlinCrossRefGoogle Scholar
  11. der Vaart AW (2000) Asymptotic statistics, vol 3. Cambridge University Press, CambridgeGoogle Scholar
  12. Dewar JA, Builder CH, Hix WM, Levin MH (1993) Assumption-based planning; a planning tool for very uncertain times, Tech rep, DTIC DocumentGoogle Scholar
  13. Funtowicz SO, Ravetz JR (1990) Uncertainty and quality in science for policy, vol 15. Springer Science & Business Media, BerlinCrossRefGoogle Scholar
  14. Goodwell AE, Kumar P (2017) Temporal information partitioning: characterizing synergy, uniqueness, and redundancy in interacting environmental variables. Water Resour Res 53(7):5920–5942CrossRefGoogle Scholar
  15. Haasnoot M, Middelkoop H, Offermans A, Van Beek E, Van Deursen WPA (2012) Exploring pathways for sustainable water management in river deltas in a changing environment. Clim Chang 115(3–4):795–819CrossRefGoogle Scholar
  16. Haasnoot M, Kwakkel JH, Walker WE, ter Maat J (2013) Dynamic adaptive policy pathways: a method for crafting robust decisions for a deeply uncertain world. Glob Environ Chang 23(2):485–498CrossRefGoogle Scholar
  17. Haasnoot M, Schellekens J, Beersma JJ, Middelkoop H, Kwadijk JCJ (2015) Transient scenarios for robust climate change adaptation illustrated for water management in The Netherlands. Environ Res Lett 10(10):105, 008CrossRefGoogle Scholar
  18. Hallegatte S (2009) Strategies to adapt to an uncertain climate change. Glob Environ Chang 19(2):240–247CrossRefGoogle Scholar
  19. Hamarat C, Kwakkel JH, Pruyt E (2013) Adaptive robust design under deep uncertainty. Technol Forecast Soc Chang 80(3):408–418CrossRefGoogle Scholar
  20. Hamarat C, Kwakkel JH, Pruyt E, Loonen ET (2014) An exploratory approach for adaptive policymaking by using multi-objective robust optimization. Simul Model Pract Theory 46:25–39CrossRefGoogle Scholar
  21. Herman JD, Giuliani M (2018) Policy tree optimization for threshold-based water resources management over multiple timescales. Environ Model Software 99:39–51CrossRefGoogle Scholar
  22. Hertzler G (2007) Adapting to climate change and managing climate risks by using real options. Aust J Agric Res 58(10):985–992CrossRefGoogle Scholar
  23. ISO ISO 5725-6 (1994) Accuracy (trueness and precision) of measurement methods and results-Part 6: Use in practice of accuracy values. International Organization for Standardization, Geneva, 1994Google Scholar
  24. Jeuland M, Whittington D (2014) Water resources planning under climate change: Assessing the robustness of real options for the blue Nile. Water Resour Res 50(3):2086–2107CrossRefGoogle Scholar
  25. Kasprzyk JR, Nataraj S, Reed PM, Lempert R (2013) Many objective robust decision making for complex environmental systems undergoing change. Environ Model Software 42:55–71CrossRefGoogle Scholar
  26. Kharin VV, Zwiers FW, Zhang X, Hegerl GC (2007) Changes in temperature and precipitation extremes in the IPCC ensemble of global coupled model simulations. J Clim 20(8):1419–1444CrossRefGoogle Scholar
  27. Kwakkel JH (2017) The exploratory modeling workbench: an open source toolkit for exploratory modeling, scenario discovery, and (multi-objective) robust decision making. Environ Model Software 96:239–250CrossRefGoogle Scholar
  28. Kwakkel JH, Pruyt E (2013) Exploratory modeling and analysis, an approach for model-based foresight under deep uncertainty. Technol Forecast Soc Chang 80(3):419–431CrossRefGoogle Scholar
  29. Kwakkel JH, Walker WE, Marchau V (2010) Adaptive airport strategic planning. EJTIR 10(3):249–273Google Scholar
  30. Kwakkel JH, Haasnoot M, Walker WE (2016) Comparing robust decision-making and dynamic adaptive policy pathways for model-based decision support under deep uncertainty. Environ Model Software 86:168–183CrossRefGoogle Scholar
  31. Lee KN (1994) Compass and gyroscope: integrating science and politics for the environment, Island Press, WashingtonGoogle Scholar
  32. Lempert R, Groves DG (2010) Identifying and evaluating robust adaptive policy responses to climate change for water management agencies in the American west. Technol Forecast Soc Chang 77(6):960–974CrossRefGoogle Scholar
  33. Lempert R, Groves DG, Popper SW, Bankes SC (2006) A general, analytic method for generating robust strategies and narrative scenarios. Manag Sci 52(4):514–528CrossRefGoogle Scholar
  34. Liu Y, Gupta HV (2007) Uncertainty in hydrologic modeling: toward an integrated data assimilation framework. Water Resour Res 43:W07401.
  35. McInerney D, Lempert R, Keller K (2012) What are robust strategies in the face of uncertain climate threshold responses? Clim Change 112(3–4):547–568CrossRefGoogle Scholar
  36. Moser SC, Ekstrom JA (2010) A framework to diagnose barriers to climate change adaptation. Proc Natl Acad Sci 107(51):22, 026–22, 031CrossRefGoogle Scholar
  37. Nearing GS, Tian Y, Gupta HV, Clark MP, Harrison KW, Weijs SV (2016) A philosophical basis for hydrological uncertainty. Hydrol Sci J 61(9):1666–1678CrossRefGoogle Scholar
  38. Pahl-Wostl C (2009) A conceptual framework for analysing adaptive capacity and multi-level learning processes in resource governance regimes. Glob Environ Chang 19(3):354–365CrossRefGoogle Scholar
  39. Panthou G, Vischel T, Lebel T, Blanchet J, Quantin G, Ali A (2012) Extreme rainfall in West Africa: a regional modeling. Water Resour Res 48:W08501.
  40. Papoulis A (1977) Signal analysis, vol 191. McGraw-Hill, New YorkGoogle Scholar
  41. Raso L, Weijs SV, Werner M (2018) Balancing costs and benefits in selecting new information: efficient monitoring using deterministic hydro-economic models. Water Resour Manage 32:339.
  42. Sims CA, Goldfeld SM, Sachs JD (1982) Policy analysis with econometric models. Brook Pap Econ Act 1982(1):107–164CrossRefGoogle Scholar
  43. Sterman JD (2006) Learning from evidence in a complex world. Am J Public Health 96(3):505–514CrossRefGoogle Scholar
  44. Tariq A, Lempert R, Riverson J, Schwartz M, Berg N (2017) A climate stress test of Los Angeles? water quality plans. Clim Chang 144(4):625–639CrossRefGoogle Scholar
  45. Walker WE, Rahman SA, Cave J (2001) Adaptive policies, policy analysis, and policy-making. Eur J Oper Res 128(2):282–289CrossRefGoogle Scholar
  46. Weijs SV (2011) Information theory for risk-based water system operation, 658 Ph.D. thesis. Delft University of Technology, Delft, The NetherlandsGoogle Scholar
  47. Weijs SV, Schoups G, Van De Giesen N (2010) Why hydrological predictions should be evaluated using information theory. Hydrol Earth Syst Sci 14(EPFL-ARTICLE-167375):2545–2558CrossRefGoogle Scholar
  48. Wilby RL, Dessai S (2010) Robust adaptation to climate change. Weather 65(7):180–185CrossRefGoogle Scholar
  49. Wilcox C, et al (2018) Trends in hydrological extremes in the Senegal and Niger rivers. J Hydrol 566:531–545CrossRefGoogle Scholar
  50. Wise RM, Fazey I, Smith MS, Park SE, Eakin HC, Van Garderen ERMA, Campbell B (2014) Reconceptualising adaptation to climate change as part of pathways of change and response. Glob Environ Chang 28:325–336CrossRefGoogle Scholar
  51. Woodward M, Kapelan Z, Gouldby B (2014) Adaptive flood risk management under climate change uncertainty using real options and optimization. Risk Anal 34(1):75–92CrossRefGoogle Scholar
  52. Zeff HB, Herman JD, Reed PM, Characklis GW (2016) Cooperative drought adaptation: integrating infrastructure development, conservation, and water transfers into adaptive policy pathways. Water Resour Res 52(9):7327–7346CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Delft University of TechnologyDelftThe Netherlands
  2. 2.Université Grenoble-AlpesGrenobleFrance

Personalised recommendations