How to evaluate a monitoring system for adaptive policies: criteria for signposts selection and their model-based evaluation
Adaptive policies have emerged as a valuable strategy for dealing with uncertainties by recognising the capacity of systems to adapt over time to new circumstances and surprises. The efficacy of adaptive policies hinges on detecting on-going change and ensuring that actions are indeed taken if and when necessary. This is operationalised by including a monitoring system composed of signposts and triggers in the design of the plan. A well-designed monitoring system is indispensable for the effective implementation of adaptive policies. Despite the importance of monitoring for adaptive policies, the present literature has not considered criteria enabling the a-priori evaluation of the efficacy of signposts. In this paper, we introduce criteria for the evaluation of individual signposts and the monitoring system as a whole. These criteria are relevance, observability, completeness, and parsimony. These criteria are intended to enhance the capacity to detect the need for adaptation in the presence of noisy and ambiguous observations of the real system. The criteria are identified from an analysis of the information chain, from system observations to policy success, focusing on how data becomes information. We illustrate how models, in particular, the combined use of stochastic and exploratory modelling can be used to assess individual signposts, and the whole monitoring system according to these criteria. This analysis provides significant insight into critical factors that may hinder learning from data. The proposed criteria are demonstrated using a hypothetical case, in which a monitoring system for a flood protection policy in the Niger River is designed and tested.
KeywordsMonitoring Climate change Adaptive policies Dynamic adaptive policy pathways Signposts Evidence based Monitoring Information Flood management Extremes Deep uncertainty Niger River
We are grateful to the anonymous reviewers whose precious comments helped improved the manuscript.
This work is partially supported by the Netherlands Organisation for Scientific Research (www.nwo.nl/en).
- Brown C, Ghile Y, Laverty M, Li K (2012) Decision scaling: linking bottom-up vulnerability analysis with climate projections in the water sector. Water Resour Res 48:W09537. https://doi.org/10.1029/2011WR011212
- Ceres RL, Forest CE, Keller K (2017) Understanding the detectability of potential changes to the 100-year peak storm surge. Clim Change 145:221. https://doi.org/10.1007/s10584-017-2075-0
- Cover TM, Thomas JA (2006) Elements of information theory, 2nd edn., Wiley-interscience, New YorkGoogle Scholar
- Cover TM, Thomas JA (2012) Elements of information theory. Wiley, New YorkGoogle Scholar
- der Vaart AW (2000) Asymptotic statistics, vol 3. Cambridge University Press, CambridgeGoogle Scholar
- Dewar JA, Builder CH, Hix WM, Levin MH (1993) Assumption-based planning; a planning tool for very uncertain times, Tech rep, DTIC DocumentGoogle Scholar
- ISO ISO 5725-6 (1994) Accuracy (trueness and precision) of measurement methods and results-Part 6: Use in practice of accuracy values. International Organization for Standardization, Geneva, 1994Google Scholar
- Kwakkel JH, Walker WE, Marchau V (2010) Adaptive airport strategic planning. EJTIR 10(3):249–273Google Scholar
- Lee KN (1994) Compass and gyroscope: integrating science and politics for the environment, Island Press, WashingtonGoogle Scholar
- Liu Y, Gupta HV (2007) Uncertainty in hydrologic modeling: toward an integrated data assimilation framework. Water Resour Res 43:W07401. https://doi.org/10.1029/2006WR005756
- Panthou G, Vischel T, Lebel T, Blanchet J, Quantin G, Ali A (2012) Extreme rainfall in West Africa: a regional modeling. Water Resour Res 48:W08501. https://doi.org/10.1029/2012WR012052
- Papoulis A (1977) Signal analysis, vol 191. McGraw-Hill, New YorkGoogle Scholar
- Raso L, Weijs SV, Werner M (2018) Balancing costs and benefits in selecting new information: efficient monitoring using deterministic hydro-economic models. Water Resour Manage 32:339. https://doi.org/10.1007/s11269-017-1813-4
- Weijs SV (2011) Information theory for risk-based water system operation, 658 Ph.D. thesis. Delft University of Technology, Delft, The NetherlandsGoogle Scholar