Skip to main content
SpringerLink
Log in

Application of Hydrological Forecast Verification Information

  • Reference work entry
  • First Online:
Handbook of Hydrometeorological Ensemble Forecasting

Abstract

Verification studies and systems often focus solely on the exercise of verifying forecasts and not on the application of verification information. This chapter discusses the potential for application of hydrological forecast verification information to improve decision-making in and around the forecast process. Decision-makers include model developers and system designers, forecasters, forecast consumers, and forecast administrators. Each of these has an important role in decisions about forecasts and/or the application of forecasts that may be improved through use of forecast verification. For each, we describe the role, the actions that could be taken to improve forecasts or their application, the context and constraints of those actions, and needs for verification information. Consistent with other studies and assessments on forecast verification, we identify the need for a routine forecast verification system to archive data, plan for operations, measure forecast performance, and group forecasts according to application. Further, we call on forecast agencies and forecast consumers to use forecast verification as a routine part of their operations in order to continually improve services and to engage others to use forecast verification to improve decision-making.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 599.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 799.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • American Geophysical Union, AGU statement: investigation of scientists and officials in L’Aquila, Italy, is unfounded. EOS Trans. Am. Geophys. Union 91(28), 248 (2010). https://doi.org/10.1029/2010EO280005

    Article  Google Scholar 

  • C. Baldwin, M. Waage, R. Steger, J. Garbrecht, T. Piechota et al., Acclimatizing water managers to climate forecasts through decision experiments, in Climate Variations, Climate Change, and Water Resources Engineering (ASCE, Reston, 2006), pp. 115–131

    Google Scholar 

  • N.D. Bennett, B.F.W. Croke, G. Guariso, J.H.A. Guillaume, S.H. Hamilton, A.J. Jakeman, S. Marsili-Libelli, L.T.H. Newham, J.P. Norton, C. Perrin, S.A. Pierce, B. Robson, R. Seppelt, A.A. Voinov, B.D. Fath, V. Andreassian, Characterising performance of environmental models. Environ. Model. Softw. 40, 1–20 (2013). https://doi.org/10.1016/j.envsoft.2012.09.011

    Article  Google Scholar 

  • A.A. Bradley, S.S. Schwartz, Summary verification measures and their interpretation for ensemble forecasts. Mon. Weather Rev. 139(9), 3075–3089 (2011). https://doi.org/10.1175/2010MWR3305.1

    Article  Google Scholar 

  • A.A. Bradley, S.S. Schwartz, T. Hashino, Distributions-oriented verification of ensemble streamflow predictions. J. Hydrometeorol. 5(3), 532–545 (2004)

    Article  Google Scholar 

  • G.W. Brier, R.A. Allen, Verification of weather forecasts. Compend, in Compendium of Meteorology (1951), pp. 841–848

    Google Scholar 

  • A.C. Bryant, T.H. Painter, Radiative forcing by desert dust in the Colorado River Basin from 2000 to 2009 inferred from MODIS data, in AGU Fall Meeting Abstracts, vol. 1 (2009), p. 0501. Online Available from http://adsabs.harvard.edu/abs/2009AGUFM.C33B0501B. Accessed 27 Jan 2015

  • Bureau of Meteorology, Verification in the Bureau. Framework Report (Bureau of Meteorology, Melbourne, 2015)

    Google Scholar 

  • L. Cuo, T.C. Pagano, Q.J. Wang, A review of quantitative precipitation forecasts and their use in short-to medium-range streamflow forecasting. J. Hydrometeorol. 12(5), 713–728 (2011)

    Article  Google Scholar 

  • J. Demargne, M. Mullusky, K. Werner, T. Adams, S. Lindsey, N. Schwein, W. Marosi, E. Welles, Application of forecast verification science to operational river forecasting in the US National Weather Service. Bull. Am. Meteorol. Soc. 90(6), 779–784 (2009)

    Article  Google Scholar 

  • Q. Duan, N.K. Ajami, X. Gao, S. Sorooshian, Multi-model ensemble hydrologic prediction using Bayesian model averaging. Adv. Water Resour. 30(5), 1371–1386 (2007)

    Article  Google Scholar 

  • B.A. Faber, J.R. Stedinger, Reservoir optimization using sampling SDP with ensemble streamflow prediction (ESP) forecasts. J. Hydrol. 249(1–4), 113–133 (2001). https://doi.org/10.1016/S0022-1694(01)00419-X

    Article  Google Scholar 

  • M.H. Glantz, Consequences and responsibilities in drought forecasting: the case of Yakima, 1977. Water Resour. Res. 18(1), 3–13 (1982)

    Article  Google Scholar 

  • N. Gordon, J. Shaykewich, Guidelines on Performance Assessment of Public Weather Services (World Meteorological Organization, Geneva, 2000). Online Available from www.wmo.int/pages/prog/hwrp/documents/FFI/expert/Guidelines_on_Performance_Assessment_of_Public_Weather_Services.pdf

  • A. Hamlet, D. Huppert, D. Lettenmaier, Economic value of long-lead streamflow forecasts for Columbia River Hydropower. J. Water Resour. Plan. Manag. 128(2), 91–101 (2002). https://doi.org/10.1061/(ASCE)0733-9496(2002)128:2(91)

    Article  Google Scholar 

  • H. Hartmann, R. Bales, S. Sorooshian, Weather, Climate, and Hydrologic Forecasting for the Southwest U.S. (The University of Arizona, Tucson, 1999). Online Available from http://www.climas.arizona.edu/publication/report/weather-climate-and-hydrologic-forecasting-southwest-us

  • H.C. Hartmann, R. Bales, S. Sorooshian, Weather, climate, and hydrologic forecasting for the US Southwest: a survey. Clim. Res. 21(3), 239–258 (2002)

    Article  Google Scholar 

  • I.T. Jolliffe, D.B. Stephenson, Forecast Verification: A Practitioner’s Guide in Atmospheric Science (Wiley, 2012). Online Available from http://books.google.nl/books?hl=en&lr=&id=DCxsKQeaBH8C&oi=fnd&pg=PT8&dq=jolliffe+stephenson&ots=3Ojk_X1AOy&sig=8hGKrwljwaUgKxxYwxzmafhUB8k. Accessed 27 Jan 2015

  • D. Kahneman, G. Klein, Conditions for intuitive expertise: a failure to disagree. Am. Psychol. 64(6), 515 (2009)

    Article  Google Scholar 

  • D. Kavetski, M.P. Clark, Numerical troubles in conceptual hydrology: approximations, absurdities and impact on hypothesis testing. Hydrol. Process. 25(4), 661–670 (2011)

    Article  Google Scholar 

  • Y. Liu, A.H. Weerts, M. Clark, H.-J. Hendricks Franssen, S. Kumar, H. Moradkhani, D.-J. Seo, D. Schwanenberg, P. Smith, A. Van Dijk et al., Advancing data assimilation in operational hydrologic forecasting: progresses, challenges, and emerging opportunities. Hydrol. Earth Syst. Sci. 16(10), 3863–3887 (2012)

    Article  Google Scholar 

  • B.M. Muir, Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems. Ergonomics 37(11), 1905–1922 (1994)

    Article  Google Scholar 

  • B.M. Muir, N. Moray, Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation. Ergonomics 39(3), 429–460 (1996)

    Article  Google Scholar 

  • A.H. Murphy, What is a good forecast? An essay on the nature of goodness in weather forecasting. Weather Forecast. 8(2), 281–293 (1993)

    Article  Google Scholar 

  • A.H. Murphy, Forecast verification, in Economic Value of Weather and Climate Forecasts (Cambridge University Press, Cambridge, UK/New York/Melbourne, 1997)

    Google Scholar 

  • N. Nicholls, Cognitive illusions, heuristics, and climate prediction. Bull. Am. Meteorol. Soc. 80(7), 1385–1397 (1999). https://doi.org/10.1175/1520-0477(1999)080<1385:CIHACP>2.0.CO;2

    Article  Google Scholar 

  • K. O’Grady, L. Shabman, Communicating the probability of Great Lakes water levels and storms, in Proceedings of Great Lakes Water Level Forecast and Statistics Symposium, Windsor (1990) pp. 197–204

    Google Scholar 

  • T.C. Pagano, Evaluation of Mekong River commission operational flood forecasts, 2000–2012. Hydrol. Earth Syst. Sci. 18(7), 2645–2656 (2014)

    Article  Google Scholar 

  • T.C. Pagano, D. Garen, S. Sorooshian, Evaluation of official western US seasonal water supply outlooks, 1922–2002. J. Hydrometeorol. 5(5), 896–909 (2004)

    Article  Google Scholar 

  • T.C. Pagano, H. Hapuarachchi, Q.J. Wang, Continuous Soil Moisture Accounting and Routing Modelling to Support Short Lead-Time Streamflow Forecasting (CSIRO Water for a Healthy Country National Research Flagship, Melbourne, 2009)

    Google Scholar 

  • T.C. Pagano, A.W. Wood, M.-H. Ramos, H.L. Cloke, F. Pappenberger, M.P. Clark, M. Cranston, D. Kavetski, T. Mathevet, S. Sorooshian, J.S. Verkade, Challenges of operational river forecasting. J. Hydrometeorol. (2014). Online Available from http://journals.ametsoc.org/doi/abs/10.1175/JHM-D-13-0188.1. Accessed 27 Jan 2015

  • F. Pappenberger, J. Thielen, M. Del Medico, The impact of weather forecast improvements on large scale hydrology: analysing a decade of forecasts of the European Flood Alert System. Hydrol. Process. 25(7), 1091–1113 (2011)

    Article  Google Scholar 

  • F. Pappenberger, M.H. Ramos, H.L. Cloke, F. Wetterhall, L. Alfieri, K. Bogner, A. Mueller, P. Salamon, How do I know if my forecasts are better? Using benchmarks in hydrological ensemble prediction. J. Hydrol. 522, 697–713 (2015a). https://doi.org/10.1016/j.jhydrol.2015.01.024

    Article  Google Scholar 

  • F. Pappenberger, H.L. Cloke, D.J. Parker, F. Wetterhall, D.S. Richardson, J. Thielen, The monetary benefit of early flood warnings in Europe. Environ. Sci. Pol. 51, 278–291 (2015b). https://doi.org/10.1016/j.envsci.2015.04.016

    Article  Google Scholar 

  • R. Parasuraman, V. Riley, Humans and automation: use, misuse, disuse, abuse. Hum. Factors J. Hum. Factors Ergon. Soc. 39(2), 230–253 (1997)

    Article  Google Scholar 

  • L. Paskus, Why the Silvery Minnow Matters, AlterNet (2003). Online Available from http://www.alternet.org/story/17152/why_the_silvery_minnow_matters. Accessed 27 Jan 2015

  • C. Perrin, C. Michel, V. Andréassian, Improvement of a parsimonious model for streamflow simulation. J. Hydrol. 279(1), 275–289 (2003)

    Article  Google Scholar 

  • R.A. Pielke Jr., Who decides? Forecasts and responsibilities in the 1997 Red River flood. Appl. Behav. Sci. Rev. 7(2), 83–101 (1999)

    Article  Google Scholar 

  • M. Pocernich, Appendix: verification software, in Forecast Verification: A Practitioner’s Guide in Atmospheric Science, 2nd edn. (Wiley, Chichester, 2012), pp. 231–240. Online Available from http://onlinelibrary.wiley.com/doi/10.1002/9781119960003.app1/summary. Accessed 27 Jan 2015

    Chapter  Google Scholar 

  • S. Rayner, D. Lach, H. Ingram, Weather forecasts are for wimps: why water resource managers do not use climate forecasts. Clim. Chang. 69(2–3), 197–227 (2005)

    Article  Google Scholar 

  • S.L. Rhodes, D. Ely, J.A. Dracup, Climate and the Colorado River: the limits of management. Bull. Am. Meteorol. Soc. 65(7), 682–691 (1984)

    Article  Google Scholar 

  • D.E. Robertson, D.L. Shrestha, Q.J. Wang, Post processing rainfall forecasts from numerical weather prediction models for short term streamflow forecasting. Hydrol. Earth Syst. Sci. Discuss. 10(5), 6765–6806 (2013). https://doi.org/10.5194/hessd-10-6765-2013

    Article  Google Scholar 

  • A. Rossa, K. Liechti, M. Zappa, M. Bruen, U. Germann, G. Haase, C. Keil, P. Krahe, The COST 731 action: a review on uncertainty propagation in advanced hydro-meteorological forecast systems. Atmos. Res. 100(2–3), 150–167 (2011). https://doi.org/10.1016/j.atmosres.2010.11.016

    Article  Google Scholar 

  • D. Sarewitz, R.A. Pielke, R. Byerly, Prediction: Science, Decision Making, and the Future of Nature (Island Press, 2000). Online Available from http://books.google.nl/books?hl=en&lr=&id=O0nxEU-deAUC&oi=fnd&pg=PR11&dq=sarewitz+pielke+prediction&ots=F3r_mNYv9p&sig=78GiOAFyglce8xbodoqOVanjRZA. Accessed 27 Jan 2015

  • L.J. Skitka, K.L. Mosier, M. Burdick, Does automation bias decision-making? Int. J. Hum. Comput. Stud. 51(5), 991–1006 (1999)

    Article  Google Scholar 

  • H.R. Stanski, L.J. Wilson, W.R. Burrows, Survey of Common Verification Methods in Meteorology (World Meteorological Organization, Geneva, 1989). Online Available from http://www.eumetcal.org/resources/ukmeteocal/verificationSAV/www/english/msg/library/SWB_Chapter1.pdf. Accessed 27 Jan 2015

  • J.S. Verkade, M.G.F. Werner, Estimating the benefits of single value and probability forecasting for flood warning. Hydrol. Earth Syst. Sci. 15(12), 3751–3765 (2011). https://doi.org/10.5194/hess-15-3751-2011

    Article  Google Scholar 

  • R.Y. Wang, D.M. Strong, Beyond accuracy: what data quality means to data consumers. J. Manag. Inf. Syst. 12, 5–33 (1996)

    Article  Google Scholar 

  • A.H. Weerts, H.C. Winsemius, J.S. Verkade, Estimation of predictive hydrological uncertainty using quantile regression: examples from the National Flood Forecasting System (England and Wales). Hydrol. Earth Syst. Sci. 15(1), 255–265 (2011). https://doi.org/10.5194/hess-15-255-2011

    Article  Google Scholar 

  • E. Welles, Verification of River Stage Forecasts (2005). Online Available from http://arizona.openrepository.com/arizona/handle/10150/195133. Accessed 27 Jan 2015

  • E. Welles, S. Sorooshian, Scientific verification of deterministic river stage forecasts. J. Hydrometeorol. 10(2), 507–520 (2009)

    Article  Google Scholar 

  • E. Welles, S. Sorooshian, G. Carter, B. Olsen, Hydrologic verification: a call for action and collaboration. Bull. Am. Meteorol. Soc. 88(4), 503–511 (2007)

    Article  Google Scholar 

  • F. Wetterhall, F. Pappenberger, H.L. Cloke, J. Thielen-del Pozo, S. Balabanova, J. Daňhelka, A. Vogelbacher, P. Salamon, I. Carrasco, A.J. Cabrera-Tordera et al., Forecasters priorities for improving probabilistic flood forecasts. Hydrol. Earth Syst. Sci. Discuss. 10(2), 2215–2242 (2013)

    Article  Google Scholar 

  • D.S. Wilks, Statistical Methods in the Atmospheric Sciences (Academic, 2011). Online Available from http://books.google.nl/books?hl=en&lr=&id=IJuCVtQ0ySIC&oi=fnd&pg=PP2&dq=wilks+statistical&ots=anHlqQBLNU&sig=w7ZsTvkiX5BaOjYRzngpdtC9l1M. Accessed 27 Jan2015

  • World Meteorological Organization, Guide to the Implementation of a Quality Management System for National Meteorological and Hydrological Services (World Meteorological Organization, Geneva, 2013). Online Available from http://www.wmo.int/pages/prog/hwrp/qmf-h/documents/ext/wmo_1100_en.pdf

  • WWRP/WGNE Joint Working Group on Forecast Verification Research, Forecast Verification: Issues, Method and FAQ (2015). Online Available from http://www.cawcr.gov.au/projects/verification/. Accessed 27 Jan 2015

  • M. Zappa, K.J. Beven, M. Bruen, A.S. Cofino, K. Kok, E. Martin, P. Nurmi, B. Orfila, E. Roulin, K. Schröter et al., Propagation of uncertainty from observing systems and NWP into hydrological models: COST-731 Working Group 2. Atmos. Sci. Lett. 11(2), 83–91 (2010)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Weather Service, National Oceanic and Atmospheric Administration, Salt Lake City, UT, USA

    Kevin Werner

  2. Deltares, Delft, The Netherlands

    Jan S. Verkade

  3. Ministry of Infrastructure and the Environment, Water Management Centre of the Netherlands, River Forecasting Service, Lelystad, The Netherlands

    Jan S. Verkade

  4. Delft University of Technology, Delft, The Netherlands

    Jan S. Verkade

  5. Bureau of Meteorology, Melbourne, VIC, Australia

    Thomas C. Pagano

Authors
  1. Kevin Werner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan S. Verkade
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas C. Pagano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Kevin Werner , Jan S. Verkade or Thomas C. Pagano .

Editor information

Editors and Affiliations

  1. Faculty of Geographical Science, Beijing Normal University, Beijing, China

    Qingyun Duan

  2. European Centre for Medium-Range Weather Forecasts, ECMWF, Reading, UK

    Florian Pappenberger

  3. National Center for Atmospheric Research, Boulder, CO, USA

    Andy Wood

  4. Department of Meteorology, Reading University, Reading, UK

    Hannah L. Cloke

  5. U.S. National Weather Service (retired), Annapolis, MD, USA

    John C. Schaake

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer-Verlag GmbH Germany, part of Springer Nature

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Werner, K., Verkade, J.S., Pagano, T.C. (2019). Application of Hydrological Forecast Verification Information. In: Duan, Q., Pappenberger, F., Wood, A., Cloke, H., Schaake, J. (eds) Handbook of Hydrometeorological Ensemble Forecasting. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39925-1_7

Download citation

Publish with us

Policies and ethics

Search

Navigation