Skip to main content
SpringerLink
Log in

NeuralHydrology – Interpreting LSTMs in Hydrology

  • Chapter
  • First Online:
Explainable AI: Interpreting, Explaining and Visualizing Deep Learning

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11700))

Abstract

Despite the huge success of Long Short-Term Memory networks, their applications in environmental sciences are scarce. We argue that one reason is the difficulty to interpret the internals of trained networks. In this study, we look at the application of LSTMs for rainfall-runoff forecasting, one of the central tasks in the field of hydrology, in which the river discharge has to be predicted from meteorological observations. LSTMs are particularly well-suited for this problem since memory cells can represent dynamic reservoirs and storages, which are essential components in state-space modelling approaches of the hydrological system. On basis of two different catchments, one with snow influence and one without, we demonstrate how the trained model can be analyzed and interpreted. In the process, we show that the network internally learns to represent patterns that are consistent with our qualitative understanding of the hydrological system.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Following [40] the baseflow is defined as: “Discharge which enters a stream channel mainly from groundwater, but also from lakes and glaciers, during long periods when no precipitation or snowmelt occurs”.

References

  1. Addor, N., Newman, A.J., Mizukami, N., Clark, M.P.: Catchment Attributes for Large-Sample Studies. UCAR/NCAR, Boulder, CO (2017)

    Google Scholar 

  2. Anderson, E.A.: National Weather Service River Forecast System - Snow Accumulation and Ablation Model. Technical report, November, US Department of Commerce, Silver Spring (1973)

    Google Scholar 

  3. Arras, L., Montavon, G., Müller, K.R., Samek, W.: Explaining recurrent neural network predictions in sentiment analysis. In: EMNLP 2017 Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis (WASSA), pp. 159–168 (2017)

    Google Scholar 

  4. Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10(7), e0130140 (2015)

    Article  Google Scholar 

  5. Beven, K.: How far can we go in distributed hydrological modelling ? Hydrol. Earth Syst. Sci. 5(1), 1–12 (2001)

    Article  Google Scholar 

  6. Bowden, G.J., Dandy, G.C., Maier, H.R.: Input determination for neural network models in water resources applications. Part 1 - Background and methodology. J. Hydrol. 301(1–4), 75–92 (2005)

    Article  Google Scholar 

  7. Brenner, C., Thiem, C.E., Wizemann, H.D., Bernhardt, M., Schulz, K.: Estimating spatially distributed turbulent heat fluxes from high-resolution thermal imagery acquired with a UAV system. Int. J. Remote Sens. 38(8–10), 3003–3026 (2017)

    Article  Google Scholar 

  8. Burnash, R.J.C., Ferral, R.L., McGuire, R.A.: A generalised streamflow simulation system-conceptual modelling for digital computers. Technical report, US Department of Commerce National Weather Service and State of California Department of Water Resources (1973)

    Google Scholar 

  9. Daniell, T.M.: Neural networks-applications in hydrology and water resources engineering. In: Proceedings of the International Hydrology and Water Resources Symposium, vol. 3, pp. 797–802. Institution of Engineers, Perth, Australia (1991)

    Google Scholar 

  10. Freeze, R.A., Harlan, R.L.: Blueprint for a physically-based, digitally-simulated hydrologic response model. J. Hydrol. 9(3), 237–258 (1969)

    Article  Google Scholar 

  11. Gupta, H.V., Sorooshian, S., Yapo, P.O.: Status of automatic calibration for hydrologic models: comparison with multilevel expert calibration. J. Hydrol. Eng. 4(2), 135–143 (1999)

    Article  Google Scholar 

  12. Half, A.H., Half, H.M., Azmoodeh, M.: Predicting runoff from rainfall using neural networks. In: ASCE, New York, USA, pp. 760–765 (1993)

    Google Scholar 

  13. Hengl, T., et al.: SoilGrids250m: global gridded soil information based on machine learning, vol. 12 (2017)

    Article  Google Scholar 

  14. Herrnegger, M., Nachtnebel, H.P., Schulz, K.: From runoff to rainfall: Inverse rainfall-runoff modelling in a high temporal resolution. Hydrol. Earth Syst. Sci. 19(11), 4619–4639 (2015)

    Article  Google Scholar 

  15. Herrnegger, M., Nachtnebel, H.P., Haiden, T.: Evapotranspiration in high alpine catchments - an important part of the water balance!. Hydrol. Res. 43(4), 460 (2012)

    Article  Google Scholar 

  16. Hochreiter, S., Heusel, M., Obermayer, K.: Fast model-based protein homology detection without alignment. Bioinformatics 23(14), 1728–1736 (2007)

    Article  Google Scholar 

  17. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  18. Karpathy, A., Johnson, J., Fei-Fei, L.: Visualizing and understanding recurrent networks. arXiv preprint arXiv:1506.02078 (2015)

  19. Kindermans, P.J., et al.: Learning how to explain neural networks: PatternNet and Pattern Attribution, pp. 1–12 (2017)

    Google Scholar 

  20. Klemeš, V.: Dilettantism in hydrology: transition or destiny? Water Resour. Res. 22(9 S), 177S–188S (1986)

    Article  Google Scholar 

  21. Klemes, V.: Stochastic models of rainfall-runoff relationship (1982)

    Google Scholar 

  22. Klotz, D., Herrnegger, M., Schulz, K.: Symbolic regression for the estimation of transfer functions of hydrological models. Water Resour. Res. 53(11), 9402–9423 (2017)

    Article  Google Scholar 

  23. Kratzert, F., Klotz, D., Brenner, C., Schulz, K., Herrnegger, M.: Rainfall-runoff modelling using Long Short-Term Memory (LSTM) networks. Hydrol. Earth Syst. Sci. 22(11), 6005–6022 (2018)

    Article  Google Scholar 

  24. Li, J., Chen, X., Hovy, E., Jurafsky, D.: Visualizing and Understanding Neural Models in NLP. arXiv preprint arXiv:1506.01066 (2015)

  25. Lindström, G., Pers, C., Rosberg, J., Strömqvist, J., Arheimer, B.: Development and testing of the HYPE (Hydrological Predictions for the Environment) water quality model for different spatial scales. Hydrol. Res. 41(3–4), 295 (2010)

    Article  Google Scholar 

  26. Montavon, G., Lapuschkin, S., Binder, A., Samek, W., Müller, K.R.: Explaining nonlinear classification decisions with deep Taylor decomposition. Pattern Recogn. 65, 211–222 (2017)

    Article  Google Scholar 

  27. Moriasi, D.N., Gitau, M.W., Pai, N., Daggupati, P.: Hydrologic and water quality models: performance measures and evaluation criteria. Trans. ASABE 58(6), 1763–1785 (2015)

    Article  Google Scholar 

  28. Murdoch, W.J., Liu, P.J., Yu, B.: Beyond word importance: contextual decomposition to extract interactions from LSTMs. In: International Conference on Learning Representations (2018)

    Google Scholar 

  29. Myneni, R.B., et al.: Global products of vegetation leaf area and fraction absorbed PAR from year one of MODIS data. Remote Sens. Environ. 83(1–2), 214–231 (2002)

    Article  Google Scholar 

  30. Nash, J.E., Sutcliffe, J.V.: River flow forecasting through conceptual models part I - a discussion of principles. J. Hydrol. 10(3), 282–290 (1970)

    Article  Google Scholar 

  31. Newman, A., Sampson, K., Clark, M., Bock, A., Viger, R., Blodgett, D.: A large-sample watershed-scale hydrometeorological dataset for the contiguous USA. UCAR/NCAR, Boulder, CO (2014)

    Google Scholar 

  32. Perrin, C., Michel, C., Andréassian, V.: Improvement of a parsimonious model for streamflow simulation. J. Hydrol. 279(1–4), 275–289 (2003)

    Article  Google Scholar 

  33. Poerner, N., Schütze, H., Roth, B.: Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 340–350 (2018)

    Google Scholar 

  34. Ribeiro, M.T., Singh, S., Guestrin, C.: Why should I trust you? Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135–1144. ACM (2016)

    Google Scholar 

  35. Samaniego, L., et al.: Toward seamless hydrologic predictions across spatial scales. Hydrol. Earth Syst. Sci. 21(9), 4323–4346 (2017)

    Article  Google Scholar 

  36. Strobelt, H., Gehrmann, S., Pfister, H., Rush, A.M.: LSTMVis: a tool for visual analysis of hidden state dynamics in recurrent neural networks. IEEE Trans. Visual Comput. Graphics 24(1), 667–676 (2018)

    Article  Google Scholar 

  37. Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 3319–3328. JMLR. org (2017)

    Google Scholar 

  38. Thielen, J., Bartholmes, J., Ramos, M.H., de Roo, A.: The European flood alert system – Part 1: concept and development. Hydrol. Earth Syst. Sci. Dis. 5(1), 257–287 (2008)

    Article  Google Scholar 

  39. Tieleman, T., Hinton, G.: Lecture 6.5 - RMSProp, COURSERA: Neural Networks for Machine Learning. Technical report (2012)

    Google Scholar 

  40. WMO, UNESCO (United Nations Educational, Scientific and Cultural Organization): International Glossary of Hydrology. No. 12, Geneva, Switzerland (1998)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Johannes Kepler University Linz, 4040, Linz, Austria

    Frederik Kratzert, Daniel Klotz, Sepp Hochreiter & Günter Klambauer

  2. University of Natural Resources and Life Sciences, Vienna, 1190, Vienna, Austria

    Mathew Herrnegger

Authors
  1. Frederik Kratzert
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mathew Herrnegger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Klotz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sepp Hochreiter
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Günter Klambauer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Frederik Kratzert or Mathew Herrnegger .

Editor information

Editors and Affiliations

  1. Fraunhofer Heinrich Hertz Institute, Berlin, Germany

    Wojciech Samek

  2. Technische Universität Berlin, Berlin, Germany

    Grégoire Montavon

  3. University of Oxford, Oxford, UK

    Andrea Vedaldi

  4. Technical University of Denmark, Kgs. Lyngby, Denmark

    Lars Kai Hansen

  5. Sekretariat MAR 4-1, Technical University of Berlin, Berlin, Berlin, Germany

    Klaus-Robert Müller

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Kratzert, F., Herrnegger, M., Klotz, D., Hochreiter, S., Klambauer, G. (2019). NeuralHydrology – Interpreting LSTMs in Hydrology. In: Samek, W., Montavon, G., Vedaldi, A., Hansen, L., Müller, KR. (eds) Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Lecture Notes in Computer Science(), vol 11700. Springer, Cham. https://doi.org/10.1007/978-3-030-28954-6_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-28954-6_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-28953-9

  • Online ISBN: 978-3-030-28954-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation