Advertisement

NeuralHydrology – Interpreting LSTMs in Hydrology

  • Frederik KratzertEmail author
  • Mathew HerrneggerEmail author
  • Daniel Klotz
  • Sepp Hochreiter
  • Günter Klambauer
Chapter
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11700)

Abstract

Despite the huge success of Long Short-Term Memory networks, their applications in environmental sciences are scarce. We argue that one reason is the difficulty to interpret the internals of trained networks. In this study, we look at the application of LSTMs for rainfall-runoff forecasting, one of the central tasks in the field of hydrology, in which the river discharge has to be predicted from meteorological observations. LSTMs are particularly well-suited for this problem since memory cells can represent dynamic reservoirs and storages, which are essential components in state-space modelling approaches of the hydrological system. On basis of two different catchments, one with snow influence and one without, we demonstrate how the trained model can be analyzed and interpreted. In the process, we show that the network internally learns to represent patterns that are consistent with our qualitative understanding of the hydrological system.

Keywords

Neural networks LSTM Interpretability Hydrology Rainfall-runoff modelling 
This is a preview of subscription content, log in to check access.

References

  1. 1.
    Addor, N., Newman, A.J., Mizukami, N., Clark, M.P.: Catchment Attributes for Large-Sample Studies. UCAR/NCAR, Boulder, CO (2017)Google Scholar
  2. 2.
    Anderson, E.A.: National Weather Service River Forecast System - Snow Accumulation and Ablation Model. Technical report, November, US Department of Commerce, Silver Spring (1973)Google Scholar
  3. 3.
    Arras, L., Montavon, G., Müller, K.R., Samek, W.: Explaining recurrent neural network predictions in sentiment analysis. In: EMNLP 2017 Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis (WASSA), pp. 159–168 (2017)Google Scholar
  4. 4.
    Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10(7), e0130140 (2015)CrossRefGoogle Scholar
  5. 5.
    Beven, K.: How far can we go in distributed hydrological modelling ? Hydrol. Earth Syst. Sci. 5(1), 1–12 (2001)CrossRefGoogle Scholar
  6. 6.
    Bowden, G.J., Dandy, G.C., Maier, H.R.: Input determination for neural network models in water resources applications. Part 1 - Background and methodology. J. Hydrol. 301(1–4), 75–92 (2005)CrossRefGoogle Scholar
  7. 7.
    Brenner, C., Thiem, C.E., Wizemann, H.D., Bernhardt, M., Schulz, K.: Estimating spatially distributed turbulent heat fluxes from high-resolution thermal imagery acquired with a UAV system. Int. J. Remote Sens. 38(8–10), 3003–3026 (2017)CrossRefGoogle Scholar
  8. 8.
    Burnash, R.J.C., Ferral, R.L., McGuire, R.A.: A generalised streamflow simulation system-conceptual modelling for digital computers. Technical report, US Department of Commerce National Weather Service and State of California Department of Water Resources (1973)Google Scholar
  9. 9.
    Daniell, T.M.: Neural networks-applications in hydrology and water resources engineering. In: Proceedings of the International Hydrology and Water Resources Symposium, vol. 3, pp. 797–802. Institution of Engineers, Perth, Australia (1991)Google Scholar
  10. 10.
    Freeze, R.A., Harlan, R.L.: Blueprint for a physically-based, digitally-simulated hydrologic response model. J. Hydrol. 9(3), 237–258 (1969)CrossRefGoogle Scholar
  11. 11.
    Gupta, H.V., Sorooshian, S., Yapo, P.O.: Status of automatic calibration for hydrologic models: comparison with multilevel expert calibration. J. Hydrol. Eng. 4(2), 135–143 (1999)CrossRefGoogle Scholar
  12. 12.
    Half, A.H., Half, H.M., Azmoodeh, M.: Predicting runoff from rainfall using neural networks. In: ASCE, New York, USA, pp. 760–765 (1993)Google Scholar
  13. 13.
    Hengl, T., et al.: SoilGrids250m: global gridded soil information based on machine learning, vol. 12 (2017)CrossRefGoogle Scholar
  14. 14.
    Herrnegger, M., Nachtnebel, H.P., Schulz, K.: From runoff to rainfall: Inverse rainfall-runoff modelling in a high temporal resolution. Hydrol. Earth Syst. Sci. 19(11), 4619–4639 (2015)CrossRefGoogle Scholar
  15. 15.
    Herrnegger, M., Nachtnebel, H.P., Haiden, T.: Evapotranspiration in high alpine catchments - an important part of the water balance!. Hydrol. Res. 43(4), 460 (2012)CrossRefGoogle Scholar
  16. 16.
    Hochreiter, S., Heusel, M., Obermayer, K.: Fast model-based protein homology detection without alignment. Bioinformatics 23(14), 1728–1736 (2007)CrossRefGoogle Scholar
  17. 17.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  18. 18.
    Karpathy, A., Johnson, J., Fei-Fei, L.: Visualizing and understanding recurrent networks. arXiv preprint arXiv:1506.02078 (2015)
  19. 19.
    Kindermans, P.J., et al.: Learning how to explain neural networks: PatternNet and Pattern Attribution, pp. 1–12 (2017)Google Scholar
  20. 20.
    Klemeš, V.: Dilettantism in hydrology: transition or destiny? Water Resour. Res. 22(9 S), 177S–188S (1986)CrossRefGoogle Scholar
  21. 21.
    Klemes, V.: Stochastic models of rainfall-runoff relationship (1982)Google Scholar
  22. 22.
    Klotz, D., Herrnegger, M., Schulz, K.: Symbolic regression for the estimation of transfer functions of hydrological models. Water Resour. Res. 53(11), 9402–9423 (2017)CrossRefGoogle Scholar
  23. 23.
    Kratzert, F., Klotz, D., Brenner, C., Schulz, K., Herrnegger, M.: Rainfall-runoff modelling using Long Short-Term Memory (LSTM) networks. Hydrol. Earth Syst. Sci. 22(11), 6005–6022 (2018)CrossRefGoogle Scholar
  24. 24.
    Li, J., Chen, X., Hovy, E., Jurafsky, D.: Visualizing and Understanding Neural Models in NLP. arXiv preprint arXiv:1506.01066 (2015)
  25. 25.
    Lindström, G., Pers, C., Rosberg, J., Strömqvist, J., Arheimer, B.: Development and testing of the HYPE (Hydrological Predictions for the Environment) water quality model for different spatial scales. Hydrol. Res. 41(3–4), 295 (2010)CrossRefGoogle Scholar
  26. 26.
    Montavon, G., Lapuschkin, S., Binder, A., Samek, W., Müller, K.R.: Explaining nonlinear classification decisions with deep Taylor decomposition. Pattern Recogn. 65, 211–222 (2017)CrossRefGoogle Scholar
  27. 27.
    Moriasi, D.N., Gitau, M.W., Pai, N., Daggupati, P.: Hydrologic and water quality models: performance measures and evaluation criteria. Trans. ASABE 58(6), 1763–1785 (2015)CrossRefGoogle Scholar
  28. 28.
    Murdoch, W.J., Liu, P.J., Yu, B.: Beyond word importance: contextual decomposition to extract interactions from LSTMs. In: International Conference on Learning Representations (2018)Google Scholar
  29. 29.
    Myneni, R.B., et al.: Global products of vegetation leaf area and fraction absorbed PAR from year one of MODIS data. Remote Sens. Environ. 83(1–2), 214–231 (2002)CrossRefGoogle Scholar
  30. 30.
    Nash, J.E., Sutcliffe, J.V.: River flow forecasting through conceptual models part I - a discussion of principles. J. Hydrol. 10(3), 282–290 (1970)CrossRefGoogle Scholar
  31. 31.
    Newman, A., Sampson, K., Clark, M., Bock, A., Viger, R., Blodgett, D.: A large-sample watershed-scale hydrometeorological dataset for the contiguous USA. UCAR/NCAR, Boulder, CO (2014)Google Scholar
  32. 32.
    Perrin, C., Michel, C., Andréassian, V.: Improvement of a parsimonious model for streamflow simulation. J. Hydrol. 279(1–4), 275–289 (2003)CrossRefGoogle Scholar
  33. 33.
    Poerner, N., Schütze, H., Roth, B.: Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 340–350 (2018)Google Scholar
  34. 34.
    Ribeiro, M.T., Singh, S., Guestrin, C.: Why should I trust you? Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135–1144. ACM (2016)Google Scholar
  35. 35.
    Samaniego, L., et al.: Toward seamless hydrologic predictions across spatial scales. Hydrol. Earth Syst. Sci. 21(9), 4323–4346 (2017)CrossRefGoogle Scholar
  36. 36.
    Strobelt, H., Gehrmann, S., Pfister, H., Rush, A.M.: LSTMVis: a tool for visual analysis of hidden state dynamics in recurrent neural networks. IEEE Trans. Visual Comput. Graphics 24(1), 667–676 (2018)CrossRefGoogle Scholar
  37. 37.
    Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 3319–3328. JMLR. org (2017)Google Scholar
  38. 38.
    Thielen, J., Bartholmes, J., Ramos, M.H., de Roo, A.: The European flood alert system – Part 1: concept and development. Hydrol. Earth Syst. Sci. Dis. 5(1), 257–287 (2008)CrossRefGoogle Scholar
  39. 39.
    Tieleman, T., Hinton, G.: Lecture 6.5 - RMSProp, COURSERA: Neural Networks for Machine Learning. Technical report (2012) Google Scholar
  40. 40.
    WMO, UNESCO (United Nations Educational, Scientific and Cultural Organization): International Glossary of Hydrology. No. 12, Geneva, Switzerland (1998)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Johannes Kepler University LinzLinzAustria
  2. 2.University of Natural Resources and Life Sciences, ViennaViennaAustria

Personalised recommendations

Cite chapter

Buy options