Skip to main content
SpringerLink
Log in

Streamflow Data Infilling Techniques Based on Concepts of Groups and Neural Networks

  • Chapter
Artificial Neural Networks in Hydrology

Part of the book series: Water Science and Technology Library ((WSTL,volume 36))

Abstract

For planning, management, and effective control of water resource systems, a considerable amount of data on numerous hydrologic variables such as rainfall, streamflow, evapotranspiration, temperature, etc. is required. Data sets of various hydrologic variables are at times not only short, but also often have gaps because of missing observations. Such deficiencies in hydrologic time series are attributable, among others, to the malfunctioning of monitoring equipment, the effects of natural phenomena, such as earthquakes, hurricanes, or landslides, and problems with data transmission, storage and retrieval processes. Deficiencies in hydrologic data series vary from 5 to 10 percent in the case of runoff data [Correll et al. (1998)] and up to 25 percent in the case of oceanic storm surges [Zhang et al. (1997)] . Time series methods, among others, do not tolerate missing observations, and thus numerous data infilling techniques have evolved in various scientific disciplines to deal with incomplete data sets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Afza, N., and Panu, U.S. (1992) Infilling Missing Monthly Streamflow Data for Rivers with Seasonal Runoff, Civil Eng. Technical Report, No. CE-92–3, Lakehead University, Ontario, Canada.

    Google Scholar 

  • Alley, W.M. Burns, A.W. (1983) Mixed-Station Extension of Monthly Streamflow Records, Jour. of Hydraulic Engineering,Vol. 109 (10), 1272–1284.

    Article  Google Scholar 

  • Beale, E.M. and Little, R.J. (1975) Missing Values in Multivariate Analysis. J.R. Stat. B, 37(1), 129–145.

    Google Scholar 

  • Beard, L. R. (1962) Statistical Methods in Hydrology, U.S. Army Engineers, Calif., pp. 5.01 - 5.05.

    Google Scholar 

  • Beard, L.R., Fredrich, A.J., and Hawkins, E.F. (1970) Estimating Monthly Streamflows within a Region. Technical paper 18, HEC, U.S. Army Corps of Engineers, 14 pages.

    Google Scholar 

  • Beauchamp, J. J., Dowing, D. J., and Railsback, S. F. (1989) Comparison of Regression and Time-Series Methods for Synthesizing Missing Streamflow Records, Water Resources Bulletin, 25(5), 961 - 975.

    Article  Google Scholar 

  • Gyau-Boakye, P. G., and Schultz, G. A. (1994) Filling Gaps in Runoff Time Series in West Africa, Hydrological Science Journal, 39(6), 621 - 636.

    Article  Google Scholar 

  • Box, G. E. P. and Jenkins, G. M. (1976) Time Series Analysis: Forecasting and Control, Holden-Day, San Francisco, California, Revised Edition.

    Google Scholar 

  • Chakraborty, K., Mehrotra, K., Mohan, C. K., and Ranka, S. (1992) Forecasting the Behavior of Multi-variate Time Series Using Neural Networks, Neural Networks, Vol. 5, 961–970.

    Article  Google Scholar 

  • Chow, V. T. (1964) Handbook of Applied Hydrology. McGraw-Hill, New York.

    Google Scholar 

  • Correll, D. L., Jordan, T. E., and Weller, D. E. (1998) Effect of Inter-annual Variation of Precipitation on Stream Discharge from Rhode River Sub-watersheds, AWRA (in printing).

    Google Scholar 

  • Dax, A. (1985) Completing Missing Groundwater Observations by Interpolation. J. Hydrol., 81, 375–399.

    Article  Google Scholar 

  • Elshorbagy, A., Simonovic, S. P. and Panu, U. S. (1998) Performance Evaluation of Artificial Neural Networks for Runoff Prediction. ASCE Journal of Hydrologic Engineering (under review).

    Google Scholar 

  • Elshorbagy, A., Panu, U. S. and Simonovic, S. P. (1999) Investigations into Group Based Data Infilling Techniques. CSCE Annual Conference, Regina, Canada.

    Google Scholar 

  • Fiering, M. B. (1962) On the Use of Correlation to Augment Data. J. Amer. Statist. Assoc., 57(297), 20–32.

    Article  Google Scholar 

  • French, M. N., Krajewski, W. F., and Cuykendall, R. R. (1992) Rainfall Forecasting in Space and Time Using a Neural Network, J. Hydrol., Vol. 137, 1–31.

    Article  Google Scholar 

  • Giiroy, E. I. (1971) Reliability of a Variance Estimate Obtained from a Sample Augmented by Multivariate Regression, Water Resources Research, Vol. 6(6), 1595–1600.

    Google Scholar 

  • Gnanadesikan, R. (1977) Methods for Statistical Data Analysis of Multivariate Observation, John Wiley, New York

    Google Scholar 

  • Goodier, C. and Panu, U.S. (1993) Applications of a Multivariate Approach for Infilling of Missing Monthly Streamflows, Civil Engineering Technical Report No. CE-93–3, Lakehead Univ., Thunder Bay, Ontario.

    Google Scholar 

  • Goodier, C. and Panu, U.S. (1994) Infilling Missing Monthly Streamflow Data Using a Multivariate Approach, Stochastic and Statistical Methods in Hydrology &; Environmental Engineering, 3, 191–202.

    Google Scholar 

  • Granger, C. W., and Newbold, P. (1986) Forecasting Economic Time Series. Orlando, Academic Press.

    Google Scholar 

  • Griffith, D. A., Haining, R. P. and Bennett, R. J. (1985) Estimating Missing Values in Space-time Data Series. in Time Series Analysis: Theory and Practice 6. Anderson, O. D., Ord, J. K. and Robinson, E.A., eds., Elsevier Science Publishers B.V., North-Holland.

    Google Scholar 

  • Grygier, J.C., Stedinger, J. R., Yin, H.B. (1989) A Generalized Maintenance of Variance Extension Procedure for Extending Correlated Series, Water Resources Research, Vol. 25(3), 345–349.

    Article  Google Scholar 

  • Gupta, A. and Lam, M. (1996) Estimating Missing Values Using N.Networks. J. Oper. Res., 47(2), 229–238.

    Google Scholar 

  • Hirsch, R.M. (1979) An Evaluation of Record Reconstruction Techniques. WRR, 15(6), 1781–1790.

    Article  Google Scholar 

  • Hirsch, R. M. (1982) A Comparison of Four Streamflow Extension Techniques. WRR, 18(4), 1081–88.

    Article  Google Scholar 

  • Hirsch, R.M. and Gilroy, E.J. (1984) Methods of Fitting a Straight Line to Data: Examples in Water Resources, Water Resources Bulletin, Vol. 20(5), 705–711.

    Article  Google Scholar 

  • Hsu, Kuo-lin, Gupta, H. V., and Sorooshian, S. (1995) Artificial Neural Network Modeling of the Rainfall-Runoff Process, WRR, Vol. 31(10), 2517–2530.

    Article  Google Scholar 

  • Hughes, D. A. and Smakhtin, V. (1996) Daily Flow Time Series Patching or Extension: A Spatial Interpolation Approach Based on Flow Duration Curves. Hydrol. Sci. J., 41(6), 851–871.

    Article  Google Scholar 

  • Ishibuchi, H., Miyazaki, A., Kwon, K. and Tanaka, H. (1993) learning from incomplete training data with missing values and medical application. Proc. Int. Joint Conf. On Neu. Net., Japan, V.2, 1871–74.

    Google Scholar 

  • Johnson, R. A., and Wichern, D. W. (1988) Applied Multivariate Statistical Analysis, Prentice Hall, N.Y.

    Google Scholar 

  • Kang, K. W., Park, C. Y., and Kim, J. H. (1993) Neural Network and its Application to Rainfall-Runoff Forecasting, Korean J. Hydrosci., Vol. 4, 1–9.

    Google Scholar 

  • Karunanithi, N., Grenney, W. J., Whitley, D., and Bovee, K. (1994) Neural Networks for River Flow Prediction. J. Computing in Civ. Eng., ASCE, 8(2), 201–220.

    Article  Google Scholar 

  • Khalil, M., Panu, U. S. Panu, and Lennox, W.C. (1998) Infilling of Missing Streamflow Values Based on Concepts of Groups and Neural Networks, Civil Engineering Technical Report No CE-98–2, Lakehead University, Thunder Bay, Ontario, Canada.

    Google Scholar 

  • Khalil, M., Panu, U. S., and Lennox, W. (1999) Streamflow Data Infilling Procedures Based on Concept of Groups and Neural Networks. 1. Development of Models, Journal of Hydrology (under review).

    Google Scholar 

  • Kuczera, G. (1987) On Maximum Likelihood Estimators for the Multisite Lag-one Streamflow Models: Complete and Incomplete Data Cases. WRR, 23(4), 641–645.

    Article  Google Scholar 

  • Lachtermacher, G., and Fuller, J. D. (1994) Back-Propagation in Hydrological Time Series Forecasting, Stochastic and Statistical Methods in Hydrology and Environment Engineering, Vol. 3, 229–242.

    Google Scholar 

  • Lettenmaier, D. P., and Burges, S. J. (1979) Operational Assessment of Hydrologic Models of Long-term Persistence, WRR, Vol. 13 (1), 113–124.

    Article  Google Scholar 

  • Makhuvha, T., Pegram, G., Sparks, R. and Zucchini, W. (1997) Patching Rainfall Data Using Regression Methods. 2. Comparisons of Accuracy, Bias and Efficiency. J. Hydrol., 198, 308–318.

    Google Scholar 

  • Matalas, N. C. (1967) Mathematical Assessment of Synthetic Hydrology, WRR,Vol. 3, 937 - 945.

    Article  Google Scholar 

  • Matalas, N.C., and Jacobs, B. (1964) A Correlation Procedure for Augmenting Hydrologic Data, U.S. Geol. Surv. Prof. Pap., 434-E, E 1–E7.

    Google Scholar 

  • McCuen, R.A.(1993) Statistical Hydrology. Prentice-Hall, Englewood Cliffs, N.J., 306 pages.

    Google Scholar 

  • Moran, M. A. (1974) On Estimators Obtained from a Sample Augmented by Multiple Regression. WRR, 10(1), 81–85.

    Article  Google Scholar 

  • Mott, P., Sammis, T.W., and Southward, G. M. (1994). Climate Data Estimation Using Climate Information from Surrounding Climate Stations. Appl. Eng. In Agric., 10(1), 41–44.

    Google Scholar 

  • Panu, U. S. (1991) Application of Some Entropic Measures in Hydrologic Data Infilling Procedures. In Entropy and Energy Dissipation in Water Resources. Singh, V. P. and Fiorentino, M., eds., Kleuwer Academic Publishers, the Netherlands, 175–192.

    Google Scholar 

  • Panu, U.S., Unny, T.E. and Ragade, R.K., 1978. (1978) A Feature Prediction Model in Synthetic Hydrology Based on Concepts of Pattern Recognition. WRR, 14(2), 335–344.

    Article  Google Scholar 

  • Panu, U. S. and Unny, T. E. (1980a) Extension and Application of Feature Prediction Model for Synthesis of Hydrologic Records. WRR, 16(1), 77–96.

    Article  Google Scholar 

  • Panu, U.S. and Unny, T.E. (1980b) Stochastic Synthesis of Hydrologic Data Based on Concepts of Pattern Recognition, l-. General Methodology of the Approach, Journal of Hydrology, Vol. 46, 5–34.

    Article  Google Scholar 

  • Panu, U.S. and Unny, T.E. (1980c) Stochastic Synthesis of Hydrologic Data Based on Concepts of Pattern Recognition, 2-. Application to Natural Watersheds, Journal of Hydrology, Vol. 46, 197–217.

    Article  Google Scholar 

  • Panu, U.S. and Unny, T.E. (1980d) Stochastic Synthesis of Hydrologic Data Based on Concepts of Pattern Recognition, 3-. Performance Evaluation of the Methodology, Journal of Hydrology, Vol. 46, 219–237.

    Article  Google Scholar 

  • Panu, U. S., and Afza, N. (1993) Entropic Evaluation of Streamflow Data Infilling Procedures, Proc. of Stochastic and Statistical Methods in Hydrology and Environmental Engineering, pp. 410–412.

    Google Scholar 

  • Panu, U. S. and A. Ku (1997) Forecasting Monthly Streamflow Patterns for Reservoir Operations. CSCE Annual Conference, Vol. 3, 159–168.

    Google Scholar 

  • Panu, U. S. and Afza, N. (1998) Development of Feature Infilling Procedures for Hydrologic Data Series. Journal of Hydrology, (to appear).

    Google Scholar 

  • Pedreira, C.E. and Parente, E. (1995) Neural networks with missing values attributes. Proc. IEEE Int. Conf. Neu. Net., V.6, 3021–23.

    Article  Google Scholar 

  • Pegram, G. (1997) Patching Rainfall Data Using Regression Methods. 3. Grouping, Patching and Outlier Detection. J. Hydrol., 198, 319–334.

    Article  Google Scholar 

  • Raman, H., Sunilkumar, N. (1995) Multivariate Modeling of Water Resources Time Series Using Artificial Neural Networks, Hydrological Sciences Journal, Vol.40(2), 145–163.

    Article  Google Scholar 

  • Salas, J.D. (1993) Analysis and Modeling of Hydrologic Time Series. In Handbook of Hydrology, Maidment, D.R. (ed.), McGraw-Hill, Inc., USA.

    Google Scholar 

  • Salas J. D., Delleur, J. W., Yevjevich, V., and Lane, W. L. (1980) Applied Modeling of Hydrologic Time Series. Water Resour. Pub. Colorado, 46 1–473.

    Google Scholar 

  • Salas, J. D., Obeysekera, J. T. B. (1992) Conceptual Basis of Seasonal Streamflow Time Series Models, Journal of Hydraulic Engineering, Vol. 118 (8), 1186–194.

    Article  Google Scholar 

  • Shih, S. F., and Cheng, K. S. (1989) Generation of Synthetic and Missing Climatic Data for Puerto Rico, Water Resources Bulletin, Vol. 25 (4), 829–836.

    Article  Google Scholar 

  • SPSS (1995) Base Systems User’s Guide (Part-II), SPSS Inc., Chicago, Illinois, USA

    Google Scholar 

  • Srikanthan, R., McMahan, T. A., Codner, G. P., and Mein, R.G. (1983) Practical Aspects of Multi-site generation of stream flow data, Paper presented at Proceeding Hydrology and Water Resources Symposium, Inst. Of Eng., Hobart, Australia.

    Google Scholar 

  • Stedinger, J. R., Lettenmaier, D. P., and Vogal, R. M (1985) Multi-site ARMA (1,1) and Disaggregation Models for Annual Streamflow Generation, WRR, Vol. 21 (4), 497–510.

    Article  Google Scholar 

  • Streit, R. L., and Luginbuhl, T. E. (1994) Maximum Likelihood Training of Probabilistic Neural Networks, IEEE Trans., Neural Networks, Vol. 5(5), 764–783.

    Article  Google Scholar 

  • Tanaka, M. (1996) Identification of Nonlinear Systems with Missing Data Using Stochastic Neural Network, Decision and Control: Proceedings of the 35th IEEE Conference; Journal: Vol. 1, 933–934.

    Google Scholar 

  • Tang, W. Y., Kassim, A. H. M., Abubakar, S. H. (1996) Comparative studies of Various Data Treatment Methods—Malaysian Experience. Atmospheric Research Journal, Vol. 42, 247–262.

    Article  Google Scholar 

  • Terry, W. R., Lee, J. B., Kumar, A. (1986) Time Series Analysis in Acid Rain Modeling: Evaluation of Filling Missing Values By Linear Interpolations, Atmospheric Environment, Vol. 20 (10), 1941–1945.

    Article  Google Scholar 

  • Tiao, G. C., and Tsay, R. S. (1989) Model Specification in Multivariate Time Series, Journal of the Royal Statistical Society, B 51, 157–213.

    Google Scholar 

  • Tokar, A. S. (1996) Rainfall—Runoff Modeling in an Uncertain Environment. Ph.D. Thesis. University of Maryland. UMI Dissertation Service. Bell and Howell Company.

    Google Scholar 

  • Tong, H. (1983) Threshold Models in Nonlinear Time Series Analysis, Lecture Notes in Statistics, 21, New York: Springer-Verlag.

    Book  Google Scholar 

  • Tong, H. (1990) Nonlinear Time Series: A Dynamical System Approach, Oxford: Oxford University Press.

    Google Scholar 

  • Unny, T.E., Panu, U.S., McInnes, C.D., and Wong, A.K.C. (1981) Pattern Analysis and Synthesis of Time Dependent Hydrologic Data. Advances in Hydrosciences, Academic Press, Vol. 12, 222–244,

    Google Scholar 

  • Vogel, R. M. and Stedinger, J. R. (1985) Minimum Variance Streamtlow Record Augmentation Procedures. WRR, 21(5), 715–723.

    Article  Google Scholar 

  • Wong, I., Lam, D., Storey, A., and Fong, P. (1994) A Neural Network Approach to predict Missing Environmental Data, World Congress in Neural Networks, Vol. 1, Conference, San Diego, CA.

    Google Scholar 

  • Young, G.K., Orlob, G.T. and Roesner, L. A. (1970) Decision Criteria for Using Stochastic Hydrology. Journal of the Hydraulics Division, ASCE, 96 (HY4), 911–926.

    Google Scholar 

  • Zhang, K., Douglas, B. C., and Leatherman, S. P. (1997) East Coast Storm Surges Provide Unique Climate Record, EOS, Trans AGU, Vol. 78(37), 389–397.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Civil Engineering, Lakehead University, Thunder Bay, Ontario, P7B-5E1, Canada

    U. S. Panu, M. Khalil & A. Elshorbagy

Authors
  1. U. S. Panu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Khalil
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. Elshorbagy
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Purdue University, West Lafayette, Indiana, USA

    R. S. Govindaraju  & A. Ramachandra Rao  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2000 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Panu, U.S., Khalil, M., Elshorbagy, A. (2000). Streamflow Data Infilling Techniques Based on Concepts of Groups and Neural Networks. In: Govindaraju, R.S., Rao, A.R. (eds) Artificial Neural Networks in Hydrology. Water Science and Technology Library, vol 36. Springer, Dordrecht. https://doi.org/10.1007/978-94-015-9341-0_13

Download citation

  • DOI: https://doi.org/10.1007/978-94-015-9341-0_13

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-90-481-5421-0

  • Online ISBN: 978-94-015-9341-0

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation