Skip to main content

Analysis and estimation of the effects of missing values on the calculation of monthly temperature indices

Abstract

Long and complete climatic data series are a fundamental resource for scientific research on climate change. Data quality is important, and missing value or data gap management is a key process that must be dealt with carefully to produce reliable datasets. Although a large variety of techniques are available for gap-filling, a widespread strategy is to consider a dataset reliable if the rate of missing data is below a given threshold. However this strategy varies from study to study. The aim of this paper is to analyze the impact of missing daily values on the estimation of monthly average temperature indices. The relationship between the error of the estimate and the presence of random or consecutive missing values, as well as data series autocorrelation is also analyzed. A theoretical, a linear and a nonlinear model to estimate the maximum error at the 95 % confidence interval are tested on data series provided by national and worldwide networks of stations. Consecutive missing values have an important effect on error estimation due to autocorrelation of temperature data series. On our dataset, the mean and standard deviation of the error for five consecutive missing values (0.27 ± 0.05 °C) on a normalized daily series (σ = 1) was higher than for five random missing values (0.14 ± 0.006 °C). A nonlinear model taking into account the number of consecutive missing values is able to estimate the error and its performance is less affected by the presence of consecutive missing values than the other proposed models.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Acock MC, Pachepsky YA (2000) Estimating missing weather data for agricultural simulations using group method of data handling. J Appl Meteorol 39:1176–1184. doi:10.1175/1520-0450(2000)039<1176:EMWDFA>2.0.CO;2

    Article  Google Scholar 

  2. Auer I, Böhm R, Jurkovic A, Lipa W, Orlik A, Potzmann R, Schöner W, Ungersböck M, Matulla C, Briffa K, Jones PD, Efthymiadis D, Brunetti M, Nanni T, Maugeri M, Mercalli L, Mestre O, Moisselin JM, Begert M, Müller-Westermeier G, Kveton V, Bochnicek O, Stastny P, Lapin M, Szalai S, Szentimrey T, Cegnar T, Dolinar M, Gajic-Capka M, Zaninovic K, Majstorovic Z, Nieplova E (2007) HISTALP — historical instrumental climatological surface time series of the Greater Alpine Region. Int J Climatol 27:17–46. doi:10.1002/joe.1377

    Article  Google Scholar 

  3. Bartolini G, Morabito M, Crisci A, Grifoni D, Torrigiani T, Petralli M, Maracchi G, Orlandini S (2008) Recent trends in Tuscany (Italy) summer temperature and indices of extremes. Int J Climatol 28:1751–1760. doi:10.1002/joc.1673

    Article  Google Scholar 

  4. Bates DM, Chambers JM (1992) Statistical models in S, chapter 10 (Nonlinear models). Chapman and Hall, Boca Raton

    Google Scholar 

  5. Bocchiola D, Diolaiuti G (2010) Evidence of climate change within the Adamello Glacier of Italy. Theor Appl Climatol 100:351–369. doi:10.1007/s00704-009-0186-x

    Article  Google Scholar 

  6. Brunetti M, Maugeri M, Monti F, Nanni T (2006) Temperature and precipitation variability in Italy in the last two centuries from homogenised instrumental time series. Int J Climatol 26(3):345–381. doi:10.1002/joc.1251

    Article  Google Scholar 

  7. Durre I, Menne MJ, Gleason BE, Houston TG, Vose RS (2010) Comprehensive automated quality assurance of daily surface observations. J Appl Meteorol Climatol 49:1615–1633. doi:10.1175/2010JAMC2375.1

    Article  Google Scholar 

  8. Eischeid JK, Pasteris PA, Diaz HF, Plantico MS, Lott NJ (2000) Creating a serially complete, national daily time series of temperature and precipitation for the Western United States. J Appl Meteorol 39(9):1580–1591. doi:10.1175/1520-0450(2000)039<1580:CASCND>2.0.CO;2

    Article  Google Scholar 

  9. El Kenawy AM, Lopez-Moreno JI, Vicente-Serrano SM, Mekld MS (2009) Temperature trends in Libya over the second half of the 20th century. Theor Appl Climatol 98:1–8. doi:10.1007/s00704-008-0089-2

    Article  Google Scholar 

  10. Feng S, Hu Q, Qian WH (2004) Quality control of daily meteorological data in China, 1951–2000: a new dataset. Int J Climatol 24(7):853–870. doi:10.1002/joc.1047

    Article  Google Scholar 

  11. Huang F, Xia Z, Guo L et al (2013) Climate change detection and annual extreme temperature analysis of the Irtysh Basin. Theor Appl Climatol 111:465–470. doi:10.1007/s00704-012-0676-0

    Article  Google Scholar 

  12. Hubbard KG (2001) Multiple station quality control procedures. Automated weather stations for applications in agriculture and water resources management. World Meteorological Organization Tech. Doc. AGM-3 WMO/TD No. 1074, 133–136

  13. Kemp WP, Burnell DG, Everson DO, Thomson AJ (1983) Estimating missing daily maximum and minimum temperatures. J Clim Appl Meteorol 22:1587–1593. doi:10.1175/1520-0450(1983)022<1587:EMDMAM>2.0.CO;2

    Article  Google Scholar 

  14. Klok EJ, Klein Tank AMG (2009) Updated and extended European dataset of daily climate observations. Int J Climatol 29:1182–1191. doi:10.1002/joc.1779

    Article  Google Scholar 

  15. Lu QQ, Lund R, Seymour L (2005) An update of U.S. temperature trends. J Clim 18(22):4906–4914. doi:10.1175/JCLI3557.1

    Article  Google Scholar 

  16. Lund RB, Seymour PL, Kafadar K (2001) Temperature trends in the United States. Environmetrics 12:673–690. doi:10.1002/env.468

    Article  Google Scholar 

  17. Moberg A, Jones PD (2005) Trends in indices for extremes in daily temperature and precipitation in central and western Europe, 1901–99. Int J Climatol 25(9):1149–1171. doi:10.1002/joc.1163

    Article  Google Scholar 

  18. Parry ML, Canziani OF, Palutikof JP, van der Linden PJ, Hanson CE (2007) Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, 2007 Cambridge University Press, Cambridge, UK

  19. Perry M, Hollis D (2005) The development of a new set of long-term climate averages for the UK. Int J Climatol 25:1023–1039. doi:10.1002/joc.1160

    Article  Google Scholar 

  20. R Development Core Team (2008) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, URL http://www.R-project.org. ISBN 3-900051-07-0

    Google Scholar 

  21. Rebetez M, Reinhard M (2008) Monthly air temperature trends in Switzerland 1901–2000 and 1975–2004. Theor Appl Climatol 91:27–34. doi:10.1007/s00704-007-0296-2

    Article  Google Scholar 

  22. Samba G, Nganga D, Mpounza M (2008) Rainfall and temperature variations over Congo-Brazzaville between 1950 and 1998. Theor Appl Climatol 91:85–97. doi:10.1007/s00704-007-0298-0

    Article  Google Scholar 

  23. Stooksbury DE, Idso CD, Hubbard KG (1999) The effects of data gaps on the calculated monthly mean maximum and minimum temperatures in the continental United States: a spatial and temporal study. J Climate 12:1524–1533. doi: 10.1175/1520-0442(1999)012<1524:TEODGO>2.0.CO;2, doi:10.1175/1520-0442%281999%29012%3C1524:TEODGO%3E2.0.CO;2

  24. Tabari H, Somee BS, Zadeh MR (2011) Testing for long-term trends in climatic variables in Iran. Atmos Res 100(1):132–140. doi:10.1016/j.atmosres.2011.01.005

    Article  Google Scholar 

  25. Tayanc M, Im U, Dogruel M, Karaca M (2009) Climate change in Turkey for the last half century. Climatic Change 94: 483–502. doi: 10.1007/s10584-008-9511-0, doi:10.1007/s10584-008-9511-0#_blank

    Google Scholar 

  26. Toreti A, Desiato F (2008) Temperature trend over Italy from 1961 to 2004. Theor Appl Climatol 91:51–58. doi:10.1007/s00704-006-0289-6

    Article  Google Scholar 

  27. Trewin B (2007) The role of climatological normals in a changing climate WCDMP - No. 61 WMO — TD No. 1377 World Climate Data and Monitoring Programme World Meteorological Organization (Geneva, March 2007). Edited by: Omar Baddour and Hama Ko

  28. Vincent LA, Gullett DW (1999) Canadian historical and homogeneous temperature datasets for climate change analyses. Int J Climatol 19:1375–1388

    Article  Google Scholar 

  29. Vose RS, Easterling DR, Gleason B (2005) Maximum and minimum temperature trends for the globe: an update through 2004. Geophys Res Lett 32, L23822. doi:10.1029/2005GL024379

    Article  Google Scholar 

  30. WMO (2008) WMO Guide to Meteorological Instruments and Methods of Observation WMO-No. 8 (Seventh edition)

  31. Xia Y, Fabian P, Stohl A, Winterhalter M (1999) Estimation of missing values for Bavaria, Germany. Agric For Meteorol 96:131–144. doi:10.1016/S0168-1923(99)00056-8

    Article  Google Scholar 

  32. You JS, Hubbard KG, Goddard S (2008) Comparison of methods for spatially estimating station temperatures in a quality control system. Int J Climatol 28:777–787. doi:10.1002/joc.1571

    Article  Google Scholar 

Download references

Acknowledgements

The author thanks Alison Garside for the revision of the text.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Luciano Massetti.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Massetti, L. Analysis and estimation of the effects of missing values on the calculation of monthly temperature indices. Theor Appl Climatol 117, 511–519 (2014). https://doi.org/10.1007/s00704-013-1024-8

Download citation

Keywords

  • Root Mean Square Error
  • Data Series
  • Maximum Error
  • Temperature Index
  • World Meteorological Organization