Abstract
Long and complete climatic data series are a fundamental resource for scientific research on climate change. Data quality is important, and missing value or data gap management is a key process that must be dealt with carefully to produce reliable datasets. Although a large variety of techniques are available for gap-filling, a widespread strategy is to consider a dataset reliable if the rate of missing data is below a given threshold. However this strategy varies from study to study. The aim of this paper is to analyze the impact of missing daily values on the estimation of monthly average temperature indices. The relationship between the error of the estimate and the presence of random or consecutive missing values, as well as data series autocorrelation is also analyzed. A theoretical, a linear and a nonlinear model to estimate the maximum error at the 95 % confidence interval are tested on data series provided by national and worldwide networks of stations. Consecutive missing values have an important effect on error estimation due to autocorrelation of temperature data series. On our dataset, the mean and standard deviation of the error for five consecutive missing values (0.27 ± 0.05 °C) on a normalized daily series (σ = 1) was higher than for five random missing values (0.14 ± 0.006 °C). A nonlinear model taking into account the number of consecutive missing values is able to estimate the error and its performance is less affected by the presence of consecutive missing values than the other proposed models.
Similar content being viewed by others
References
Acock MC, Pachepsky YA (2000) Estimating missing weather data for agricultural simulations using group method of data handling. J Appl Meteorol 39:1176–1184. doi:10.1175/1520-0450(2000)039<1176:EMWDFA>2.0.CO;2
Auer I, Böhm R, Jurkovic A, Lipa W, Orlik A, Potzmann R, Schöner W, Ungersböck M, Matulla C, Briffa K, Jones PD, Efthymiadis D, Brunetti M, Nanni T, Maugeri M, Mercalli L, Mestre O, Moisselin JM, Begert M, Müller-Westermeier G, Kveton V, Bochnicek O, Stastny P, Lapin M, Szalai S, Szentimrey T, Cegnar T, Dolinar M, Gajic-Capka M, Zaninovic K, Majstorovic Z, Nieplova E (2007) HISTALP — historical instrumental climatological surface time series of the Greater Alpine Region. Int J Climatol 27:17–46. doi:10.1002/joe.1377
Bartolini G, Morabito M, Crisci A, Grifoni D, Torrigiani T, Petralli M, Maracchi G, Orlandini S (2008) Recent trends in Tuscany (Italy) summer temperature and indices of extremes. Int J Climatol 28:1751–1760. doi:10.1002/joc.1673
Bates DM, Chambers JM (1992) Statistical models in S, chapter 10 (Nonlinear models). Chapman and Hall, Boca Raton
Bocchiola D, Diolaiuti G (2010) Evidence of climate change within the Adamello Glacier of Italy. Theor Appl Climatol 100:351–369. doi:10.1007/s00704-009-0186-x
Brunetti M, Maugeri M, Monti F, Nanni T (2006) Temperature and precipitation variability in Italy in the last two centuries from homogenised instrumental time series. Int J Climatol 26(3):345–381. doi:10.1002/joc.1251
Durre I, Menne MJ, Gleason BE, Houston TG, Vose RS (2010) Comprehensive automated quality assurance of daily surface observations. J Appl Meteorol Climatol 49:1615–1633. doi:10.1175/2010JAMC2375.1
Eischeid JK, Pasteris PA, Diaz HF, Plantico MS, Lott NJ (2000) Creating a serially complete, national daily time series of temperature and precipitation for the Western United States. J Appl Meteorol 39(9):1580–1591. doi:10.1175/1520-0450(2000)039<1580:CASCND>2.0.CO;2
El Kenawy AM, Lopez-Moreno JI, Vicente-Serrano SM, Mekld MS (2009) Temperature trends in Libya over the second half of the 20th century. Theor Appl Climatol 98:1–8. doi:10.1007/s00704-008-0089-2
Feng S, Hu Q, Qian WH (2004) Quality control of daily meteorological data in China, 1951–2000: a new dataset. Int J Climatol 24(7):853–870. doi:10.1002/joc.1047
Huang F, Xia Z, Guo L et al (2013) Climate change detection and annual extreme temperature analysis of the Irtysh Basin. Theor Appl Climatol 111:465–470. doi:10.1007/s00704-012-0676-0
Hubbard KG (2001) Multiple station quality control procedures. Automated weather stations for applications in agriculture and water resources management. World Meteorological Organization Tech. Doc. AGM-3 WMO/TD No. 1074, 133–136
Kemp WP, Burnell DG, Everson DO, Thomson AJ (1983) Estimating missing daily maximum and minimum temperatures. J Clim Appl Meteorol 22:1587–1593. doi:10.1175/1520-0450(1983)022<1587:EMDMAM>2.0.CO;2
Klok EJ, Klein Tank AMG (2009) Updated and extended European dataset of daily climate observations. Int J Climatol 29:1182–1191. doi:10.1002/joc.1779
Lu QQ, Lund R, Seymour L (2005) An update of U.S. temperature trends. J Clim 18(22):4906–4914. doi:10.1175/JCLI3557.1
Lund RB, Seymour PL, Kafadar K (2001) Temperature trends in the United States. Environmetrics 12:673–690. doi:10.1002/env.468
Moberg A, Jones PD (2005) Trends in indices for extremes in daily temperature and precipitation in central and western Europe, 1901–99. Int J Climatol 25(9):1149–1171. doi:10.1002/joc.1163
Parry ML, Canziani OF, Palutikof JP, van der Linden PJ, Hanson CE (2007) Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, 2007 Cambridge University Press, Cambridge, UK
Perry M, Hollis D (2005) The development of a new set of long-term climate averages for the UK. Int J Climatol 25:1023–1039. doi:10.1002/joc.1160
R Development Core Team (2008) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, URL http://www.R-project.org. ISBN 3-900051-07-0
Rebetez M, Reinhard M (2008) Monthly air temperature trends in Switzerland 1901–2000 and 1975–2004. Theor Appl Climatol 91:27–34. doi:10.1007/s00704-007-0296-2
Samba G, Nganga D, Mpounza M (2008) Rainfall and temperature variations over Congo-Brazzaville between 1950 and 1998. Theor Appl Climatol 91:85–97. doi:10.1007/s00704-007-0298-0
Stooksbury DE, Idso CD, Hubbard KG (1999) The effects of data gaps on the calculated monthly mean maximum and minimum temperatures in the continental United States: a spatial and temporal study. J Climate 12:1524–1533. doi: 10.1175/1520-0442(1999)012<1524:TEODGO>2.0.CO;2, doi:10.1175/1520-0442%281999%29012%3C1524:TEODGO%3E2.0.CO;2
Tabari H, Somee BS, Zadeh MR (2011) Testing for long-term trends in climatic variables in Iran. Atmos Res 100(1):132–140. doi:10.1016/j.atmosres.2011.01.005
Tayanc M, Im U, Dogruel M, Karaca M (2009) Climate change in Turkey for the last half century. Climatic Change 94: 483–502. doi: 10.1007/s10584-008-9511-0, doi:10.1007/s10584-008-9511-0#_blank
Toreti A, Desiato F (2008) Temperature trend over Italy from 1961 to 2004. Theor Appl Climatol 91:51–58. doi:10.1007/s00704-006-0289-6
Trewin B (2007) The role of climatological normals in a changing climate WCDMP - No. 61 WMO — TD No. 1377 World Climate Data and Monitoring Programme World Meteorological Organization (Geneva, March 2007). Edited by: Omar Baddour and Hama Ko
Vincent LA, Gullett DW (1999) Canadian historical and homogeneous temperature datasets for climate change analyses. Int J Climatol 19:1375–1388
Vose RS, Easterling DR, Gleason B (2005) Maximum and minimum temperature trends for the globe: an update through 2004. Geophys Res Lett 32, L23822. doi:10.1029/2005GL024379
WMO (2008) WMO Guide to Meteorological Instruments and Methods of Observation WMO-No. 8 (Seventh edition)
Xia Y, Fabian P, Stohl A, Winterhalter M (1999) Estimation of missing values for Bavaria, Germany. Agric For Meteorol 96:131–144. doi:10.1016/S0168-1923(99)00056-8
You JS, Hubbard KG, Goddard S (2008) Comparison of methods for spatially estimating station temperatures in a quality control system. Int J Climatol 28:777–787. doi:10.1002/joc.1571
Acknowledgements
The author thanks Alison Garside for the revision of the text.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Massetti, L. Analysis and estimation of the effects of missing values on the calculation of monthly temperature indices. Theor Appl Climatol 117, 511–519 (2014). https://doi.org/10.1007/s00704-013-1024-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00704-013-1024-8