Advertisement

Comparison of Imputation Methods for Missing Values in Air Pollution Data: Case Study on Sydney Air Quality Index

  • W. M. L. K. N. WijesekaraEmail author
  • Liwan Liyanage
Conference paper
  • 9 Downloads
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1130)

Abstract

Missing values in air quality data may lead to a substantial amount of bias and inefficiency in modeling. In this paper, we discuss six methods for dealing with missing values in univariate time series and compare their performances. The methods we discuss here are Mean Imputation, Spline Interpolation, Simple Moving Average, Exponentially Weighted Moving Average, Kalman Smoothing on Structural Time Series Models and Kalman Smoothing on Autoregressive Integrated Moving Average (ARIMA) models. The performances of these methods were compared using three different performance measures; Mean Squared Error, Coefficient of Determination and the Index of Agreement. Kalman Smoothing on Structural Time Series method is the best method among the methods considered, for imputing missing values in the context of air quality data under Missing Completely at Random (MCAR) mechanism. Kalman Smoothing on ARIMA, and Exponentially Weighted Moving Average methods also perform considerably well. Performance of Spline Interpolation decreases drastically with increased percentage of missing values. Mean Imputation performs reasonably well for smaller percentage of missing values; however, all the other methods outperform Mean Imputation regardless the number of missing values.

Keywords

Imputation Smoothing Missing completely at random 

References

  1. 1.
    Nakagawa, S., Freckleton, R.P.: Missing inaction: the dangers of ignoring missing data. Trends Ecol. Evol. 23(11), 592–596 (2008)CrossRefGoogle Scholar
  2. 2.
    Norazian, M.N., et al.: Estimation of missing values in air pollution data using single imputation techniques. ScienceAsia 34(3), 341–345 (2008)CrossRefGoogle Scholar
  3. 3.
    Zakaria, N.A., Noor, N.M.: Imputation methods for filling missing data in urban air pollution data for Malaysia. Urbanism 9(2), 159–166 (2018)Google Scholar
  4. 4.
    Junninen, H., et al.: Methods for imputation of missing values in air quality data sets. Atmos. Environ. 38(18), 2895–2907 (2004)CrossRefGoogle Scholar
  5. 5.
    Wyzga, R.E.: Note on a air method to estimate missing pollution data. J. Air Pollut. Control Assoc. 23(3), 207–208 (1973)CrossRefGoogle Scholar
  6. 6.
    Junger, W.L., de Leon, A.P.: Imputation of missing data in time series for air pollutants. Atmos. Environ. 102, 96–104 (2015)CrossRefGoogle Scholar
  7. 7.
    Lei, K.S., Wan, F.: Pre-processing for missing data: a hybrid approach to air pollution prediction in Macau. In: Proceedings of the 2010 IEEE International Conference on Automation and Logistics (2010)Google Scholar
  8. 8.
    Shahbazi, H., et al.: A novel regression imputation framework for Tehran air pollution monitoring network using outputs from WRF and CAMx models. Atmos. Environ. 187, 24–33 (2018)CrossRefGoogle Scholar
  9. 9.
    Yuan, H.W., et al.: Imputation of missing data in time series for air pollutants using long short-term memory recurrent neural networks. In: Proceedings of the 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2018 ACM International Symposium on Wearable Computers (Ubicomp/Iswc 2018 Adjunct), pp. 1293–1300 (2018)Google Scholar
  10. 10.
    Rubright, J.D., Nandakumar, R., Glutting, J.J.: A simulation study of missing data with multiple missing X’s. Pract. Assess. Res. Eval. 19(10) (2014)Google Scholar
  11. 11.
    Abril, J.C.: Structural time series models. In: Lovric, M. (ed.) International Encyclopedia of Statistical Science, pp. 1555–1558. Springer, Heidelberg (2011)Google Scholar
  12. 12.
    Moritz, S., Bartz-Beielstein, T.: imputeTS: time series missing value imputation in R. R J. 9(1), 207–218 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.School of Computing, Engineering and MathematicsWestern Sydney UniversitySydneyAustralia

Personalised recommendations