Effect of Filtering in Big Data Analytics for Load Forecasting in Smart Grid

  • Sneha Rai
  • Mala DeEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1240)


With the introduction of smart metering infrastructure in the power system, the availability of real-time data for every node in a smart grid has become possible. This has led to the availability of big data in power system. The analysis of this huge set of data requires a different method of treatment. Machine learning-based tools are being used in this environment for load forecasting. But this huge data set requires appropriate pre-processing for bad data removal, missing data identification, normalization of largely varying datasets, etc. to enable the load forecaster to perform better. The present paper focuses on the analysis of different filters or pre-processors in the performance of multiple load forecasting methods in the case of smart grid. The paper uses a practical smart grid data to implement the same.


Data filtering/Pre-processing Smart grid Load forecasting Regression Big data 


  1. 1.
    Wen, L., Zhou, K., Yang, S., Li, L.: Compression of smart meter big data: a survey. Renew. Sustain. Energy Rev. 91, 59–69 (2018)CrossRefGoogle Scholar
  2. 2.
    Majidi, M., Zare, K.: Integration of smart energy hubs in distribution networks under uncertainties and demand response concept. In: IEEE Transactions on Power Systems (2018)Google Scholar
  3. 3.
    Hayes, B., Gruber, J., Prodanovic, M.: Short-term load forecasting at the local level using smart meter data. In: IEEE Eindhoven PowerTech, Eindhoven, pp. 1–6 (2015)Google Scholar
  4. 4.
    Nose-Filho, K., Lotufo, A.D.P., Minussi, C.R.: Preprocessing data for short-term load forecasting with a general regression neural network and a moving average filter. Presented at the IEEE Trondheim Power Tech, Trondheim, Norway, 19–23 (2011)Google Scholar
  5. 5.
    Zhang, P., Wu, X., Wang, X.: Short-term load forecasting based on big data technologies. CSEE J. Power Energy Syst. 1(3), 59–67 (2015)CrossRefGoogle Scholar
  6. 6.
    Moshkbar-Bakhshayesh, K., Ghofrani, M.B.: Development of a robust identifier for NPPs transients combining ARIMA model and EBP algorithm. IEEE Trans. Nuclear Sci. 61(4), 2383–2391 (2014)CrossRefGoogle Scholar
  7. 7.
    Ji, P.R., Xiong, D., Wang, P., Chen, J.: A study on exponential smoothing model for load forecasting. In: Proceedings of 2012 Power and Energy Engineering Conference (APPEEC), Shanghai, China, 1–4 (2012)Google Scholar
  8. 8.
    Hagan, M.T., Behr, S.M.: The time series approach to short term load forecasting. IEEE Trans. Power Syst. 2(3), 785–791 (1987)CrossRefGoogle Scholar
  9. 9.
    Blissing, D., Klein, M.T., Chinnathambi, R.A., Selvaraj, D.F., Ranganathan, P.: A hybrid regression model for day-ahead energy price forecasting. IEEE Access 7, 36833–36842 (2019)CrossRefGoogle Scholar
  10. 10.
    Singh, P., Dwivedi, P.: Integration of new evolutionary approach with artificial neural network for solving short term load forecast problem. Appl. Energy 217, 537–549 (2018)CrossRefGoogle Scholar
  11. 11.
    Buitrago, J., Asfour, S.: Short-term forecasting of electric loads using nonlinear autoregressive artificial neural networks with exogenous vector inputs. Energies 10, 40 (2017)CrossRefGoogle Scholar
  12. 12.
    Tian, C., Ma, J., Zhang, C., Zhan, P.: A deep neural network for short-term load forecast based on LSTM and convolution neural network. Energies 11, 3493 (2018)CrossRefGoogle Scholar
  13. 13.
    Zhang, X., Wang, J., Zhang, K.: Short-term electric load forecasting based on singular spectrum analysis and support vector machine optimized by Cuckoo search algorithm. Electr. Power Syst. Res. 146, 270–285 (2017)CrossRefGoogle Scholar
  14. 14.
    Selakov, A., Cvijetinovic, D., Milovic, L.: Hybrid PSO-SVM method for short-term load forecasting during periods with significant temperature variations in the city of Burbank. Appl. Soft Comput. 16(3), 80–88 (2013)Google Scholar
  15. 15.
    Wen, L., Zhou, K., Yang, S.: Load demand forecasting of residential buildings using a deep learning model. Electr. Power Syst. Res. 179 (2020)Google Scholar
  16. 16.
    Nose-Filho, K., Lotufo, A.D.P., Minussi, C.R.: Short-term multinodal load forecasting using a modified general regression neural network. IEEE Trans. Power Deliv. 26(4), 2862–2869 (2011)CrossRefGoogle Scholar
  17. 17.
    Hussein, E.M.A.: Preprocessing of Measurements, pp. 97–123. Elsevier (2011)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  1. 1.Department of Electrical EngineeringNIT PatnaPatnaIndia

Personalised recommendations