Detection of Abnormal Load Consumption in the Power Grid Using Clustering and Statistical Analysis

  • Matúš Cuper
  • Marek LódererEmail author
  • Viera Rozinajová
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11871)


Nowadays, the electricity load profiles of customers (consumers and prosumers) are changing as new technologies are being developed, and therefore it is necessary to correctly identify new trends, changes and anomalies in data. Anomalies in load consumption can be caused by abnormal behavior of customers or a failure of smart meters in the grid. Accurate identification of such anomalies is crucial for maintaining stability in the grid and reduce electricity loss of distribution companies. Smart meters produce huge amounts of load consumption measurements every day and analyzing all the measurements is computationally expensive and very inefficient. Therefore, the aim of this work is to propose an anomaly detection method, that addresses this issue. Our proposed method firstly narrows down potential anomalous customers in large datasets by clustering discretized time series, and then analyses selected profiles using statistical method S-H-ESD to calculate final anomaly score. We evaluated and compared our method to four state-of-the-art anomaly detection methods on created synthetic dataset of load consumption time series containing collective anomalies. Our method outperformed other evaluated methods in terms of accuracy.


Anomaly detection Time series Clustering Statistical analysis 



This work was partially supported by the Slovak Research and Development Agency under the contract APVV-16-0213, and by the Scientific Grant Agency of the Slovak Republic VEGA, grant No. VG 1/0759/19. The authors would also like to thank for financial assistance from the STU Grant scheme for Support of Excellent Teams of Young Researchers (Grant No. 1391).


  1. 1.
    Carter, K.M., Streilein, W.W.: Probabilistic reasoning for streaming anomaly detection. In: 2012 IEEE Statistical Signal Processing Workshop (SSP). IEEE, August 2012.
  2. 2.
    Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. 41(3), 15:1–15:58 (2009). Scholar
  3. 3.
    Cleveland, W.S.: Robust locally weighted regression and smoothing scatterplots. J. Am. Stat. Assoc. 74(368), 829–836 (1979)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Fernandes Jr., G., Rodrigues, J.J., Carvalho, L.F., Al-Muhtadi, J.F., Proença Jr., M.L.: A comprehensive survey on network anomaly detection. Telecommun. Syst. 70(3), 447–489 (2019). Scholar
  5. 5.
    Hasani, Z.: Robust anomaly detection algorithms for real-time big data: comparison of algorithms. In: 2017 6th Mediterranean Conference on Embedded Computing (MECO), pp. 1–6, June 2017.
  6. 6.
    Hochenbaum, J., Vallis, O.S., Kejariwal, A.: Automatic anomaly detection in the cloud via statistical learning. CoRR abs/1704.07706 (2017)Google Scholar
  7. 7.
    Ishimtsev, V., Nazarov, I., Bernstein, A., Burnaev, E.: Conformal k-NN anomaly detector for univariate data streams (2017)Google Scholar
  8. 8.
    Laurinec, P., Lucká, M.: Interpretable multiple data streams clustering with clipped streams representation for the improvement of electricity consumption forecasting. Data Min. Knowl. Disc. 33(2), 413–445 (2019)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Lima, M.F., Zarpelão, B.B., Sampaio, L.D.H., Rodrigues, J.J.P.C., Abrão, T., Proença, M.L.: Anomaly detection using baseline and k-means clustering. In: SoftCOM 2010, 18th International Conference on Software, Telecommunications and Computer Networks, pp. 305–309, September 2010Google Scholar
  10. 10.
    Maaten, V.D.L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)zbMATHGoogle Scholar
  11. 11.
    Qi, J., Chu, Y., He, L.: Iterative anomaly detection algorithm based on time series analysis. In: 2018 IEEE 15th International Conference on Mobile Ad Hoc and Sensor Systems (MASS), pp. 548–552, October 2018Google Scholar
  12. 12.
    Raza, H., Prasad, G., Li, Y.: EWMA model based shift-detection methods for detecting covariate shifts in non-stationary environments. Pattern Recogn. 48(3), 659–669 (2015). Scholar
  13. 13.
    Salehi, M., Rashidi, L.: A survey on anomaly detection in evolving data: [with application to forest fire risk prediction]. SIGKDD Explor. Newsl. 20(1), 13–23 (2018). Scholar
  14. 14.
    Salvador, S., Chan, P.: Learning states and rules for detecting anomalies in time series. Appl. Intell. 23(3), 241–255 (2005)CrossRefGoogle Scholar
  15. 15.
    Shao, X., Zhang, M., Meng, J.: Data stream clustering and outlier detection algorithm based on shared nearest neighbor density. In: 2018 International Conference on Intelligent Transportation, Big Data Smart City (ICITBS), pp. 279–282 (2018)Google Scholar
  16. 16.
    Tan, P.N., Steinbach, M., Kumar, V.: Introduction to Data Mining. Addison Wesley, Boston (2005)Google Scholar
  17. 17.
    Vallis, O., Hochenbaum, J., Kejariwal, A.: A novel technique for long-term anomaly detection in the cloud. In: Proceedings of the 6th USENIX Conference on Hot Topics in Cloud Computing, p. 15. USENIX Association, Berkeley (2014)Google Scholar
  18. 18.
    Wu, H.: A survey of research on anomaly detection for time series. In: 2016 13th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), pp. 426–431, December 2016Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Matúš Cuper
    • 1
  • Marek Lóderer
    • 1
    Email author
  • Viera Rozinajová
    • 1
  1. 1.Faculty of Informatics and Information TechnologiesSlovak University of TechnologyBratislavaSlovakia

Personalised recommendations