Advertisement

ADSaS: Comprehensive Real-Time Anomaly Detection System

  • Sooyeon LeeEmail author
  • Huy Kang KimEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11402)

Abstract

Since with massive data growth, the need for autonomous and generic anomaly detection system is increased. However, developing one stand-alone generic anomaly detection system that is accurate and fast is still a challenge. In this paper, we propose conventional time-series analysis approaches, the Seasonal Autoregressive Integrated Moving Average (SARIMA) model and Seasonal Trend decomposition using Loess (STL), to detect complex and various anomalies. Usually, SARIMA and STL are used only for stationary and periodic time-series, but by combining, we show they can detect anomalies with high accuracy for data that is even noisy and non-periodic. We compared the algorithm to Long Short Term Memory (LSTM), a deep-learning-based algorithm used for anomaly detection system. We used a total of seven real-world datasets and four artificial datasets with different time-series properties to verify the performance of the proposed algorithm.

Keywords

Anomaly detection SARIMA STL Real-time Data stream 

Notes

Acknowledgements

This work was supported under the framework of international cooperation program managed by National Research Foundation of Korea (No. 2017K1A3A1A17 092614).

References

  1. 1.
    The numenta anomaly benchmark. https://github.com/numenta/NAB
  2. 2.
    Ahmad, S., Lavin, A., Purdy, S., Agha, Z.: Unsupervised real-time anomaly detection for streaming data. Neurocomputing 262, 134–147 (2017)CrossRefGoogle Scholar
  3. 3.
    Chauhan, S., Vig, L.: Anomaly detection in ECG time signals via deep long short-term memory networks. In: IEEE International Conference on Data Science and Advanced Analytics (DSAA), pp. 1–7. IEEE (2015)Google Scholar
  4. 4.
    Cleveland, R.B., Cleveland, W.S., Terpenning, I.: STL: a seasonal-trend decomposition procedure based on loess. J. Off. Stat. 6(1), 3 (1990)Google Scholar
  5. 5.
    Dickey, D.A., Fuller, W.A.: Distribution of the estimators for autoregressive time series with a unit root. J. Am. Stat. Assoc. 74(366a), 427–431 (1979)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Goh, J., Adepu, S., Tan, M., Lee, Z.S.: Anomaly detection in cyber physical systems using recurrent neural networks. In: IEEE 18th International Symposium on High Assurance Systems Engineering (HASE), pp. 140–145. IEEE (2017)Google Scholar
  7. 7.
    Hamilton, J.: Time Series Analysis. Princeton University Press, Princeton (1994)zbMATHGoogle Scholar
  8. 8.
    Han, M.L., Lee, J., Kang, A.R., Kang, S., Park, J.K., Kim, H.K.: A statistical-based anomaly detection method for connected cars in internet of things environment. In: Hsu, C.-H., Xia, F., Liu, X., Wang, S. (eds.) IOV 2015. LNCS, vol. 9502, pp. 89–97. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-27293-1_9CrossRefGoogle Scholar
  9. 9.
    Kwon, H., Kim, T., Yu, S.J., Kim, H.K.: Self-similarity based lightweight intrusion detection method for cloud computing. In: Nguyen, N.T., Kim, C.-G., Janiak, A. (eds.) ACIIDS 2011. LNCS (LNAI), vol. 6592, pp. 353–362. Springer, Heidelberg (2011).  https://doi.org/10.1007/978-3-642-20042-7_36CrossRefGoogle Scholar
  10. 10.
    Laptev, N., Amizadeh, S., Flint, I.: Generic and scalable framework for automated time-series anomaly detection. In: Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1939–1947. ACM (2015)Google Scholar
  11. 11.
    Laxhammar, R., Falkman, G., Sviestins, E.: Anomaly detection in sea traffic-a comparison of the Gaussian mixture model and the kernel density estimator. In: 12th International Conference on Information Fusion, FUSION 2009, pp. 756–763. IEEE (2009)Google Scholar
  12. 12.
    Leung, K., Leckie, C.: Unsupervised anomaly detection in network intrusion detection using clusters. In: Proceedings of the Twenty-Eighth Australasian Conference on Computer Science, vol. 38, pp. 333–342. Australian Computer Society, Inc. (2005)Google Scholar
  13. 13.
    Malhotra, P., Vig, L., Shroff, G., Agarwal, P.: Long short term memory networks for anomaly detection in time series. In: Proceedings, p. 89. Presses universitaires de Louvain (2015)Google Scholar
  14. 14.
    Mills, T.C., Mills, T.C.: Time Series Techniques for Economists. Cambridge University Press, Cambridge (1991)zbMATHGoogle Scholar
  15. 15.
    T.D.P. Studio: Cygnus research international. https://www.cygres.com/0cnPageE/Glosry/SpecE.html
  16. 16.
    Wang, Y., Wang, J., Zhao, G., Dong, Y.: Application of residual modification approach in seasonal ARIMA for electricity demand forecasting: a case study of china. Energy Policy 48, 284–294 (2012)CrossRefGoogle Scholar
  17. 17.
    Yaacob, A.H., Tan, I.K., Chien, S.F., Tan, H.K.: ARIMA based network anomaly detection. In: Second International Conference on Communication Software and Networks, ICCSN 2010, pp. 205–209. IEEE (2010)Google Scholar
  18. 18.
    Yu, S.J., Koh, P., Kwon, H., Kim, D.S., Kim, H.K.: Hurst parameter based anomaly detection for intrusion detection system. In: 2016 IEEE International Conference on Computer and Information Technology (CIT), pp. 234–240. IEEE (2016)Google Scholar
  19. 19.
    Zhang, Z.K., Cho, M.C.Y., Wang, C.W., Hsu, C.W., Chen, C.K., Shieh, S.: IoT security: ongoing challenges and research opportunities. In: IEEE 7th International Conference on Service-Oriented Computing and Applications (SOCA), pp. 230–234. IEEE (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Graduate School of Information SecurityKorea UniversitySeoulSouth Korea

Personalised recommendations