Daedalus: Network Anomaly Detection on IDS Stream Logs

  • Aniss ChohraEmail author
  • Mourad DebbabiEmail author
  • Paria ShiraniEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11358)


In this paper, we propose a scalable framework, called Daedalus, to analyze streams of NIDS (network-based intrusion detection system) logs in near real-time and to extract useful threat security intelligence. The proposed system pre-processes huge amounts of BRO NIDS logs received from different participating organizations and applies an elaborated anomaly detection technique in order to distinguish between normal and abnormal or anomalous network behaviors. As such, Daedalus detects network traffic anomalies by extracting a set of features of interest from the connection logs and then applying a time series-based technique in order to detect abnormal behavior in near real-time. Moreover, we correlate IP blocks extracted from the logs with some external security signature-based feeds that detect factual malicious activities (e.g., malware families and hashes, ransomware distribution, and command and control centers) in order to validate the proposed approach. Performed experiments demonstrate that Daedalus accurately identifies the malicious activities with an average \(F_{1}\) score of \(92.88\%\). We further compare our proposed approach with existing K-Means approaches and demonstrate the accuracy and efficiency of our system.


  1. 1.
    Antonakakis, M., et al.: Understanding the mirai botnet. In: Proceedings of the 26th USENIX Security Symposium (2017)Google Scholar
  2. 2.
    Eberhart, R., Kennedy, J.: A new optimizer using particle swarm theory. In: Proceedings of the Sixth International Symposium on Micro Machine and Human Science. MHS 1995, pp. 39–43. IEEE (1995)Google Scholar
  3. 3.
    Goldberg, D., Shan, Y.: The importance of features for statistical anomaly detection. In: HotCloud (2015)Google Scholar
  4. 4.
    Hamamoto, A.H., Carvalho, L.F., Sampaio, L.D.H., Abrão, T., Proença Jr., M.L.: Network anomaly detection system using genetic algorithm and fuzzy logic. Expert Syst. Appl. 92, 390–402 (2018)CrossRefGoogle Scholar
  5. 5.
    Hu, W., Liao, Y., Vemuri, V.R.: Robust anomaly detection using support vector machines. In: Proceedings of the International Conference on Machine Learning, pp. 282–289 (2003)Google Scholar
  6. 6.
    Lakhina, A., Crovella, M., Diot, C.: Mining anomalies using traffic feature distributions. In: ACM SIGCOMM Computer Communication Review, vol. 35, pp. 217–228. ACM (2005)Google Scholar
  7. 7.
    Machaka, P., Bagula, A., Nelwamondo, F.: Using exponentially weighted moving average algorithm to defend against ddos attacks. In: 2016 Pattern Recognition Association of South Africa and Robotics and Mechatronics International Conference (PRASA-RobMech), pp. 1–6. IEEE (2016)Google Scholar
  8. 8.
    Maimo, L.F., Gomez, A.L.P., Clemente, F.J.G., Pérez, M.G., Pérez, G.M.: A self-adaptive deep learning-based system for anomaly detection in 5g networks. IEEE Access 6, 7700–7712 (2018)CrossRefGoogle Scholar
  9. 9.
    Marini, F., Walczak, B.: Particle swarm optimization (PSO). A tutorial. Chemom. Intell. Lab. Syst. 149, 153–165 (2015)CrossRefGoogle Scholar
  10. 10.
    Mendel, J.M.: Fuzzy logic systems for engineering: a tutorial. Proc. IEEE 83(3), 345–377 (1995)CrossRefGoogle Scholar
  11. 11.
    Mirsky, Y., Doitshman, T., Elovici, Y., Shabtai, A.: Kitsune: an ensemble of autoencoders for online network intrusion detection. In: 25th Annual Network and Distributed System Security Symposium, NDSS 2018, San Diego, California, USA, February 18–21, 2018 (2018)Google Scholar
  12. 12.
    Mushtaq, R.: Augmented dickey fuller testGoogle Scholar
  13. 13.
    Sbert, M., Shen, H.-W., Viola, I., Chen, M., Bardera, A., Feixas, M.: Tutorial on information theory in visualization. In: SIGGRAPH Asia 2017 Courses, p. 17. ACM (2017)Google Scholar
  14. 14.
    Sharafaldin, I., Lashkari, A.H., Ghorbani, A.A.: Toward generating a new intrusion detection dataset and intrusion traffic characterization. In: ICISSP, pp. 108–116 (2018)Google Scholar
  15. 15.
    Shinde, R., et al.: Survey on ransomware: a new era of cyber attackGoogle Scholar
  16. 16.
    Shirani, P., Azgomi, M.A., Alrabaee, S.: A method for intrusion detection in web services based on time series. In: 2015 IEEE 28th Canadian Conference on Electrical and Computer Engineering (CCECE), pp. 836–841. IEEE (2015)Google Scholar
  17. 17.
    Anomaly detection with k-means clustering (2015).
  18. 18.
    An exponentially weighted moving average implementation that decays based on the elapsed time since the last update, approximating a time windowed moving average (2017).
  19. 19.
    htop(1) - linux man page
  20. 20.
    How to check if time series data is stationary with python (2016).
  21. 21.
    Ransomware tracker website (2018).
  22. 22.
    The bro network security monitor.
  23. 23.
  24. 24.
    Exploring the exponentially weighted moving average (2018).
  25. 25.
    UNBCIC 2017 IDS Dataset (2017).
  26. 26.
    Wang, X., Zhang, H., Zhang, C., Cai, X., Wang, J., Ye, M.: Time series prediction using LS-SVM with particle swarm optimization. In: Wang, J., Yi, Z., Zurada, J.M., Lu, B.-L., Yin, H. (eds.) ISNN 2006. LNCS, vol. 3972, pp. 747–752. Springer, Heidelberg (2006). Scholar
  27. 27.
    Zhang, X., Gu, C., Lin, J.: Support vector machines for anomaly detection. In: The Sixth World Congress on Intelligent Control and Automation. WCICA 2006, vol. 1, pp. 2594–2598. IEEE (2006)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Security Research Centre, Gina Cody School of Engineering and Computer ScienceConcordia UniversityMontrealCanada

Personalised recommendations