An Attack Detection System for Multiple Web Applications Based on Big Data Platform

  • Xiaohui JinEmail author
  • Congxian Yin
  • Pengpeng Yang
  • Baojiang Cui
Conference paper
Part of the Lecture Notes on Data Engineering and Communications Technologies book series (LNDECT, volume 2)


Considering the protection requirements of large organizations for multiple web applications, we design and implement an attack detection system. The system is built on the big data platform, which is highly scalable. It adopts the network-traffic-based detection, capturing, parsing and analyzing the HTTP packets passing by in real time. By analyzing historical data, we are able to get application-specific access patterns, which can help domain experts find out anomalies efficiently. Besides, based on the labels given by domain experts, semi-supervised learning is applied to build attack detection classifier. The system is deployed in the real network of our university and has detected dozens of attacks.


Intrusion Detection Domain Expert Intrusion Detection System Attack Detection Computer Security Foundation Workshop 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    J Mchugh,A Christie,J Allen. Defending Yourself: The Role of Intrusion Detection Systems. IEEE Software, 2000, 17(5):42-51Google Scholar
  2. 2.
    D Barbará,J Couto,S Jajodia,L Popyack,N Wu. ADAM: Detecting Intrusions by Data Mining. Proceedings of the IEEE Workshop on Information Security, 2001:11–16Google Scholar
  3. 3.
    Y Gu, A Mccallum, D Towsley. Detecting anomalies in network traffic using maximum entropy estimation. ACM Sigcomm Conference on Internet Measurement, 2005:345-350Google Scholar
  4. 4.
    J Yu, H Lee, MS Kim, D Park. Traffic flooding attack detection with SNMP MIB using SVM. Computer Communications, 2008, 31(17):4212-4219Google Scholar
  5. 5.
    J Zhang, M Zulkernine. A Hybrid Network Intrusion Detection Technique Using Random Forests. International Conference on Availability, 2006, 37(8):262-269Google Scholar
  6. 6.
    G Jia, G Cheng, DM Gangahar,DK Agrawal. Traffic anomaly detection using k-means clustering. In. GI/ITG workshop MMBnetGoogle Scholar
  7. 7.
    SR Gaddam, VV Phoha, KS Balagani. K-Means+ID3: A Novel Method for Supervised Anomaly Detection by Cascading K-Means Clustering and ID3 Decision Tree Learning Methods. IEEE Transactions on Knowledge & Data Engineering, 2007, 19(3):345-354Google Scholar
  8. 8.
    X. Zhu. Semi-supervised learning literature survey. Technical Report 1530, Department of Computer Sciences, University of Wisconsin at Madison, Madison, WI, Apr. 2006.Google Scholar
  9. 9.
    O. Chapelle, B. Schölkopf, A. Zien, eds. Semi-Supervised Learning, Cambridge, MA: MIT Press, 2006Google Scholar
  10. 10.
    M Almgren, E Jonsson. Using active learning in intrusion detection. Proceedings of the 17th IEEE Computer Security Foundations Workshop (CSFW’04)Google Scholar
  11. 11.
    J Kreps, L Corp, N Narkhede, J Rao, L Corp: Kafka: a distributed messaging system for log processing. NetDB’11, Athens, 2011Google Scholar
  12. 12.
  13. 13.
    S Ghemawat: The Google file system. ACM SIGOPS Operating Systems Review, 2003, 37(5):29-43Google Scholar
  14. 14.
    F Chang, J Dean, S Ghemawat, WC Hsieh, DA Wallach: Bigtable:a distributed storage system for structured data. ACM Transactions on Computer Systems, 2008, 26(2):205–218Google Scholar
  15. 15.

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Xiaohui Jin
    • 1
    • 2
    Email author
  • Congxian Yin
    • 1
    • 2
  • Pengpeng Yang
    • 1
    • 2
  • Baojiang Cui
    • 1
    • 2
  1. 1.School of Computer Science and TechnologyBeijing University of Posts and TelecommunicationsBeijingChina
  2. 2.National Engineering Laboratory for Mobile Network SecurityBeijingChina

Personalised recommendations