A Novel Approach for Anomaly Detection over High-Speed Networks

Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 30)


This paper provides a new framework for efficient detection and identification of network anomalies over high speed links, in early stage of its occurrence to quickly react by taking the appropriate countermeasures. The proposed framework is based on change point detection in counters value of reversible sketch, which aggregates multiple data streams from high speed links in a stretched database. To detect network anomalies, we apply the cumulative sum (CUSUM) algorithm at the counter value of each bucket in the proposed reversible sketch, to detect change point occurrence and to uncover culprit flows via a new approach for sketch inversion. Theoretical framework for attacks detection is presented. We also give the results of our experiments analysis over two real data traces containing anomalies, and extensively analyzed in OSCAR French research project. Our analysis results from real-time internet traffic and online implementation over Endace DAG 3.6ET card show that our proposed architecture is able to detect culprit flows quickly with a high level of accuracy.


Hash Function Anomaly Detection Attack Detection Heavy Hitter Anomaly Detection Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Charikar M, Chen K, Farach-Colton M (2002) Finding frequent items in data streams. In: 29th International Colloquium on Automata, Languages and Programming (ICALP ’02), London, UK, pp. 693–703.Google Scholar
  2. [2]
    Cisco Systems Inc: Cisco netflow. http://www.cisco.com/wrap/public/732/Tech/netflow
  3. [3]
    Cormode G, Korn F, Muthukrishnan S, Srivastava D (2004) Diamond in the rough: Finding hierarchical heavy hitters in multi-dimensional data. In: 23rd ACM SIGMOD, pp. 155–166.Google Scholar
  4. [4]
    Cormode G, Muthukrishnan S (2004) What’s new: Finding significant differences in network data streams. In: IEEE Infocom’04, pp. 1534–1545.Google Scholar
  5. [5]
    Cormode G, Muthukrishnan S (2005). An improved data stream summary: The count-min sketch and its applications. Journal of Algorithms, 55(1):58–75.MATHCrossRefMathSciNetGoogle Scholar
  6. [6]
    Feng W, Zhang Z, Jia Z., Fu Z (2006). Reversible sketch based on the xor-based hashing. In: Asia-Pacific Conference on Services Computing (APSCC ’06), Guangzhou, Guangdong, China, pp. 93–98.Google Scholar
  7. [7]
    Fluhrer S, McGrew D (2001). Statistical analysis of the alleged RC4 keystream generator. In: 7th International Workshop on Fast Software Encryption (FSE ’00), London, UK, pp. 19–30.Google Scholar
  8. [8]
    Gutmann P (1996), Optimized RC4 code. http://www.zengl.net/freeswan/.
  9. [9]
    Jung J, Paxson V, Berger A, Balakrishnan H (2004) Fast portscan detection using sequential hypothesis testing, in: IEEE Symposium on Security and Privacy, pp. 9–12.Google Scholar
  10. [10]
    Kim H, Rozovskii B, Tartakovsky A (2004) A nonparametric multichart cusum test for rapid intrusion detection. Journal of Computing and Information Science, 2(3):149–158.Google Scholar
  11. [11]
    Krishnamurthy B, Sen S, Zhang Y, Chen Y (2003) Sketch-based change detection: methods, evaluation, and applications. In: 3rd ACM SIGCOMM Conference on Internet Measurement (IMC’03), New York, USA, pp. 234–247.Google Scholar
  12. [12]
    Li X, Bian F, Crovella M, Diot C, Govindan R, Iannaccon G, Lakhina A (2006) Detection and identification of network anomalies using sketch subspaces. In: 6th ACM SIGCOMM on Internet Measurement (IMC ’06), New York, USA, pp. 147–152.Google Scholar
  13. [13]
    Li Y, Yang J, An C, Zhang H (2007) Finding hierarchical heavy hitters in network measurement system. In: ACM Symposium on Applied Computing (SAC ’07), New York, USA, pp. 232–236.Google Scholar
  14. [14]
    Massive Data Analysis Lab: MassDal: Count-min sketch source code. http://www.cs.rutgers.edu/7Emuthu/massdal-code-index.html
  15. [15]
    Moore D, Voelker G, Savage S (2001) Inferring internet denial-of-service activity. In: Usenix Security Symposium, pp. 9–22.Google Scholar
  16. [16]
    National Laboratory of Applied Network Research: NLANR: Traces archive. http://pma.nlanr.net/Special/.
  17. [17]
    Paxson V (1999). Bro: A system for detecting network intruders in real-time. Journal of Computer Networks, 31(23–24):2435–2463.CrossRefGoogle Scholar
  18. [18]
    Roesch M (1999) Snort – lightweight intrusion detection for networks. In: USENIX Lisa ’99, Seattle, WA, USA.Google Scholar
  19. [19]
    Schweller R, Li Z, Chen Y, Gao Y, Gupta A, Parsons E, Zhang Y, Dinda P, Kao M.-Y, Memik G (2006) Reverse hashing for high-speed network monitoring: Algorithms, evaluation, and applications. In: INFOCOM 06, pp. 1–12.Google Scholar
  20. [20]
    Siris V. A, Papagalou F (2004) Application of anomaly detection algorithms for detecting Syn flooding attacks. In: GLOBECOM ’04, vol 4, Dallas, USA, pp. 2050–2054.Google Scholar
  21. [21]
    Tartakovsky A (2005) Asymptotic performance of a multichart cusum test under false alarm probability constraint. In: 44th IEEE Conference on Decision and Control and the European Control Conference, Seville, Spain, pp. 320–325.Google Scholar
  22. [22]
    Tartakovsky A, Rozovskii B, Blazek R, Kim H (2006) A novel approach to detection of intrusions in computer networks via adaptive sequential and batch-sequential change-point detection methods. Journal of IEEE Transactions on Signal Processing, 54(9):3372–3382.CrossRefGoogle Scholar
  23. [23]
    Thorup M, Zhang Y (2004) Tabulation based 4-universal hashing with applications to second moment estimation. In: ACM-SIAM Symposium on Discrete Algorithms (SODA ’04), New Orleans, LA, USA.Google Scholar
  24. [24]
    Wang H, Zhang D, Shin K. G (2002) Syn-dog: Sniffing syn flooding sources. In: 22nd International Conference on Distributed Computing Systems (ICDCS’02), Washington, DC, USA, pp. 421–429.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.Department of Computer ScienceENST BretagneBrestFrance

Personalised recommendations