Skip to main content
SpringerLink
Log in

Diagnosing Traffic Anomalies Using a Two-Phase Model

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Network traffic anomalies are unusual changes in a network, so diagnosing anomalies is important for network management. Feature-based anomaly detection models (ab)normal network traffic behavior by analyzing packet header features. PCA-subspace method (Principal Component Analysis) has been verified as an efficient feature-based way in network-wide anomaly detection. Despite the powerful ability of PCA-subspace method for network-wide traffic detection, it cannot be effectively used for detection on a single link. In this paper, different from most works focusing on detection on flow-level traffic, based on observations of six traffic features for packet-level traffic, we propose a new approach B6-SVM to detect anomalies for packet-level traffic on a single link. The basic idea of B6-SVM is to diagnose anomalies in a multi-dimensional view of traffic features using Support Vector Machine (SVM). Through two-phase classification, B6-SVM can detect anomalies with high detection rate and low false alarm rate. The test results demonstrate the effectiveness and potential of our technique in diagnosing anomalies. Further, compared to previous feature-based anomaly detection approaches, B6-SVM provides a framework to automatically identify possible anomalous types. The framework of B6-SVM is generic and therefore, we expect the derived insights will be helpful for similar future research efforts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. http://www.symantec.com/.

  2. Lakhina A, Crovella M, Diot C. Mining anomalies using traffic feature distributions. In Proc. ACM SIGCOMM, Philadelphia, USA, Aug. 22-26, 2005, pp.217–228.

  3. Ahmed T, Coates M, Lakhina A. Multivariate online anomaly detection using kernel recursive least squares. In Proc. IEEE INFOCOM, Anchorage, Alaska, USA, May 6-12, 2007, pp.625–633.

  4. Brauckhoff D, Salamatian K, May M. Applying PCA for traffic anomaly detection: Problems and solutions. In Proc. INFOCOM, Rio de Janeiro, Brazil, Apr. 19-25, 2009, pp.2866–2870.

  5. Li X, Bian F, Crovella M, Diot C, Govindan R, Iannaccone G, Lakhina A. Detection and identification of network anomalies using sketch subspaces. In Proc. IMC, Rio de Janeiro, Brazil, Oct. 25-27, 2006, pp.147–152.

  6. Liu Y, Zhang L, Guan Y. Sketch-based streaming PCA algorithm for network-wide traffic anomaly detection. In Proc. the 30th International Conference on Distributed Computing Systems, Genova, Italy, Jun. 21-25, 2010, pp.807–816.

  7. Rubinstein B I P, Nelson B, Huang L et al. Antidote: Understanding and defending against poisoning of anomaly detectors. In Proc. the 9th Internet Measurement Conference, Chicago, USA, Nov. 4-6, 2009, pp.1–14.

  8. Feinstein L, Schnackenberg D, Balupari R, Kindred D. Statistical approaches to DDos attack detection and response. In Proc. DARPA Information Survivability Conference and Exposition (DISCEX), Washington DC, USA, Apr. 22-24, 2003, pp.303–314.

  9. Nychis G, Sekar V, Andersen D G, Kim H, Zhang H. An empirical evaluation of entropy-based traffic anomaly detection. In Proc. the 8th IMC, Vouliagmeni, Greece, Oct. 20-22, 2008, pp.151–156.

  10. Vapnik V. The Nature of Statistical Learning Theory. New York: Springer, 1995.

    MATH  Google Scholar 

  11. Burges C J C. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 1998, 2(2): 121–167.

    Article  Google Scholar 

  12. Kim H, Claffy K, Fomenkov M et al. Internet traffic classification demystified: Myths, caveats, and the best practices. In Proc. ACM CoNEXT, Madrid, Spain, Dec. 9-12, 2008, Article No.11.

  13. Scholkopf B, Platt J C, Shawe-Taylor J C et al. Estimating the support of a high-dimensional distribution. Neural Computation, 2001, 13(7): 1443–1471.

    Article  Google Scholar 

  14. Lin C H, Liu J C, Ho C H. Anomaly detection using LibSVM training tools. In Proc. International Conference on Information Security and Assurance, Busan, Korea, Apr. 24-26, 2008, pp.166–171.

  15. Keerthi S S, Lin C. Asymptotic behaviors of support vector machines with Gaussian kernel. Neural Computation, 2003, 15(7): 1667–1689.

    Article  MATH  Google Scholar 

  16. Chang C C, Lin C J. LIBSVM: A library for support vector machines, 2010, http://www.csie.ntu.edu.tw/»cjlin/libsvm/.

  17. Jung J, Paxson V, Berger A, Balakrishnan H. Fast portscan detection using sequential hypothesis testing. In Proc. IEEE Symposium on Security and Privacy, Berkeley, CA, USA, May 9-12, 2004, pp.211–225.

  18. Li Z, Wang L, Chen Y, Fu Z. Network-based and attack-resilient length signature generation for zero-day polymorphic worms. In Proc. the 15th IEEE International Conference on Network Protocols (ICNP), Beijing, China, Oct. 16-19, 2007, pp.164–173.

  19. Liu Z, Shu G, Li N, Lee D. Defending against instant messaging worms. In Proc. GLOBECOM, San Francisco, USA, Nov. 27-Dec. 1, 2006.

  20. Zhong Z, Ramaswamy L, Li K. ALPACAS: A large-scale privacy-aware collaborative anti-spam system. In Proc. IEEE INFOCOM, Phoenix, USA, Apr. 13-18, 2008, pp.556–564.

  21. Luo X, Chang R. On a new class of pulsing denial-of-service attacks and the defense. In Proc. Network and Distributed System Security Symposium, San Diego, California, USA, Feb. 2005.

  22. Ning P, Liu A, Du W. Mitigating DoS attacks against broadcast authentication in wireless sensor networks. ACM Transactions on Sensor Networks, 2008, 4(1): 1–31.

    Article  Google Scholar 

  23. Jung J, Krishnamurthy B, Rabinovich M. Flash crowds and denial of service attacks: Characterization and implications for CDNs and Web sites. In Proc. the 11th WWW, Honolulu, Hawaii, USA, May 7-11, 2002, pp.293–304.

  24. Krishnamurthy B, Sen S, Zhang Y, Chen Y. Sketch-based change detection: Methods, evaluation, and applications. In Proc. the 3 rd ACM IMC, Miami, Florida, USA, Oct. 27-29, 2003, pp.234–247.

  25. Won Y J, Choi M J, Hong J W K, Kim M S, Hwang H, Lee J H, Lee S G. Fault detection and diagnosis in IP-base mission critical industrial process control networks. IEEE Communications Magazine, 2008, 46(5): 172–180.

    Article  Google Scholar 

  26. Barford P, Kline J, Plonka D, Ron A. A signal analysis of network traffic anomalies. In Proc. the 2nd ACM SIGCOMM Internet Measurement Workshop, Marseille, France, Nov. 6-8, 2002, pp.71–82.

  27. Brutlag J D. Aberrant behavior detection in time series for network monitoring. In Proc. the 14th Systems Administration Conference, New Orleans, Dec. 3-8, 2000, pp.139–146.

  28. Zhang Y, Ge Z, Greenberg A, Roughan M. Network anomography. In Proc. the 5th ACM SIGCOMM Internet Measurement Conference, Berkeley, CA, USA, Oct. 19-21, 2005, pp.317–330.

  29. Gu Y, McCallum A, Towsley D. Detecting anomalies in network traffic using maximum entropy estimation. In Proc. Internet Measurement Conference, Berkeley, CA, USA, Oct. 19-21, 2005, pp.45–50.

  30. Wagner A, Plattner B. Entropy based worm and anomaly detection in fast IP networks. In Proc. the 14th IEEE International Workshops Enabling Technologies: Infrastructure Collaborative Enterprise, Washington DC, USA, June 13-15, 2005, pp.172–177.

  31. Ringberg H, Soule A, Rexford J. Webclass: Adding rigor to manual labeling of traffic anomalies. SIGCOMM Comput. Commun. Rev., 2008, 38(1): 35–38.

    Article  Google Scholar 

  32. Soule A, Larsen H, Silveira F, Rexford J, Diot C. Detectability of traffic anomalies in two adjacent networks. In Proc. the 8th Int. Conf. Passive and Active Network Measurement, Louvain-la-neuve, Belgium, Apr. 5-6, 2007, pp.22–31.

  33. Brauckhoff D, Tellenbach B, Wagner A, May M, Lakhina A. Impact of packet sampling on anomaly detection metrics. In Proc. the 6th ACM SIGCOMM Conference on Internet Measurement, ACM Press, Oct. 25-27, 2006, pp.159–164.

  34. Scherrer A, Larrieu N, Owezarski P, Borgnat P, Abry P. Non-Gaussian and long memory statistical characterizations for Internet traffic with anomalies. IEEE/ACM Trans. Dependable and Secure Computing, 2007, 4(1): 56–70.

    Article  Google Scholar 

  35. Kind A, Stoecklin M P, Dimitropoulos X. Histogram-based traffic anomaly detection. IEEE Transactions on Network and Service Management, 2009, 6(2): 110–121.

    Article  Google Scholar 

  36. Silveira F, Diot C, Taft N, Govindan R. Astute: Detecting a different class of traffic anomalies. In Proc. SIGCOMM, New-Delhi, India, Aug. 30-Sept. 3, 2010, pp.267–278.

  37. Lakhina A, Crovella M, Diot C. Diagnosing network-wide traffic anomalies. In Proc. SIGCOMM, Portland, OR, USA, Aug. 30-Sept. 3, 2004, pp.219–230.

  38. Ringberg H, Soule A, Rexford J, Diot C. Sensitivity of PCA for traffic anomaly detection. In Proc. ACM SIGMETRICS International Conf. Measurement and Modeling of Computer Systems, San Diego, CA, Jun. 12-16, 2007, pp.109–120.

  39. Ma J, Perkins S. Online novelty detection on temporal sequences. In Proc. the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Washington DC, USA, Aug. 24-27, 2003, pp.613–618.

  40. Li K, Teng G. Unsupervised SVM based on p-kernels for anomaly detection. In Proc. Innovative Computing, Information and Control, Beijing, China, Aug. 30-Sept. 1, 2006, pp.59–62.

  41. Brauckhoff D, Dimitropoulos X, Wagner A, Salamatian K. Anomaly extraction in backbone networks using association rules. In Proc. the 9th IMC, Chicago, Illinois, USA, Nov. 4-6, 2009, pp.28–34.

  42. Paredes-Oliva I, Dimitropoulos X, Molina M, Barlet-Ros P, Brauckhoff D. Automating root-cause analysis of network anomalies using frequent itemset mining. In Proc. SIGCOMM (Poster), New Delhi, India, Aug. 30-Sep. 3, 2010, pp.467–468.

  43. Silveira F, Diot C. URCA: Pulling out anomalies by their root causes. In Proc. the 29th INFOCOM, San Diego, USA, Mar. 14-19, 2010, pp.722–730.

Download references

Author information

Authors and Affiliations

  1. Network Research Center, Tsinghua University, Beijing, 100084, China

    Bin Zhang, Jia-Hai Yang (Member, CCF, ACM, IEEE), Jian-Ping Wu (Fellow, IEEE, Member, CCF, ACM) & Ying-Wu Zhu

  2. Tsinghua National Laboratory for Information Science and Technology (TNList), Beijing, China

    Bin Zhang, Jia-Hai Yang (Member, CCF, ACM, IEEE), Jian-Ping Wu (Fellow, IEEE, Member, CCF, ACM) & Ying-Wu Zhu

  3. Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China

    Bin Zhang, Jia-Hai Yang (Member, CCF, ACM, IEEE), Jian-Ping Wu (Fellow, IEEE, Member, CCF, ACM) & Ying-Wu Zhu

Authors
  1. Bin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jia-Hai Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jian-Ping Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ying-Wu Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Zhang.

Additional information

This work is supported by the National Basic Research 973 Program of China under Grant No. 2009CB320505, the National Science and Technology Supporting Plan of China under Grant No. 2008BAH37B05, the National Natural Science Foundation of China under Grant No. 61170211, the Ph.D. Programs Foundation of Ministry of Education of China under Grant No. 20110002110056, and the National High Technology Research and Development 863 Program of China under Grant Nos. 2008AA01A303 and 2009AA01Z251.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

(PDF 108 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhang, B., Yang, JH., Wu, JP. et al. Diagnosing Traffic Anomalies Using a Two-Phase Model. J. Comput. Sci. Technol. 27, 313–327 (2012). https://doi.org/10.1007/s11390-012-1225-0

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-012-1225-0

Keywords

Search

Navigation