Most Memory Efficient Distributed Super Points Detection on Core Networks

  • Jie XuEmail author
  • Wei Ding
  • Xiaoyan Hu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11334)


The super point, a host which communicates with lots of others, is a kind of special hosts gotten great focus. Mining super point at the edge of a network is the foundation of many network research fields. In this paper, we proposed the most memory efficient super points detection scheme. This scheme contains a super points reconstruction algorithm called short estimator and a super points filter algorithm called long estimator. Short estimator gives a super points candidate list using thousands of bytes memory and long estimator improves the accuracy of detection result using millions of bytes memory. Combining short estimator and long estimator, our scheme acquires the highest accuracy using the smallest memory than other algorithms. There is no data confliction and floating operation in our scheme. This ensures that our scheme is suitable for parallel running and we deploy our scheme on a common GPU to accelerate processing speed. Experiments on several real-world core network traffics show that our algorithm acquires the highest accuracy with only consuming littler than one-fifth memory of other algorithms.


Super points detection Distributed computing GPU computing Network measurement 


  1. 1.
    The Center for Applied Internet Data Analysis: The caida anonymized internet traces (2017). Accessed 2017
  2. 2.
    Bernaschi, M., Bisson, M., Rossetti, D.: Benchmarking of communication techniques for GPUS. J. Parallel Distrib. Comput. 73(2), 250–255 (2013). Scholar
  3. 3.
    Bhuyan, M.H., Bhattacharyya, D., Kalita, J.: Surveying port scans and their detection methodologies. Comput. J. 54(10), 1565–1581 (2011). Scholar
  4. 4.
    Cao, J., Jin, Y., Chen, A., Bu, T., Zhang, Z.L.: Identifying high cardinality internet hosts. IEEE INFOCOM 2009, 810–818 (2009). Scholar
  5. 5.
    Carter, J., Wegman, M.N.: Universal classes of hash functions. J. Comput. Syst. Sci. 18(2), 143–154 (1979). Scholar
  6. 6.
  7. 7.
    Estan, C., Varghese, G., Fisk, M.: Bitmap algorithms for counting active flows on high-speed links. IEEE/ACM Trans. Netw. 14(5), 925–937 (2006). Scholar
  8. 8.
    Harang, R.E., Mell, P.: Evasion-resistant network scan detection. Secur. Inf. 4(1), 4 (2015). Scholar
  9. 9.
    Jonker, M., Sperotto, A., van Rijswijk-Deij, R., Sadre, R., Pras, A.: Measuring the adoption of DDoS protection services. In: Proceedings of the 2016 Internet Measurement Conference, IMC 2016, pp. 279–285. ACM, New York (2016).
  10. 10.
    Kane, D.M., Nelson, J., Woodruff, D.P.: An optimal algorithm for the distinct elements problem. In: Proceedings of the Twenty-Ninth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, PODS 2010, pp. 41–52. ACM, New York (2010).
  11. 11.
    Krotofil, M., Cárdenas, A.A., Manning, B., Larsen, J.: CPS: driving cyber-physical systems to unsafe operating conditions by timing dos attacks on sensor signals. In: Proceedings of the 30th Annual Computer Security Applications Conference, ACSAC 2014, pp. 146–155. ACM, New York (2014).
  12. 12.
    Liu, W., Qu, W., Gong, J., Li, K.: Detection of superpoints using a vector bloom filter. IEEE Trans. Inf. Forensics Secur. 11(3), 514–527 (2016). Scholar
  13. 13.
    Liu, Y., Chen, W., Guan, Y.: Identifying high-cardinality hosts from network-wide traffic measurements. IEEE Trans. Depend. Secure Comput. 13(5), 547–558 (2016). Scholar
  14. 14.
    Moraes, D.M., Duarte, Jr, E.P.: A failure detection service for internet-based multi-as distributed systems. In: 2011 IEEE 17th International Conference on Parallel and Distributed Systems, pp. 260–267, December 2011.
  15. 15.
    Roesch, M.: Snort - lightweight intrusion detection for networks. In: Proceedings of the 13th USENIX Conference on System Administration, LISA 1999, pp. 229–238. USENIX Association, Berkeley (1999).
  16. 16.
    Rossow, C., et al.: SoK: P2PWNED - modeling and evaluating the resilience of peer-to-peer botnets. In: 2013 IEEE Symposium on Security and Privacy, pp. 97–111, May 2013.
  17. 17.
    Shin, S.H., Im, E.J., Yoon, M.: A grand spread estimator using a graphics processing unit. J. Parallel Distrib. Comput. 74(2), 2039–2047 (2014). Scholar
  18. 18.
    Silber-Chaussumier, F., Muller, A., Habel, R.: Generating data transfers for distributed GPU parallel programs. J. Parallel Distrib. Comput. 73(12), 1649–1660 (2013). Heterogeneity in Parallel and Distributed ComputingCrossRefGoogle Scholar
  19. 19.
    Snyder, P., Ansari, L., Taylor, C., Kanich, C.: Browser feature usage on the modern web. In: Proceedings of the 2016 Internet Measurement Conference, IMC 2016, pp. 97–110. ACM, New York (2016).
  20. 20.
    Venkataraman, S., Song, D., Gibbons, P.B., Blum, A.: New streaming algorithms for fast detection of superspreaders. In: Proceedings of Network and Distributed System Security Symposium (NDSS), pp. 149–166 (2005)Google Scholar
  21. 21.
    Wang, B., Zheng, Y., Lou, W., Hou, Y.T.: DDoS attack protection in the era of cloud computing and software-defined networking. Comput. Netw. 81, 308–319 (2015). Scholar
  22. 22.
    Wang, P., Guan, X., Qin, T., Huang, Q.: A data streaming method for monitoring host connection degrees of high-speed links. IEEE Trans. Inf. Forensics Secur. 6(3), 1086–1098 (2011). Scholar
  23. 23.
    Whang, K.Y., Vander-Zanden, B.T., Taylor, H.M.: A linear-time probabilistic counting algorithm for database applications. ACM Trans. Database Syst. 15(2), 208–229 (1990). Scholar
  24. 24.
    Xiao, P., Qu, W., Qi, H., Li, Z.: Detecting DDoS attacks against data center with correlation analysis. Comput. Commun. 67, 66–74 (2015). Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Computer Science and EngineeringSoutheast UniversityNanjingChina
  2. 2.School of Cyber Science and EngineeringSoutheast UniversityNanjingChina

Personalised recommendations