Abstract
The super point, a host which communicates with lots of others, is a kind of special hosts gotten great focus. Mining super point at the edge of a network is the foundation of many network research fields. In this paper, we proposed the most memory efficient super points detection scheme. This scheme contains a super points reconstruction algorithm called short estimator and a super points filter algorithm called long estimator. Short estimator gives a super points candidate list using thousands of bytes memory and long estimator improves the accuracy of detection result using millions of bytes memory. Combining short estimator and long estimator, our scheme acquires the highest accuracy using the smallest memory than other algorithms. There is no data confliction and floating operation in our scheme. This ensures that our scheme is suitable for parallel running and we deploy our scheme on a common GPU to accelerate processing speed. Experiments on several real-world core network traffics show that our algorithm acquires the highest accuracy with only consuming littler than one-fifth memory of other algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
The Center for Applied Internet Data Analysis: The caida anonymized internet traces (2017). http://www.caida.org/data/passive. Accessed 2017
Bernaschi, M., Bisson, M., Rossetti, D.: Benchmarking of communication techniques for GPUS. J. Parallel Distrib. Comput. 73(2), 250–255 (2013). https://doi.org/10.1016/j.jpdc.2012.09.006. http://www.sciencedirect.com/science/article/pii/S0743731512002213
Bhuyan, M.H., Bhattacharyya, D., Kalita, J.: Surveying port scans and their detection methodologies. Comput. J. 54(10), 1565–1581 (2011). https://doi.org/10.1093/comjnl/bxr035
Cao, J., Jin, Y., Chen, A., Bu, T., Zhang, Z.L.: Identifying high cardinality internet hosts. IEEE INFOCOM 2009, 810–818 (2009). https://doi.org/10.1109/INFCOM.2009.5061990
Carter, J., Wegman, M.N.: Universal classes of hash functions. J. Comput. Syst. Sci. 18(2), 143–154 (1979). https://doi.org/10.1016/0022-0000(79)90044-8. http://www.sciencedirect.com/science/article/pii/0022000079900448
Cisco: Global IP traffic forecast (2017). http://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/vni-hyperconnectivity-wp.pdf
Estan, C., Varghese, G., Fisk, M.: Bitmap algorithms for counting active flows on high-speed links. IEEE/ACM Trans. Netw. 14(5), 925–937 (2006). https://doi.org/10.1109/TNET.2006.882836
Harang, R.E., Mell, P.: Evasion-resistant network scan detection. Secur. Inf. 4(1), 4 (2015). https://doi.org/10.1186/s13388-015-0019-7
Jonker, M., Sperotto, A., van Rijswijk-Deij, R., Sadre, R., Pras, A.: Measuring the adoption of DDoS protection services. In: Proceedings of the 2016 Internet Measurement Conference, IMC 2016, pp. 279–285. ACM, New York (2016). https://doi.org/10.1145/2987443.2987487
Kane, D.M., Nelson, J., Woodruff, D.P.: An optimal algorithm for the distinct elements problem. In: Proceedings of the Twenty-Ninth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, PODS 2010, pp. 41–52. ACM, New York (2010). https://doi.org/10.1145/1807085.1807094
Krotofil, M., Cárdenas, A.A., Manning, B., Larsen, J.: CPS: driving cyber-physical systems to unsafe operating conditions by timing dos attacks on sensor signals. In: Proceedings of the 30th Annual Computer Security Applications Conference, ACSAC 2014, pp. 146–155. ACM, New York (2014). https://doi.org/10.1145/2664243.2664290
Liu, W., Qu, W., Gong, J., Li, K.: Detection of superpoints using a vector bloom filter. IEEE Trans. Inf. Forensics Secur. 11(3), 514–527 (2016). https://doi.org/10.1109/TIFS.2015.2503269
Liu, Y., Chen, W., Guan, Y.: Identifying high-cardinality hosts from network-wide traffic measurements. IEEE Trans. Depend. Secure Comput. 13(5), 547–558 (2016). https://doi.org/10.1109/TDSC.2015.2423675
Moraes, D.M., Duarte, Jr, E.P.: A failure detection service for internet-based multi-as distributed systems. In: 2011 IEEE 17th International Conference on Parallel and Distributed Systems, pp. 260–267, December 2011. https://doi.org/10.1109/ICPADS.2011.5
Roesch, M.: Snort - lightweight intrusion detection for networks. In: Proceedings of the 13th USENIX Conference on System Administration, LISA 1999, pp. 229–238. USENIX Association, Berkeley (1999). https://dl.acm.org/citation.cfm?id=1039834.1039864
Rossow, C., et al.: SoK: P2PWNED - modeling and evaluating the resilience of peer-to-peer botnets. In: 2013 IEEE Symposium on Security and Privacy, pp. 97–111, May 2013. https://doi.org/10.1109/SP.2013.17
Shin, S.H., Im, E.J., Yoon, M.: A grand spread estimator using a graphics processing unit. J. Parallel Distrib. Comput. 74(2), 2039–2047 (2014). https://doi.org/10.1016/j.jpdc.2013.10.007. http://www.sciencedirect.com/science/article/pii/S0743731513002189
Silber-Chaussumier, F., Muller, A., Habel, R.: Generating data transfers for distributed GPU parallel programs. J. Parallel Distrib. Comput. 73(12), 1649–1660 (2013). https://doi.org/10.1016/j.jpdc.2013.07.022. http://www.sciencedirect.com/science/article/pii/S0743731513001603. Heterogeneity in Parallel and Distributed Computing
Snyder, P., Ansari, L., Taylor, C., Kanich, C.: Browser feature usage on the modern web. In: Proceedings of the 2016 Internet Measurement Conference, IMC 2016, pp. 97–110. ACM, New York (2016). https://doi.org/10.1145/2987443.2987466
Venkataraman, S., Song, D., Gibbons, P.B., Blum, A.: New streaming algorithms for fast detection of superspreaders. In: Proceedings of Network and Distributed System Security Symposium (NDSS), pp. 149–166 (2005)
Wang, B., Zheng, Y., Lou, W., Hou, Y.T.: DDoS attack protection in the era of cloud computing and software-defined networking. Comput. Netw. 81, 308–319 (2015). https://doi.org/10.1016/j.comnet.2015.02.026. http://www.sciencedirect.com/science/article/pii/S1389128615000742
Wang, P., Guan, X., Qin, T., Huang, Q.: A data streaming method for monitoring host connection degrees of high-speed links. IEEE Trans. Inf. Forensics Secur. 6(3), 1086–1098 (2011). https://doi.org/10.1109/TIFS.2011.2123094
Whang, K.Y., Vander-Zanden, B.T., Taylor, H.M.: A linear-time probabilistic counting algorithm for database applications. ACM Trans. Database Syst. 15(2), 208–229 (1990). https://doi.org/10.1145/78922.78925
Xiao, P., Qu, W., Qi, H., Li, Z.: Detecting DDoS attacks against data center with correlation analysis. Comput. Commun. 67, 66–74 (2015). https://doi.org/10.1016/j.comcom.2015.06.012. http://www.sciencedirect.com/science/article/pii/S0140366415002285
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Xu, J., Ding, W., Hu, X. (2018). Most Memory Efficient Distributed Super Points Detection on Core Networks. In: Vaidya, J., Li, J. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2018. Lecture Notes in Computer Science(), vol 11334. Springer, Cham. https://doi.org/10.1007/978-3-030-05051-1_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-05051-1_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-05050-4
Online ISBN: 978-3-030-05051-1
eBook Packages: Computer ScienceComputer Science (R0)