A Space-Efficient Fair Packet Sampling Algorithm

  • Jin Zhang
  • Xiaona Niu
  • Jiangxing Wu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5297)


Due to the high-skewed nature of network flow size distributions, uniform packet sampling concentrates too much on a few large flows and ignores the majority of small ones. To overcome this drawback, recently proposed Sketch Guided Sampling (SGS) selects each packet at a probability that is decreasing with its current flow size, which results in better flow wide fairness. However, the pitfall of SGS is that it needs a large, high-speed memory to accommodate flow size sketch, making it impractical to be implemented and inflexible to be deployed. We refined the flow size sketch using a multi-resolution d-left hashing schema, which is both space-efficient and accurate. A new fair packet sampling algorithm which is named Space-Efficient Fair Sampling (SEFS) is proposed based on this novel flow size sketch. We compared the performance of SEFS with that of SGS in the context of flow traffic measurement and large flow identification using real-world traffic traces. The experimental results show that SEFS outperforms SGS in both application contexts while a reduction of 65 percent in space complexity can be achieved.


Packet sampling flow size estimation multi-resolution sampling d-left hashing 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Claffy, K.C., Polyzos, G.C., Braun, H.-W.: Application of Sampling Methodologies to Network Traffic Characterization. In: Proc. ACM SIGCOMM (1993)Google Scholar
  2. 2.
    Cisco System White Paper. NetFlow Services Solutions GuideGoogle Scholar
  3. 3.
    Estan, C., Varghese, G.: New directions in traffic measurement and accounting. In: Proc. ACM SIGCOMM (August 2002)Google Scholar
  4. 4.
    Kodialam, M., Lakshman, T.V., Mohanty, S.: Runs bAsed Traffic Estimator (RATE): A simple, Memory Efficient Scheme for Per-Flow Rate Estimation. In: IEEE Proceedings of INFOCOM (2004)Google Scholar
  5. 5.
    Raspall, F., Sallent, S., Yufera, J.: Shared State Sampling. In: Proc. ACM Internet Measurement Conference (2006)Google Scholar
  6. 6.
    Hohn, N., Veitch, D.: Inverting Sampled Traffic. In: ACM Internet Measurement Conference (2003)Google Scholar
  7. 7.
    Duffield, N., Lund, C., Thorup, M.: Properties and Prediction of Flow Statistics from Sampled Packet Streams. In: Proc. ACM Internet Measurement Conference (2002)Google Scholar
  8. 8.
    Barakat, C., Iannaccone, G., Diot, C.: Ranking flows from sampled traffic. In: Proceedings of the 2005 ACM conference on Emerging network experiment and technology, pp. 188–199 (2005)Google Scholar
  9. 9.
    Brauckhoff, D., Tellenbach, B.: Impact of Packet Sampling on Anomaly Detection Metrics. In: ACM Internet Measurement Conference (2006)Google Scholar
  10. 10.
    Mai, J., Chua, C.-N.: Is Sampled Data Sufficient for Anomaly Detection. In: ACM Internet Measurement Conference (2006)Google Scholar
  11. 11.
    Chen, G., Gong, J., Ding, W.: A Real-Time Anomaly Detection Model Based on Sampling Measurement in a High-Speed Network. Chinese Journal of Software 3 (2003)Google Scholar
  12. 12.
    Duffield, N., Lund, C., Thorup, M.: Learn More, Sample Less: Control of Volume and Variance in Network Measurement. IEEE Transactions on Information Theory 51(5) (May 2005)Google Scholar
  13. 13.
    Estan, C., Keys, K.: Building a Better NetFlow. In: Proc. ACM SIGCOMM (2005)Google Scholar
  14. 14.
    Wang, J., Yang, J., Zhou, H., Xie, G., Zhou, M.: Adaptive Sampling Methodology in Network Measurements. Chinese Journal of Software 15(8) (2004)Google Scholar
  15. 15.
    Kumar, A., Xu, J.J.: Sketch Guided Sampling – Using On-Line Estimates of Flow Size for Adaptive Data Collection. In: IEEE Infocom 2006 (2006)Google Scholar
  16. 16.
    Kong, S., He, T., Shao, X., Li, X.: Time-out Bloom Filter: A New Sampling Method for Recording More Flows. In: Chong, I., Kawahara, K. (eds.) ICOIN 2006. LNCS, vol. 3961, pp. 590–599. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  17. 17.
    Bonomi, F., Mitzenmacher, M., Panigrahy, R., Singh, S., Varghese, G.: An Improved Construction for Counting Bloom Filters. In: European Symposium on Algorithms (2006)Google Scholar
  18. 18.
    Estain, C.: Bitmap Algorithms for Counting Active Flows on High Speed Links. In: Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement, pp. 153–166 (2003)Google Scholar
  19. 19.
    Kumar, K., Xu, J., Jia, W., Spatschek, O., Li, L.: Space-Code bloom filter for efficient per-flow traffic measurement. In: Proc. of the INFOCOM 2004 (2004)Google Scholar
  20. 20.
    Zhang, J.: Performance Evaluation and Comparison of Three Counting Bloom Filter Schemes. Techinical Report (2008)Google Scholar
  21. 21.
  22. 22.
    Shah, D., Iyer, S., Prahhakar, B., McKeown, N.: Maintaining statistics counters in router line cards. IEEE Micro. 22(1), 76–81 (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Jin Zhang
    • 1
  • Xiaona Niu
    • 1
  • Jiangxing Wu
    • 1
  1. 1.National Digital Switching System Engineering and Technology Research Center (NDSC)ZhenzhouChina

Personalised recommendations