© 2017

Traffic Measurement for Big Network Data


Part of the Wireless Networks book series (WN)

Table of contents

  1. Front Matter
    Pages i-vii
  2. Shigang Chen, Min Chen, Qingjun Xiao
    Pages 1-9
  3. Shigang Chen, Min Chen, Qingjun Xiao
    Pages 11-45
  4. Shigang Chen, Min Chen, Qingjun Xiao
    Pages 47-76
  5. Shigang Chen, Min Chen, Qingjun Xiao
    Pages 77-104

About this book


This book presents several compact and fast methods for online traffic measurement of big network data. It describes challenges of online traffic measurement, discusses the state of the field, and provides an overview of the potential solutions to major problems.

The authors introduce the problem of per-flow size measurement for big network data and present a fast and scalable counter architecture, called Counter Tree, which leverages a two-dimensional counter sharing scheme to achieve far better memory efficiency and significantly extend estimation range. 

Unlike traditional approaches to cardinality estimation problems that allocate a separated data structure (called estimator) for each flow, this book takes a different design path by viewing all the flows together as a whole: each flow is allocated with a virtual estimator, and these virtual estimators share a common memory space. A framework of virtual estimators is designed to apply the idea of sharing to an array of cardinality estimation solutions, achieving far better memory efficiency than the best existing work. 

To conclude, the authors discuss persistent spread estimation in high-speed networks. They offer a compact data structure called multi-virtual bitmap, which can estimate the cardinality of the intersection of an arbitrary number of sets. Using multi-virtual bitmaps, an implementation that can deliver high estimation accuracy under a very tight memory space is presented. 

The results of these experiments will surprise both professionals in the field and advanced-level students interested in the topic. By providing both an overview and the results of specific experiments, this book is useful for those new to online traffic measurement and experts on the topic.


Big network data Per-flow traffic measurement Cardinality estimation Persistent spread estimation Traffic measurement Network traffic measurement Network planning Big data Counter tree Multi virtual bitmap Virtual data structures

Authors and affiliations

  1. 1.Department of Computer & Information ScienceUniversity of FloridaGainesvilleUSA
  2. 2.Department of Computer & Information ScienceUniversity of FloridaGainesvilleUSA
  3. 3.School of Computer Science and EngineeringSoutheast University of ChinaNanjingChina

Bibliographic information

Industry Sectors
IT & Software
Materials & Steel
Finance, Business & Banking
Energy, Utilities & Environment
Oil, Gas & Geosciences