Frequent Itemset Mining over Data Streams

Abstract

We study the problem of computing frequent elements in a data-stream. Given support threshold $s \in [0, 1]$, an element is said to be frequent if it occurs more than $sN$ times, where $N$ denotes the current length of the stream. If we maintain a list of counters of the form 〈element, count〉, one counter per unique element encountered, we need $N$ counters in the worst-case. Many distributions are heavy-tailed in practice, so we would need far fewer than $N$ counters. However, the number would still exceed $1/s$, which is the maximum possible number of frequent elements. If we insist on identifying exact frequency counts, then $\varOmega(N)$ space is necessary. This observation motivates the design of streaming techniques based on $\epsilon$ -approximate frequency counts. We also discuss the extension of the ideas to the problem of mining frequent itemsets over streams, and relevant applications.