Finding Frequent Elements in Compressed 2D Arrays and Strings

Abstract

We show how to store a compressed two-dimensional array such that, if we are asked for the elements with high relative frequency in a range, we can quickly return a short list of candidates that includes them. More specifically, given an m ×n array A and a fraction α > 0, we can store A in $\ensuremath{\mathcal{O}\!\left( {m n (H + 1) \log^2 (1 / \alpha)} \right)}$ bits, where H is the entropy of the elements’ distribution in A, such that later, given a rectangular range in A and a fraction β ≥ α, in $\ensuremath{\mathcal{O}\!\left( {1 / \beta} \right)}$ time we can return a list of $\ensuremath{\mathcal{O}\!\left( {1 / \beta} \right)}$ distinct array elements that includes all the elements that have relative frequency at least β in that range. We do not verify that the elements in the list have relative frequency at least β, so the list may contain false positives. In the case when m = 1, i.e., A is a string, we improve this space bound by a factor of log(1/α), and explore a space-time trade off for verifying the frequency of the elements in the list. This leads to an $\ensuremath{\mathcal{O}\!\left( {n \min(\log(1/\alpha), H+1)\log n} \right)}$ bit data structure for strings that, in $\ensuremath{\mathcal{O}\!\left( {1/\beta} \right)}$ time, can return the $\ensuremath{\mathcal{O}\!\left( {1/\beta} \right)}$ elements that have relative frequency at least β in a given range, without false positives, for β ≥ α.