Frequent Item Mining When Obtaining Support Is Costly

Abstract

Suppose there are n users and m items, and the preference of each user for the items is revealed only upon probing, which takes time and is therefore costly. How can we quickly discover all the frequent items that are favored individually by at least a given number of users? This new problem not only has strong connections with several well-known problems, such as the frequent item mining problem, it also finds applications in fields such as sponsored search and marketing surveys. Unlike traditional frequent item mining, however, our problem assumes no prior knowledge of users’ preferences, and thus obtaining the support of an item becomes costly. Although our problem can be settled naively by probing the preferences of all n users, the number of users is typically enormous, and each probing itself can also incur a prohibitive cost. We present a sampling algorithm that drastically reduces the number of users needed to probe to $O(\log m)$—regardless of the number of users—as long as slight inaccuracy in the output is permitted. For reasonably sized input, our algorithm needs to probe only $0.5\%$ of the users, whereas the naive approach needs to probe all of them.

Appendices

A Appendix: Probabilistic Inequalities

We outline several probabilistic inequalities that will be employed in our proofs.

Proposition 3

(Chernoff’s bound [9] and Hoeffding’s inequality [14]). Let $X_1,X_2,\dots ,X_s$ be independent Bernoulli variables and $\mathbf{Pr }\left[ X_i=1\right] =p$, where $0<p<1$, for $i=1,2,\dots ,s$. Let $X=\sum _{i=1}^{s}X_i$ and $\mathbf E \left[ X\right] =\mu$ such that $\mu _\ell \le \mu \le \mu _h$, where $\mu _\ell ,\mu _h\in \mathbb {R}$. Then, for any $\varepsilon <1$,

$$\begin{aligned} \mathbf{Pr }\left[ X\ge (1+\varepsilon )\mu _h\right]\le & {} e^{\frac{-\mu _{\!h}\varepsilon ^2}{3}}, \quad \mathbf{Pr }\left[ X\le (1-\varepsilon )\mu _\ell \right] \le e^{\frac{-\mu _{\!\ell }\varepsilon ^2}{2}}, \end{aligned}$$

(1)

$$\begin{aligned} \mathbf{Pr }\left[ X\ge \mu +\varepsilon \right]\le & {} e^{-\frac{2\varepsilon ^2}{s}}, \quad \mathbf{Pr }\left[ X\le \mu -\varepsilon \right] \le e^{-\frac{2\varepsilon ^2}{s}}~. \end{aligned}$$

(2)

B Appendix: Proofs

Proof

(Lemma 1). Let X be the discovered support of a given frequent item $t$ (i.e., an item whose support is at least pn). For $i=1,2,\dots ,s$, let $X_i$ be the indicator random variable such that $X_i=1$ if the ith sampled user favors item $t$, and $X_i=0$ otherwise. We first note that $X=\sum _{i=1}^{s}X_i$. In addition, since our algorithm samples users independently and uniformly at random, we have So, we conclude that $\mathbf E \left[ X\right] =\mathbf E \left[ \sum _{i=1}^{s}X_i\right] =\sum _{i=1}^{s}\mathbf E \left[ X_i\right] \ge ps.$

Now, we bound the probability that item t is not returned. According to our algorithm, this event occurs if and only if the discovered support of item $t$ is less than $(p-\varepsilon /2)s$:

$$\begin{aligned} \mathbf{Pr }\left[ X<\left( p-\frac{\varepsilon }{2}\right) s\right]= & {} \mathbf{Pr }\left[ X<\left( 1-\frac{\varepsilon }{2p}\right) ps\right] \le e^{-\frac{ps\left( \frac{\varepsilon }{2p}\right) ^2}{2}} = e^{-\frac{s\varepsilon ^2}{8p}} ~. \end{aligned}$$

The inequality above is established by Chernoff’s bound (Proposition 3). Now, without loss of generality, suppose that there are k frequent items in the item set $\mathcal {T}$, where $1\le k\le m$ is unknown. As will be clear shortly, the value of k is irrelevant to the analysis of our algorithm. By using the union bound (a.k.a Boole’s inequality), therefore, the probability that at least one of the k frequent items is not returned by our algorithm is bounded above by    $\square$

Proof

(Lemma 2). Now, let Y be the discovered support of a given infrequent item (i.e., an item whose support is less than $(p-\varepsilon )n$). By using the same line of argument for random variable X in Lemma 1, we can show that $\mathbf E \left[ Y\right] <(p-\varepsilon )s$.

Now, we bound the probability that the given infrequent item is returned. By the design of our algorithm, this event occurs if and only if the discovered support of that item is at least $(p-\varepsilon /2)s$:

$$\begin{aligned} \mathbf{Pr }\left[ Y\ge \left( p-\frac{\varepsilon }{2}\right) s\right]= & {} \mathbf{Pr }\left[ Y\ge \left( 1+\frac{\varepsilon }{2(p-\varepsilon )}\right) (p-\varepsilon )s \right] \le e^{-\frac{(p-\varepsilon )s \left[ \frac{\varepsilon }{2(p-\varepsilon )}\right] ^2 }{3}}\\= & {} e^{-\frac{s\varepsilon ^2}{12(p-\varepsilon )}} ~. \end{aligned}$$

The inequality is established by Chernoff’s bound (Proposition 3). Since there are k items whose support is at least pn, it follows that there are at most $m-k$ items whose support is less than $(p-\varepsilon )n$ (because there are m items). By the union bound, the probability that our algorithm returns at least one of the infrequent items is at most $(m-k)e^{-\frac{s\varepsilon ^2}{12(p-\varepsilon )}}~.$    $\square$

Proof

(Lemma 3). From Lemmas 1 and 2, we know that the probability that our algorithm fails to achieve Property P1 (resp. Property P2) of the $\varepsilon$-approximation guarantee is at most $ke^{-\frac{s\varepsilon ^2}{8p}}$ (resp. $(m-k)e^{-\frac{s\varepsilon ^2}{12(p-\varepsilon )}}$). By appealing to the union bound again, the probability that our algorithm fails to achieve the $\varepsilon$-approximation guarantee is at most $ke^{-\frac{s\varepsilon ^2}{8p}} + (m-k)e^{-\frac{s\varepsilon ^2}{12(p-\varepsilon )}}~.$ We simplify the expression to identify the sample size s:

$$\begin{aligned} e^{-\frac{s\varepsilon ^2}{8p}} \le e^{-\frac{s\varepsilon ^2}{12(p-\varepsilon )}} \iff 12(p-\varepsilon )\ge 8p \iff p\ge 3\varepsilon ~. \end{aligned}$$

(3)

By referring to Eq. (3), we know that $e^{-\frac{s\varepsilon ^2}{8p}}> e^{-\frac{s\varepsilon ^2}{12(p-\varepsilon )}}$. Hence

$$\begin{aligned} ke^{-\frac{s\varepsilon ^2}{8p}} + (m-k)e^{-\frac{s\varepsilon ^2}{12(p-\varepsilon )}}< & {} ke^{-\frac{s\varepsilon ^2}{8p}} + (m-k)e^{-\frac{s\varepsilon ^2}{8p}} = me^{-\frac{s\varepsilon ^2}{8p}}. \end{aligned}$$

Again by Eq. (3), we have

$$\begin{aligned} ke^{-\frac{s\varepsilon ^2}{8p}} + (m-k)e^{-\frac{s\varepsilon ^2}{12(p-\varepsilon )}}\le & {} ke^{-\frac{s\varepsilon ^2}{12(p-\varepsilon )}} + (m-k)e^{-\frac{s\varepsilon ^2}{12(p-\varepsilon )}} = me^{-\frac{s\varepsilon ^2}{12(p-\varepsilon )}}. \end{aligned}$$

Taking the maximum of both cases, we finish the proof.   $\square$

Proof

(Theorem 1). We use the result of Lemma 3.

In this case, the probability that Support-Sampling fails is at most $me^{-\frac{s\varepsilon ^2}{8p}}$. By setting this quantity to $\delta$ and then solve for s, we have $s=\frac{8p}{\varepsilon ^2}\ln \frac{m}{\delta }~.$

Similarly, the probability that Support-Sampling fails in this case is at most $me^{-\frac{s\varepsilon ^2}{12(p-\varepsilon )}}$. By setting this quantity to $\delta$ and then solve for s, we have $s=\frac{12(p-\varepsilon )}{\varepsilon ^2}\ln \frac{m}{\delta }~.$ Taking the maximum of both cases, we finish the proof.   $\square$

Proof

(Lemma 4). For $i=1,2,\dots ,s$, let $X_i$ and $Z_i$ be Bernouilli distributions with mean $p_h$ and $p_\ell /p_h$, respectively. Let $Y_i:=X_iZ_i$ so that $Y_i\le X_i$ with probability 1, and that $Y_i$ is a Bernouilli distribution with mean $p_\ell$. Now, notice that $X=\sum _{i=1}^{n}X_i$ and $Y=\sum _{i=1}^{n}Y_i$. Also, since $Y_i\le X_i$, it follows that $Y\le X$ with probability 1. Therefore, the event $Y\ge c$ implies the event $X\ge c$. Consequently, we have $\mathbf{Pr }\left[ Y\ge c\right] \le \mathbf{Pr }\left[ X\ge c\right]$.    $\square$

Proof

(Proposition 1). Let Y be the discovered support of a given potentially-frequent item $t$ with $\text {support}(t)=pn-r$, where $\varepsilon n/2<r\le \varepsilon n$. Then, using the same argument in Lemma 1, we can show that $\mathbf E \left[ Y\right] =\left( \frac{pn-r}{n}\right) s=\left( p-\frac{r}{n}\right) s.$ Hence, the probability that item $t$ is returned by Support-Sampling is given by $\mathbf{Pr }\left[ Y\ge \left( p-\frac{\varepsilon }{2}\right) s\right] = \mathbf{Pr }\left[ Y\ge \left( p-\frac{\varepsilon }{2}+\frac{r}{n}-\frac{r}{n}\right) s\right] = \mathbf{Pr }\left[ Y\ge \left( p-\frac{r}{n}\right) s+\left( \frac{r}{n}-\frac{ \varepsilon }{2}\right) s\right] = \mathbf{Pr }\left[ Y\ge \mathbf E \left[ Y\right] +\left( \frac{r}{n}-\frac{ \varepsilon }{2}\right) s\right] .$ Now, since $r>\frac{\varepsilon n}{2}$, it follows that $\left( \frac{r}{n}-\frac{\varepsilon }{2}\right) s>0$. Hence, we can apply Hoeffding’s inequality (Proposition 3) to bound the equation: $\mathbf{Pr }\left[ Y\ge \mathbf E \left[ Y\right] +\left( \frac{r}{n}-\frac{ \varepsilon }{2}\right) s\right] \le e^{- \frac{2\left( \frac{r}{n}-\frac{ \varepsilon }{2}\right) ^2s^2}{s} } = e^{-2s\left( \frac{r}{n}-\frac{\varepsilon }{2}\right) ^2}.$    $\square$

Proof

(Proposition 2). The proof is similar to that of Proposition 1 and is omitted due to the space constraint.   $\square$