BPMiner: Algorithms for Large-Scale Private Analysis

Abstract

An abundance of data generated from a multitude of sources, and intelligence derived by analyzing the same, has become an important asset across many walks of life. Simultaneously, it raises serious concerns about privacy. Differential privacy has become a popular way to reason about the amount of information about individual entries of a dataset that is divulged upon giving out a perturbed result for a query on a given data-set. However, current differentially-private algorithms are computationally inefficient, and do not explicitly exploit the abundance of data, thus wearing out the privacy budget irrespective of the volume of data. In this paper, we propose BPMiner, a solution that is both private and accurate, while simultaneously addressing the computation and budget challenges of very big datasets. The main idea is a non-trivial combination between differential privacy, sample-and-aggregation, and a classical statistical methodology called sequential estimation. Rigorous proof regarding the privacy and asymptotic accuracy of our solution are provided. Furthermore, experimental results over multiple datasets demonstrate that BPMiner outperforms current private algorithms in terms of computational and budget efficiencies, while achieving comparable accuracy. Overall, BPMiner is a practical solution based on strong theoretical foundations for privacy-preserving analysis on big datasets.

A Proof of Theorem 1

For estimators from [18, 29, 37], the Laplace noise Y is added directly to $\bar{Z}$ as in Algorithm  1. We have the following useful lemma.

Lemma 5

Suppose the Laplace noise is given by $Y = Y_k = \mathrm {Laplace}\left( \mathsf {Range}/k\epsilon \right)$. Then $Y_k \xrightarrow {P} 0$ as $k \rightarrow \infty$.

Proof

(of Lemma 5 ). Given $\epsilon > 0$, consider the probability $\Pr (|Y_k - 0| < \epsilon )$. This probability equals

$$\begin{aligned} \Pr (|Y_k| < \epsilon ) = \Pr (-\epsilon < Y_k < \epsilon ) = F_{Y_k}(\epsilon ) - F_{Y_k}(-\epsilon ). \end{aligned}$$

The CDF of a Laplace random variable X is given by $F_X(x) = \tfrac{1}{2} \exp \left( \tfrac{x-\mu }{\sigma } \right)$ if $x < \mu$, and $F_X(x) = 1 - \tfrac{1}{2} \exp \left( - \tfrac{x-\mu }{\sigma } \right)$ otherwise. Since $Y_k$ has a Laplace distribution with parameters $\mu = 0$ and $\sigma = \mathsf {Range}/(k\epsilon )$, computing $F_{Y_k}(\epsilon )$ and $F_{Y_k}(-\epsilon )$ (with $\epsilon > 0 = \mu$) gives us

$$\begin{aligned} \Pr (|Y_k - \mu | < \epsilon ) = F_{Y_k}(\epsilon ) - F_{Y_k}(-\epsilon ) = 1 - \exp \left( - \tfrac{k\epsilon ^2}{\mathsf {Range}} \right) \rightarrow 1 \end{aligned}$$

as $k \rightarrow \infty$. Therefore, it follows from the definition of convergence in probability that $Y_k \xrightarrow {P} 0$ as $k \rightarrow \infty$.

Using Lemma 5, the proof is straightforward for the statistics proposed in [18, 29, 37]: $\Pr ( |\hat{\theta }_T(X) - \theta | \le \delta )$ $=$ $\Pr ( | \bar{Z} + Y_k - \theta |$ $\le \delta )$ $\rightarrow$ $\Pr ( |\bar{Z} - \theta | \le \delta )$.

For the estimator from [38], the noise Y is added to the winsorized mean rather than to $\bar{Z}$, so the above argument does not apply. Nevertheless, the results extracted from [38] are particularly useful in deriving the proof. First, we obseve that

$$\begin{aligned} \nonumber \Pr ( |\hat{\theta }_T(X) - \theta | \le \delta ) \nonumber&= \Pr ( \theta - \delta \le \hat{\theta }_T(X) \le \theta + \delta ) \\&= F_{\hat{\theta }_T}(\theta + \delta ) - F_{\hat{\theta }_T}(\theta - \delta ), \end{aligned}$$

(22)

where $F_X$ denotes the CDF of X. By Corollary 10 in [38], the $\mathsf {KS}$ distance between $\hat{\theta }_T$ and $\bar{Z}$ goes to 0 as k goes to infinity. This leads to the fact that $\hat{\theta }_T$ converges in distribution to $\bar{Z}$. In other words, when $k \rightarrow \infty$, $F_{\hat{\theta }_T}(t) \rightarrow F_{\bar{Z}}(t)$, and the expression (22) converges to

$$\begin{aligned} F_{\bar{Z}}(\theta + \delta ) - F_{\bar{Z}}(\theta - \delta )&= \Pr ( \theta - \delta \le \bar{Z} \le \theta + \delta ) \\&= \Pr ( |\bar{Z} - \theta | \le \delta ), \end{aligned}$$

which completes the proof.