Efficient Algorithms for Privately Releasing Marginals via Convex Relaxations

Abstract

Differential privacy is a definition giving a strong privacy guarantee even in the presence of auxiliary information. In this work, we pursue the application of geometric techniques for achieving differential privacy, a highly promising line of work initiated by Hardt and Talwar (Proceedings of the 42nd ACM Symposium on Theory of Computing, STOC’10, pp 705–714. ACM Press, New York, 2010). We apply these techniques to the problem of marginal release. Here, a database refers to a collection of the data of $n$ individuals, each characterized by $d$ binary attributes. A $k$-way marginal query is specified by a subset $S$ of $k$ attributes, together with a $|S|$-dimensional binary vector $\beta$ specifying their values. The true answer to this query is a count of the number of people in the database whose attribute vector restricted to $S$ agrees with $\beta$. Information theoretically, the error complexity of marginal queries—how “wrong” do the answers have to be in order to preserve differential privacy—is well understood: the per-query additive error is known to be at least $\varOmega (\min \{ \sqrt{n},d^{k/2}\})$ and at most $\tilde{O}(\sqrt{n}d^{{\lceil k/2\rceil /4}})$. However, no polynomial time algorithm with error complexity as low as the information-theoretic upper bound is known for small $n$. We present a polynomial time algorithm that matches the best known information-theoretic bounds when $k=2$; more generally, by reducing to the case $k=2$, for any distribution on marginal queries, our algorithm achieves average error at most $\tilde{O}(\sqrt{n}d^{{\lceil k/2\rceil /4}})$, an improvement over previous work when $k$ is small and when error $o(n)$ is desirable. Using private boosting, we are also able to give nearly matching worst-case error bounds. Our algorithms are based on the geometric techniques of Nikolov et al. (Proceedings of the 45th Annual ACM Symposium on Theory of Computing, STOC’13, pp 351–360. ACM Press, New York, 2013), wherein a vector of “sufficiently noisy” answers is projected onto a particular convex body. We reduce the projection step, which is expensive, to a simple geometric question: given (a succinct representation of) a convex body $K$, find a containing convex body $L$ that one can efficiently optimize over, while keeping the Gaussian width of $L$ small. This reduction is achieved by a careful use of the Frank–Wolfe algorithm.