Almost Perfect Privacy for Additive Gaussian Privacy Filters

Abstract

We study the maximal mutual information about a random variable Y (representing non-private information) displayed through an additive Gaussian channel when guaranteeing that only $\varepsilon$ bits of information is leaked about a random variable X (representing private information) that is correlated with Y. Denoting this quantity by $g_\varepsilon (X,Y)$, we show that for perfect privacy, i.e., $\varepsilon =0$, one has $g_0(X,Y)=0$ for any pair of absolutely continuous random variables (X, Y) and then derive a second-order approximation for $g_\varepsilon (X,Y)$ for small $\varepsilon$. This approximation is shown to be related to the strong data processing inequality for mutual information under suitable conditions on the joint distribution $P_{XY}$. Next, motivated by an operational interpretation of data privacy, we formulate the privacy-utility tradeoff in the same setup using estimation-theoretic quantities and obtain explicit bounds for this tradeoff when $\varepsilon$ is sufficiently small using the approximation formula derived for $g_\varepsilon (X,Y)$.

S. Asoodeh—This work was supported in part by NSERC of Canada.

Appendices

A Connection Between Mutual Information and Non-Gaussianness

For any pair of random variables (U, V) with $I(U; V){<\infty }$, let $P_{V|U}(\cdot |u)$ be the conditional density of V given $U=u$. Then, we have

$$\begin{aligned} I(U; V)= & {} {\mathbb {E}}_{UV}\left[ \log \frac{P_{V|U}(V|U)}{P_V(V)}\right] \nonumber \\= & {} {\mathbb {E}}_{UV}\left[ \log \frac{P_{V|U}(V|U)}{P_{V_{\mathsf {G}}|U_{\mathsf {G}}}(V|U)}\right] +{\mathbb {E}}_{UV}\left[ \log \frac{P_{V_{\mathsf {G}}|U_{\mathsf {G}}}(V|U)}{P_{V_{\mathsf {G}}}(V)}\right] -{\mathbb {E}}_{UV}\left[ \log \frac{P_{V}(V)}{P_{V_{\mathsf {G}}}(V)}\right] \nonumber \\= & {} I(U_{{\mathsf {G}}}; V_{\mathsf {G}}) +D(V|U)-D(V), \end{aligned}$$

(37)

where $(U_{\mathsf {G}}, V_{\mathsf {G}})$ is a pair of Gaussian random variable having the same means, variances and correlation coefficient as (U, V), and $P_{V_{\mathsf {G}}|U_{\mathsf {G}}}(\cdot |u)$ is the conditional density of $V_{\mathsf {G}}$ given $U_{\mathsf {G}}=u$, and the quantity D(V|U) is defined in (34). Replacing U and V with X and $Z_\gamma$, respectively, the decomposition (37) allows us to conclude that

$$\begin{aligned} I(X; Z_\gamma )=I(X_{{\mathsf {G}}}; \sqrt{\gamma }Y_{{\mathsf {G}}}+N_{{\mathsf {G}}})+D(Z_{\gamma }|X)-D(Z_{\gamma }), \end{aligned}$$

and therefore, if $Y=Y_{\mathsf {G}}$ is Gaussian, we have

$$\begin{aligned} I(X; Z_\gamma )=I(X_{{\mathsf {G}}}; Z_\gamma )+D(Z_\gamma |X)\ge I(X_{{\mathsf {G}}}; Z_\gamma ), \end{aligned}$$

from which we conclude that when Y is Gaussian then $I(X; Z_\gamma )\le \varepsilon$ implies that $I(X_{\mathsf {G}}; Z_\gamma )\le \varepsilon$ and hence $g_\varepsilon (X, Y_{\mathsf {G}})\le g_\varepsilon (X_{\mathsf {G}}, Y_{\mathsf {G}})$.

B Completion of the Proof of Theorem 4

Lemma 1

For Gaussian $X_{\mathsf {G}}$ and absolutely continuous Y with unit variance, we have

$$\begin{aligned} D(Z_\gamma |X_{\mathsf {G}})\le \frac{\gamma }{2}\left[ {\mathsf {mmse}}(Y_{\mathsf {G}}|X_{\mathsf {G}})-{\mathsf {mmse}}(Y|X_{\mathsf {G}})\right] +o(\gamma ). \end{aligned}$$

Proof

Let E be an auxiliary random variable defined as

$$\begin{aligned} E= {\left\{ \begin{array}{ll} 1,~~~|Y|\le L\\ 0,~~~\text {otherwise}, \end{array}\right. } \end{aligned}$$

for some real number $M>0$. Note that

$$\begin{aligned} D(Z_\gamma |X_{\mathsf {G}}=x)&=h(\sqrt{\gamma }Y_{\mathsf {G}}+N_{\mathsf {G}}|X_{\mathsf {G}}=x)-h(Z_\gamma |X_{\mathsf {G}}=x) \nonumber \\&\le h(\sqrt{\gamma }Y_{\mathsf {G}}+N_{\mathsf {G}}|X_{\mathsf {G}}=x)-h(Z_\gamma |X_{\mathsf {G}}=x, E)\nonumber \\&= \frac{1}{2}\log (2\pi e(1+\gamma {\mathsf {var}}(Y_{\mathsf {G}}|X_{\mathsf {G}}=x)))\nonumber \\&\quad -\Pr (E=1)h(Z_\gamma |X_{\mathsf {G}}=x, E=1)-\Pr (E=0)h(Z_\gamma |X_{\mathsf {G}}=x, E=0) \nonumber \\&\mathop {\le }\limits ^{(a)} \frac{1}{2}\log (2\pi e(1+\gamma {\mathsf {var}}(Y_{\mathsf {G}}|X_{\mathsf {G}}=x))-\Pr (E=0)h(N_{\mathsf {G}})\nonumber \\&\quad -\Pr (E=1)h(Z_\gamma |X_{\mathsf {G}}=x, E=1) \end{aligned}$$

(38)

where (a) follows from the fact that $h(Z_\gamma |X_{\mathsf {G}}=x, E=0)\ge h(N_{\mathsf {G}})$.

Prelov [24] showed that for any random variable Y such that

$$\begin{aligned} {\mathbb {E}}[|Y|^{2+\alpha }]\le K{<\infty }, \end{aligned}$$

(39)

for some $\alpha >0$, then

$$\begin{aligned} h(\sqrt{\gamma }Y+N_{\mathsf {G}})=\frac{1}{2}\log (2\pi e)+\frac{{\mathsf {var}}(Y)}{2}(\gamma +o(\gamma )), \end{aligned}$$

(40)

where $o(\gamma )$ term depends only on K. Since $Y|\{E=1\}$ satisfies (39), we can use (40) to evaluate $h(Z_\gamma |X_{\mathsf {G}}=x, E=1)$ in (38) which yields

$$\begin{aligned} D(Z_\gamma |X_{\mathsf {G}}=x)\le & {} \frac{1}{2}\log (2\pi e(1+\gamma {\mathsf {var}}(Y_{\mathsf {G}}|X_{\mathsf {G}}=x))-\Pr (E=0)\frac{1}{2}\log (2\pi e)\nonumber \\&-\Pr (E=1)\left[ \frac{1}{2}\log (2\pi e)+\frac{{\mathsf {var}}(Y|X_{\mathsf {G}}=x, E=1)}{2}(\gamma +o(\gamma ))\right] \nonumber \\= & {} \frac{1}{2}\log (1+\gamma {\mathsf {var}}(Y_{\mathsf {G}}|X_{\mathsf {G}}=x))\nonumber \\&-\frac{{\mathsf {var}}(Y|X_{\mathsf {G}}=x, E=1)}{2}(\gamma +o(\gamma ))\Pr (E=1). \end{aligned}$$

(41)

Note that since ${\mathsf {var}}(Y){<\infty }$ and $X_{\mathsf {G}}$ has a positive density, ${\mathsf {var}}(Y|X_{\mathsf {G}}=x){<\infty }$ for almost all x (except for x in a set of zero Lebesgue measure). Hence, we can choose L sufficiently large such that for any given $\delta >0$,

$$\begin{aligned} \Pr (E=1)\ge 1-\delta , \end{aligned}$$

and

$$\begin{aligned} {\mathsf {var}}(Y|X_{\mathsf {G}}=x, E=1)\ge {\mathsf {var}}(Y|X_{\mathsf {G}}=x)-\delta . \end{aligned}$$

Therefore, invoking the inequality $\log (1+u)\le u$ for $u>0$, we can write

$$\begin{aligned} D(Z_\gamma |X_{\mathsf {G}}=x)\le \frac{\gamma }{2}\left[ {\mathsf {var}}(Y_{\mathsf {G}}|X_{\mathsf {G}}=x)-({\mathsf {var}}(Y|X_{\mathsf {G}}=x)-\delta )(1-\delta )\right] +o(\gamma ), \end{aligned}$$

from which and the fact the $\delta$ is arbitrarily small the result follows. $\square$