A Secure Data Deduplication Scheme for Cloud Storage

Abstract

As more corporate and private users outsource their data to cloud storage providers, recent data breach incidents make end-to-end encryption an increasingly prominent requirement. Unfortunately, semantically secure encryption schemes render various cost-effective storage optimization techniques, such as data deduplication, ineffective. We present a novel idea that differentiates data according to their popularity. Based on this idea, we design an encryption scheme that guarantees semantic security for unpopular data and provides weaker security and better storage and bandwidth benefits for popular data. This way, data deduplication can be effective for popular data, whilst semantically secure encryption protects unpopular content. We show that our scheme is secure under the Symmetric External Decisional Diffie-Hellman Assumption in the random oracle model.

Appendices

Appendix A: Proof of Lemma 1

SXDH assumes two groups of prime order $q$, $\mathbb {G}_1$, and $\mathbb {G}_2$, such that there is not an efficiently computable distortion map between the two; a bilinear group $\mathbb {G}_{T}$, and an efficient, non-degenerate bilinear map $\hat{e}: \mathbb {G}_1 \times \mathbb {G}_2 \rightarrow \mathbb {G}_{T}$. In this setting, the Decisional Diffie-Hellman (DDH) holds in both $\mathbb {G}_1$, and $\mathbb {G}_2$, and that the bilinear decisional Diffie-Hellman (BDDH) holds given the existence of $\hat{e}$ [19].

Challenger $\mathcal {C}$ is given an SXDH context $\mathbb {G}_1 ', \mathbb {G}_2 ', \mathbb {G}_{T} ', \hat{e}'$ and an instance of the DDH problem $\langle \mathbb {G}_1 ', g ', A=(g ')^a, B=(g ')^b, W \rangle$ in $\mathbb {G}_1$’. $\mathcal {C}$ simulates an environment in which $\mathcal {A}$ operates, using its advantage in the game $\mathsf{DS }_{\mu }$-$\mathsf{IND }$ to decide whether $W = g '^{ab}$. $\mathcal {C}$ interacts with $\mathcal {A}$ in the $\mathsf{DS }_{\mu }$-$\mathsf{IND }$ game as follows:

• Setup Phase $\mathcal {C}$ sets $\mathbb {G}_1 \leftarrow \mathbb {G}_1 '$, $\mathbb {G}_2 \leftarrow \mathbb {G}_2 '$, $\mathbb {G}_{T} \leftarrow \mathbb {G}_{T} '$, $\hat{e} = \hat{e}'$, $g \leftarrow g '$; picks a random generator $\bar{g}$ of $\mathbb {G}_2$ and sets $\bar{g} _{pub} = (\bar{g})^\mathsf{sk}$, where $\mathsf{sk}\leftarrow _R \mathbb {Z}_q^*$. $\mathcal {C}$ also generates the set of user identities $\mathsf{U}= \{{\mathsf {U}}_i \}_{i=0}^{n-1}$. The public key $\mathsf{pk}= \{q, \mathbb {G}_1, \mathbb {G}_2, \mathbb {G}_{T} \, \hat{e}, \mathcal {O}_{\mathsf{H}_\mathsf{1}}, \mathcal {O}_{\mathsf{H}_\mathsf{2}}, \bar{g}, \bar{g} _{pub}\}$ and $\mathsf{U}$ are forwarded to $\mathcal {A}$. $\mathcal {A}$ declares the list $\mathsf {U}_\mathcal {A}$ of $n_\mathcal {A} < t-1$ user identities that will later on be subject to $\mathcal {O}_{\mathsf{Corrupt}}$ calls. Let $\mathsf {U}_\mathcal {A} =\{{\mathsf {U}}_i \}_{i=0}^{n_\mathcal {A}-1}$. To generate key-shares $\{\mathsf{sk_{i}}\}_{i=0}^{n-1}$, $\mathcal {C}$ constructs a $t-1$-degree Lagrange polynomial $\mathrm {P} ()$ with interpolation points $\mathrm {I_P} = \{ (0, \mathsf{sk_{}}) \cup \{ (\mathsf{r_{i}}, y_i) \}_{i=0}^{t-2} \},$ where $\mathsf{r_{i}}, y_i\leftarrow _R \mathbb {Z}_q^*$, for $i \in [0,t-3]$, and $\mathsf{r_{t-2}} \leftarrow _R \mathbb {Z}_q^*$, $y_{t-2} \leftarrow a$. Secret key-shares are set to $\mathsf{sk_{i}} \leftarrow y_i, i \in [0,n-1]$. Since $a$ is not known to $\mathcal {C}$, $\mathcal {A}$ sets the corrupted key-shares to be $\mathsf{sk_{i}}$ for $i \in [0, n_\mathcal {A}-1]$.

• Access to Oracles $\mathcal {C}$ simulates oracles $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$, $\mathcal {O}_{\mathsf{H}_\mathsf{2}}$, $\mathcal {O}_{\mathsf{Corrupt}}$ and $\mathcal {O}_{\mathsf{DShare}}$ :

  • $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$: to respond to $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$-queries, $\mathcal {C}$ maintains a list of tuples $\{\mathsf{H_1}, v, h_{v}, \rho _{v}, \mathrm {c}_v\}$ as explained below. We refer to this list as $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$ list, and it is initially empty. When $\mathcal {A}$ submits an $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$ query for $v$, $\mathcal {C}$ checks if $v$ already appears in the $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$ list in a tuple $\{v, h_{v}, \rho _{v}, \mathrm {c}_{v}\}$. If so, $\mathcal {C}$ responds with $\mathsf{H_1}(v) = h_{v}$. Otherwise, $\mathcal {C}$ picks $\rho _{v} \leftarrow _R \mathbb {Z}_q^*$, and flips a coin $\mathrm {c}_{v}$; $\mathrm {c}_{v}$ flips to $'0'$ with probability $\delta$ for some $\delta$ to be determined later. If $\mathrm {c}_{v}$ equals $'0'$, $\mathcal {C}$ responds $\mathsf{H_1}(v) = h_{v} = g^{\rho _{v}}$ and stores $\{v, h_{v}, \rho _{v}, \mathrm {c}_{v}\}$; otherwise, she returns $\mathsf{H_1}(v) = h_{v} = B^{\rho _{v}}$ and stores $\{v, h_{v}, \rho _{v}, \mathrm {c}_{v}\}$.

  • $\mathcal {O}_{\mathsf{H}_\mathsf{2}}$: The challenger $\mathcal {C}$ responds to a newly submitted $\mathcal {O}_{\mathsf{H}_\mathsf{2}}$ query for $v$ with a randomly chosen $h_{v} \in \mathbb {G}_{T}$. To be consistent in her $\mathcal {O}_{\mathsf{H}_\mathsf{2}}$ responses, $\mathcal {C}$ maintains the history of her responses in her local memory.

  • $\mathcal {O}_{\mathsf{Corrupt}}$: $\mathcal {C}$ responds to a $\mathcal {O}_{\mathsf{Corrupt}}$ query involving user ${\mathsf {U}}_i \in \mathsf {U}_\mathcal {A}$, by returning the coordinate $y_i$ chosen in the Setup Phase.

  • $\mathcal {O}_{\mathsf{DShare}}$: simulation of $\mathcal {O}_{\mathsf{DShare}}$ is performed as follows. As before, $\mathcal {C}$ keeps track of the submitted $\mathcal {O}_{\mathsf{DShare}}$ queries in her local memory. Let $\langle m, {\mathsf {U}}_i \rangle$ be a decryption query submitted for message $m$ and user identity ${\mathsf {U}}_i$. If there is no entry in $\mathsf{H_1}$-list for $m$, then $\mathcal {C}$ runs the $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$ algorithm for $m$. Let $\{m, h_{m}, \rho _{m}, \mathrm {c}_m\}$ be the $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$ entry in $\mathcal {C}$’s local memory for message $m$. Let $\mathrm {I_P} ' \leftarrow \mathrm {I_P} \setminus (\mathsf{r_{t-2}}, \mathsf{sk_{t-2}})$. $\mathcal {C}$ responds with ${\mathsf {ds}}_{m,i} = \left( g^{\sum \limits _{(\mathsf{r_{j}}, \mathsf{sk_{j}}) \in \mathrm {I_P} '} \mathsf{sk_{j}} \lambda _{\mathsf{r_{i}},\mathsf{r_{j}}}^{\mathrm {I_P} '}} X^{\lambda _{\mathsf{r_{i}} ,\mathsf{r_{t-2}}}^{\mathrm {I_P} '}}\right) ^{\rho _m}$ where $X \leftarrow A$ iff $\mathrm {c}_m = 0$, and $X \leftarrow W$ iff $\mathrm {c}_m = 1$. In both cases, $\mathcal {C}$ keeps a record of her response in her local memory.

• Challenge Phase $\mathcal {A}$ selects the challenge message $m_*$. Let the corresponding entry in the $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$ list be $\{m_*, h_{m_*}, \rho _{m_*}, \mathrm {c}_{m_*}\}$. If $\mathrm {c}_{m_*} = 0$, then $\mathcal {C}$ aborts.

• Guessing Phase $\mathcal {A}$ outputs one bit ${\mathsf {b}}'_{m_*}$ representing the guess for ${\mathsf {b}}_{m_*}$. $\mathcal {C}$ responds positively to the DDH challenger if ${\mathsf {b}}_{m_*}'=0$, and negatively otherwise.

It is easy to see, that if $\mathcal {A}$’s answer is $'0'$, it means that the $\mathcal {O}_{\mathsf{DShare}}$ responses for $m_*$ constitute properly structured decryption shares for $m_*$. This can only be if $W = g^{ab}$ and $\mathcal {C}$ can give a positive answer to the SXDH challenger. Clearly, if $\mathrm {c}_{m_*} = 1$ and $\mathrm {c}_{m} = 0$ for all other queries to $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$ such that $m \ne m_*$, the execution environment is indistinguishable from the actual game $\mathsf{DS }_{\mu }$-$\mathsf{IND }$. This happens with probability $\mathrm {Pr}[\mathrm {c}_{m_*} = 1\ \wedge \ (\forall m \ne m_*: \mathrm {c}_{m} = 0)] = \delta (1 - \delta )^{\mathcal {Q}_{H_1} - 1},$ where $\mathcal {Q}_{H_1}$ is the number of distinct $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$ queries. By setting $\delta \approx \frac{1}{\mathcal {Q}_{H_1} - 1}$ the above probability becomes greater than $\frac{1}{e\cdot (\mathcal {Q}_{H_1}-1)}$ and the success probability of the adversary can be bounded as $\mathsf {Adv}_{{{\mathsf{DS }_{\mu }}}-{\mathsf{IND }}}^{\mathcal {A}} \le e \cdot (\mathcal {Q}_{H_1}-1) \cdot \mathsf {Adv}_{\mathsf {SXDH}}^\mathcal {C}$.

Appendix B: Proof of Lemma 2

Challenger $\mathcal {C}$ is given an instance $\langle q'$, $\mathbb {G}_1 ', \mathbb {G}_2 ', \mathbb {G}_{T} ', \hat{e}', g ', \bar{g} ', A=(g ')^a, B=(g ')^b, C=(g ')^c, \bar{A}=(\bar{g} ')^a, \bar{B}=(\bar{g} ')^b, \bar{C}=(\bar{g} ')^c, W \rangle$ of the SXDH problem and wishes to use $\mathcal {A}$ to decide if $W = \hat{e}\left( g ', \bar{g} ' \right) ^{abc}$. The algorithm $\mathcal {C}$ simulates an environment in which $\mathcal {A}$ operates, using its advantage in the game $\mathsf{IND }_{\mu }$-$\mathsf{CPA }$ to help compute the solution to the BDDH problem, as described before. $\mathcal {C}$ interacts with $\mathcal {A}$ within an $\mathsf{IND }_{\mu }$-$\mathsf{CPA }$ game:

• Setup Phase $\mathcal {C}$ sets $q \leftarrow q'$, $\mathbb {G}_1 \leftarrow \mathbb {G}_1 '$, $\mathbb {G}_2 \leftarrow \mathbb {G}_2 '$, $\mathbb {G}_{T} \leftarrow \mathbb {G}_{T} '$, $\hat{e} = \hat{e}'$, $g \leftarrow g '$, $\bar{g} \leftarrow \bar{g} '$, $\bar{g} _{pub} = \bar{A}$. Notice that the secret key $\mathsf{sk}= a$ is not known to $\mathcal {C}$. $\mathcal {C}$ also generates the list of user identities $\mathsf{U}$. $\mathcal {C}$ sends $\mathsf{pk}= \{q, \mathbb {G}_1, \mathbb {G}_2, \mathbb {G}_{T} \, \hat{e}, \mathcal {O}_{\mathsf{H}_\mathsf{1}}, \mathcal {O}_{\mathsf{H}_\mathsf{2}}, \bar{g}, \bar{g} _{pub}\}$ to $\mathcal {A}$. At this point, $\mathcal {A}$ declares the list of corrupted users $\mathsf {U}_\mathcal {A}$ as in $\mathsf{DS }_{\mu }$-$\mathsf{IND }$. Let $\mathsf {U}_\mathcal {A} =\{{\mathsf {U}}_i \}_{i=0}^{n_\mathcal {A}-1}$. To generate key-shares $\{\mathsf{sk_{i}}\}_{i=0}^{n-1}$, $\mathcal {C}$ picks a $t-1$ degree Lagrange polynomial $\mathrm {P} ()$ assuming interpolation points $\mathrm {I_P} = \left\{ (0, a)\ \cup \ \{(\mathsf{r_{i}}, y_{i})\}_{i=0}^{t-2} \right\} ,$ where $\mathsf{r_{i}}, y_{i} \leftarrow _R \mathbb {Z}_q^*$. She then sets the key-shares to $\mathsf{sk_{i}} \leftarrow y_i, i \in [0, n-1]$ and assigns $\mathsf{sk_{i}}$ for $i \in [0, n_\mathcal {A}-1]$ to corrupted users.

• Access to Oracles $\mathcal {C}$ simulates oracles $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$, $\mathcal {O}_{\mathsf{H}_\mathsf{2}}$, $\mathcal {O}_{\mathsf{Corrupt}}$ and $\mathcal {O}_{\mathsf{DShare}}$ :

  • $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$, $\mathcal {O}_{\mathsf{H}_\mathsf{2}}$, $\mathcal {O}_{\mathsf{Corrupt}}$: $\mathcal {C}$ responds to these queries as in $\mathsf{DS }_{\mu }$-$\mathsf{IND }$.

  • $\mathcal {O}_{\mathsf{DShare}}$: $\mathcal {C}$ keeps track of the submitted $\mathcal {O}_{\mathsf{DShare}}$ -queries in her local memory. Let $\langle m, {\mathsf {U}}_i \rangle$ be a decryption query submitted for message $m$ and user identity ${\mathsf {U}}_i$. If there is no entry in $\mathsf{H_1}$-list for $m$, then $\mathcal {C}$ runs the $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$ algorithm for $m$. Let $\{m, h_{m}, \rho _{m}, \mathrm {c}_m\}$ be the $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$ entry in $\mathcal {C}$’s local memory for message $m$. If $\mathrm {c}_m = 1$, and $\mathcal {A}$ has already submitted $t-n_{\mathcal {A}}-1$ queries for $m$, $\mathcal {C}$ aborts. If the limit of $t-n_{\mathcal {A}}-1$ queries has not been reached, $\mathcal {C}$ responds with a random ${\mathsf {ds}}_{m,i} \in \mathbb {G}_1$ and keeps a record for it. From Lemma 1, this step is legitimate as long as less than $t$ decryption shares are available for $m$. Let $\mathrm {I_P} ' \leftarrow \mathrm {I_P} \setminus (0, a)$. If $\mathrm {c}_m = 0$, $\mathcal {C}$ responds with ${\mathsf {ds}}_{m,i} = \left( g^{\sum \limits _{(\mathsf{r_{j}}, \mathsf{sk_{j}}) \in \mathrm {I_P} '} \mathsf{sk_{j}} \lambda _{\mathsf{r_{i}},\mathsf{r_{j}}}^{\mathrm {I_P} '}} A^{\lambda _{\mathsf{r_{i}},0}^{\mathrm {I_P} '}}\right) ^{\mathsf{r_{m}}}$.

• Challenge Phase $\mathcal {A}$ submits $m_*$ to $\mathcal {C}$. $\mathcal {A}$ has not submitted $\mathcal {O}_{\mathsf{DShare}}$ -queries for the challenge message with more than $t-n_{\mathcal {A}}-1$ distinct user identities. Next, $\mathcal {C}$ runs the algorithm for responding to $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$-queries for $m_*$ to recover the entry from the $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$-list. Let the entry be $\{m_*, h_{m_*}, \rho _{m_*}, \mathrm {c}_{m_*}\}$. If $\mathrm {c}_{m_*} = 0$, $\mathcal {C}$ aborts. Otherwise, $\mathcal {C}$ computes $e_{*} \leftarrow W^{\rho _{m_*}}$, sets $c_{*} \leftarrow \langle m_* \oplus \mathsf{H_2}(e_{*}), \bar{C}\rangle$ and returns $c_{*}$ to $\mathcal {A}$.

• Guessing Ph. $\mathcal {A}$ outputs the guess ${\mathsf {b}}'$ for ${\mathsf {b}}$. $\mathcal {C}$ provides ${\mathsf {b}}'$ for its SXDH challenge.

If ${\mathcal {A}}$’s answer is ${\mathsf {b}}' = 1$, it means that she has recognized the ciphertext $c_*$ as the encryption of $m_*$; $\mathcal {C}$ can then give the positive answer to her SXDH challenge. Indeed, $W^{\rho _{m_*}} = \hat{e}\left( g, \bar{g} \right) ^{abc\rho _{m_*}} = \hat{e}\left( (B^{\rho _{m_*}})^a, \bar{g} ^c \right) = \hat{e}\left( H_1(m_*)^{\mathsf{sk}}, \bar{C} \right) .$ Clearly, if $\mathrm {c}_{m_*} = 1$ and $\mathrm {c}_{m} = 0$ for all other queries to $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$ such that $m \ne m_*$, then the execution environment is indistinguishable from the actual game $\mathsf{IND }_{\mu }$-$\mathsf{CPA }$. This happens with probability $\mathrm {Pr}[\mathrm {c}_{m_*} = 1\ \wedge \ (\forall m \ne m_*: \mathrm {c}_{m} = 0)] = \delta (1 - \delta )^{\mathcal {Q}_{H_1} - 1}$, where $\mathcal {Q}_{H_1}$ is the number of different $\mathcal {O}_{\mathsf{H}_\mathsf{1}}$-queries. By setting $\delta \approx \frac{1}{\mathcal {Q}_{H_1} - 1}$, the above probability becomes greater than $\frac{1}{e\cdot (\mathcal {Q}_{H_1}-1)}$, and the success probability of the adversary $\mathsf {Adv}_{{{\mathsf{IND }_{\mu }}}-{\mathsf{CPA }}}^{\mathcal {A}}$ is bounded as $\mathsf {Adv}_{{{\mathsf{IND }_{\mu }}}-{\mathsf{CPA }}}^{\mathcal {A}} \le e \cdot (\mathcal {Q}_{H_1}-1) \cdot \mathsf {Adv}_{\mathsf {SXDH}}^\mathcal {C}$.