1 Introduction

The NTRU encryption scheme designed by Hoffstein et al. [6] is considered as a reasonable alternative to the public key encryption schemes based on either integer factorization or discrete logarithm. Since its first introduction, minor changes of the parameter to avoid known attacks have been added. Even with its computational efficiency and standardization of the NTRU [11], a provably secure version was not known until Stehl\(\acute{\text {e}}\) et al. proposed a modification of the original NTRU in the year 2011 [10]. The IND-CPA security of their modification is proven in the standard model under the hardness assumption of standard worst-case problems over ideal lattices [10]. Reflecting the continued progress in the research on quantum computing, researches on transitioning to quantum resistant algorithms become very active. Moreover, NIST has initiated a standardization process in post-quantum cryptography. The IND-CPA secure version of NTRU could be a strong candidate for the standardization of post-quantum public key encryption. The security proof of the IND-CPA secure NTRU was given in [10] under the assumption that the public key is an invertible polynomial in \(R_q=\mathbb {Z}[x]/\langle q, x^n+1 \rangle \), however, no such result is known for ‘non-invertible’ public key. L\(\acute{\text {o}}\)pez-Alt et al. observed that the IND-CPA secure NTRU can be made fully homomorphic and proposed the first multikey homomorphic encryption scheme for a bounded number of users [8]. Notably, the homomorphic NTRU [8] and its subsequent versions [3, 9] do not assume invertible public keys. If q is a prime number and n is a power of 2 with \(q=1 \bmod {2n}\), then there is a ring isomorphism between \(R_q\) and \(\mathbb {Z}_q^n\) and the number of non-invertible elements in \(R_q\) is \(q^n-(q-1)^n\).

In this paper, we investigate the security influence of using non-invertible public key in the homomorphic NTRU. We present a very effective lattice attack for message recovery on the homomorphic NTRU when the public key is not invertible. The message space of the homomorphic NTRU is \(\{0,1\}\) which implies that the IND-CPA security is equivalent to the security against the message recovery attack. We interpret the message recovery attack as solving a system of linear equations under some condition over a finite field \(\mathbb {Z}_q\) using \(\beta (x)=\frac{x^n+1}{\gcd {(h(x), x^n+1)}}\in \mathbb {Z}_q[x]\) for any non-invertible public key \(pk=h(x)\). For a proof of successful message recovery in general, we used a sequence of sublattices of the target lattice and showed that there is an optimal sublattice which gives the desired short vector by the LLL algorithm if the degree of \(\deg {\beta (x)}\le \frac{ \log q}{4}\) in the homomorphic NTRU. Moreover, it is known that the actual shortest output vector of the LLL algorithm could be much shorter than its theoretical bound. In fact, our experiments using MLLL(Modified LLL) in [4] give much shorter vector than the theoretical bound and this suggests that avoiding \(\beta (x)\) to have small degree is not enough to guarantee the security of the homomorphic NTRU under message recovery attack. Therefore we conclude that setting the public key of the homomorphic NTRU as an invertible polynomial in \(R_q\) is desirable since the security against message recovery attack is a minimal requirement for encryption scheme. We note that some lattice attacks called by the subfield attacks on NTRU cryptosystem were proposed by Cheon et al. [5] and Albrecht et al. [1] and the goal of the subfield attack is to recover private key which can be understood as a short vector of the NTRU lattice. Their subfield attacks are based on the fact that there exist subfields that allow to reduce the dimension of the NTRU lattice and successful when the modulus q is exponential in n. Contrary to [1, 5], the goal of our lattice attack is the message recovery when the public key is non-invertible.

The rest of the paper is organized as follows. In Sect. 2, we review some basics of this paper. In Sect. 3, we show that how to mount the message recovery attack to be successful if the public key is not invertible. In Sect. 4, we conclude our paper.

2 Preliminaries

2.1 The Basic Scheme of Homomorphic NTRU

The homomorphic NTRU is defined on the ring \(R_q=\mathbb {Z}[x]/\langle q, x^n+1 \rangle \) for q is a prime number and n is a power of two. Any element \(k(x)\in R_q\) is represented as \(k(x)=\sum _{i=0}^{n-1}k_ix^i\), where \(-\frac{q}{2}<k_i<\frac{q}{2}\). For the ring \(R=\mathbb {Z}[x]/\langle x^n+1 \rangle \), we denote \(k(x)\leftarrow \chi _\epsilon \) for an appropriate distribution \(\chi _\epsilon \) and each coefficient \(|k_i|\le \epsilon \) of k(x) if \(k(x)\leftarrow \chi _\epsilon \). In the homomorphic version in [8], it is assumed that \(q=2^{n^{\delta }}\) with \(0<\delta <1\) and the message space is \(\{0,1\}\) while it was considered that \(q=poly(n)\) with the message space \(\{0,1\}^n\) in the proven IND-CPA secure version [10]. The basic scheme of the homomorphic NTRU consists of three polynomial time algorithms \(\textsf {KeyGen}, \textsf {Enc}, \textsf {Dec})\).

  • \(\textsf {KeyGen}(1^\kappa )\): Sample polynomials \(\tilde{f}(x),\;g(x)\leftarrow \chi _\epsilon \), repeat sampling \(\tilde{f}(x)\) until \(f(x):=2\tilde{f}(x)+1\) is invertible in \(R_q\) and denote the inverse of f(x) in \(R_q\) as \((f(x))^{-1}\). Output \(pk=h(x):=2g(x)(f(x))^{-1}\pmod {q, x^n+1}\) and \(sk=f(x)\).

  • \(\textsf {Enc}(pk,m\in \{0,1\})\): Sample polynomials \(s(x),\;e(x)\leftarrow \chi _\epsilon \), and output \(\quad \quad c(x):=h(x)s(x)+2e(x)+m \pmod {q, x^n+1}\).

  • \(\textsf {Dec}(sk,c)\): Compute \(\mu (x)=f(x)c(x) \pmod {q, x^n+1}\), and output \(\quad \quad m'=\mu (x) \pmod {2}\).

2.2 Lattices and LLL Algorithm

The lattice L is an additive subgroup of \(\mathbb {R}^m\) that is \(\mathbb {Z}\)-generated by a set of n linearly independent vectors \(\{\mathbf {b}_1, ...,\mathbf {b}_n \}\) in \(\mathbb {R}^m\). We say n as the dimension of the lattice L which is denoted by dim(L). For a given lattice L, there is a geometric invariant called the minimum of the lattice and there are several computational problems related to the minimum.

Definition 1

(Minimum). The (first) minimum of a lattice L is the norm of a shortest non-zero vector in L and denoted as \(\lambda _{1}(L)= {\mathop {\min }\nolimits _{\mathbf {v}\in L \setminus \{\mathbf {0}\}}}\Vert \mathbf {v}\Vert _2\) where \(\Vert \cdot \Vert _2\) is the Euclidean norm of the vector.

In [2], Ajtai proved that for a given lattice L, the problem of finding a vector of the minimum norm, which is called as the Shortest Vector Problem(SVP), is NP-hard. A relaxed SVP is a problem of finding a vector which is no longer than a factor of \(\gamma \) to the first minimum and these problems are often refer to as the approximate SVP\(_{\gamma }\). Note that if \(\gamma \) increases, the problem gets easier. There is no known efficient algorithm solving the SVP\(_{\gamma }\) for small \(\gamma \) in a lattice in arbitrary dimension even in quantum computer. The LLL algorithm is a polynomial time algorithm for SVP\(_{\gamma }\) with \(\gamma =2^{\frac{n-1}{2}}\) [7]. Moreover, in theoretical view, the shortest vector \(\mathbf {v}\) of the output vector of LLL algorithm for n dimensional lattice L satisfies that \(||\mathbf {v}||_2\le 2^{\frac{n-1}{4}}\det (L)^{1/n}\). We note that the input of LLL algorithm should be a basis of the lattice. The MLLL is modified from LLL so that it works on any set of generating set of vectors of integer lattices [4].

3 Message Recovery of Homomorphic NTRU with Non-invertible Public Keys

The IND-CPA security of homomorphic NTRU was proven when the public key \(h(x)=\frac{2g(x)}{f(x)}\in R_q\) is invertible in [10]. In this section, we consider the case that the public key h(x) is not invertible in \(R_q\). Because q is prime, we see that \(\mathbb {Z}_q[x]\) is a unique factorization domain. If h(x) is not invertible in \(R_q\), then \(\gcd {(h(x), x^n+1)}=d(x)\ne 1\) in \(\mathbb {Z}_q[x]\). Therefore, we see that \(x^n+1=\beta (x)d(x)\) and \(\gcd {(\beta (x), h(x))}=1\) in \(\mathbb {Z}_q[x]\). Since \(x^n+1\) divides \({\beta }(x)h(x)\), we see that \({\beta }(x)h(x)=0 \) in \(R_q\). For a given ciphertext \(c(x)=h(x)s(x)+2e(x)+m\), we see that \(w(x)=\beta (x)c(x)\bmod {(q, x^n+1)} =\beta (x)(2e(x)+m)\bmod {(q, x^n+1)}\). In the homomorphic NTRU, the plaintext is chosen from \(\{0,1\}\), and therefore, its IND-CPA security is equivalent to the security in message recovery attack. Therefore, the IND-CPA adversarial goal is to recover \(m\in \{0,1\}\) from

$$\begin{aligned} w(x)=\beta (x)(2e(x)+m)\bmod {(q, x^n+1)}, \end{aligned}$$
(1)

while m and e(x) are unknown and w(x) and \(\beta (x)\) are known.

3.1 A Sufficient Condition for Message Recovery

For \(\beta (x)=\frac{x^n+1}{\gcd {(h(x), x^n+1)}}=\sum _{i=0}^{\ell }\beta _ix^i\in \mathbb {Z}_q[x]\), we consider the following matrix \([B]\in \mathbb {Z}^{n\times n}\):

$$\begin{aligned}{}[B]=\begin{bmatrix} \beta _0&\cdots&\cdots&\beta _{\ell }&\cdots&0 \\ \vdots&\ddots&&\ddots&\vdots \\ 0&\cdots&\beta _0&\cdots&\cdots&\beta _{\ell } \\ -\beta _{\ell }&\cdots&0&\beta _0&\cdots&\beta _{\ell -1} \\&\ddots&&\ddots&\\ -\beta _1&\cdots&-\beta _{\ell }&0&\cdots&\beta _0 \\ \end{bmatrix} =\begin{bmatrix} \mathbf {b}_{n-1} \\ \vdots \\ \mathbf {b}_{\ell } \\ \mathbf {b}_{\ell -1} \\ \vdots \\ \mathbf {b}_0 \\ \end{bmatrix} \end{aligned}$$
(2)

Note that the Eq. (1) can be represented by using matrices over \(\mathbb {Z}_q\) for \(e(x)=\sum _{i=0}^{n-1}e_ix^i\) and \( w(x)=\sum _{i=0}^{n-1}w_ix^i\):

$$\begin{aligned} \mathbf {w}= [B]\cdot (2\mathbf {e}+\mathbf {m}) \bmod {q} \end{aligned}$$
(3)

with \(\mathbf {w}=[w_{n-1},\ldots , w_0]^T;\quad 2\mathbf {e}+\mathbf {m}=[2e_{n-1},\ldots ,2e_1,2e_0+m]^T.\) Again, Eq. (3) of matrices can be written as

$$ w_i=\langle \mathbf {b}_i, 2\mathbf {e}+\mathbf {m}\rangle \bmod { q} \text { for all }i=0,...,n-1, $$

where \(\langle \cdot ,\cdot \rangle \) is the usual inner product of two vectors in \(\mathbb {Z}^n\).

Theorem 1

Suppose that \(\mathbf {b}_i\)’s are given as in Eq. (2) and a vector \(\eta = (\eta _0,\ldots ,\eta _{n-1}) \in \mathbb {Z}^n\) is known to satisfy the following condition:

$$\begin{aligned}{\textsf {Condition(*)}} \left\{ \begin{array}{l} (i) \quad \eta ={\mathop {\sum }\nolimits _{i=0}^{n-1}}\lambda _i\mathbf {b}_i \bmod {q}\text { for }\lambda _i\in \mathbb {Z}\\ (ii)\quad |\eta _i|< \frac{q}{4n\epsilon +2}\text { for all }i=0,1,\ldots ,n-1 \\ (iii)\quad \eta _{n-1}=1 \bmod {2} \end{array}\right. \end{aligned}$$

For any given ciphertext c(x), the plaintext \(m\in \{0,1\}\) can be recovered by \( m=(\sum _{i=0}^{n-1}\lambda _i w_i \bmod {q})\bmod {2}\), where \(w(x)=\beta (x)c(x)\bmod {(q, x^n+1)}=\sum _{i=0}^{n-1}w_ix^i\).

Proof

For a given vector \(\eta =(\eta _0,\ldots ,\eta _{n-1})=\sum _{i=0}^{n-1}\lambda _i\mathbf {b}_i \bmod {q}\) with the Condition(*) holds, we have

$$\begin{aligned} \sum _{i=0}^{n-1}\lambda _i w_i\bmod {q}= & {} \langle \sum _{i=0}^{n-1}\lambda _i\mathbf {b}_i, 2\mathbf {e}+\mathbf {m}\rangle \bmod { q} =(\sum _{i=0}^{n-1}2e_i\eta _{i}) +m\eta _{n-1} \bmod { q}. \end{aligned}$$

From the assumptions \(|\eta _i|<\frac{q}{4n\epsilon +2}\), \(|e_i|\le \epsilon \) and \(m\in \{0,1\}\), we see that

$$\begin{aligned} |(\sum _{i=0}^{n-1}2e_i\eta _{i})+m\eta _{n-1}|< & {} 2n\epsilon \frac{q}{4n\epsilon +2} +\frac{q}{4n\epsilon +2}=\frac{q(2n\epsilon +1)}{4n\epsilon +2}= q/2. \end{aligned}$$

Therefore, we have \(\sum _{i=0}^{n-1}\lambda _i w_i \bmod {q} =\left( \sum _{i=0}^{n-1}2e_i\eta _{i}\right) +m\eta _{n-1}\), which implies that \((\sum _{i=0}^{n-1}\lambda _i w_i \bmod {q})\bmod {2} =m\). \(\quad \square \)

Note that Theorem 1 works for any solution \((\lambda _i)_{0\le i\le n-1}\) which is easy to compute from \(\eta \) by a simple linear algebra over \(\mathbb {Z}_q\). Therefore, for a successful message recovery attack, it is enough to get a vector \(\eta \in \mathbb {Z}^n\) that satisfies Condition(*).

3.2 A Lattice Attack for the Message Recovery

Now we present how to apply a lattice reduction algorithm, to find such a short vector \(\eta \) that is described in Theorem 1.

For the vectors \(\mathbf {b}_i\)’s as given in Eq. (2), we consider the lattice \( L_B=\{ \zeta \in \mathbb {Z}^n | \zeta =\sum _{i=0}^{n-1}x_i\mathbf {b}_i \bmod {q}\text { for some }x_i \in \mathbb {Z} \}\). Now we describe the process of finding a short vector in \(L_B\) that satisfies Condition(*) in two ways. Firstly, we apply a lattice reduction algorithm MLLL [4] for the linearly dependent generating set of vectors

$$ S=\{ (q,0,...,0), (0,q,0,...,0),..., (0,...,0,q), \mathbf {b}_{n-1}, ..., \mathbf {b}_0\}\subset \mathbb {Z}^{n}. $$

From our experiments, we see that the algorithm MLLL outputs a short vector with Condition(*) holds if the degree \(\ell \) of \(\beta (x)\) is small. However, the only thing we can prove on the size of the shortest vector of the output of MLLL is that it is at least smaller than \(2^{\frac{n-1}{4}} (\det {L_B})^{\frac{1}{n}}\le 2^{\frac{n-1}{4}} q^{\frac{\ell }{n}}\) from the LLL reducedness of the output. This does not give enough reason why a short vector from the output of MLLL satisfies the Condition(*).

Now we present a method of finding a short vector in \(L_B\) that with Condition(*) holds provably if the degree \(\ell \le \frac{\log _2 q}{4}\). We consider a sequence of sublattices \(L_{\ell +1}\subset L_{\ell +2}\subset \cdots \subset L_n\subset L_B)\), where \(L_i (\ell +1 \le i \le n)\) is generated by the row vectors of \(B_i\in \mathbb {Z}^{i\times n}\) which are defined as follows:

$$ B_i=\left[ \begin{matrix} 0 &{}\cdots &{}0 &{} q &{}\cdots &{} 0&{} 0 &{} \cdots &{}0 \\ \vdots &{} &{} \vdots &{} &{} \ddots &{} &{}\vdots &{} &{} \vdots \\ 0 &{}\cdots &{}0 &{}0 &{} \cdots &{} q &{}0 &{} \cdots &{} 0 \\ 0 &{}\cdots &{}0 &{} \beta _0 &{} \cdots &{} \beta _{\ell -1} &{}1 &{} \cdots &{}0 \\ \vdots &{} &{}\vdots &{} &{}\ddots &{}\ddots &{} \ddots &{}\ddots &{} \\ 0 &{}\cdots &{}0 &{} 0&{} \cdots &{} \beta _0 &{} \cdots &{} \beta _{\ell -1} &{}1 \\ \end{matrix} \right] =[ 0_{i\times (n-i)} | B'_{i,\textsf {red}} ],\quad B'_{i,\textsf {red}}\in \mathbb {Z}^{i\times i} $$

Let \(L_{B'_{i,\textsf {red}}}\subset \mathbb {Z}^i\) be the lattice generated by the row vectors of \(B'_{i,\textsf {red}}\) for \(i=\ell +1, ..., n\). If \(\eta _\textsf {red}=(\eta '_{n-i}, ..., \eta '_{n-1})\in L_{B'_{i,\textsf {red}}}\) is a short vector that satisfies Condition(*) then \(\eta =(\eta _j)_{0\le j\le (n-1)}\) is a short vector in \(L_B\) that satisfies Condition(*), where \(\eta _j=0\) if \(0\le j\le (n-i-1)\) and \(\eta _{j}=\eta '_{j}\) if \(n-i \le j\le n-1\). From [4], we see that the shortest vector \(\mathbf {v}'_i\in L_{B'_{i,\textsf {red}}}\) of the output of the LLL algorithm for the lattice generated by the row vectors of \(B'_{i,\textsf {red}}\) satisfies that

$$ ||\mathbf {v}'_i||\le ||\mathbf {v}_i'||_2\le 2^{\frac{i-1}{4}} \det (B'_{i,\textsf {red}})^{1/i}= 2^{\frac{i-1}{4}}q^{\frac{\ell }{i}}. $$

By setting \(\log _2 q=\tau \), we have a sequence of vectors \(\mathbf {v}_i\in L_B\) with \(||\mathbf {v}_i||\le 2^{\frac{i-1}{4}+\frac{\ell \tau }{i}}\) for \(i=\ell +1, ..., n\). From a simple calculation over real numbers using the derivatives, we see that the function \(f(i)=2^{\frac{i-1}{4}+\frac{\ell \tau }{i}}\) has its minimum \(2^{-1/4 +\sqrt{\ell \tau }}\) at \(i=2\sqrt{\ell \tau }\). For simplicity, we assume that \(\kappa =2\sqrt{\ell \tau }\) is an integer. Therefore, the LLL algorithm applied on \(L_{B'_{\kappa ,\textsf {red}}}\) on the basis consists of the row vectors of \(B'_{\kappa ,\textsf {red}}\) gives a vector \(\mathbf {v}\in L_B\) with \(||\mathbf {v}||\le 2^{-1/4 +\sqrt{\ell \tau }}\).

Now we want to show that this vector satisfies Condition(*) as long as the last component is an odd number. For this, it is enough to show that \(2^{-1/4 +\sqrt{\ell \tau }}\le \frac{q}{4n\epsilon +2}\). From the equality \(2^{-1/4 +\sqrt{\ell \tau }}=2^{-1/4} q^{\sqrt{\frac{\ell }{\tau }}}\), it is enough to show that \(2^{-1/4}(4n\epsilon +2) \le q^{1-\sqrt{\frac{\ell }{\tau }}}\). In particular, if q is subexponential in n as in the homomorphic NTRU, one can assume that \(2^{-1/4}(4n\epsilon +2) \le q^{1/2}\). Moreover, if \(\ell \le \frac{\log _2 q}{4}=\frac{\tau }{4}\), we clearly have \(q^{1/2}\le q^{1-\sqrt{\frac{\ell }{\tau }}}\) and thus \(2^{-1/4}(4n\epsilon +2) \le q^{1-\sqrt{\frac{\ell }{\tau }}}\). Therefore, we conclude that \(2^{-1/4 +\sqrt{\ell \tau }}\le \frac{q}{4n\epsilon +2}\) if \(\ell \le \frac{\log _2 q}{4}\).

Note that the condition \(\ell \le \frac{\log _2 q}{4}\) to guarantee the desired shortness of the vector \(\mathbf {v}\) is deduced from the theoretical bound of the shortest vector of the output of the LLL algorithm. It is known that the actual shortest vector of the LLL algorithm is shorter than the theoretical bound in general. Moreover, as in the example of the following section, the method using MLLL gives a shorter vector than the method using the sublattice. This suggests that the message recovery attack can be successful for much larger \(\ell \)’s. Therefore, setting h(x) as an invertible polynomial in \(R_q\) is more appropriate than avoiding \(\beta (x)\) with successful lattice reduction attack using sublattice as described above.

4 Conclusion

The IND-CPA security of the homomorphic NTRU is proven when the public key is invertible in \(R_q\) [10]. However, no result on the security of the homomorphic NTRU is known when the public key is not invertible. In this paper, we show that if the public key is not invertible in the homomorphic NTRU, then one can use a lattice reduction algorithm effectively to recover the plaintext of any ciphertext. Therefore, we conclude that the public key of homomorphic NTRU should be invertible in the ring \(R_q\) to guarantee the IND-CPA security of homomorphic variants of NTRU [3, 8, 9].