Attacks on the AJPS Mersenne-Based Cryptosystem

Abstract

Aggarwal, Joux, Prakash and Santha recently introduced a new potentially quantum-safe public-key cryptosystem, and suggested that a brute-force attack is essentially optimal against it. They consider but then dismiss both Meet-in-the-Middle attacks and LLL-based attacks. Very soon after their paper appeared, Beunardeau et al. proposed a practical LLL-based technique that seemed to significantly reduce the security of the AJPS system. In this paper we do two things. First, we show that a Meet-in-the-Middle attack can also be made to work against the AJPS system, using locality-sensitive hashing to overcome the difficulty that Aggarwal et al. saw for such attacks. We also present a quantum version of this attack. Second, we give a more precise analysis of the attack of Beunardeau et al., confirming and refining their results.

A Experiments

Since our MITM attack is not fully provable due to the presence of Heuristics 1 and 2, we provide some experimental verifications. The python scripts of those experiments are available at https://github.com/lducas/MiTM-Mersenne.

One tweak in our implementation is that when w is odd, we do not split our space exactly into two equal parts. Instead we choose $w_1 = \lfloor w/2 \rfloor$, $w_2 = w - w_1$, and then choose $n_1, n_2$, such that $\left( {\begin{array}{c}n_1\\ w_1\end{array}}\right) \approx \left( {\begin{array}{c}n_2\\ w_2\end{array}}\right)$. We will also simulate the quantum case, and choose $w_1 = \lceil w/3 \rceil$, $w_2 = w - w_1$, and then choose $n_1, n_2$, such that $\left( {\begin{array}{c}n_1\\ w_1\end{array}}\right) ^2 \approx \left( {\begin{array}{c}n_2\\ w_2\end{array}}\right)$. In both the classical and quantum case, we set $B = \lfloor \log _2 \left( {\begin{array}{c}n_1\\ w_1\end{array}}\right) \rfloor$.

1.1 A.1 Verification of Heuristic 2

We recall that Heuristic 2 states that the number of collisions $c = |\{(g_1,g_2) \in S_1\times S_2: \mathcal {H}(g_1h)=\mathcal {H}(-g_2h)\}|$ is approximately given by $c' = |S_1|\cdot |S_2| 2^{-B}$. We measure the ratio $r = c / c'$ experimentally, over 100 samples for each dimension n. Infrequently, this ratio may get as large as 3, yet for 90% of the experiments, it was very close to 1. Figure 1 below shows the $9^{th}$ decile of r as n grows.

Fig. 1.

$9^{th}$ decile of the ratio between the measured number of collisions c and expected number of collisions $c'$ according to Heuristic 2, over 100 experiments per dimension.

Fig. 2.

Success rate of the attack over 100 trials (in blue), compared to the theoretical success rate $(1-2w/(n-B))^B$ (in red). The rather discontinuous shape of the red curve is due to the rounding of $w = \lfloor \sqrt{n} / 2\rfloor$ and $B = \lfloor \log _2 \left( {\begin{array}{c}n_1\\ w_1\end{array}}\right) \rfloor$. (Color figure online)

1.2 A.2 Running time and success probability

In Figs. 2 and 3, we report on the practical efficiency of our attack and compare it to our heuristic prediction. Note that in the quantum regime, the success probability of this MITM attack in practice is sometimes significantly larger than the theoretical prediction. This is most likely due to the fact that our analysis is done for one particular solution, while certain rotations of the same key may be found as well if its bits are properly balanced with respect to the split $\mathbb F_2^n = G_1 \oplus G_2$.

Fig. 3.

Average running time of the classical attack over 100 trials in comparison with the function $\sqrt{\left( {\begin{array}{c}n\\ w\end{array}}\right) }$, which is the dominant factor in our asymptotic complexity.