Telepathic Headache: Mitigating Cache Side-Channel Attacks on Convolutional Neural Networks

Abstract

Convolutional Neural Networks (CNNs) are the target of several side-channel attacks aiming at recovering their parameters and hyper-parameters. Attack vectors include monitoring of the cache, power consumption analysis and execution time measurements. These attacks often rely on the knowledge of a certain – large – set of hyper-parameters among which the victim model lies. The goal of the potential attacker is then to reduce that search space or even deduce the correct architecture. One such attack, Cache Telepathy by Yan et al., monitors access to a common matrix multiplication algorithm, GeMM (Generalized Matrix Multiply), in order to determine the victim model’s hyper-parameters. In this paper, we propose to change the order in which the computations are made and add randomness to the said computations in order to mitigate Cache Telepathy. The security analysis of our protection shows that the Cache Telepathy attack on a protected VGG-16 has an increased search space: from 16 to $2^{22}$.

Appendices

A Protected Case: When the Attacker can Distinguish between Convolutional Layers

As explained in Sect. 7, we consider layers 4 (Conv4), 5 (Conv5) and 6 (MaxPool) in a VGG-16 architecture [25] (depicted in Fig. 5). Conv4 and Conv5 are computed according to the process in Fig. 1. In that process, inputs $I_4$ and $I_5$ are reshaped into $R_4$ and $R_5$ respectively. Let us note that in order to have an output size equal to the input size, the input needs to be padded. We consider $I_4$ and $I_5$ to be the padded inputs. Their shape is (64, 114, 114) and (128, 114, 114) respectively. Let $O^{(j)}$ denote the reshaped output of layer j – i.e. the output of the layer’s matrix multiplication.

Let $H^{(4)} = \{H^{(4)}_1, ..., H^{(4)}_t\}$ be the number of blocks height-wise for each width-wise column of blocks. Let us denote $w^{(4)} = \{w_1^{(4)}, ..., w_{t'}^{(4)}\}$ – respectively $h^{(4)} = \{h_1^{(4)}, ..., h_t^{(4)}\}$ – the sizes of the blocks width-wise – respectively height-wise. These are determined as described in Sect. 5. Further, let $\{M^{(4)}_{i, j}\}_{1 \le i \le t, 1 \le t'}$ denote the number of filter blocks associated with each of Conv4’s input blocks. The reshaped filters are $F_4$ and $F_5$ respectively. The blocks we consider are submatrices in $R_{4,5}$ and $F_{4, 5}$. $R_4$ and $R_5$ both have width $W_{4, 5} = 112 \times 112$. $R_4$ has height $3 \times 3 \times 64$ while $R_5$ has height $3 \times 3 \times 128$.

In our case, each block has size between $32 \times 32$ and $64 \times 64$. Let $(x_b, y_b)$ be the coordinates of the last element in the first block in $R_5$. If we take the minimal block size, then $(x_{\min }, y_{\min }) = (32, 32)$. In the case of the maximal block size, $(x_{\max }, y_{\max }) = (64, 64)$.

Let $tot_{x, y}^{i, j}$ denote the number of matrix multiplications in layer i required so that the element (x, y) in $R_j$ is ready. Since our goal is to determine the number of multiplications in Conv4 necessary to have the first block in Conv5 ready, we actually need to compute $tot_{x_b, y_b}^{4, 5}$. We will consider block size extremes to give a range of values for $tot_{x_b, y_b}^{4, 5}$:

$$ tot_{x_{\max }, y_{\max }}^{4, 5} \le tot_{x_b, y_b}^{4, 5} \le tot_{x_{\min }, y_{\min }}^{4, 5} $$

In order to compute that value, we need to find the coordinates of element $e = (x, y)$ in $I_5$ rather than $R_5$. Indeed, this will provide us with the blocks that need to be computed in Conv4 to obtain e.

First, let us find which input channel ch element e belongs to. Because of the way $R_5$ is obtained, we have that:

$$ ch = \left\lfloor {\frac{x}{k^{(4)} \times k^{(4)}}}\right\rfloor $$

Thus:

$$\begin{aligned} ch_{e_{\min }}&= \left\lfloor {\frac{32}{k^{(4)} \times k^{(4)}}}\right\rfloor = 3 \\ ch_{e_{\max }}&= \left\lfloor {\frac{64}{k^{(4)} \times k^{(4)}}}\right\rfloor = 7 \end{aligned}$$

where $e_{\min } = (x_{\min }, y_{\min })$ and $e_{\max } = (x_{\min }, y_{\min })$ indicate the maximal and minimal block sizes respectively.

Let us now find the coordinates (row, col) of element e in channel ch. Let $n' = n + 2 \cdot p = 114$ be the padded input width and height. With an input of shape $(n', n')$, a convolution results in an output of size $(n' - k^{(4)} + 1, n' - k^{(4)} + 1) = (n, n)$ here. Because of the way $I_5$ is reshaped, each column in $R_5$ is a $k^{(5)} \times k^{(5)}$ window in $I_5$. Thus,

$$\begin{aligned} row&= \left\lfloor {\frac{y}{n}}\right\rfloor + \left\lfloor {\frac{x \mod (k^{(5)} \times k^{(5)})}{k^{(5)}}}\right\rfloor \\ col&= y \mod n + (x \mod (k^{(5)} \times k^{(5)})) \mod k^{(5)} \end{aligned}$$

As mentioned before, these coordinates include the padding. We need to remove said padding to find out how many computations from the previous layer need to have been made. If (row, col) does not correspond to a padding value, we have:

$$\begin{aligned}&row_{unpadded} = row - p \\&col_{unpadded} = col - p \end{aligned}$$

If $row < p$ with $ch = 0$, then no value from the previous layer needs to be computed for the first block to be executed. This results in $tot_{x, y}^{4, 5} = 0$. The same goes for $row = p$, $ch = 0$ and $col < p$.

If (x, y) is a padding value outside of the two previous cases, then we consider the previous non-padding value in $I_5$. This corresponds to:

$$\begin{aligned}&row_{unpadded} = row - 1 \\&col_{unpadded} = n' \end{aligned}$$

Indeed, we then need to take the last (non-padding) element in the previous row.

With $(x_{\min }, y_{\min })$ and $(x_{\max }, y_{\max })$, we have that:

Because $e_{\max } = (x_{\max }, y_{\max })$ is in the first row of $I_5$, it corresponds to a padding element. However, not all elements in that block stem from padding elements. Instead, we can take the last non-padding element in the previous channel. We therefore need to consider channel $ch_{e_{\max }}' = ch_{e_{\max }} - 1 = 6$. We also need to select the last element of the $(3 \times 3)$ input window, meaning:

$$\begin{aligned}&row'_{\max } = 2\\&col'_{\max } = 66 \end{aligned}$$

We then remove the padding from $e_{\min } = (x_{\min }, y_{\min })$ so as to find the coordinates in Conv4’s output. This gives us:

$$\begin{aligned}&row'_{\min } = 1 \\&col'_{\min } = 33 \\&ch_{e_{\min }}' = 3 \\&row'_{\max } = 2 \\&col'_{\max } = 66 \\&ch_{e_{\max }}' = 6 \\ \end{aligned}$$

These are the coordinates of the last element we need in the output of Conv4. The coordinates c of that element in the said output are:

$$\begin{aligned}&c_{\min } = (ch_{e_{\min }}', row'_{\min } \times n + col'_{\min }) = (3, 145) \\&c_{\max } = (ch_{e_{\max }}', row'_{\max } \times n + col'_{\max }) = (6, 290) \end{aligned}$$

We now have enough information to compute $tot_{e}^{4, 5}$.

Let $b = \arg \min _{b'} \left( \sum _{q=0}^{b'} w_1 \ge c \right)$. b is the number of the block width-wise containing c. For each block number $b' < b$, we need to multiply all the blocks height-wise with their associated block filters, meaning $H^{(4)}_{b'} \times M^{(4)}_{b'}$ multiplications for each block $b'$. For b, we need to compute all the height-wise block multiplications up to the block containing ch. This is equal to: $\left\lceil {\frac{ch}{h_1^{(4)}}}\right\rceil \times H_b^{(4)}$. The full formula is then:

$$\begin{aligned} tot_{e}^{1, 2} = \left( \sum _{q=0}^{b-1} H^{(4)}_{q} \times M^{(4)}_{q}\right) + \left\lceil {\frac{ch}{h_1^{(4)}}}\right\rceil \times H_b^{(4)} \end{aligned}$$

(2)

We can apply that to $e_{\max }$ and $e_{\min }$, considering constant block shapes of either (32, 32) for $e_{\min }$ or (64, 64) for $e_{\max }$:

$$\begin{aligned}&b_{e_{\min }}&= \left\lceil {\frac{145}{32}}\right\rceil = 5 \end{aligned}$$

(3)

$$\begin{aligned}&H^{(4)}_{q, \min }&= \left\lceil {\frac{3 \times 3 \times 64}{32}}\right\rceil = 18 \text { } \forall q \le b \end{aligned}$$

(4)

$$\begin{aligned}&M^{(4)}_{q, \min }&= \left\lceil {\frac{128}{32}}\right\rceil = 4 \text { } \forall q \le b\end{aligned}$$

(5)

$$\begin{aligned}&\left\lceil {\frac{ch_{e_{\min }'}}{32}}\right\rceil \times H^{(4)}_{b, \min }&= 4 \end{aligned}$$

(6)

$$\begin{aligned}&tot_{e_{\min }}^{4, 5}&= b_{e_{\min }} \times H^{(4)}_{q} \times M^{(4)}_{q} + \left\lceil {\frac{ch_{e_{\min }'}}{32}}\right\rceil \times H_b^{(4)} \end{aligned}$$

(7)

$$\begin{aligned}&= 292 \end{aligned}$$

(8)

$$\begin{aligned}&b_{e_{\max }}&= \left\lceil {\frac{290}{64}}\right\rceil = 5 \end{aligned}$$

(9)

$$\begin{aligned}&H^{(4)}_{\max }&= H^{(4)}_{q, \max } = \left\lceil {\frac{3 \times 3 \times 64}{64}}\right\rceil = 9 \text { } \forall q \le b \end{aligned}$$

(10)

$$\begin{aligned}&M^{(4)}_{\max }&= M^{(4)}_{q, \max } = \left\lceil {\frac{128}{64}}\right\rceil = 2 \end{aligned}$$

(11)

$$\begin{aligned}&\left\lceil {\frac{ch_{e_{\max }'}}{64}}\right\rceil \times H_{b, \max }^{(4)}&= 2 \end{aligned}$$

(12)

$$\begin{aligned}&tot_{e_{\max }}^{4, 5}&= (b_{e_{\max }} - 1) \times H^{(4)}_{\max } \times M^{(4)}_{\max } + \left\lceil {\frac{ch_{e_{\max }'}}{64}}\right\rceil \times H^{(4)}_{b, \max } \end{aligned}$$

(13)

$$\begin{aligned}&= 74 \end{aligned}$$

(14)

Thus, the total number of matrix multiplications required for one block in Conv5 to be ready ranges between 74 and 292 depending on the block sizes. This range of values can be obtained because we know the padding and filter sizes. A potential attacker, however, does not have access to this information. She only knows the number of input channels and the input shape.

Table 5. Maximal and minimal number of multiplications depending on the filter size, when the attacker can distinguish between convolutional layers.

Let us compute all the possible values for $tot_{x, y}^{1, 2}$ $(x'_{\min }, y'_{\min }) = (32, 32)$ and $(x'_{\max }, y'_{\max }) = (64, 64)$, given $out_4$, $out_5$ and $in_5$ and depending on $k^{(4)} = k^{(5)}$. Table 5 shows the resulting values.

B Protected Case: When the Attacker Cannot Distinguish Between Convolutional Layers

Since the attacker cannot distinguish between convolutional layers, we need to compute $tot^{4, 6}$. If we denote (x, y) the last element in the first MaxPool block $R_B$, and $(x', y')$ the last element in Conv5 that needs to be computed for $R_B$ to be ready, then: $tot^{4, 6}_{x, y} = tot^{4,5}_{x', y'} + tot^{5, 6}_{x, y}$. In our case, MaxPool has window size $k^{(6)} = 2$, and does not require any padding. Furthermore, all channels need to be computed at once in the pooling case. Thus, the coordinates of (x, y) in the output of Conv5 are:

$$\begin{aligned}&row = \left\lfloor {\frac{k^{(6)} \cdot y}{n}}\right\rfloor + \left\lfloor {\frac{x}{k^{(6)}}}\right\rfloor \\&col = \left( y \mod \frac{n}{k^{(6)}}\right) \times k^{(6)} + x \mod k^{(6)} \\&c = row \times n + col \\ \end{aligned}$$

We also have that $b = \arg \min _{b'} \left( \sum _{q=0}^{b'} w_1 \ge c \right)$. Applying it to our case for $(x_{\min }, y_{\min }) = (32, 32)$ and $(x_{\max }, y_{\max }) = (64, 64)$, we have:

In $O^{(5)}$, we need the element with coordinates $(ch, B \cdot b)$ where ch is the last filter of Conv5, since MaxPool requires all channels to be completed. That element requires element $(x', y') = (in_5 \times k^{(5)} \times k^{(5)}, B \cdot b)$ in $I^{(5)}$. Indeed, the column number in $O^{(5)}$ and $I^{(5)}$ must be the same. Furthermore, the whole column must be computed, which is why $x' = in_5 \times k^{(5)} \times k^{(5)}$. Thus, the previous formulas directly provide us with $(x', y')$:

$$\begin{aligned} (x'_{\min }, y'_{\min })&= (in_5 \times k^{(5)} \times k^{(5)} - 1, 32 \cdot b_{\min })&= (128 \times 9 - 1, 1856) \\ (x'_{\max }, y'_{\max })&= (in_5 \times k^{(5)} \times k^{(5)} - 1, 64 \cdot b_{\max })&= (128 \times 9 - 1, 3712) \end{aligned}$$

Thus, given Conv5 has 128 input and output channels:

$$\begin{aligned} H^{(5)}_{\min }&= \left\lceil {\frac{128 \times 9 - 1}{32}}\right\rceil&= 36 \\ H^{(5)}_{\max }&= \left\lceil {\frac{128 \times 9 - 1}{64}}\right\rceil&= 18 \\ M^{(5)}_{\min }&= \left\lceil {\frac{128}{32}} \right\rceil&= 4 \\ M^{(5)}_{\max }&= \left\lceil {\frac{128}{64}} \right\rceil&= 2 \\ tot_{32, 32}^{5, 6}&= b_{\min } \times H^{(5)}_{\min } \times M^{(5)}_{\min }&= 8352\\ tot_{64, 64}^{5, 6}&= b_{\max } \times H^{(5)}_{\max } \times M^{(5)}_{\max }&= 2088 \end{aligned}$$

Here, we have $H^{(5)} \times M^{(5)}$ for width-wise block b as well because the whole b-th block width-wise needs to be computed, as well as all channels.

Applying the same process as previously to compute $tot^{4, 5}_{e_{\min }}$ and $tot^{4, 5}_{e_{\max }}$, we have:

$$\begin{aligned} tot_{x'_{\min }, y'_{\min }}^{4, 5}&= 4338 \\ tot_{x'_{\max }, y'_{\max }}^{4, 5}&= 1053 \end{aligned}$$

Thus, we have:

$$\begin{aligned} tot_{x'_{\min }, y'_{\min }}^{4, 6}&= tot_{x_{\min }, y_{\min }}^{5, 6} + tot_{in_5 \times k^{(5)} \times k^{(5)}, e_{\min } \cdot b_{\min }}^{4, 5}&= 12690 \\ tot_{x'_{\max }, y'_{\max }}^{4, 6}&= tot_{x_{\max }, y_{\max }}^{5, 6} + tot_{in_5 \times k^{(5)} \times k^{(5)}, e_{\max } \cdot b_{\max }}^{4, 5}&= 3141 \\ \end{aligned}$$

Thus, the total number of multiplications for one MaxPool block to be ready ranges between 3, 141 and 12, 690 in the set of layers we study from the VGG-16 architecture.

Once again, the attacker does not know $k^{(4)}, k^{(5)}, k^{(6)}$ or the padding values. For each layer, $k^{(i)} \in \{1, ..., 11\}$. We also suppose, like before, that $k^{(4)} = k^{(5)}$. The padding p between Conv4 and Conv5 (the only one that intervenes in the computations) is such that $p < k^{(4)}$. We get tables of possible values depending on $k^{(4)} = k^{(5)}$, p and $k^{(6)}$. As before, we take the maximum and minimal values over all possible p. Table 6 considers the range of $tot^{4, 6}$ depending on $k^{(4)}$.

Table 6. Maximal and minimal number of multiplications depending on the filter size, when the attacker cannot distinguish between convolutional layers but knows $out_4$, $in_5$ and $out_5$.

Once again, the ranges for all possible $tot^{4, 6}$ values overlap, making it harder for the attacker to determine the architecture. In our case, we had $tot^{4, 6} \in [3 141, 12 041]$. An attacker could therefore only deduce that $1< k^{(4)} < 11$. The range can be further deduced depending on the value actually observed, but there are at least two filter size values in every case. Furthermore, for most $k^{(6)}$ values, $2< k^{(4)} < 10$.

So far, we had assumed, for simplicity, that the number of input and output channels are known for all layers. But it is generally not the case. Taking this last fact into account, the possible $tot^{4, 6}$ values are given in Table 7.

Table 7. Maximal and minimal number of multiplications depending on the filter size, when the attacker cannot distinguish between convolutional layers.

C  Reshaping

See Fig. 7 and Fig. 8.

Fig. 7.

Example of a subdivision of a reshaped input

Fig. 8.

Reshaping the input to turn a convolution into a matrix multiplication. The input size is $n \times n$ and the filter size is $k \times k$.