Replica Symmetry Breaking in Multi-species Sherrington–Kirkpatrick Model

Abstract

In the Sherrington–Kirkpatrick (SK) and related mixed p-spin models, there is interest in understanding replica symmetry breaking at low temperatures. For this reason, the so-called AT line proposed by de Almeida and Thouless as a sufficient (and conjecturally necessary) condition for symmetry breaking, has been a frequent object of study in spin glass theory. In this paper, we consider the analogous condition for the multi-species SK model, which concerns the eigenvectors of a Hessian matrix. The analysis is tractable in the two-species case with positive definite variance structure, for which we derive an explicit AT temperature threshold. To our knowledge, this is the first non-asymptotic symmetry breaking condition produced for a multi-species spin glass. As possible evidence that the condition is sharp, we draw further parallel with the classical SK model and show coincidence with a separate temperature inequality guaranteeing uniqueness of the replica symmetric critical point.

A Proof of Lemma 3.1

Proof of Lemma 3.1

Here we consider (1.6) when $k = 1$, and

$$\begin{aligned} \zeta _1 = \zeta \in (0,1), \quad q_1^s = q^s, \quad q_2^s = p^s. \end{aligned}$$

Recall that with these choices, we have $Q_0 = Q_0^s = 0$ and

$$\begin{aligned} Q_1^s&= 2\sum _t \Delta _{st}^2\lambda _tq^t&Q_1&= \sum _{s,t}\Delta _{st}^2\lambda _s\lambda _tq^sq^t \\ Q_2^s&= 2\sum _{t}\Delta _{st}^2\lambda _tp^t&Q_2&= \sum _{s,t}\Delta _{st}^2\lambda _s\lambda _tp^sp^t \\ Q_3^s&= 2\sum _{t}\Delta _{st}^2\lambda _t&Q_3&= \sum _{s,t}\Delta _{st}^2\lambda _s\lambda _t. \end{aligned}$$

We have

$$\begin{aligned} \mathscr {P}_\mathrm {1RSB}(q,p,\zeta )&{:}{=}\log 2 + \sum _s \lambda _s X_0^s - \frac{\beta ^2}{2}\sum _{\ell =1}^2 \zeta _\ell (Q_{\ell +1}-Q_{\ell }) \\&= \log 2 + \sum _s \lambda _s X_0^s - \frac{\beta ^2}{2}(Q_3 - Q_2+\zeta (Q_2-Q_1)), \end{aligned}$$

where

$$\begin{aligned} X_3^s&= \log \cosh (\beta \eta _3\sqrt{Q_3^s-Q_2^s} + \beta \eta _2\sqrt{Q_2^s-Q_1^s}+\beta \eta _1\sqrt{Q_1^s}+h) \\ \Rightarrow X_2^s&= \log \mathbf {E}_3 \cosh (\beta \eta _3\sqrt{Q_3^s-Q_2^s} + \beta \eta _2\sqrt{Q_2^s-Q_1^s}+\beta \eta _1\sqrt{Q_1^s}+h) \\&= \frac{\beta ^2}{2}(Q_3^s-Q_2^s) + \log \cosh (\beta \eta _2\sqrt{Q_2^s-Q_1^s}+\beta \eta _1\sqrt{Q_1^s}+h) \\ \Rightarrow X_1^s&= \frac{1}{\zeta }\log \mathbf {E}_2 \exp (\zeta _2 X_2^s) \\&= \frac{\beta ^2}{2}(Q_3^s-Q_2^s) + \frac{1}{\zeta }\log \mathbf {E}_2 \cosh ^\zeta (\beta \eta _2\sqrt{Q_2^s-Q_1^s} + \beta \eta _1\sqrt{Q_1^s}+h) \\ \Rightarrow X_0^s&= \mathbf {E}X_1^s = \frac{\beta ^2}{2}(Q_3^s-Q_2^s) + \frac{1}{\zeta } \mathbf {E}_1\log \mathbf {E}_2 \cosh ^\zeta (\beta \eta _2\sqrt{Q_2^s-Q_1^s} + \beta \eta _1\sqrt{Q_1^s}+h). \end{aligned}$$

In simplifying $X_2^s$, we have used the fact that for $\eta \sim \mathcal {N}(0,1)$ and $\sigma >0$,

$$\begin{aligned} \mathbf {E}\cosh (\sigma \eta +h)&= \frac{1}{2}\mathbf {E}(e^{\sigma \eta +h} + e^{-\sigma \eta -h}) \nonumber \\&= \frac{1}{2}(e^{\sigma ^2/2 + h} + e^{\sigma ^2/2 - h}) = e^{\sigma ^2/2}\cosh (h). \end{aligned}$$

(A.1)

In summary,

$$\begin{aligned} \mathscr {P}_\mathrm {1RSB}(q,p,\zeta )&= \log 2+ \sum _s \lambda _s \frac{1}{\zeta } \mathbf {E}_1\log \mathbf {E}_2 \cosh ^\zeta (\beta \eta _2\sqrt{Q_2^s-Q_1^s} + \beta \eta _1\sqrt{Q_1^s}+h) \nonumber \\&\quad +\frac{\beta ^2}{2}\sum _s \lambda _s (Q_3^s - Q_2^s) - \frac{\beta ^2}{2}(Q_3-Q_2+\zeta (Q_2-Q_1)). \end{aligned}$$

(A.2)

Notice that when $\zeta = 1$, we recover the replica symmetric expression (1.9):

$$\begin{aligned} \mathscr {P}_\mathrm {1RSB}(q,p,1)&= \log 2 + \sum _s\lambda _s \mathbf {E}_1\log \mathbf {E}_2\cosh (\beta \eta _2\sqrt{Q_2^s-Q_1^s} + \beta \eta _1\sqrt{Q_1^s}+h)\nonumber \\&\quad + \frac{\beta ^2}{2} \sum _s \lambda _s(Q_3^s - Q_2^s)- \frac{\beta ^2}{2}(Q_3-Q_2+Q_2-Q_1) \nonumber \\&= \log 2 + \sum _s \lambda _s \mathbf {E}_1\bigg [\frac{\beta ^2}{2}(Q_2^s-Q_1^s) + \log \cosh (\beta \eta _1\sqrt{Q_1^s}+h)\bigg ] \nonumber \\&\quad + \frac{\beta ^2}{2} \sum _s \lambda _s(Q_3^s - Q_2^s) - \frac{\beta ^2}{2}(Q_3-Q_1) \nonumber \\&= \log 2 + \sum _s \lambda _s \mathbf {E}_1\log \cosh (\beta \eta _1\sqrt{Q_1^s}+h) \nonumber \\&\quad + \frac{\beta ^2}{2}\sum _s \lambda _s(Q_3^s-Q_1^s) - \frac{\beta ^2}{2}(Q_3-Q_1) = \mathscr {P}_\mathrm {RS}(q). \end{aligned}$$

(A.3)

Henceforth fix an RS critical point $q = q_*\in \mathcal {C}(\beta ,h)$. For ease of notation, let us write

$$\begin{aligned} Y_1^s&:= \beta \eta _1\sqrt{Q_1^s}+h, \qquad Y_2^s := \beta \eta _2\sqrt{Q_2^s-Q_1^s} +Y_1^s, \end{aligned}$$

so that

$$\begin{aligned} \mathscr {P}_\mathrm {1RSB}(q,p,\zeta )&= \log 2+ \sum _s \lambda _s \frac{1}{\zeta } \mathbf {E}_1\log \mathbf {E}_2 \cosh ^\zeta Y_2^s +\frac{\beta ^2}{2}\sum _s \lambda _s (Q_3^s - Q_2^s) \\&\quad - \frac{\beta ^2}{2}(Q_3-Q_2+\zeta (Q_2-Q_1)). \end{aligned}$$

We can then calculate

$$\begin{aligned} \frac{\partial \mathscr {P}_\mathrm {1RSB}(q_*,p,\zeta )}{\partial \zeta }&= \sum _s \lambda _s\bigg [\frac{-\mathbf {E}_1\log \mathbf {E}_2 \cosh ^\zeta Y_2^s}{\zeta ^2} +\mathbf {E}_1\bigg (\frac{\mathbf {E}_2\log (\cosh Y_2^s)\cosh ^\zeta Y_2^s}{\zeta \mathbf {E}_2 \cosh ^\zeta Y_2^s}\bigg )\bigg ] \\&\quad - \frac{\beta ^2}{2}(Q_2-Q_1), \end{aligned}$$

which gives

$$\begin{aligned} V(p)&= \sum _s\lambda _s\bigg [-\mathbf {E}_1\log \mathbf {E}_2 \cosh Y_2^s + \mathbf {E}_1\bigg (\frac{\mathbf {E}_2\log (\cosh Y_2^s)\cosh Y_2^s}{\mathbf {E}_2\cosh Y_2^s}\bigg )\bigg ]\nonumber \\&\quad - \frac{\beta ^2}{2}(Q_2-Q_1). \end{aligned}$$

(A.4)

When $p = q_*$, we have $Y_2^s = Y_1^s$ and $Q_2 = Q_1$. In particular, $Y_2^s$ has no dependence on $\eta _2$, and so the above expression reduces to $V(q_*) = 0$. This proves claim (a).

The next step is to take partial derivatives with respect to the $p^t$. First, we use (A.1) to make the simple calculation

$$\begin{aligned} \mathbf {E}_2 \cosh Y_2^s = \exp \Big (\frac{\beta ^2}{2}(Q_2^s-Q_1^s)\Big )\cosh Y_1^s. \end{aligned}$$

(A.5)

Since $Y_1^s$ has no dependence on p, and

$$\begin{aligned} \frac{\partial }{\partial p^t}(Q_2^s-Q_1^s) = \frac{\partial }{\partial p^t}Q_2^s = 2\Delta _{st}^2\lambda _t, \end{aligned}$$

(A.6)

we find

$$\begin{aligned} \frac{\partial }{\partial p^t} \mathbf {E}_1\log \mathbf {E}_2\cosh Y_2^s =\frac{\partial }{\partial p^t}\Big [\frac{\beta ^2}{2}(Q_2^s-Q_1^s)+\mathbf {E}_1\log \cosh Y_1^s\Big ] = \beta ^2\Delta _{st}^2\lambda _t. \end{aligned}$$

(A.7)

Next we observe that for any twice differentiable function whose derivatives have at most exponential growth at infinity, (A.6) and Gaussian integration by parts together give

$$\begin{aligned} \frac{\partial }{\partial p^t} \mathbf {E}_2 f(Y_2^s) = \beta \Delta _{st}^2\lambda _t\mathbf {E}_2\bigg (f^{\prime }(Y_2^s)\frac{\eta _2}{\sqrt{Q_2^s-Q_1^s}}\bigg )&= \beta ^2\Delta _{st}^2\lambda _t\mathbf {E}_2 f^{\prime \prime }(Y_2^s). \end{aligned}$$

(A.8)

Hence

$$\begin{aligned}&\frac{\partial }{\partial p^t} \mathbf {E}_1\bigg (\frac{\mathbf {E}_2f(Y_2^s)}{\mathbf {E}_2\cosh Y_2^s}\bigg ) {\mathop {=}\limits ^{(A.5)}} \frac{\partial }{\partial p^t}\bigg [ \exp \Big (-\frac{\beta ^2}{2}(Q_2^s-Q_1^s)\Big )\mathbf {E}_1\bigg (\frac{\mathbf {E}_2 f(Y_2^s)}{\cosh Y_1^s}\bigg )\bigg ]\nonumber \\&\quad {\mathop {=}\limits ^{{(A.6),(A.8)}}} \beta ^2\Delta _{st}^2\lambda _t\exp \Big (-\frac{\beta ^2}{2}(Q_2^s-Q_1^s)\Big )\bigg [- \mathbf {E}_1\bigg (\frac{\mathbf {E}_2 f(Y_2^s)}{\cosh Y_1^s}\bigg ) + \mathbf {E}_1\bigg (\frac{\mathbf {E}_2f^{\prime \prime }(Y_2^s)}{\cosh Y_1^s}\bigg )\bigg ] \nonumber \\&\quad {=} \beta ^2\Delta _{st}^2\lambda _t\exp \Big (-\frac{\beta ^2}{2}(Q_2^s-Q_1^s)\Big )\mathbf {E}_1\bigg (\frac{\mathbf {E}_2[f^{\prime \prime }(Y_2^s)-f(Y_2^s)]}{\cosh Y_1^s}\bigg )\nonumber \\&\quad {\mathop {=}\limits ^{{(A.5)}}} \beta ^2\Delta _{st}^2\lambda _t\mathbf {E}_1\bigg (\frac{\mathbf {E}_2[f^{\prime \prime }(Y_2^s)-f(Y_2^s)]}{\mathbf {E}_2\cosh Y_2^s}\bigg ). \end{aligned}$$

(A.9)

We now apply (A.9) to $f(x) = \log (\cosh x)\cosh x$, for which

$$\begin{aligned} f^{\prime \prime }(x)-f(x)&= \cosh x + \sinh x \tanh x, \end{aligned}$$

to obtain

$$\begin{aligned} \frac{\partial }{\partial p^t}\mathbf {E}_1\bigg (\frac{\mathbf {E}_2 \log (\cosh Y_2^s) \cosh Y_2^s}{\mathbf {E}_2\cosh Y_2^s}\bigg )\bigg ] = \beta ^2\Delta _{st}^2\lambda _t\bigg [1 + \mathbf {E}_1\bigg (\frac{\mathbf {E}_2 \sinh Y_2^s \tanh Y_2^s}{\mathbf {E}_2 \cosh Y_2^s}\bigg )\bigg ]. \end{aligned}$$

(A.10)

Finally, we have

$$\begin{aligned} \frac{\partial }{\partial p^t}(Q_2 - Q_1) = \frac{\partial }{\partial p^t}Q_2 = 2\sum _s\Delta _{st}^2\lambda _s\lambda _t p^s. \end{aligned}$$

(A.11)

Using (A.7), (A.10), and (A.11) in (A.4), we arrive at

$$\begin{aligned} \frac{\partial }{\partial p^t}V(p) = \beta ^2\lambda _t\sum _s \Delta _{st}^2\lambda _s\bigg [\mathbf {E}_1\bigg (\frac{\mathbf {E}_2 \sinh Y_2^s \tanh Y_2^s}{\mathbf {E}_2 \cosh Y_2^s}\bigg ) - p^s\bigg ]. \end{aligned}$$

Once more, if $p = q_*$, then $Y_2^s = Y_1^s$ has no dependence on $\eta _2$, in which case

$$\begin{aligned} \mathbf {E}_1\bigg (\frac{\mathbf {E}_2 \sinh Y_2^s \tanh Y_2^s}{\cosh Y_1^s}\bigg ) - p^s = \mathbf {E}_1(\tanh ^2 Y_1^s) - q_*^s = q_*^s - q_*^s = 0 \quad \text {for all }s. \end{aligned}$$

Consequently, claim (b) holds: $\nabla V(q_* ) = 0$.

Our final step is to compute the Hessian of V. We have

$$\begin{aligned} \frac{\partial ^2}{\partial p^{t^{\prime }}\partial p^t}V(p) = \beta ^2\lambda _t\sum _{s}\Delta ^2_{st}\lambda _s\bigg [\frac{\partial }{\partial p^{t^{\prime }}} \mathbf {E}_1\bigg (\frac{\mathbf {E}_2\sinh Y_2^s \tanh Y_2^s}{\mathbf {E}_2\cosh Y_2^s}\bigg ) - \delta _{st^{\prime }}\bigg ], \end{aligned}$$

where $\delta _{st^{\prime }} = 1$ if $s = t^{\prime }$ and 0 zero otherwise. To determine the derivative of the expectation, we apply (A.9) with $f(x) = \sinh x \tanh x$, for which

$$\begin{aligned} f^{\prime \prime }(x)-f(x)&= 2\mathrm {sech}^3\,x. \end{aligned}$$

After doing so, we arrive at

$$\begin{aligned} \frac{\partial ^2}{\partial p^{t^{\prime }}\partial p^{t}}V(p)&= \beta ^2\lambda _t\sum _s \Delta ^2_{st}\lambda _s\bigg [2\beta ^2\Delta _{st^{\prime }}^2\lambda _{t^{\prime }}\mathbf {E}_1\bigg (\frac{\mathbf {E}_2{{\,\mathrm{sech}\,}}^3 Y_2^s}{\mathbf {E}_2\cosh Y_2^s}\bigg ) - \delta _{st^{\prime }}\bigg ]. \end{aligned}$$

As before, the expression simplifies when $p=q^*$, since then $Y_2^s=Y_1^s$ has no dependence on $\eta _2$. Namely,

$$\begin{aligned} \frac{\partial ^2}{\partial p^{t^{\prime }}\partial p^{t}}V(q_*)&= \beta ^2\lambda _t\sum _{s}\Delta _{st}^2\lambda _s\big [2\beta ^2\Delta _{st^{\prime }}^2\lambda _{t^{\prime }}\mathbf {E}_1\, \mathrm {sech}^4\, Y_1^s - \delta _{st^{\prime }}\big ] \\&= \bigg (2\beta ^4\lambda _t\lambda _{t^{\prime }}\sum _s \Delta _{st}^2\Delta _{st^{\prime }}^2\lambda _s \mathbf {E}_1\, \mathrm {sech}^4\, Y_1^s\bigg ) - \beta ^2\lambda _t\lambda _{t^{\prime }}\Delta _{tt^{\prime }}^2. \end{aligned}$$

Rewriting the expression in terms of matrices yields claim (c). $\square$