1 Introduction

Weak scale supersymmetry (SUSY) is one of the most interesting candidates for new physics beyond the standard model (SM). Its low energy phenomenology is determined mainly by the relevant soft SUSY breaking parameters which are required to preserve flavor and CP with a good accuracy. Such soft SUSY breaking parameters can be predicted by the mediation mechanism of SUSY breaking. So it is crucial to understand various well-motivated mediation mechanisms, for example, the gauge mediated SUSY breaking (GMSB) [1,2,3,4,5,6,7], the anomaly mediated SUSY breaking (AMSB) [8, 9] mechanisms.

Minimal AMSB, which is determined solely by the parameter \(F_\phi \simeq m_{3/2}\), is insensitive to the UV theory [10] and predicts a flavor conservation soft SUSY breaking spectrum. Unfortunately, negative slepton squared masses will appear and the minimal scenario must be extended. Although there are many possible ways to tackle such tachyonic slepton problem, the most elegant solution from aesthetical point of view is the deflected AMSB [11,12,13,14] (dAMSB) scenario. In deflected AMSB, additional messenger sectors are introduced to deflect the AMSB trajectory and additional gauge mediation contributions can possibly push the negative slepton squared masses to positive values [15, 16]. On the other hand, \(N\ge 4\) species are always needed to give positive slepton squared masses with naturally negative deflection parameters, possibly leading to strong gauge couplings below GUT scale or Landau pole below Planck scale. Besides, electroweak naturalness and the discovered 125 GeV Higgs boson by both the ATLAS [17] and CMS collaborations [18] of LHC, may indicate respectively the lightness of stop and large trilinear coupling \(A_t\); null search results of sparticles with 36 \(fb^{-1}\) of data at the (13 TeV) LHC by the ATLAS and CMS collaborations [19,20,21,22] suggest that the low energy SUSY spectrum should display an intricate pattern, for example, the first two generation squarks need to be heavy to avoid the stringent constraints from LHC. However, spectrum of such a type will in general not be predicted in ordinary (d)AMSB scenarios.

We had proposed to introduce general messenger–matter interactions in dAMSB to solve the previous problems [23] which can be advantageous in various aspects. The previous formalism in [23, 24], within which the messenger–matter interactions are introduced in the superpotential, has an analog in GMSB [25,26,27,28,29,30,31]. In this paper, we propose an alternative approach to including messenger–matter interactions in (d)AMSB from typical holomorphic terms in the Kahler potential. Such an approach, with the messenger scales set by the compensator vacuum expectation value (VEV), can only be possible in AMSB type scenarios.

Next-to-minimal supersymmetric standard model (NMSSM) [32, 33] is a singlet extension of MSSM that has various advantages. The SUSY preserving \(\mu \) parameter, which (however) needs to lie near the soft SUSY breaking scale to trigger EWSB, could be generated with the correct scale once the singlet scalar acquires a VEV. Besides, with additional tree-level contributions or through doublet-singlet mixing, NMSSM can easily accommodate the discovered 125 GeV Higgs boson without much fine-tuning. Low energy NMSSM from typical SUSY breaking mechanism, such as GMSB, is always bothered by the requirement to achieve successful EWSB with suppressed trilinear couplings \(A_\kappa ,A_\lambda \) and \(m_S^2\), rendering the model building non-trivial [34, 35]. Such difficulty always persists in ordinary (d)AMSB-type scenarios. We find that phenomenological interesting NMSSM spectrum can be successfully generated from non-trivial holomorphic messenger–matter mixing terms in the Kahler potential. Besides, the discrepancy between the theoretical predictions for the muon anomalous magnetic momentum and the experiments, can possibly be solved in our scenario.

This paper is organized as follows. In Sect. 2, we propose our new scenario and discuss the general methods to obtain the soft SUSY parameters. The soft SUSY parameters for NMSSM are also given. The relevant numerical results are discussed in Sect. 3. Section 4 contains our conclusions.

2 Messenger–matter interactions from Kahler potential

There are two possible ways to deflect the AMSB trajectory with the presence of messengers, by pseudo-moduli field or holomorphic terms (for messengers) in the Kahler potential. In our previous work [23], messenger–matter interactions are introduced in the superpotential involving the pseudo-moduli field. We find that messenger–matter interactions can also be consistently generated by holomorphic messenger–matter mixing terms in the Kahler potential. In order to show the most general features of this approach, we discuss the relevant soft parameters in the framework of NMSSM.

As mentioned previously, the low energy soft SUSY breaking spectrum of NMSSM obtained from typical SUSY breaking mechanism, such as GMSB and ordinary extended (d)AMSB, is always bothered by the requirement to achieve successful electroweak symmetry breaking (EWSB), rendering the model building non-trivial [34, 35]. In fact, EWSB within NMSSM in general requires a large VEV for the singlet. This prefers a negative \(m^2_S\) and/or large A-terms for the singlet superpotential interactions, \(A_\lambda \) and \(A_\kappa \) [36]. AMSB always predicts a positive value for \(m^2_S\) and \(A_\lambda ,A_\kappa \) can be sizable only if \(\lambda ,\kappa \) are also large which would induce a larger \(m^2_S\), suppressing the singlet VEV. Such difficulty can possibly be ameliorated in AMSB-type scenarios with enhanced trilinear couplings. We find that phenomenological interesting NMSSM spectrum can be successfully generated by introducing holomorphic messenger–matter mixing terms in the Kahler potential.

We propose to introduce new holomorphic term in the Kahler potential in addition to ordinary canonical Kahler kinetic terms and \(Z_3\) invariant NMSSM superpotential

$$\begin{aligned}&{K_{h}}\supseteq {\phi ^\dagger \phi }\left[ \sum \limits _{i=1}^2c_{S,i} T S_i +c_P \tilde{P}P+c_Q \tilde{Q}Q\nonumber \right. \\&\left. +\sum \limits _{m=1}^3 \left( c_{m,a}^P \tilde{P}_{m,a} P+c_{m,a}^Q \tilde{Q}Q_{m,a}\right) \right] +h.c. ,\nonumber \\&{W}={\phi ^3}\left[ W_M+\tilde{\lambda } S_1 H_u H_d+\frac{1}{3}\tilde{\kappa } S_1^3+W_{\overline{MSSM}}\right] ,\nonumber \\&W_M =\sum \limits _{a=1,2,3}\lambda _P S_1 \tilde{P}_{m,a} P+\lambda _Q S_1 \tilde{Q} Q_{m,a}. \end{aligned}$$
(2.1)

Here \(\phi \) is the conformal compensator field and \(S_1,S_2,T\) are gauge singlet superfields; \(\tilde{P}_{m,a}\) and \(Q_{m,a}\) are the standard model matter superfields in the \(\bar{\mathbf{5}}\) and \(\mathbf{10}\) representations of SU (5) with ‘\(a=1,2,3\)’ the family index. Additional vector-like messengers in the \(\tilde{P},P\) \({\varvec{\overline{5}}} \oplus \mathbf{5} \)) and \(\tilde{Q},Q\) (\({\varvec{\overline{10}}} \oplus \mathbf{10} \)) representations of SU (5) are introduced to solve the tachyonic slepton problems and at the same time deflect the ordinary AMSB trajectory. In this paper, we assume \(c_P,c_Q,c_{m,a}^{P,Q}\) are real and \(c_{m,a}^{P,Q}\ne 0\) only for \(a=3\) for simply. Such mixing between the third generation fermions and additional vector-like fermions always appear in new physics models, such as top (top-bottom ) seesaw model or extra dimension models. We should note that \(c_{m,a}^{P,Q}\ne 0\) for the first two generations \(a=1,2\) are also possible and such possibility will be discussed subsequently. Besides, the choice of superpotential \(W_M\) is not unique. For example, we can adopt couplings between S and messengers used in [37] or [38] (with double messenger species).

With only \(c_{m,3}^{P,Q}\ne 0\), the holomorphic terms in the Kahler potential reduce to

$$\begin{aligned} K_h \supseteq&\frac{\phi ^\dagger }{\phi } \bigg [T \left( \sum \limits _{i=1,2} c_{S,i} S_i\right) +P \left( c_P \tilde{P}+ c_{m,3}^P \tilde{P}_{m,3}\right) \nonumber \\&\quad +\tilde{Q}\left( c_Q Q+ c_{m,3}^Q Q_{m,3}\right) \bigg ]+h.c., \end{aligned}$$
(2.2)

after rescaling each superfield with the compensator field \(\phi \), namely \(\Phi \rightarrow \phi \Phi \). With the F-term VEVs of compensator \(\phi =1+F_\phi \theta ^2\), we have the potential for the singlets

$$\begin{aligned} V \supseteq&c_{S,i}|F_\phi |^2 T S_i+|F_\phi |^2\left( \sum \limits _{i} c_{S,i}^2\right) |T|^2\nonumber \\&\quad +|F_\phi |^2 |\left( \sum \limits _{i}c_{S_i} S_i \right) |^2+\cdots , \end{aligned}$$
(2.3)

where the first term are obtained by picking out the \(F_\phi \) terms. We thus arrive at the mass matrix for the scalar components \(T,S_1,S_2\)

$$\begin{aligned}&(~T,~S_1^*,~S_2^*) \nonumber \\&\quad \left( \begin{array}{ccc} (c_{S,1}^2+c_{S,2}^2)|F_\phi |^2&{}c_{S,1}|F_\phi |^2&{}c_{S,2}|F_\phi |^2\\ c_{S,1}|F_\phi |^2&{}c_{S,1}^2|F_\phi |^2&{}c_{S,1}c_{S,2}|F_\phi |^2\\ c_{S,2}|F_\phi |^2&{}c_{S,1}c_{S,2}|F_\phi |^2&{}c_{S,2}^2|F_\phi |^2 \end{array} \right) \nonumber \\&\quad \left( \begin{array}{c}T^*\\ S_1\\ S_2\end{array}\right) \end{aligned}$$
(2.4)

It can be seen that the mass matrix has vanishing determinant.

We can define \(c_{S}\equiv \sqrt{c_{S,1}^2+c_{S,2}^2}\) and redefine the fields

$$\begin{aligned} \tilde{S}_0\equiv & {} \frac{1}{c_S}\left( c_{S,1} S_1+c_{S,2} S_2\right) ,\nonumber \\ \tilde{S}_1\equiv & {} \frac{1}{c_S}\left( -c_{S,2} S_1+c_{S,1} S_2\right) , \end{aligned}$$
(2.5)

The mixing angle can be given as

$$\begin{aligned} \cos (-\theta )= & {} \frac{c_{S,1}}{\sqrt{c_{S,1}^2+c_{S,2}^2}},\nonumber \\ \sin (-\theta )= & {} -\frac{c_{S,2}}{\sqrt{c_{S,1}^2+c_{S,2}^2}}. \end{aligned}$$
(2.6)

A minus sign for the angle is kept for future convenience. The zero eigenvalue of the scalar mass matrix corresponds to the combination \(\tilde{S}_1\). The fermionic component of \(\tilde{S}_1\), which is orthogonal to \(\tilde{S}_0\), can also be seen to be massless from the Kahler potential. The non-vanishing mass eigenstates for scalar matrix are given by

$$\begin{aligned} \mathcal{L}\supseteq & {} -\frac{ (c_S^2-c_S)}{2c_S^2}F^2_\phi \left| -c_S T+c_{S,1}S_1^*+c_{S,2}S_2^*\right| ^2 \nonumber \\&-\frac{ (c_S^2+c_S)}{2 c_S^2}F^2_\phi \left| c_S T+c_{S,1}S_1^*+c_{S,2}S_2^*\right| ^2,\nonumber \\= & {} -{ (c_S^2-c_S)} F^2_\phi \left| \frac{-T+\tilde{S}_0^*}{\sqrt{2}}\right| ^2 \nonumber \\&\quad -{ (c_S^2+c_S)} F^2_\phi \left| \frac{T+\tilde{S}_0^*}{\sqrt{2}}\right| ^2. \end{aligned}$$
(2.7)

Such expressions are analog to that of the GMSB with \(T,\tilde{S}_0\) the messenger-like fields. After integrated out the messengers, we can obtain their contributions to the low energy soft SUSY breaking spectrum. \(S_1\) can be written as the combination

$$\begin{aligned} S_1=\cos \theta \tilde{S}_0+\sin \theta \tilde{S}_1, \end{aligned}$$
(2.8)

which, after substituting into the superpotential, can lead to couplings between the massless fields \(\tilde{S}_1\) and heavy massive messenger-type fields \(\tilde{S}_0\).

Similar to the gauge singlet case, we can define the massive combinations \({\bar{K}}_1,K_1\) for P, Q within Eq. (2.2) and their massless orthogonal combinations \({\bar{K}}_2,K_2\) as

$$\begin{aligned} {\bar{K}}_1\equiv & {} \frac{1}{\sqrt{c_P^2+ (c_{m,3}^P)^2 }} \left[ c_P \tilde{P}+ c_{m,3}^P \tilde{P}_{m,3}\right] ,\nonumber \\ {\bar{K}}_2\equiv & {} \frac{1}{\sqrt{c_P^2+ (c_{m,3}^P)^2 }} \left[ -c_{m,3}^P \tilde{P}+ c_P \tilde{P}_{m,3}\right] ,\nonumber \\ K_1\equiv & {} \frac{1}{\sqrt{c_Q^2+ (c_{m,3}^Q)^2 }}\left[ c_Q Q+ c_{m,3}^Q Q_{m,3}\right] ,\nonumber \\ K_2\equiv & {} \frac{1}{\sqrt{c_Q^2+ (c_{m,3}^Q)^2 }}\left[ -c_{m,3}^Q Q+ c_Q Q_{m,3}\right] . \end{aligned}$$
(2.9)

So \(\tilde{P}_{m,3} (Q_{m,3})\) can be written as the combination of massive state \({\bar{K}}_1 (K_1)\) and massless state \({\bar{K}}_2 (K_2)\)

$$\begin{aligned} \tilde{P}_{m,3}= & {} \sin \psi _1\bar{K_1}+\cos \psi _1{\bar{K}}_2,\nonumber \\ Q_{m,3}= & {} \sin \psi _2{K_1}+\cos \psi _2{K}_2, \end{aligned}$$
(2.10)

with

$$\begin{aligned} \tan \psi _1=c_{m,3}^P/c_P,~~~~ \tan \psi _2=c_{m,3}^Q/c_Q. \end{aligned}$$
(2.11)

The SUSY breaking effects from compensator F-term VEVs can be taken into account by introducing a spurion superfields R

$$\begin{aligned} W=\int d^2 \theta \left( c_P R \tilde{P}P+ c_Q R \tilde{Q} Q +\cdots \right) , \end{aligned}$$
(2.12)

with the spurion VEV as

$$\begin{aligned} R\equiv M_R+\theta ^2 F_R= F_\phi (1-\theta ^2 F_\phi ), \end{aligned}$$
(2.13)

which gives the deflection parameter

$$\begin{aligned} d\equiv \frac{F_R}{M_R F_\phi }-1=-2. \end{aligned}$$
(2.14)

The spurion messenger–matter interactions will affect the AMSB RGE trajectory after integrating out the heavy modes.

The superpotential involving the matter and singlet superfields at the \(F_\phi \) scale are given as

$$\begin{aligned} W\supseteq & {} \tilde{\lambda } S_1 H_u H_d+\frac{1}{3}\tilde{\kappa } S_1^3\nonumber \\&\quad +\sum \limits _{a=1,2,3}\left[ \lambda _P S_1 \tilde{P}_{m,a} P+\lambda _Q S_1 \tilde{Q} Q_{m,a}\right] \nonumber \\&\quad +y_{ij}^{\bar{\mathbf{5}}}\tilde{P}_{m,i}Q_{m,j}{\bar{H}}_{\bar{\mathbf{5}}}+ {y}^\mathbf{5}_{ij}Q_{m,i}Q_{m,j}H_\mathbf{5}\nonumber \\&\quad +\sqrt{c_P^2+ (c_{m,3}^P)^2}{\bar{K}}_1 P R+\sqrt{c_Q^2+ (c_{m,3}^Q)^2}\tilde{Q} K_1 R ,\nonumber \\\supseteq & {} (\cos \theta \tilde{S}_0+\sin \theta \tilde{S}_1){\tilde{\lambda }} H_u H_d+ \frac{\tilde{\kappa }}{3} (\cos \theta \tilde{S}_0+\sin \theta \tilde{S}_1)^3\nonumber \\&\quad + \left[ {\lambda _P}\left( \sin \psi _1{\bar{K}}_1+\cos \psi _1 {\bar{K}}_2\right) P\right. \nonumber \\&\quad \left. +{\lambda _Q}\tilde{Q}\left( \sin \psi _2 {K}_1+\cos \psi _2 K_2\right) \right] (\cos \theta \tilde{S}_0+\sin \theta \tilde{S}_1),\nonumber \\&\quad +\sum \limits _{a=1,2}\left( {\lambda _P} \tilde{P}_{m,a} P+{\lambda _Q}\tilde{Q} Q_{m,a}\right) \nonumber \\&\quad (\cos \theta \tilde{S}_0+\sin \theta \tilde{S}_1)+\cdots . \end{aligned}$$
(2.15)

The terms containing the \(\mathbf{10},{\bar{\mathbf{5}}}\) representations of SU (5) reduce to the sum of their components below the GUT scale and should be understood as the abbreviation of this sum at low energy

$$\begin{aligned}&\sum \limits _{a=1,2,3}\left[ \lambda _P S_1 \tilde{P}_{m,a} P+\lambda _Q S_1 \tilde{Q}Q_{m,a} \right] \nonumber \\&\quad +y_{ij}^{ \bar{\mathbf{5}}}\tilde{P}_{m,i}Q_{m,j}{\bar{H}}_{\bar{\mathbf{5}}}+ {y}^\mathbf{5}_{ij}Q_{m,i}Q_{m,j}H_\mathbf{5}\nonumber \\&\quad \Longrightarrow \sum \limits _{a=1,2,3}S_1\left[ \lambda _{D,a} (D_{L,a}^c) D+\lambda _{L,a} (L_{L,a}) L\nonumber \right. \\&\quad \left. +\,\lambda _{Q,a} (Q_{L,a}) Q+\lambda _{U,a} (U_{L,a}^c) U+\lambda _{E,a} (E_{L,a}^c) E\right] \nonumber \\&\quad +y_{ab}^U (Q_{L,a}) (U_{L,b}^c) H_u+y_{ab}^D (Q_{L,a}) (D_{L,b}^c) H_d \nonumber \\&\quad + y_{ab}^E (L_{L,a}) (E_{L,b}^c) H_d. \end{aligned}$$
(2.16)

The same holds for terms containing \(K_a,{\bar{K}}_a\). Besides, terms involving the triplet components of \(H,{\bar{H}}\) are integrated out by assuming proper doublet-triplet splitting mechanism to generate heavy triplet Higgs masses.

After integrated out all the heavy modes including \(T,\tilde{S}_0;{\bar{K}}_1,K_1;P,\tilde{Q}\) etc, the low energy theory will reduce to NMSSM. The effects of integrating out the messengers can be taken into account by using Giudice-Rattazi’s wavefunction renormalization approach [39]. The messenger threshold \(M_R\) can be further promoted to the other chiral spurion field X with \(M_R=\sqrt{X^\dagger X}\). The superfield \(\tilde{S}_1\) will act as the singlet S appearing in ordinary NMSSM superpotential and \({\bar{K}}_2,K_2\) as the third generation superfields. Note that there is a scaling of various couplings appeared in NMSSM

$$\begin{aligned} \lambda= & {} \tilde{\lambda }\sin \theta ,~\kappa = \tilde{\kappa }\sin ^3\theta ,~y_{b,\tau }=y_{33}^{D,L}\cos \psi _1\cos \psi _2,\nonumber \\ y_t= & {} y_{33}^U\cos ^2\psi _2. \end{aligned}$$
(2.17)

In the subsequent studies, we must ensure that correct scaled (or unscaled) couplings are used in the expressions. New flavor dependent interactions involving the messengers from \(y_{ij}^{D,L}\tilde{P}_{m,i}Q_{m,j}{\bar{H}}_{\bar{\mathbf{5}}}\) and \({y}^U_{ij}Q_{m,i}Q_{m,j}H_\mathbf{5}\) are not dangerous because these new flavor dependent interactions are aligned with the MSSM Yukawa coupling. Diagonalizing the MSSM Yukawa couplings will simultaneously diagonalize these additional Yukawa couplings.

We should briefly discuss the most general case with \(c^P_{m;1,2}\ne 0\). Similar to Eq. (2.9), we can define the mixing matrix \(U^P_{AB}\) with the indices \(A,B= (0,a)\quad [a=1,2,3]\)

$$\begin{aligned}&\left( \begin{array}{c}{{\bar{K}}}_0\\ {{\bar{K}}}_1\\ {\bar{K}}2\\ {\bar{K}}3 \end{array}\right) \nonumber \\\equiv & {} \frac{1}{\sqrt{c_P^2+\sum \limits _{a} (c_{m,a}^P)^2}} \left( \begin{array}{cccc} c_P&{}c^P_{m,1}&{}c^P_{m,2}&{}c^P_{m,3}\\ -c^P_{m,1}&{}c_P&{}c^P_{m,3}&{}-c_{m,2}^P\\ -c^P_{m,2}&{}-c^P_{m,3} &{}c_P&{}c_{m,1}^P\\ -c^P_{m,3}&{}c_{m,2}^P&{}-c_{m,1}^P&{}c_P\end{array}\right) \nonumber \\&\left( \begin{array}{c}\tilde{P}\\ \tilde{P}_{m,1}\\ \tilde{P}_{m,2}\\ \tilde{P}_{m,3} \end{array}\right) . \end{aligned}$$
(2.18)

within which the three orthogonal combinations \({\bar{K}}a (a=1,2,3)\) are determined up to an arbitrary rotation under SO(3) transformation \(\mathcal{O}^P\). The scalar components of the three combinations \({\bar{K}}a (a=1,2,3)\) correspond to the massless eigenvalues of the \(5\times 5\) sfermion mass matrix for \( (P^*,\tilde{P},\tilde{P}_1^m,\tilde{P}_2^m,\tilde{P}_3^m)\) that can be identified to be the three generation squark/sleptons in MSSM. Similar conclusions hold for \(Q^m_a\). Besides, the relations between \(y_t,y_b,y_\tau \) and \(y^U_{33},y^D_{33},y^L_{33}\) will be non-trivial and depends on various parameters in the mixing matrix

$$\begin{aligned} \left( \mathcal{O^Q} y_{SM}^U\mathcal{O^Q}\right) _{ab}= & {} (U^Q)^{-1}_{a c} y^U_{cd} (U^Q)^{-1}_{db},\nonumber \\ \left( \mathcal{O^P} y_{SM}^D\mathcal{O^Q}\right) _{ab}= & {} (U^P)^{-1}_{a c} y^D_{cd} (U^Q)^{-1}_{db},\nonumber \\ \left( \mathcal{O^P} y_{SM}^L\mathcal{O^Q}\right) _{ab}= & {} (U^P)^{-1}_{a c} y^L_{cd} (U^Q)^{-1}_{db}. \end{aligned}$$
(2.19)

with

$$\begin{aligned} (U^P)^{-1}=\frac{ (U^P)^T}{c_P^2+\sum \limits _{a} (c_{m,a}^P)^2}. \end{aligned}$$

After fixing the rotation matrix \(\mathcal{O}^P\) and \(\mathcal{O}^Q\), we can obtain precisely the relations between SM Yukawa couplings and the couplings in the superpotential. For new flavor dependent interactions involving one messengers, such as \( (y^U)^\prime _{0 b}\bar{K}_{Q;0}{\bar{K}}_{U_L^c;b}H_u\), the coupling can be seen to satisfy

$$\begin{aligned} (y^U)^\prime _{0 c}\mathcal{O^Q}_{c b}= & {} (U^Q)^{-1}_{0 c} y^U_{cd} (U^Q)^{-1}_{db},\nonumber \\ (y^U)^\prime _{0 c}= & {} (U^Q)^{-1}_{0 a} U^Q_{a e}\mathcal{O}^Q_{e d} (y^U_\mathbf{SM})_{d c}. \end{aligned}$$
(2.20)

by combining with Eq. (2.19).

So the flavor constraints are no-dangerous if \(U^Q_{ae} \mathcal{O}_{ed}^Q\) is diagonal so that \( (y^U)^\prime \) can align to \(y^U_\mathbf{SM}\) or the coefficients of the new Yukawa couplings

$$\begin{aligned} (U^Q)^{-1}_{0 a}= -\frac{c^Q_{m,a}}{c_Q^2+\sum \limits _{a} (c_{m,a}^Q)^2}, \end{aligned}$$
(2.21)

are small. The SO (3) rotation matrix can be parameterized by three Euler angles, so the three diagonal elements of the diagonalized \(U^Q_{ab} \mathcal{O}_{bc}^Q\) matrix can match the three rotation freedom. Therefore, it is possible to align \( (y^U)^\prime \) and \(y^U_{SM}\) by proper chosen \(\mathcal{O}^Q\) so that we need not worry too much about the flavor constraints. On the other hand, if we insist on small \(c_{1,2}^{P,Q}\), such small numbers can be the consequence of suppressions from additional Froggatt-Nielsen [40, 41] type mechanism with additional horizontal flavor symmetry.

After integrating out the messenger fields, the wavefunction will depend on the messenger threshold set by the spurion superfield R. The soft gaugino masses are given at the messenger scale by

$$\begin{aligned} M_{i} (M_{mess})= & {} g_i^2\left( \frac{F_\phi }{2}\frac{\partial }{\partial \ln \mu }-\frac{d F_\phi }{2}\frac{\partial }{\partial \ln |X|}\right) \frac{1}{g_i^2} (\mu ,|X|,T),\nonumber \\ \end{aligned}$$
(2.22)

with

$$\begin{aligned} \frac{\partial }{\partial \ln |X|} g_i (\alpha ; |X|)=\frac{\Delta b_i}{16\pi ^2} g_i^3, \end{aligned}$$
(2.23)

Because of the non-renormalization of the superpotential, the trilinear soft terms will be determined by the wavefunction normalization as

$$\begin{aligned} A_0^{ijk}\equiv & {} \frac{A_{ijk}}{y_{ijk}}\nonumber \\= & {} \sum \limits _{i}\left( -\frac{F_\phi }{2}\frac{\partial }{\partial \ln \mu }+{d F_\phi }\frac{\partial }{\partial \ln X}\right) \ln \left[ Z_i (\mu ,X,T)\right] ,\nonumber \\= & {} \sum \limits _{i} \left( -\frac{F_\phi }{2} G_i^- +d F_\phi \frac{\Delta G_i}{2}\right) . \end{aligned}$$
(2.24)

The anomalous dimension are expressed in the holomorphic basis [42] as

$$\begin{aligned} G^i\equiv & {} \frac{d Z_{ij}}{d\ln \mu }\nonumber \\\equiv & {} -\frac{1}{8\pi ^2}\left( \frac{1}{2}d_{kl}^i\lambda ^*_{ikl}\lambda _{jmn}Z_{km}^{-1*}Z_{ln}^{-1*}-2c_r^iZ_{ij}g_r^2\right) .\nonumber \\ \end{aligned}$$
(2.25)

with \(\Delta G\equiv G^+-G^-\) the discontinuity across the messenger threshold. Here \(G^+ (G^-)\) denotes the value above (below) the messenger threshold, respectively.

The soft SUSY breaking scalar masses are given by

$$\begin{aligned} m^2_{soft}= & {} -\left| -\frac{F_\phi }{2}\frac{\partial }{\partial \ln \mu }+d F_\phi \frac{\partial }{\partial \ln X}\right| ^2 \ln \left[ Z_i (\mu ,X,T)\right] ,\nonumber \\= & {} -\left( \frac{F_\phi ^2}{4}\frac{\partial ^2}{\partial (\ln \mu )^2}+\frac{d^2F^2_\phi }{4}\frac{\partial }{\partial (\ln |X|)^2}\nonumber \right. \\&\left. -\frac{d F^2_\phi }{2}\frac{\partial ^2}{\partial \ln |X|\partial \ln \mu }\right) \ln \left[ Z_i (\mu ,X,T)\right] , \end{aligned}$$
(2.26)

The dAMSB soft scalar masses can be divided into several parts, namely the gauge-anomaly interference part, the pure gauge mediation part as well as the ordinary anomaly mediation part.

3 Numerical results

From the previous general formulas for soft SUSY breaking parameters, we can obtain the analytical expressions in our scenario at the scale \(F_\phi \) after integrating out the messenger fields. Some of the lengthy expressions are given explicitly in the appendix. We can see that new contributions will be given to the trilinear couplings \(A_\kappa .A_\lambda ,m_S^2\) which could be helpful to trigger EWSB in NMSSM.

We use NMSSMTools5.1.2 [43, 44] to scan the whole parameter space. The free parameters of our inputs are chosen to satisfy

$$\begin{aligned} 10 \mathrm{TeV}< & {} F_\phi< 1000\,\mathrm{TeV},\nonumber \\ 0.1< & {} \cot \theta ,\tan \psi _1,\tan \psi _2< 10,~\nonumber \\ 0< & {} \lambda _{D,a},\lambda _{L,a},\lambda _{Q,a},\lambda _{U,a},\nonumber \\ \lambda _{E,a}< & {} \sqrt{4\pi },~~~~~~~0<\lambda ,\kappa <0.7,~~ \end{aligned}$$
(3.1)

with the range of the mixing angle \(-\pi /2<\theta ,\psi _1,\psi _2<\pi /2\) and we require that \(\lambda ^2+\kappa ^2\lesssim 0.7\) to satisfy the perturbative bounds. The soft SUSY mass \(m_{H_u}^2,m_{H_d}^2,m_S^2\) can be recast into \(\mu ,\tan \beta ,M_Z^2\) by the minimization conditions of the scalar potential. Usually, the parameter \(M_A\) can be used to replace \(A_\kappa \) by

$$\begin{aligned} M_A^2= & {} \frac{2\mu _{eff}}{\sin 2\beta }B_{eff}, ~~\mu _{eff}\equiv \lambda \langle s\rangle ,\nonumber \\ B_{eff}= & {} (A_\lambda +\kappa \langle s\rangle ). \end{aligned}$$
(3.2)

We note that \(\kappa \) is a free parameter while the \(\tan \beta \equiv v_u/v_d\) is not. This choice is different to ordinary numerical setting from top-down in which \(\tan \beta \) is free while \(\kappa \) is a derived quantity [45]. Such choice is convenient for those predictable NMSSM models with a UV completion. Firstly, we need to guess a value for \(\tan \beta \) to obtain the relevant Yukawa \(y_t,y_b\) couplings at the electroweak (EW) scale. After renormalization group equation (RGE) evolution of \(g_i,y_t,y_b\) couplings from EW scale to the messenger scale as the theory inputs, the whole soft SUSY breaking parameters at the messenger scale can be obtained. Low energy \(\tan \beta \) can be obtained iteratively from the minimization condition of the Higgs potential [46]. The purpose of deflection in AMSB is to solve the notorious tachyonic slepton problem. So non-tachyonic slepton should be obtained at the EW scale.

In our scan, we also impose the following collider constraints

  1. (i)

    The lower bounds from current LHC constraints on SUSY particles [47,48,49,50]. In particular, the gluino mass is bounded \(m_{\tilde{g}} \gtrsim 1.85\sim 2.0 \) TeV from a search for gluino pair production, assuming \(\tilde{g}\rightarrow q{\bar{q}}\chi _1^0\) with a massless LSP and decoupled squarks; or \(m_{\tilde{g}} \gtrsim 1.96\sim 2.05 \) TeV, assuming \(\tilde{g}\rightarrow t{\bar{t}}\chi _1^0\). The squark \(m_{\tilde{t}_1}\gtrsim 0.95\sim 1.05\) TeV for light third generation sfermions, and \(m_{\tilde{q}}\gtrsim 1.6\) TeV for the first two generations.

  2. (ii)

    Flavor constraints from the rare decays of B meson. We adopt the recent experimental results [51]:

    $$\begin{aligned}&0.85\times 10^{-4}< Br (B^+\rightarrow \tau ^+\nu )< 2.89\times 10^{-4},\nonumber \\&2.99\times 10^{-4}< Br (B_S\rightarrow X_s \gamma )< 3.87\times 10^{-4},\nonumber \\&1.7\times 10^{-9}< Br (B_s\rightarrow \mu ^+ \mu ^-) < 4.5\times 10^{-9}, \nonumber \\ \end{aligned}$$
    (3.3)
  3. (iii)

    The CP-even component \(S_2\) in the Goldstone-\('eaten'\) combination of \(H_u\) and \(H_d\) doublets corresponds to the SM-like Higgs. The \(S_2\) dominated CP-even scalar should lie in the combined mass range: \(122 \, \mathrm{GeV}<M_h <128 \,\mathrm{GeV}\) from ATLAS and CMS data. Note that the uncertainty is 3 GeV instead of default 2 GeV in NMSSMTools because large \(\lambda \) may induce additional \(\mathcal{O} (1)\) GeV correction to \(m_h\) at two-loop level [52].

  4. (iv)

    The EW precision observables [53] and the lower bounds for the neutralino and chargino masses, including the invisible decay bounds for Z boson. The most stringent LEP bounds require \(m_{\tilde{\chi }^\pm }> 103.5\) GeV and the invisible decay width \(\Gamma (Z\rightarrow \tilde{\chi }_0\tilde{\chi }_0)<1.71~\mathrm{MeV}\), which is consistent with the \(2\sigma \) precision EW measurement \(\Gamma ^{non-SM}_{inv}< 2.0~\mathrm{MeV}\) [54].

A possible hint of new physics beyond the SM is the muon \(g-2\) anomaly. The E821 experimental result of the muon anomalous magnetic moment at the Brookhaven AGS [55] is given by

$$\begin{aligned} a_\mu ^{\mathrm{expt}} =116592089 (63)\times 10^{-11}, \end{aligned}$$
(3.4)

which is larger than the SM prediction [56, 57]

$$\begin{aligned} a^{\mathrm{SM}}_\mu =116591834 (49)\times 10^{-11}. \end{aligned}$$
(3.5)

We adopt the conservative estimation \(4.7\times 10^{-10}\lesssim \Delta a_\mu \lesssim 52.7\times 10^{-10}\) in our scenario. If possible, we would like our theory to explain such muon g-2 anomaly.

To address the strong CP problem in SUSY, the axino, which is the fermionic superpartner of axion, can be predicted. If the existence of axino could be confirmed by experiments, the two theoretical hypotheses, which are designed to solve two respective hierarchy problems: the strong CP problem by a very light axion and the gauge hierarchy problem by SUSY, could be validated.

The SUSY version of the Kim-Shifman-Vainshtein-Zakharov (KSVZ) [58, 59] axion model, which introduces a SM singlet and a pair of extra vector-like quarks that carry \(U (1)_{PQ}\) charges while keeps the SM fermions and Higgs fields neutral under \(U (1)_{PQ}\) symmetry, could be introduced in NMSSM with additional singlets and messengers. The decoupling theorem [60] of anomaly mediation, which states that pure mass threshold will not deflect the AMSB trajectory, guarantee that such new messengers, with their masses determined by the singlet VEVs between \(10^{10}\mathrm{GeV}\lesssim \langle S\rangle \lesssim 10^{12}\) GeV (to be compatible with the experimental bound on \(f_a\) [61]) will not change the original AMSB predictions upon \(F_\phi \).

The axino mass, which is strongly model dependent, can be much smaller or much larger than the SUSY scale. Some models predict \(m_{\tilde{a}}\) of the order of gravitino mass \(m_{3/2}\) [61] while the spontaneously broken global SUSY predict it to lie of order \(m_{\tilde{a}} \sim \mathcal{O} (M^2_{SUSY}/f_a)\) [62]. So one usually treats axino mass as a free parameter and assumes the axino interactions to be given by the \(U (1)_{PQ}\) symmetry. If axino is heavy, the lightest neutralino will be DM candidate; if the axino is the LSP and acts as DM particle, colored or charged NLSP can decay into axino and non-thermally generate the observed axino DM relic density.

Fig. 1
figure 1

Numerical scan for scenario I with neutralino LSP. All points can satisfy the collider constraints from (i) to (iv). The allowed values for \(\lambda \) vs \(\kappa \), which are derived from EWSB conditions of NMSSM, are given in the upper left panel; in the upper right panel, the allowed ranges of \(F_\phi \) vs \(m_{Higgs}\) are shown explicitly. The corresponding values of SUSY contributions to \(\Delta a_\mu \) are also shown; in the lower left panel and lower right panel, the Spin-Independent and Spin-Dependent direct detection cross sections for neutralino dark matter are given, respectively. The exclusion lines from from LUX and PANDAX experiments are also shown

Full size image

Depending on the nature of the LSP, we have the following discussions from our numerical results

  • Scenario I–The lightest neutralino \(\chi _1^0\) as the LSP: Successful EWSB conditions impose stringent constrains on the input parameters of NMSSM, especially when such low energy inputs are determined by a UV-completed theory. Random scan in this case indicates that still some points can survive the EWSB conditions by leading to a iteratively stable value of \(\tan \beta \). The allowed range for characteristic NMSSM parameters \(\kappa \) and \(\lambda \) can be seen in the upper left panel of Fig. 1. Besides, the discovered 125 GeV Higgs can be successfully interpreted in our scenario. The mass scales of the soft spectrum are uniquely determined by the gravitino mass \(F_\phi \) in AMSB-type scenarios. The plot of \(F_\phi \) versus Higgs boson mass is shown in the upper right panel of Fig. 1. In the NMSSM, the SM-like Higgs can be pushed to 125 GeV by additional tree-level contributions for relatively large \(\lambda \) and small \(\tan \beta \). If the lightest CP-even scalar is mostly singlet-like, mixing between it and the SM-like Higgs can lead to an increase of the SM-like Higgs mass. Although the possibility of SM-like Higgs being the second lightest CP-even scalar is attractive, we find that almost all the survived points predict the lightest CP-even scalar as the 125 GeV Higgs. Numerical results indicate that the muon \(g-2\) anomaly cannot be explained in this scenario. The SUSY contributions to \(\Delta a_\mu \) are shown in different color in the upper right panel of Fig. 1. This result can be understood because the required \(\Delta a_\mu \) can be achieved only if the relevant sparticles ( \(\tilde{\mu },\tilde{\nu }_\mu , \tilde{B},\tilde{W}, \tilde{H}\)) are lighter than \(600\sim 700\) GeV for \(\tan \beta \sim 10\) in MSSM [63,64,65,66,67,68]. The inclusion of singlino in NMSSM will not give sizable contributions to \(\Delta a_\mu \) because the coupling of singlino to MSSM sector is suppressed. Although the two loop contributions involving the Higgs is negligible in SM, the new higgs bosons in NMSSM could have an important impact on \(a_\mu \) if the lightest neutral CP-odd Higgs scalar is very light [69]. As noted there, positive two-loop contribution is numerically more important for a light CP-odd Higgs heavier than 3 GeV and the sum of both one-loop and two-loop contributions is maximal around \(m_{a_1}\sim 6\) GeV. In our scenario, the lightest CP-odd Higgs \(a_1\) is not light enough to give sizable contributions to \(\Delta a_\mu \). So the main contributions are similar to that in the MSSM and are not large enough to account for the discrepancy. In the case of \(\chi _1^0\) as the LSP, the dark matter (DM) relic density is required to satisfy the bounds set by the Planck data [70] in combination with the 5\(\sigma \) WMAP data [71]:

    $$\begin{aligned} 0.0913\lesssim \Omega _{DM}\lesssim 0.1363. \end{aligned}$$

    Numerical scan indicates that the LSP is almost pure Bino-like. This fact can be understood from the gaugino ratio at the \(F_\phi \) scale \(M_1{:}M_2{:}M_3=14.6{:}9{:}5\) which will lead to the ratio at the EW scale \(M_1{:}M_2{:}M_3\approx 14.6{:}18{:}30\). The Higgsino components within the LSP are tiny and the effective \(\mu \) parameter is relatively high in our scenario. It is well known that the Bino LSP will lead to over-abundance of DM unless co-annihilation with sfermions or s-channel resonance increase the annihilation rates. In our scenario, the Bino-like DM always co-annihilates with the lightest stau or sbottom and leads to the right DM relic density. The mass differences between \(\tilde{\tau }_1 (\tilde{b}_1)\) NLSP and \(\tilde{\chi }_1^0\) LSP are typically of order \(\mathcal{O} (GeV)\). Interactions between Bino dark matter and the nucleons are primarily mediated by t-channel scalar Higgses or by s-channel squarks. The Higgs exchange diagrams dominate the Spin-Independent (SI) cross section of \(\chi _1^0\) on nucleon for sufficiently heavy squarks. The Spin-Dependent (SD) cross section, which is dominated by the Z-boson exchange diagram as long as the squarks are sufficiently heavy, can provide a good probe of the gaugino and Higgsino parameters. Large SD cross sections can appear only if \(\chi _1^0\) DM contains a large Higgsino component. Although the SD direct detection cross section is generally larger than the SI cross section, it is much more difficult to probe in the experiment, as the SD cross section does not scale directly with the mass of the nuclei. In fact, current bounds on the neutron SD cross section are less stringent by a factor of \(10^6\). The SI and SD direct detection results are shown in the lower right panel of Fig. 1. In our scenario, the neutralino DM is almost pure Bino with very tiny Higgsino and Wino contents. Such (almost pure) Bino-like DM can survive the direct detection constraints from LUX [72] and PANDAX [73] because of the suppressed Higgsino component.

  • Scenario II–The axino as the LSP: As mentioned before, the axino can be the LSP and act as the DM candidate. So the previously forbidden charged or colored LSP region can be revived. In some of the allowed parameter space, numerical scan indicate that lightest ordinary supersymmetric particle (LOSP) will be the lightest sbottom, stau, tau-sneutrino etc. Even in this scenario, successful EWSB conditions can impose stringent constrains on the input parameters of NMSSM, especially for characteristic NMSSM parameters \(\kappa \) and \(\lambda \). Their allowed range can be seen in the upper left panel of Fig. 2. The plot of \(F_\phi \) versus Higgs boson mass is shown in the upper right panel of Fig. 2. We can see that the 125 GeV Higgs mass can also be easily accommodated in this scenario. The muon \(g-2\) discrepancy, however, prefer this scenario. The most stringent constraint, as noted in our previous papers [74], is the gluino mass bound. We can see that the SUSY contributions to \(\Delta a_\mu \) can reach \(14\times 10^{-10}\) in the case of slepton (\(\tilde{\tau }_1,\tilde{e}_R\)) LOSP with \(m_{\tilde{g}}\approx 1.8\) TeV. The mass bound of gluino can be understood as follows: as the whole low energy spectrum is determined by the value of \(F_\phi \), the mass scales of \(\tilde{\mu }\) etc determine the upper bound of \(F_\phi \), which, on the other hand, sets a bound on gluino mass. We find that the gluino mass is bounded to less than 2.2 TeV if we adopt the lower limit of the required \(\Delta a_\mu \). Such light gluino will soon be tested by the future LHC searching results. The explanation of the muon \(g-2\) discrepancy prefers the effective \(\mu \) parameter to be light, which could reduce the fine tuning (FT) involved in our theory. Besides, the stop mass can also be predicted to lie just upon the experimental bounds from recent LHC and be fairly natural [75]. We show the Barberi-Giudice (BG) FTs [76] for the survived points in the figures. We can see that the FT involved in this scenario can be as low as 50. DM particle in this scenario is the axino from LOSP decay (neglecting the effects of saxion condensate [77]). As the non-thermally produced relic density of axino can be related to that of NLSP via

    $$\begin{aligned} \Omega _{\tilde{a}}=\frac{m_{\tilde{a}}}{m_{NLSP}}\Omega _{NLSP}, \end{aligned}$$
    (3.6)

    the relic density of DM will not impose stringent constraints on our parameter space. We show the relic density of NLSP in Fig. 2. The thermal production of axino, which may give important contributions (or even dominate over the non-thermally production) to the observed relic density and be model dependent, will impose strong bound on the reheating temperature after inflation by requiring such production should not overclose the universe.Footnote 1 Relevant discussions on the reheating temperature and cosmological consequences of axino DM can be found in [79, 80]. The extremely weak interaction strength of axino makes its detection in DM direct detection experiments, as well as at collider experiments, rather hopeless. The hint of the axino DM may show up from the properties of the LOSP. In the case of axino LSP, the LOSP typically decays with a lifetime of less than one second and practically be stable inside the collider detector. The (electrically) charged or colored particle as a LOSP would appear as a stable particle inside the detector. The injection of high-energetic hadronic and electromagnetic particles, produced from late decays of an NLSP to axino (with lifetime less than one second), will not affect the abundance of light elements produced during Big Bang Nucleosynthesis (BBN).

Fig. 2
figure 2

Numerical scan for scenario II with axino LSP. All points can satisfy the collider constraints (i) to (iv). In the upper left panel, the allowed values for \(\lambda \) vs \(\kappa \) are derived from EWSB condition of NMSSM; in the upper right panel, the allowed ranges of \(F_\phi \) vs \(m_{Higgs}\) are shown explicitly. The involved Barbieri-Giudice fine tuning are also shown; in the lower left panel, the SUSY contributions to \(\Delta a_\mu \) vs the gluino mass \(m_{\tilde{g}}\) are given for various NLSP (\(\tilde{\tau }_1,\tilde{e}_R,\tilde{b}_1\) etc); The freeze-out relic density for NLSP particles (before later decaying into axino DM) are shown in the lower right panel

Full size image

4 Conclusions

We propose a new approach to generate messenger–matter interactions in deflected anomaly mediated SUSY breaking mechanism from typical holomorphic messenger–matter mixing terms in the Kahler potential. This approach is a unique feature of AMSB and has no analog in GMSB-type scenarios. New coupling strengths obtained from the scaling of the (already known) Yukawa couplings, such as \(y_t\tan \psi _2,y_t\tan ^2\psi _2,y_b\tan \psi _2\tan \psi _1,\kappa \cot \theta ,\kappa \cot ^2\theta \), always appear in this approach, which is a salient feature of this scenario. With messenger–matter interactions in deflected AMSB, we can generate a realistic soft SUSY breaking spectrum for next-to-minimal supersymmetric standard model (NMSSM). Successful electroweak symmetry breaking conditions, which is not easy to satisfy in NMSSM for ordinary AMSB-type scenario, can be satisfied in a large portion of parameter space in our scenarios. We study the relevant phenomenology for scenarios with (Bino-like) neutralino and axino LSP, respectively. In the case of Bino-like LSP, most of parameter space can survive the recent SI and SD DM direct detection experiment; in the case of axino LSP, the SUSY contributions to \(\Delta a_\mu \) can possibly account for the muon g-2 discrepancy. The corresponding gluino masses (for those points that can explain the muon g-2 anomaly), which are found to below 2.2 TeV, could be tested soon at LHC.