Departures from neutrality induced by niche and relative fitness differences

Abstract

Breaking the core assumption of ecological equivalence in Hubbell’s “neutral theory of biodiversity” requires a theory of species differences. In one framework for characterizing differences between competing species, non-neutral interactions are said to involve both niche differences, which promote stable coexistence, and relative fitness differences, which promote competitive exclusion. We include both in a stochastic community model in order to determine if relative fitness differences compensate for changes in community structure and dynamics induced by niche differences, possibly explaining neutral theory’s apparent success. We show that species abundance distributions are sensitive to both niche and relative fitness differences, but that certain combinations of differences result in abundance distributions that are indistinguishable from the neutral case. In contrast, the distribution of species’ lifetimes, or the time between speciation and extinction, differs under all combinations of niche and relative fitness differences. The results from our model experiment are inconsistent with the hypothesis of “emergent neutrality” and support instead a hypothesis that relative fitness differences counteract effects of niche differences on distributions of abundance. However, an even more developed theory of interspecific variation appears necessary to explain the diversity and structure of non-neutral communities.

Introduction

Empirical studies of resource competition routinely show that niche differences promote coexistence in nature (e.g., Tilman 1977; Silvertown et al. 1999; Adler et al. 2010; Harms et al. 2000; Narwani et al. 2013). Mathematical models of competition with just one possible niche could therefore be expected to perform badly. Models of this type are called neutral community models (NCMs) and, contrary to expectation, actually do reproduce several observed species abundance distributions (SADs; Bell 2001; Hubbell 2001; McGill 2003; Volkov 2003; Etienne 2005). But SADs can be remarkably similar under both neutral and non-neutral assumptions (Volkov et al. 2005; Ruokolainen et al. 2009; Haegeman and Loreau 2011; Noble et al. 2011; Chisholm and Pacala 2010; Zillio and Condit 2007). To help understand the conditions allowing neutral and non-neutral communities to appear similar, we develop theory distinguishing between two different pathways to degeneracy that have garnered attention: so-called “emergent neutrality” (Holt 2006) and compensation for niche differentiation through “destabilizing” mechanisms (Chave 2004). To help establish patterns more indicative of non-neutrality in competitive communities, we also investigate how species lifetime distributions respond to the addition of both niche differences and destabilizing mechanisms.

The argument behind emergent neutrality is that neutral, or at least nearly neutral, communities arise from ecological or evolutionary processes that promote similarity between species (Holt 2006). The concept principally applies to speciation events driven by minor genetic mutations; new species persist because they scarcely differ (Scheffer and van Nes 2006). More broadly, emergent neutrality may describe any situation where trade-offs between traits allow species to achieve equivalent fitness (Ostling 2012). The resulting community dynamics are considered nearly neutral whenever the vital rates of individuals become roughly equivalent, potentially allowing communities with heterogeneous traits to obtain neutral patterns of abundance. It is not guaranteed, however, that nearly neutral models will behave much like exactly neutral models (Zhou and Zhang 2008; Ostling 2012).

In contrast to emergent neutrality, Chave (2004) hypothesized that neutral-like patterns occur when niche differences are accompanied by some destabilizing mechanism. Du et al. (2011) give a concrete example of contrasting ecological mechanisms that supports Chave’s hypothesis. Niche differences, modeled as frequency-dependent survivorship, are included along with interspecific variation in fecundity, which Du et al. (2011) incorporate as a destabilizing process. As predicted, neutral-like SADs do indeed occur when differences in fecundity are tuned to a level that offsets the stabilizing effect of frequency-dependent survivorship (Du et al. 2011).

Emergent neutrality and Chave’s hypothesis both violate the neutral assumption of strict ecological equivalence at the individual level, so a formal comparison requires a quantitative treatment of departures from neutrality. A framework for such a treatment exists in the review of coexistence mechanisms by Chesson (2000), which mirrors the hypothesis of Chave (2004) in recognizing two, opposing ways of breaking the assumption of neutrality (Adler et al. 2007). Ecological trade-offs classically associated with niche differentiation create a kind of non-neutral interaction that stabilizes species coexistence (Chesson 2000). A destabilizing process implies a kind of departure from neutrality that increases “relative fitness differences” between species and thereby promotes competitive exclusion: it is the reverse of an equalizing mechanism of coexistence (Chesson 2000).

When mapped onto the niche vs. relative fitness differences framework, the two proposed scenarios under which non-neutral communities may appear neutral become qualitatively distinct. To attribute any degeneracy to emergent neutrality, the community in question should exhibit both strong equalizing mechanisms (which eliminate relative fitness differences) and weak stabilizing mechanisms (precluding niche differences). Under this hypothesis, departures from neutrality along either axis would reduce the similarity of non-neutral communities to neutral ones. Alternatively, the ratio of stabilizing to equalizing mechanisms, rather than the magnitude of the departure from neutrality, may determine whether non-neutral communities exhibit neutral-like patterns. Even large deviations from neutrality, so long as they maintain a more or less careful balance between the two types of species difference, allow the similarity between neutral and non-neutral communities to persist under this alternative hypothesis. Unlike the first scenario, this one raises the question of whether observed similarities between neutral and non-neutral communities is a quirk peculiar to SADs alone. Confirmation of the second scenario should embolden attempts to identify ecological patterns that are not degenerate in the presence of strong niche and relative fitness differences (McGill et al. 2007).

Here, we relate the magnitude of niche differences (ND) and of relative fitness differences (RFD) to the appearance of neutral-like patterns of diversity in a stochastic Lotka-Volterra competition model, one similar to the model of Haegeman and Loreau (2011). To quantify species’ differences in a reproducible way, we extend the approach of Carroll et al. (2011), which is shown to give precisely the same measures of ND and RFD identified by Chesson (2013) in a deterministic, two-species version of the model. Our objective is to determine whether the combination of niche and relative fitness differences controls whether the community exhibits properties also observed under the neutral parameterization of the model. In addition to comparing SADs, we also examine the effect of departures from neutrality on the distribution of species’ lifetimes. In this way, our study contributes to the objective of identifying types of observational or monitoring data that would reliably characterize ecological interactions in an empirical setting.

Departures from neutrality

The dynamic signature of increasing ND is the simultaneous increase of multiple species’ invasion fitness (Metz et al. 1992; Metz 2008) or their population growth rates when rare (Chesson 2000; Carroll et al. 2011). Reducing overlap in the resources consumed by two species, the classic example of ND, will hasten recovery of both species from low densities. In contrast, the consequence of RFD is an increase in the invasion fitness of one species that comes at some cost to the invasion fitness of another (Chesson 2000; Carroll et al. 2011). Variation in fecundity, for example, can bring about a RFD when it leads to competitive exclusion in the absence of a sufficient stabilizing mechanism (Adler et al. 2010). Considering these effects of interspecific competition on community-wide trends in invasion fitness led Carroll et al. (2011) to propose community-level indices of niche and relative fitness differences. In short, the index of ND tracks the geometric mean effect of competition on low-density population growth rates, while RFD is measured by the geometric standard deviation of these effects. As detailed next, this approach to quantifying ND and RFD independently arrives at the established result for a two-species Lotka-Volterra (LV) model (Chesson 2013).

The first step is to calculate the “sensitivity to competition” for each of n species, denoted by S _i for i in {1,…,n}, and defined to be the relative change in the species’ population growth rate when invading an equilibrium community of heterospecific competitors (Carroll et al. 2011). To write down an explicit formula for S _i in a LV competition model, let the competition coefficients be α _i,j and let $y_{j,i}^{*}$ be the abundance, relative to the carrying capacity, of species j at equilibrium in the absence of species i (Appendix A). With just two species, for example, $y_{2,1}^{*} = y_{1,2}^{*} = 1$. For the LV model,

$$ S_{i} = \sum\limits_{j \neq i} \frac{\alpha_{i,j}}{\alpha_{j,j}} y_{j,i}^{*}. $$

(1)

The equation assigns zero sensitivity to a species unaffected by competition and increasing sensitivity with either larger competition coefficients or greater abundance of competitors. A sensitivity that exceeds one corresponds to a negative invasion fitness, or the inability of species i to invade an established community of competitors.

The set {S ₁,S ₂,…,S _n } is subsequently used to calculate ND _n and RFD _n , the subscripted abbreviations indicating these specific metrics for ND and RFD among n species. We equate ND _n to one minus the geometric mean of the sensitivities, which reflects an average effect of competition on invasion fitness. Likewise, RFD _n is the geometric standard deviation of the sensitivities, which reflects variability in competition’s effect on different species. In the LV model with two competitors,

$$ \text{ND}_{2} = 1 - \sqrt{\frac{\alpha_{1,2}}{\alpha_{1,1}} \frac{\alpha_{2,1}}{\alpha_{2,2}}}, $$

(2)

which Chesson (2013) denotes by 1−ρ while describing ρ as a measure of niche overlap. Likewise, assuming without loss of generality that α _2,1/α _1,1≥α _1,2/α _2,2,

$$ \text{RFD}_{2} = \sqrt{\frac{\alpha_{2,1}}{\alpha_{1,1}}\frac{\alpha_{2,2}}{\alpha_{1,2}}}. $$

(3)

The assumption makes RFD ₂≥1, as required of a geometric standard deviation. This is the “average fitness ratio” that Chesson (2013) denotes by κ ₁/κ ₂. An increase in ND _n means the average sensitivity to competition has been reduced, which corresponds to a stabilizing mechanism of coexistence; a process that brings RFD _n closer to one is an equalizing mechanism because it makes stable coexistence possible with a smaller ND _n (Carroll et al. 2011).

The preceding steps are directly applicable to deterministic models with a fixed community composition. NCMs and their non-neutral alternatives typically do not have a static set of species: extinctions and introductions change the number and identity of extant species over time, and the entire community is modeled as a stochastic process. Neither do such models have a perfect analog for the population growth rate Carroll et al. (2011) used to define S _i . For a given richness, however, the model presented next is routinely approximated by the LV model just described. A straightforward approach to quantifying ND and RFD in this stochastic setting is to independently sample communities from the process, calculate ND _n and RFD _n values according to an LV model parameterized as in each sampled community, and average these values across samples. For large samples, we obtain estimates of the expected values denoted by 〈ND〉 and 〈RFD〉. We drop n in this notation because all richness levels contribute to the expectation.

Model

We examine the consequence of 〈ND〉 and 〈RFD〉 variation in a Lotka-Volterra-like model that admits demographic stochasticity. The model is similar to one studied by Haegeman and Loreau (2011), except that it describes a closed “metacommunity” (sensu Hubbell 2001). Including dispersal can improve the fit of NCMs to empirical SADs (Etienne 2005), but our narrower purpose of comparing models only to each other justifies the simplifying assumption of a well-mixed community, closed to immigration, that only gains new species through unique speciation events. Hewing close to the model of Hubbell (2001), we also ignore the realities of population age or size structure and inhomogeneous environments. As in the deterministic LV model, we depict competition as the linear dependence of each individual’s per capita mortality rate on the population density of its competitors. We admit demographic stochasticity by modeling the community as a continuous-time Markov chain, or more precisely as a multivariate birth-death process (Nisbet and Gurney 1982; Gardiner 2009). Introductions of novel species are incorporated into the model as a Poisson process.

The state of the community is denoted by a random vector N whose ith element is an integer abundance, changing by plus or minus one with each birth or death in the ith species. During any short time interval of duration dt, the probability of a birth of one individual into the ith species is B _i (N)dt. Likewise, a death of one individual in the ith species has probability D _i (N)dt. We assign the same per capita birth rate, λ, to all individuals, giving

$$ B_{i}(\boldsymbol{N}) = \lambda N_{i}. $$

(4)

Density dependence is incorporated in the species-specific death rates

$$ D_{i}(\boldsymbol{N}) = \left( \mu + \delta \sum\limits_{j} \alpha_{i,j} \frac{N_{j}}{\Omega} \right) N_{i}, $$

(5)

where Ω is the area or volume of the system, μ is the density-independent mortality rate, α _i,i is fixed at one so δ becomes the amount of density dependence each species would experience in isolation, and the competition coefficient, α _i,j , modulates the impact of the jth species on density dependence within the ith species. All parameters are non-negative. Taking dt infinitesimally small guarantees that at most one birth or death can occur at any moment within the entire community and provides, in the usual way, a system of ordinary differential equations whose solution is the probability distribution on N over time (Appendix A).

Every species in the model eventually goes extinct. The transition from a positive population size to zero cannot be reversed because there is no immigration into the metacommunity. Introductions of novel species, i.e., “speciation” events, prevent eventual extinction of the entire community. We parody the drawn-out processes of macroevolution by allowing a single individual of a novel species to appear with probability νdt at any point in time. This is almost the “point mutation mode” of speciation described by Hubbell (2001), but the speciation rate here is not per capita. How novel species are assigned competition coefficients will be explained after distinguishing neutral from non-neutral parameterizations of the model.

Neutral parameterization

Setting every competition coefficient (i.e., every α _i,j ) equal to one corresponds to the definition given by Hubbell (2001) of neutrality at the level of the individual. The per capita mortality rate in Eq. 5 becomes the same for every individual in the community. Even though the rate increases with community size, or the total number of individuals of all species, every individual is identically affected. The per capita birth rate is the constant λ in Eq. 4, so every individual also has the same chance of reproducing. This parameterization makes individuals demographically indistinguishable and causes the model to behave similarly to previously published NCMs.

Although community size is not fixed, the model’s stationary SAD is consistent with the sampling formula that Hubbell (2001) and many others have established under the “zero-sum” assumption of a fixed community size. Caswell (1976) and Etienne et al. (2007) both present NCMs with a variable community size whose SAD is distributed according to the sampling formula of Ewens (1972) when conditioned on a given number of individuals. Our model differs from these NCMs in two ways: (1) species abundances are not independent and (2) the birth rate for species with zero individuals is zero, making the speciation process distinct from recovery of extirpated species. These features also distinguish our model from the type presented by Volkov et al. (2003), in which a fixed number of species independently repeat a random walk away from zero and then back to it. In our model, the probability distribution on the total community size is sharply peaked near Ω(λ−μ)/δ, while the relative abundances of component species are variable and negatively correlated. With density-dependent mortality regulating community size without holding it constant, the model is intermediate between a zero-sum model and a model with independent and interchangeable species.

Despite these differences, we can follow the approach of Kelly (1976) to show that relative abundances remain driven by Ewens’s sampling formula. Let the SAD be a random vector denoted by Φ, in which Φ _k is the number of species with k individuals. The total number of individuals is equal to ${\sum }_{k=1}^{\infty } k {\Phi }_{k}$ and is given the label J. In a closely related neutral model, Kelly (1976) observed that Φ itself evolves as a Markov chain and found its stationary probability distribution, denoted by P _Φ . From this solution, the distribution P _J may be calculated by marginalizing out the relative abundances of each species, and these together define the conditional distribution satisfying P _Φ =P _Φ|J P _J . As shown in Appendix B, the probability distribution on SADs for communities of size J, our P _Φ|J , is exactly Ewens’s sampling formula. In addition, P _J is the stationary distribution for a birth-death process with birth rate ν+λ J and death rate (μ+δ JΩ⁻¹)J. The total number of individuals follows a stochastic logistic process with immigration, while the relative abundance of species has the same distribution found in prior NCMs.

Non-neutral parameterization

Changing the competition coefficients so α _i,j ≠1 for j≠i breaks the neutral assumption. Obtaining values of ND _n >0 and RFD _n >1 can be achieved with a minimal number of parameters by setting α _i,j =η for i<j, and setting α _i,j =β≥η for i>j. The competition matrix in a community of, for example, three species then becomes

$$ \left(\begin{array}{ccc} 1 & \eta & \eta \\ \beta & 1 & \eta \\ \beta & \beta & 1 \end{array}\right). $$

(6)

At every speciation or extinction event, the competition matrix increases or decreases, respectively, by one row and column. This highly structured matrix constrains the generality of the model but provides simple control parameters for our modeling experiment.

For a given richness, but otherwise regardless of the number of preceding speciation and extinction events, the competition matrix is invariant. Applying this matrix to Eq. 1, the value of ND _n and RFD _n becomes unique for a given n. The expected niche and fitness difference in the stochastic model can therefore be written as a combination of known ND _n and RFD _n values with an unknown distribution on species richness. In communities composed of n species with probability P _n , we can now write

$$ (\langle \text{ND} \rangle, \langle \text{RFD} \rangle) = \sum\limits_{n=0}^{\infty} (\text{ND}_{n}, \text{RFD}_{n}) P_{n}, $$

(7)

noting that an arbitrary specification of ND ₀=1 and RFD ₀=1 has little impact given the negligible probability of zero species. While changing η or β has an exactly calculable impact on ND _n and RFD _n , the values of 〈ND〉 and 〈RFD〉 cannot be determined without obtaining the steady state distribution on species richness.

The final element of the model is the assignment of competition coefficients to each new species. Given the above competition matrix, the model need only specify how many extant species any new species will outrank in competitive ability. When β>η, competition is asymmetric and favors species with smaller indices in the vector N of abundances. Each new species’ insertion point in N establishes its rank in the hierarchy of competitive dominance. We assume a uniform distribution on this insertion point, so a speciation event among n species requires sampling a rank from n+1 integers with equal probability.

Analytic solutions pertaining to the evolution of N could not be obtained outside the neutral case (β=η=1). Instead, we generated realizations at particular values of β and η (Table 1) using the Gillespie algorithm (Gillespie 1977). We characterize different communities in terms of the compound parameters ND$_{2} = 1 - \sqrt {\eta \beta }$ and RFD$_{2} = \sqrt {\beta /\eta }$ in order to clarify the parameter ranges chosen for study. We used values that take ND ₂ from its minimum value of 0 up to 0.2 and take RFD ₂ between its minimum of 1 and, for a given value of ND ₂, a maximum of (1−ND₂)⁻¹. This maximum RFD ₂ is the greatest value that, with the corresponding ND ₂, allows stable coexistence of two species in the LV model. Preliminary simulations showed that setting RFD ₂ above this threshold is of little interest because the resulting community usually consists of a single species. We set the remaining parameters to balance computational demands against the aim of simulating communities with large numbers of individuals and species. Holding ν constant while changing values of β and η was computationally manageable for simulations having around 10⁴ individuals and with ν just large enough to avoid monocultures under the neutral parameterization. At the maximum value of ND ₂ used, exploratory simulations showed that up to 30 species could result with λ=1, μ=0.2, δ=0.08, Ω=1000, and ν=1, and these parameter choices apply to all simulations unless otherwise noted. We initiated each realization with 10⁴ individuals of a single founder species and ran time forward through two full community replacements (i.e., until every species that originated before the founder became extinct had itself also gone extinct). For each parameterization, we archived output on (i) the final state of 100 independent realizations for SAD estimation and (ii) the time between speciation and extinction for 10⁴ species in each of three additional realizations, giving three samples of the species’ lifetime distribution. Expected SADs were estimated as the average number of species at each abundance in the final state of each realization.

Table 1 Parameter values, displayed as (η, β) and rounded to two decimals, that differed between simulations. The value of ND₂ increases towards the right, and the value of RFD₂ increase towards the top. Panels in Figs. 3 and 5 use the parameterization from the corresponding location in this table. The labels ∗, † and ‡ indicate quantities in Fig. 4

Approximation in the non-neutral case

Models of non-neutral competition often ignore demographic stochasticity, on the assumption that deterministic approximations capture the important dynamics. Mathematical justification for this assumption can proceed from a linear noise approximation (LNA) to the exact Markov chain model (van Kampen 2007). The standard result is that fluctuations fade out as system size increases; for example, the coefficient of variation on the population density of each of n species in the present model becomes proportional to Ω^−1/2 as Ω→∞ (Appendix A). By increasing Ω, the effect of demographic stochasticity on a community with pre-defined richness can always be erased in the non-neutral case.

In the full model, however, non-zero speciation and extinction probabilities have the potential to make population fluctuations a driver of community structure. The LNA only proceeds in the usual way when the number of species is held constant. With richness itself a stochastic process parameterized by Ω, we can no longer guarantee that fluctuations decline like Ω^−1/2 with increasing system size. Indeed, for the community to achieve a stochastic equilibrium, fluctuations must be large enough to cause extinctions at the same frequency as speciation events. Moreover, the LNA yields an incomplete deterministic approximation to our model because it does not give an expected richness.

We attempt to specify a complete, deterministic approximation in the non-neutral case along with a fast, numerical approximation to the effects of stochasticity. In brief, our approach is to couple the LNA for a system with given richness to a new approximation for the stationary distribution on species richness (Appendix C). We first apply an asymptotic method (Kessler and Shnerb 2007; Gottesman and Meerson 2012; Ovaskainen and Meerson 2010) to the problem of estimating the probability of successful establishment, i.e., the frequency of speciation events not immediately followed by extinction (Grimm and Wissel 2004). Next, we assume a separation of time scales between population dynamics and the speciation process, which should only be expected with ν much smaller than the average birth and death rates, and seek a stationary solution to the resulting Markov chain on species richness. We use the mode of this distribution to estimate the most likely species richness, which becomes the dimension of the system of ordinary differential equations given by the LNA and completes the deterministic approximation. Combining the Gaussian process also generated by the LNA with our distribution on species richness gives a numerical approximation for SADs that includes stochasticity. The covariance matrix of the Gaussian process is the solution of a Lyapunov equation, which presents the main impediment to providing an analytical approximation. See (Rossberg 2013, ch. 14) for an alternative approach to approximating SADs in a LV model using stochastic species packing.

Results

The main results from our model experiment are as follows:

• 1. The observed magnitudes of 〈ND〉 and 〈RFD〉 cover a gradient of departures from neutrality but maintain a fixed ratio between the two kinds of species’ differences.

• 2. Community dynamics respond dramatically when competition coefficients deviate slightly from the neutral parameterization.

• 3. Expected species abundance distributions (SADs) are sometimes, but not always, indistinguishable between neutral and non-neutral communities.

• 4. Deterministic approximations partly explain the SADs, but fail to describe the abundance of rare species or their impact on diversity.

• 5. Lifetime distributions in non-neutral communities consistently deviate from neutral ones.

We quantified niche and fitness differences in simulated communities by calculating 〈ND〉 and 〈RFD〉 (Fig. 1). In two species communities, the model parameters ND ₂ and RFD ₂ have a precise interpretation, but one we could not assume to be accurate more generally. Increasing the parameter RFD ₂ always increased the value of 〈RFD〉, demonstrating a straightforward effect of asymmetric competition coefficients on relative fitness differences. Increasing the parameter ND ₂ had either positive or negligible effects on the value of 〈ND〉, but the magnitude of niche differences was also affected by RFD ₂. This result demonstrates that parameters appearing to have a straightforward relationship to niche and fitness differences may have surreptitious effects on invasion fitness.

Fig. 1

The departure from neutrality, as niche and relative fitness differences, for different parameterizations of our stochastic community model. For each pair of points connected by a line, the filled circle indicates the parameters (by reference to the compound parameters ND ₂ and RFD ₂) while the open circle is the resulting 〈ND〉 and 〈RFD〉. The location of the filled circles matches the location of sub-plots in Figs. 3 and 5, connecting the SADs and lifetime distributions to corresponding 〈ND〉 and 〈RFD〉. The position of the open circles is calculated by averaging the value of ND _n and RFD _n over the frequency distribution of n, the final species richness in realizations having a given parameterization (7)

The parameter ranges employed did translate to a gradient from neutral to non-neutral interactions, but we consistently saw convergence to similar ratios between 〈ND〉 and 〈RFD〉 (Fig. 1). Both 〈ND〉 and 〈RFD〉 fall below ND ₂ and RFD ₂, respectively, with 〈ND〉 showing less sensitivity to ND ₂ than anticipated. We numerically confirmed that ND _n+1≤ND _n in our parameter range, which implies that 〈ND〉 will decrease in communities tending to have more species since it is a weighted average of ND _n . The resistance of 〈ND〉 to changes in ND ₂ arises because the reduction in the strength of competition with any one heterospecific individual is offset by the increased diversity and abundance of heterospecifics. It remains surprising that the two kinds of species’ differences tended towards one particular ratio. That is, across parameterizations with a particular value of RFD ₂, the distribution on species richness adjusted until the values of 〈ND〉 roughly converged. Overall, our simulations do include variability in the magnitude of both departures from neutrality but are also constrained to a narrow band within the 〈ND〉 and 〈RFD〉 plane.

Deviations from neutrality obtained by perturbing β and η slightly away from one had large effects on community dynamics. In each of three non-neutral parameterizations that could all be considered nearly neutral, time series plots from individual simulations do not show random walks, or “drift,” without a deterministic trend towards extinction or coexistence (Fig. 2). Drift, a characteristic feature of NCMs, is a meandering fluctuation over large ranges in abundance, limited only by the size of the entire community (Fig. 2a). Setting β equal to η but slightly less than one, which only moves 〈ND〉 slightly away from zero, drastically reduces the range of fluctuations (Fig. 2b). Rather than drifting to large abundances, species invade and persist with several competitors about a mean abundance (around 10³ individuals per species) until particularly large fluctuations cause an extinction (Fig. 2e). Holding η equal to 1/β but slightly increasing RFD ₂ brings on a parade of “selective sweeps” in which newly created species rapidly replace the currently dominant one (Fig. 2c). Finally, reducing β alone, which makes both 〈ND〉 and 〈RFD〉 slightly greater than zero and one, respectively, appears very similar to the previous case (compare Fig. 2c, d). None of these nearly neutral (Fig. 2b-d) communities exhibit time series that lack deterministic tendencies as in the exactly neutral communities (Fig. 2a).

Fig. 2

Time series from one realization of the model from each of four different parameterizations of β and η. Each color trajectory follows a single population, and only populations with more than 100 individuals are shown for clarity. a The neutral case with η=β=1. b Niche differences only, with η=β=0.95. c Relative fitness differences only, with η=0.95,β=1.05. d Both kinds of differences, with η=0.9,β=1. e A haphazard selection of trajectories from panel b on a more revealing scale. Panel a shows a longer time scale to capture the slow “random walk” that drives the neutral community dynamics. The remaining parameters are ν=1,λ=1,μ=0.2,δ=0.08,Ω=1000

Expected SADs are neither diagnostic of neutral versus non-neutral parameterizations nor are SADs robust to all departures from neutrality (Fig. 3). With RFD ₂ equal to the maximum value that allows deterministic coexistence of two species for a given ND ₂, the resulting SADs appeared indistinguishable from those produced under the neutral parameterization (plots on the diagonal of Fig. 3). The log-linear expectation for the cumulative SAD in the neutral case is particular to the value of ν/λ, which corresponds to the “fundamental biodiversity number” used when fitting the neutral SADs to empirical observations. In contrast, when ND ₂ was increased without adding a relative fitness difference, the non-neutral SAD grew distinct from the neutral SAD and indicated a more uniform distribution of individuals among a greater number of species (plots along the bottom row of Fig. 3). Parameterizations falling between these extremes demonstrated a smooth transition in the shape of the SAD (remaining off-diagonal plots in Fig. 3), demonstrating that non-neutral SADs are likely to be at least as flexible as neutral ones, as expected of a model with two additional parameters.

Fig. 3

Expected species abundance distributions (SADs) for the combinations of niche and relative fitness differences given in Table 1. The figure shows a cumulative distribution on abundance or the number of species with abundance less than or equal to the value given on the horizontal axis. The distribution stops at the highest abundance recorded for at least one species, and the total number of individuals in the community is printed in the corner of each panel. Abundance is shown on a logarithmic scale in the style of Preston plots, but with base ten for ease of interpretation. The result for the neutral case is repeated in each panel (dark gray line) for comparison. Details on the determinisitc approximation (light gray line) can be found in Appendix C. All parameters aside from β and η are as in Fig. 2

The behavior of non-neutral SADs may be partly explained by our deterministic approximation to equilibrium population densities (Appendix A). When η≤β<1, there is a unique interior equilibrium among n species with the density of species i proportional to

$$ (1-\beta)^{i} (1-\eta)^{n-i} $$

(8)

in the deterministic approximation. On a logarithmic abundance scale, density decreases linearly with i which corresponds to the species’ competitive rank. The value of n used approximates the mode of the true distribution on species richness. Plotted as a cumulative SAD, the result is n equally spaced steps covering a broader range of abundances for larger RFD ₂ (off diagonal plots in Fig. 3). Like the simulation results, this stepwise function varies from log-linear to sigmoid across the range of ND ₂ and RFD ₂ values employed. Quantitative disagreement with the simulation results are particularly pronounced for rare species, which are both mistakenly absent (bottom row in Fig. 3) or too abundant (diagonal in Fig. 3) in the deterministic approximation.

The SADs described so far come from simulations with unrealistically frequent speciation events; we need the numerical approximation to extend our scope to arbitrarily low speciation rates. Large values of ν were needed to maintain multiple species in the neutral case while reducing computational demands. For communities of the same size (i.e., same Ω), but with much smaller values of ν, the neutral community reduces to a monoculture and the effect of fluctuations among abundant species on non-neutral SADs becomes minimal. The numerical approximation to the expected SADs shows a smoothed version of the stepwise function (Fig. 4). In two of the three choices for η and β tested, the underlying distribution on richness spiked at a single value of n and the numerical approximation performed well. Demographic stochasticity adds noise to the precise location of each step, but the spacing between steps is as given by expression (8). In the third case, multiple richness levels had non-trivial probabilities that were not as accurately approximated, a problem likely to worsen as the average number of species increases. Multiple smoothed-step functions are averaged in this case and, in comparison to results from stochastic simulations, the number of rare species gets overestimated.

Fig. 4

The cumulative expected SAD from the numerical approximation (solid) compared to results from stochastic simulations (dashed). The top, middle, and bottom panels correspond to diminishing values of ν: 10⁻², 10⁻³, and 10⁻⁴. The three pairs of lines in each panel have β and η values marked in Table 1 by ∗ (thin), ‡ (thicker) and † (thickest)

Unlike SADs, the distribution of species’ lifetimes, i.e., the time between the origination and extinction of a species, under non-neutral parameterizations is never neutral-like. Figure 5 displays the upper tail of the cumulative distribution of species’ lifetimes. Only values above the 95th percentile are shown because short-lived species had similar lifetimes across all parameterizations. In neutral communities, this upper tail is a smooth continuation of the distribution on short-lived species. This contrasts with the non-neutral communities, in which the upper tail is no longer concave and shows that stronger departures from neutrality produce a second mode in the lifetime distribution.

Fig. 5

The upper tail of the cumulative distribution for species’ lifetimes within a model realization. For each realization, following a period of transient dynamics, 10⁴ sequential extinction events were recorded and the durations of all those above the 95th percentile displayed. Each panel shows the results from three realizations, between which there is little variation. The lifetimes are on a log scale on the horizontal axis and the vertical axis shows the percentile. The average of the three neutral realizations is printed in each panel for comparison (gray line). The parameters, aside from β and η which are given in Table 1, are as in Fig. 2

Species’ lifetimes in non-neutral communities appear to arise from a mixture distribution in which species that fail to invade form one mixture component and species that successfully establish form a second. In the neutral case, every species follows the same random walk, so its lifetime is drawn from a single distribution whether or not the species persists very long. Deviating from neutrality along the ND ₂ and RFD ₂ axes in Fig. 5 influences (i) the probability that a new species will establish and thereby achieve a longer lifetime and (ii) the variance of lifetimes among species that do become established. Increasing ND ₂ predominantly increases the probability that a new species becomes established, as evidenced by the larger fraction of populations in the upper tail of the lifetime distribution (Fig. 5; bottom row). Increasing RFD ₂ also raises the probability of a successful invasion, but it decreases the variance around the long-lived mixture component, dramatically shortening the tail of the distribution. For the maximum ND ₂ tested, shifting $\sqrt {\beta /\eta }$ from 1.0 to 1.25 drops the maximum lifetime by nearly two orders of magnitude, sharply truncating the distribution. The clearest separation between a short-lived mixture component and a long-lived one (i.e., a flat transition between steep climbs in Fig. 5) occurs when a large ND ₂ is coupled to an intermediate value of RFD ₂.

Discussion

In this study of a stochastic, Lotka-Volterra-like model of competition, we describe a framework for quantifying departures from neutrality that distinguishes between two competing theories on the remarkable similarities encountered in neutral and non-neutral communities. We found that neutral-like SADs arise in non-neutral communities even when the concept of emergent neutrality does not apply. A key feature of emergent neutrality is small departures from neutrality, including both 〈ND〉, which relates to the expected invasion fitness of species experiencing competition, and 〈RFD〉, which is governed by the variability of invasion fitness within the stochastic community model. In our simulations, it was not necessary to suppose that some evolutionary or ecological mechanism had mostly eliminated both niche and relative fitness differences in the community, making it nearly neutral, in order to obtain matching neutral and non-neutral SADs. Instead, we observe near perfect alignment to a neutral SAD across a gradient in the magnitude of departures from neutrality, as quantified by the values of 〈ND〉 and 〈RFD〉.

We found mixed results in support of an alternative to the concept of emergent neutrality, which is that a particular balance of stabilizing and equalizing mechanisms is necessary to make non-neutral communities appear neutral. Non-neutral communities did obtain a neutral-like SAD in multiple cases where 〈ND〉 and 〈RFD〉 reached different magnitudes while maintaining roughly equivalent ratios. However, the same ratio of niche to relative fitness differences was also achieved in simulations where the non-neutral SAD deviated strongly from the neutral one. In other words, communities that obtained roughly the same 〈ND〉 to 〈RFD〉 ratio only aligned with the neutral SAD in a subset of parameterizations. Either the hypothesis that destabilizing species’ differences will counteract niche difference requires refinement, or our method for quantifying the magnitudes of the two departures from neutrality is invalid. Only a one-to-one correspondence between the shape of the SAD and a combination of niche and relative fitness differences would convincingly demonstrate that departures from neutrality are controlled by just these two variables.

The most surprising result is that communities with different settings for ND ₂, roughly corresponding to the average interspecific competition coefficient, converge on the same 〈ND〉 for a given 〈RFD〉. In n species communities, the value of ND _n is a community-wide average invasion fitness (Metz 2008), the growth rate of one invading species against n−1 residents. For a given ND ₂, this invasion fitness decreases in more diverse communities, so the expected niche difference, 〈ND〉, decreases as more weight is given to ND _n values with greater n. To achieve stationarity, richness must adjust until the probability of a successful invasion balances the probability of an extinction. What we observed in our model is that communities with smaller competition coefficients require more species to achieve the ND _n that meets this threshold, and this threshold is itself determined by the relative fitness difference. Greater richness, and hence total community size, compensated for weaker competition coefficients to keep 〈ND〉 constant over a range of ND ₂ settings. Interestingly, when there is no variability between species in their invasion fitness, the threshold value for ND _n appears to become so close to zero that these communities might all be described as nearly neutral. Emergent neutrality would seem to be an appropriate label for such small departures from neutrality, but SADs for these communities are the least neutral-like of all.

When modeling non-neutral communities, rigorously accounting for demographic stochasticity offers few rewards in describing any one species’ dynamics, but it cannot be ignored as a determinant of species richness. A generations-old adage in population dynamics says that demographic stochasticity is negligible for sufficiently abundant species. For any given number of species, every one of them could reach this sufficient abundance in a suitably large system, i.e., for a big enough Ω. For a given Ω, however, so long as 0<η≤β<1, increasing n reduces each species’ average abundance. Similar to the classic and much debated species-packing argument of May and MacArthur 1972, stochasticity induces a limit to niche differences, although emerging here from intrinsic rather than environmental randomness (which is not included). The limit is clearly not a hard limit (sensu Nisbet 1978), as it is sensitive to RFD ₂ and our stylized competition matrix, among other structural constraints. Nonetheless, while we can assemble a deterministic approximation reflecting the most likely richness of the stochastic community, that model does not ignore stochasticity. Were stochasticity completely ignored, there would be no limit to the number of coexisting species in our non-neutral communities. Perhaps more importantly, no remotely similar ODE model would capture a feature of the stochastic simulations discussed next: species turnover.

Comparing distributions of species’ lifetimes, or the time between speciation and extinction, also demonstrates that disparate dynamics can give indistinguishable SADs. As 〈RFD〉 increases, the lifetime distributions become increasingly distinct from the neutral case (Fig. 5). The differences remain even for the parameterizations shown on the diagonal which produce neutral-like SADs. In comparison to the neutral case, replacement of dominant species occurs much more rapidly in the presence of relative fitness differences. The effect of variation in ND ₂ on lifetime distributions is less pronounced, which may be due to the convergence of 〈ND〉 values for a given fitness difference. Nonetheless, in contrast to the behavior of SADs, consistent differences between neutral and non-neutral parameterizations are apparent in the lifetimes of long-lived species. In particular, eliminating equalizing mechanisms drastically reduces the longest lifetimes seen in our model, and this effect is not compensated for by the presence of stabilizing mechanisms.

Species’ lifetimes have previously entered the debate over neutral theory. An early concern was that average extinction times predicted by NCMs are too short under the point mutation mode (Ricklefs 2003). In our model, and not only under the neutral parameterization, introducing a single, founding individual for each new species leads to many rapid extinctions. These failed introductions can drive the average species’ lifetime to just a few generations, but adjustments to the speciation mode, independent of the assumption of neutrality, can provide more reasonable expectations (Rosindell et al. 2010). Excessively long lifetimes are a more pernicious issue for NCMs (Ricklefs 2006; Nee 2005) and are on display here as the long tail of the neutral lifetime distribution (Fig. 5). Our results are consistent with the power-law behavior of the right-hand tail that (Pigolotti et al. 2005) identified in a similar NCM. Neutral theory predicts implausibly large extinction times because the decline of abundant species occurs too slowly under ecological drift alone (Ricklefs 2006, 2012).

Our simulations of a non-neutral model show that overly long lifetimes can be eliminated without deforming the SAD from a superficially neutral, and thus apparently plausible, shape. Ricklefs (2003) argues that truncating the lifespan of long-lived species “would require the appearance of competitively superior species that occasionally swept communities and caused the extinction of other species.” This feature is present in our non-neutral model: when a newly introduced species achieves the highest position in the competitive hierarchy, it will, with high probability, sweep to dominance and displace the most abundant species in the community. Under the various parameterizations of our model, we see that this effect grows stronger as the RFD increases (i.e., the tails in Fig. 5 are shortest in the top panel) in line with Ricklefs’ 2003 intuition. For every magnitude of 〈RFD〉, however, there always remains a setting for ND ₂ for which the SAD becomes neutral-like. In principle, simultaneously fitting the model to data on current species abundances and historical extinction times would eliminate the potential for ambiguity arising from fitting them to SADs alone. Although dynamic models have been fit to extinction time distributions (e.g., Suweis et al. 2012), we are unaware of any analysis that has included both kinds of data simultaneously. Jabot and Chave (2009) demonstrate a computational approach for fitting NCMs to SADs and phylogenetic relatedness simultaneously, and it is likely possible to transform such phylogenetic data into an age distribution among extant species, a small step away from the lifetime distributions of extinct species discussed here.

Neutral theory was partly motivated by an appeal to simplicity in model construction (Hubbell 2005), prompting several authors (Volkov et al. 2005; Ruokolainen et al. 2009; Haegeman and Loreau 2011; Noble et al. 2011; Zhou and Zhang 2008; Chisholm and Pacala 2010; Zillio and Condit 2007; Du et al. 2011) to respond in the same spirit by adding complexity gradually. While an infinite variety of ecological trade-offs and life history variation could provide this complexity, the framework of Chesson (2000) offers the tantalizing possibility that departures from neutrality can be conceptually organized into two categories. Alternatives to neutral theory’s null model may, if our model has any generality, in fact require no more than two kinds of species’ differences to maintain accuracy. Relative fitness differences are needed to incur replacement of numerically dominant species, and niche differences must be present to maintain species diversity. Unfortunately, computationally demanding simulations hinder progress in determining how departures from neutrality affect community structure and make data fitting exercises all but impossible. The alternative to “brute force” simulations is to find approximations that rely on asymptotic methods (Ovaskainen and Meerson 2010; Eriksson et al. 2013), for which our treatment of the present model brings into focus several topics for further study. The simplifying assumptions of neutral community models have bought remarkable theoretical advances (e.g., Etienne and Olff 2005; O’Dwyer and Green 2010), and a simplifying framework for describing departures from neutrality shows potential for similarly advancing theory for non-neutral communities.

Appendices

Appendix A: Linear noise approximation

Multivariate birth-death processes are sometimes very well described by a set of solvable equations that arise when the size, either area or volume, of the system under consideration is large (Gardiner 2009). These equations are obtained by a routine procedure called the “system size expansion,” which we briefly step through here using the birth and death rates from the main text (following van Kampen 2007, ch. X). The procedure gives both a deterministic approximation of the mean population densities, as a system of ordinary differential equations (ODEs), as well as a linear Fokker-Planck equation approximating the fluctuations due to demographic stochasticity. The procedure is straightforward when the deterministic system approaches a stable interior equilibrium, so the method is well suited for the non-neutral parameterizations of our model. Unfortunately, we could not adapt it to directly include transitions due to speciation, in which a new species is admitted according to a Poisson process with rate parameter ν. We take up changes in species richness in Appendix C and restrict our attention here to the multivariate birth-death process with a fixed number of species.

Note that in what follows, we distinguish time-dependent probability distributions, e.g., P _N|J , from their stationary solutions, $P_{\boldsymbol {N}|J}^{*}$, with an asterisk, but we do not employ this notation in the main text or Appendix B, where all distributions are stationary. Random vectors (scalars) are written in bold uppercase (normal uppercase), e.g., N (J), while corresponding arguments to a probability distribution are written in lowercase, e.g., P _N|J (n,j). Defining P _N|S (n,t;s) as the probability that the vector of population abundances is n at time t, conditioned on having started with s species of known abundance at time 0, the master equation (Gardiner 2009) for the multivariate birth-death process is

$$ \begin{array}{lll} \frac{\mathrm{d}}{\mathrm{d}t}P_{\boldsymbol{N}|S}(\boldsymbol{n},t;s)\! &=&\! \sum\limits_{i=1}^{s} \left\{(\mathbb{E}_{i}^{-} - 1) B_{i}(\boldsymbol{n}) P_{\boldsymbol{N}|S}(\boldsymbol{n},t;s)\right.\\ &&\quad\!+\left.\left(\mathbb{E}_{i}^{+}\! -\! 1\right)\! D_{i}(\boldsymbol{n}) P_{\boldsymbol{N}|S}(\boldsymbol{n},t;s) \right\}, \end{array} $$

(9)

where B _i and D _i are defined in Eqs. 4 and 5 and the shift operator $\mathbb {E}_{i}^{\pm }$ takes n _i to n _i ±1. To carry out the system size expansion, assume there is a smooth probability density P _Z|S (z,t;s) that will be equated to P _N|S (n,t;s)Ω^1/2 upon applying the change of variables

$$ \boldsymbol{z} = \frac{\boldsymbol{n} - {\Omega} \boldsymbol{x}(t)}{{\Omega}^{1/2}}. $$

(10)

Note that this substitution is time-dependent, but we put off defining the function x(t). Now, assume that the smooth density is proportional to the distribution P _N|S at every n:

$$ P_{\boldsymbol{N}|S}(\boldsymbol{n},t;s){\Omega}^{1/2} = P_{\boldsymbol{Z}|S}\left(\frac{\boldsymbol{n}-{\Omega}\boldsymbol{x}(t)}{{\Omega}^{1/2}},t;s\right). $$

(11)

Differentiating both sides with respect to time brings in the right-hand side of the master equation, in which n is again replaced using Eq. 10. Because P _Z|S is smooth, we can substitute a Taylor series for the shift operator. Altogether, dropping the arguments to P _Z|S for clarity, the substitutions yield

$$ \begin{array}{ll} \sum\limits_{i=1}^{s} \dot{x}_{i}(t) \frac{\partial}{\partial z_{i}} P_{\boldsymbol{Z}|S} - \frac{\partial}{\partial t}P_{\boldsymbol{Z}|S} {\Omega}^{-1/2} = \\ \sum\limits_{i=1}^{s} \left\{ \left(\tilde{B}_{i}(\boldsymbol{x}(t)) - \tilde{D}_{i}(\boldsymbol{x}(t))\right) \frac{\partial}{\partial z_{i}} P_{\boldsymbol{Z}|S} \right.\\ +\left. \nabla_{\boldsymbol{x}} \left(\tilde{B}_{i}(\boldsymbol{x}(t)) - \tilde{D}_{i}(\boldsymbol{x}(t)) \right) \cdot \frac{\partial}{\partial z_{i}}\boldsymbol{z} P_{\boldsymbol{Z}|S} {\Omega}^{-1/2} \right.\\ \,-\,\left.\! \frac{1}{2}\! \left(\!\tilde{B}_{i}(\boldsymbol{x}(t))\! +\! \tilde{D}_{i}(\boldsymbol{x}(t))\! \right)\! \frac{\partial^{2}}{\partial{z_{i}^{2}}} P_{\boldsymbol{Z}|S} {\Omega}^{-1/2} \right\}\! +\! O\left({\Omega}^{-1}\right)\!, \end{array} $$

(12)

where the tilde over the birth and death functions indicates the substitution $\tilde {F}_{i}(\boldsymbol {x}) = {\Omega }^{-1} F_{i}({\Omega } \boldsymbol {x})$, and the operator ∇ _x is the gradient of the birth and death rates with respect to the vector x evaluated at x(t). Note the expansion has been truncated to leave off terms smaller than O(Ω^−1/2), which become trivial as Ω→∞, i.e., as the system’s size expands. The time-dependent vector x(t) is now defined with the express purpose of eliminating the O(1) terms:

$$ \dot{x}_{i}(t) = \tilde{B}_{i}(\boldsymbol{x}(t)) - \tilde{D}_{i}(\boldsymbol{x}(t)) \quad \text{for } i = 1 \hdots s. $$

(13)

What emerges, however, is the analogous deterministic model for the density of each population in the community. Using the birth and death rates from the main text gives

$$ \begin{array}{l} \dot{x}_{i}(t) = (\lambda - \mu) \left(1 - \frac{\delta}{\lambda - \mu} \sum\limits_{j=1}^{s} \alpha_{i,j} x_{j}(t) \right) x_{i}(t) \\ \text{for } i = 1 \hdots s, \end{array} $$

(14)

which are just the Lotka-Volterra competition equations. A dimensionless variable called the relative yield of the population, denoted y _i , is obtained by dividing every x _i by its own equilibrium in the absence of competitors. With this change of variables,

$$ \dot{y}_{i}(t) \! =\! r \left(1 - \sum\limits_{j=1}^{s} \frac{\alpha_{i,j}}{\alpha_{j,j}} y_{j}(t) \right) y_{i}(t) \quad \text{for }i = 1 \hdots s, $$

(15)

where we have also equated λ−μ to r to match the notation of Chesson (2013). The $y_{j,i}^{*}$ of Eq. 1 is the equilibrium solution to Eq. 15 with y _i =0 and the other variables non-zero, which is unique when α _i,j <α _j,j for all i≠j. Except that we have not yet chosen n, this ODE system is called the “deterministic approximation” in the main text.

Matching the O(Ω^−1/2) terms in Eq. 12 leaves a linear Fokker-Planck equation:

$$ \begin{array}{lll} \frac{\partial}{\partial t} P_{\boldsymbol{Z}|S}\! &=&\! -\sum\limits_{i=1}^{s} \left\{ \nabla_{\boldsymbol{x}} \left(\tilde{B}_{i}(\boldsymbol{x}(t)) - \tilde{D}_{i}(\boldsymbol{x}(t)) \right) \cdot \frac{\partial}{\partial z_{i}}\boldsymbol{z} P_{\boldsymbol{Z}|S} \right. \\ &&\qquad\left.-\frac{1}{2} \left(\tilde{B}_{i}(\boldsymbol{x}(t)) + \tilde{D}_{i}(\boldsymbol{x}(t)) \right) \frac{\partial^{2}}{\partial {z_{i}^{2}}} P_{\boldsymbol{Z}|S} \right\}. \end{array} $$

(16)

The equation is uniquely satisfied by a zero-mean multidimensional Ornstein-Uhlenbeck process. For our purpose, it is sufficient to follow instructions for calculating the covariance matrix of its stationary, Gaussian solution (Section VIII.6 van Kampen 2007). That is, we seek the stationary solution to Eq. 16 where each x _i (t) is replaced with an equilibrium solution to Eq. 13, denoted $x_{i}^{*}$. The covariance matrix, Σ, of this stationary solution, $P_{\boldsymbol {Z}|S}^{*}$, is calculated by numerically solving the Lyapunov equation

$$ \quad\quad 0 = \boldsymbol{A}\boldsymbol{\varSigma}+\boldsymbol{\varSigma}\boldsymbol{A}^{T}+\boldsymbol{C}, $$

(17)

in which C is diagonal with entries $C_{i,i} = 2 \lambda x_{i}^{*}$ and $A_{i,j} = -\delta x_{i}^{*} \alpha _{i,j}$. Back substituting for N, the distribution sought is approximated by

$$ \quad\quad P^{*}_{\boldsymbol{N}|S}(\boldsymbol{n};s) \quad \approx \mathcal{N}(\boldsymbol{n}; {\varOmega} \boldsymbol{x}^{*}, {\varOmega} \boldsymbol{\varSigma}), $$

(18)

where $\mathcal {N}(\cdot )$ is the multinormal probability density function. In Appendix C, we use this result to complete a numerical approximation to the effect of demographic stochasticity on SADs. Note that x ^∗ and Σ do not depend on Ω, so the coefficient of variation for N _i is asymptotically proportional to Ω^−1/2.

Appendix B: P _Φ, the probability on species abundance distributions in the neutral case

Analytical solutions for the stationary probability of a species abundance distribution (SAD) have previously been reported for neutral models similar, but not identical, to our own. Unlike Hubbell (2001) but like Volkov et al. (2003), the model we present does not assume “zero-sum” dynamics. But unlike Volkov et al. (2003), the death rate in our model increases with total community size. This form of density dependence makes the abundance of every species inter-dependent, and describing the SAD requires a different approach. Kelly (1976) points to a general class of NCMs where the time trajectory of the SAD is itself a Markov chain. We replay Kelly’s observations here, only to recover the same result after introducing our fixed, rather than community size dependent, speciation rate.

The transition probabilities of a Markov chain must be the functions of the current state, in our case random vector Φ. In the neutral case (α _i,j =1), the probability of a death in any one population with k individuals is

$$ \quad\quad \sum\limits_{i \in \mathbb{Z}} \delta_{N_{i},k} D_{i}(\boldsymbol{N}) \mathrm{d}t = k {\Phi}_{k} \left(\mu + \delta \frac{J}{\Omega} \right) \mathrm{d}t, $$

(19)

using the Kronecker symbol, $\delta _{N_{i},k}$, to sum over the Φ _k species that have k individuals. A death in any population with k individuals shifts Φ to $\mathbb {E}_{k}^{-}\mathbb {E}_{k-1}^{+}\boldsymbol {\Phi }$, where the shift operator $\mathbb {E}_{k}^{\pm }$ takes Φ _k to Φ _k ±1. Similarly, the transition from Φ to $\mathbb {E}_{k}^{-}\mathbb {E}_{k+1}^{+}\boldsymbol {\Phi }$ that occurs when any population with k individuals produces a new individual has probability

$$ \quad\quad\sum\limits_{i \in \mathbb{Z}} \delta_{N_{i},k} B_{i}(\boldsymbol{N}) \mathrm{d}t = k {\Phi}_{k} \lambda \mathrm{d}t \quad \text{for } k > 0. $$

(20)

A speciation event occurs with probability νdt and shifts Φ to $\mathbb {E}_{1}^{+}\boldsymbol {\Phi }$.

Denoting the stationary probability on Φ by P _Φ , detailed balance (Gardiner 2009) is satisfied if

$$ \nu P_{\boldsymbol{\Phi}}(\boldsymbol{\phi})= (\phi_{1}+1)\left(\mu + \delta\frac{j+1}{\Omega}\right) P_{\boldsymbol{\Phi}}(\mathbb{E}_{1}^{+}\boldsymbol{\phi})\quad \text{and} $$

(21)

$$ \begin{array}{ll} k \phi_{k} \lambda P_{\boldsymbol{\Phi}}(\boldsymbol{\phi}) = (k+1)(\phi_{k+1}+1)\left(\mu + \delta\frac{j+1}{\Omega}\right) P{\boldsymbol{\Phi}}(\mathbb{E}_{k}^{-}\mathbb{E}_{k+1}^{+}\boldsymbol{\phi})\\ \text{for } k > 0, \end{array} $$

(22)

where $j = {\sum }_{k} k \phi _{k}$. Direct substitution confirms that

$$ P_{\boldsymbol{\Phi}}(\boldsymbol{\phi}) = b \frac{({\Omega}\lambda/\delta)^{j}}{\Gamma(j+{\Omega}\mu/\delta+1)} \prod\limits_{k=1}^{\infty} \frac{(\nu/\lambda )^{\phi_{k}}}{k^{\phi_{k}}\phi_{k}!}, $$

(23)

is the desired solution, where b is a constant with respect to ϕ.

To interpret the solution for P _Φ , we distinguish two factors on the right-hand side,

$$ P_{\boldsymbol{\Phi}}(\boldsymbol{\phi}) = \underbrace{ b \frac{({\Omega}\lambda/\delta)^{j}}{\Gamma(j+{\Omega}\mu/\delta+1)} \frac{(\nu / \lambda)_{j}}{j!} }_{P_{J}(j)} \underbrace{ \frac{j!}{(\nu / \lambda)_{j}} \prod\limits_{k=1}^{\infty} \frac{(\nu/\lambda )^{\phi_{k}}}{k^{\phi_{k}}\phi_{k}!} }_{P_{\boldsymbol{\Phi}|J}(\phi)}, $$

(24)

where (ν/λ) _j uses the Pochhammer notation for the rising factorial. The conditional distribution, P _Φ|J , is equivalent to Ewens’s sampling formula (Etienne and Alonso 2005, Eq. 2) with 𝜃, the “fundamental biodiversity number,” substituted for ν/λ. The community size distribution, P _J , satisfies a different detailed balance condition,

$$ \quad(\nu + \lambda j)P_{J}(j) = (j + 1) \left( \mu + \delta \frac{j+1}{\Omega} \right) P_{J}(j + 1), $$

(25)

which arises from a logistic-like birth-death process with immigration. Fluctuations in total community size do not affect the species’ relative abundance, which still follows from Ewens’s sampling formula.

Appendix C: Approximate solution for the expected non-neutral SAD

Define $P_{N_{i}|S}$ to be the marginal distribution for the ith species obtained from the distribution P _N|S discussed in Appendix 1. The expected value of the species abundance distribution is (cf. Volkov et al. 2003 Eq. 4):

$$ \quad\quad\langle {\Phi}_{k}(t)|s \rangle = \sum\limits_{i=1}^{s} P_{N_{i}|S}(k,t;s). $$

(26)

We suppose that extinction and speciation events are infrequent, which requires us to have set ν very small and permits us to use the linear noise approximation

$$ \quad\quad\langle {\Phi}_{k}(t)|s \rangle \approx \sum\limits_{i=1}^{s} \mathcal{N}(k; {\Omega} x_{i}^{*}, {\Omega} {\Sigma}_{i,i}), $$

(27)

so long as t, chosen so that the last speciation event occurred at t=0, is sufficiently large. In our model, because of the highly structured competition matrix (6), the values of $x_{i}^{*}$ and Σ _i,i are the same for every community with s species, so the identical SAD is expected at any time there are s species (except for immediately after a speciation or extinction event, but these are rare). This is one component of our estimate of the stationary solution for 〈Φ _k (t)〉, which must be completed by averaging 〈Φ _k (t)|s〉 over the distribution on richness, denoted by P _S .

Continuing to ignore the transient dynamics surrounding changes in s, we approximate the distribution on S by solving the detailed balance equation

$$ \quad W(s,s-1) P_{S}(s-1) = W(s-1,s) P_{S}(s), $$

(28)

where W(s,s ^′)dt is the probability of transitioning from s ^′→s species during a short time interval. The formal solution is

$$ \quad P_{S}(s) = P_{S}(0) \prod\limits_{i=1}^{s} \frac{W(i,i-1)}{W(i-1,i)}, $$

(29)

in which the value of P _S (0) can be determined by the required normalization. The extinction rate is just the probability that the last member of a rare species dies, for which we have the approximation

$$ W(s-1,s) \approx \sum\limits_{i=1}^{s} \sum\limits_{\boldsymbol{n}^{(\sim i)}} D_{i}(\boldsymbol{n}^{(\sim i)}) \mathcal{N}(\boldsymbol{n}^{(\sim i)}; {\Omega} \boldsymbol{x^{*}}, {\Omega} \boldsymbol{\Sigma}). $$

(30)

The second summation is over all possible vectors with n _i equal to 1, denoted n ^(∼i), which are all the states from which there is a non-zero probability that the next event will be an extinction. The transition rate W(s+1,s), given that we want to ignore transient dynamics, has to correspond not just to speciation events but to speciation events that lead to the establishment of a new species. To approximate the probability of a successful introduction, we look to the left eigenvector of the generator of the multivariate birth-death process associated with the smallest non-zero eigenvalue (Grimm and Wissel 2004). The left eigenvector problem for the multivariate birth-death process, denoting the eigenvector by π _s (n) and the corresponding eigenvalue by 𝜖, is

$$ \epsilon \pi_{s}(\boldsymbol{n})\! =\!\! \sum\limits_{i=1}^{s} \!\left\{ B_{i}(\boldsymbol{n}) (\mathbb{E}_{i}^{+} \,-\, 1) \,+\, D_{i}(\boldsymbol{n}) (\mathbb{E}_{i}^{-} \,-\, 1) \right\} \pi_{s}(\boldsymbol{n}). $$

(31)

Applications of WKB theory to birth-death processes (Ovaskainen and Meerson 2010) have been used to give excellent approximations to right eigenvectors with vanishingly small eigenvalues (Assaf and Meerson 2010). We employ the following ansatz to make an approximation to the left eigenvector:

$$ \pi_{s}(\boldsymbol{n}) = 1 - e^{- {\Omega} U_{s}(\boldsymbol{n}/{\Omega})}. $$

(32)

We only need to evaluate the eigenvector at states immediately following the introduction of a species. If species i is the new species, and x ^(∼i) is a vector with the density of i at zero and the other s−1 species at equilibrium, we want the value of

$$ \begin{array}{ll} \mathbb{E}_{i} \pi_{s}(\boldsymbol{n}) &= \mathbb{E}_{i}^{{\Omega}^{-1}} \pi_{s}\left({\Omega} \boldsymbol{x}_{s}^{(\sim i)}\right) \\ &\approx 1 - e^{-{\Omega} U_{s}(\boldsymbol{x}_{s}^{(\sim i)}) - \frac{\partial}{\partial x_{i}} U_{s}\left(\boldsymbol{x}_{s}^{(\sim i)}\right) + O({\Omega}^{-1})}, \end{array} $$

(33)

making the substitution n/Ω→x. With this target in mind, we proceed from Eq. 32 in the usual way, substituting 𝜖→0 and arriving at the leading order equation

$$ 0 = \sum\limits_{i=1}^{s} \left\{ \tilde{B}_{i}(\boldsymbol{x}) \left( e^{-\frac{\partial}{\partial x_{i}} U_{s}(\boldsymbol{x})} - 1 \right) + \tilde{D}_{i}(\boldsymbol{x}) \left( e^{\frac{\partial}{\partial x_{i}} U_{s}(\boldsymbol{x})} - 1 \right) \right\}, $$

(34)

where the tilde over the birth and death functions indicates the substitution $\tilde {F}_{i}(\boldsymbol {x}) = {\Omega }^{-1} F_{i}({\Omega } \boldsymbol {x})$ as in Appendix 1. Now let the RHS define a surface, typically denoted H(q,p), in the coordinate system with q=x and p=−∇U _s (x). A parameterized curve will follow the zero contour on this surface if it satisfies the equations of motion,

$$ \begin{array}{ll} \dot{q}_{j} &= \frac{\partial H}{\partial p_{j}} = \tilde{B}_{i}(\boldsymbol{q}) e^{p_{j}} - \tilde{D}_{i}(\boldsymbol{q}) e^{-p_{j}} \\ \dot{p}_j &= - \frac{\partial H}{\partial q_{j}} = \sum\limits_{k=1}^{s} \frac{\partial}{\partial q_{j}} \left( \tilde{B}_{k}(\boldsymbol{q}) \left( e^{p_{k}} - 1 \right) - \tilde{D}_{k}(\boldsymbol{q}) \left( e^{-p_{k}} - 1 \right) \right) \end{array} $$

(35)

for j∈{1,2,...,s} and is initialized with H=0. The system has an equilibrium solution at the value of interest q=x ^(∼i) if p satisfies p _j =0 for j≠i and

$$ p_{i} = \ln \left(\frac{\partial}{\partial q_{i}} \tilde{D}_{i}(\boldsymbol{x}^{(\sim i)}) \right) - \ln \left( \frac{\partial}{\partial q_{i}} \tilde{B}_{i}(\boldsymbol{x}^{(\sim i)}) \right). $$

(36)

Fixing the boundary values of U _s (x) at zero in Eq. 33, we apply this equilibrium solution to achieve

$$ \mathbb{E}_{i} \pi_{s}(\boldsymbol{n}) \approx 1 - \frac{\mu + \delta \beta {\sum}_{j=1}^{i-1} x_{j}^{(\sim i)} + \delta \eta {\sum}_{j=i+1}^{s} x_{j}^{(\sim i)}}{\lambda}, $$

(37)

recognizing that x ^(∼i) does depend on s despite it having fallen out from our notation. The estimate for W(s,s−1) is the probability of a speciation event times the average over i of the value just calculated as an estimate for the probability of successful invasion:

$$ W(s,s-1) \approx \nu \frac{1}{s} \sum\limits_{i=1}^{s} \left(1 - \frac{\mu + \delta \beta {\sum}_{j=1}^{i-1} x_{j}^{*} + \delta \eta {\sum}_{j=i+1}^{s} x_{j}^{*}}{\lambda} \right). $$

(38)

Comparisons of the approximate non-neutral SAD to results from stochastic simulations with ν equal to 10⁻², 10⁻³, and 10⁻⁴ are shown in Fig. 4 and again as Preston plots in Fig. 6.

Fig. 6

The same data from Fig. 4 displayed as Preston plots with the approximation in black and simulation results in gray. The top, middle, and bottom panels correspond to diminishing values of ν: 10⁻², 10⁻³, and 10⁻⁴. The values of β and η are marked in Table 1 by ∗ (left), ‡ (middle) and † (right)