Handbook of Dynamic Game Theory pp 1039-1077 | Cite as

# Biology and Evolutionary Games

## Abstract

This chapter surveys some evolutionary games used in biological sciences. These include the Hawk–Dove game, the Prisoner’s Dilemma, Rock–Paper–Scissors, the war of attrition, the Habitat Selection game, predator–prey games, and signaling games.

## Keywords

Battle of the Sexes Foraging games Habitat Selection game Hawk–Dove game Prisoner’s Dilemma Rock–Paper–Scissors Signaling games War of attrition## 1 Introduction

Evolutionarily game theory (EGT) as conceived by Maynard Smith and Price (1973) was motivated by evolution. Several authors (e.g., Lorenz 1963; Wynne-Edwards 1962) at that time argued that animal behavior patterns were “for the good of the species” and that natural selection acts at the group level. This point of view was at odds with the Darwinian viewpoint where natural selection operates on the individual level. In particular, adaptive mechanisms that maximize a group benefit do not necessarily maximize individual benefit. This led Maynard Smith and Price (1973) to develop a mathematical model of animal behavior, called the Hawk–Dove game, that clearly shows the difference between group selection and individual selection. We thus start this chapter with the Hawk–Dove game.

Today, evolutionary game theory is one of the milestones of evolutionary ecology as it put the concept of Darwinian evolution on solid mathematical grounds. Evolutionary game theory has spread quickly in behavioral and evolutionary biology with many influential models that change the way that scientists look at evolution today. As evolutionary game theory is noncooperative, where each individual maximizes its own fitness, it seemed that it cannot explain cooperative or altruistic behavior that was easy to explain on the grounds of the group selection argument. Perhaps the most influential model in this respect is the Prisoner’s Dilemma (Poundstone 1992), where the evolutionarily stable strategy leads to a collective payoff that is lower than the maximal payoff the two individuals can achieve if they were cooperating. Several models within evolutionary game theory have been developed that show how mutual cooperation can evolve. We discuss some of these models in Sect. 3. A popular game played by human players across the world, which can also be used to model some biological populations, is the Rock–Paper–Scissors game (RPS; Sect. 4). All of these games are single-species matrix games, so that their payoffs are linear, with a finite number of strategies. An example of a game that cannot be described by a matrix and that has a continuum of strategies is the war of attrition in Sect. 5.1 (or alternatively the Sir Philip Sidney game mentioned in Sect. 10). A game with nonlinear payoffs which examines an important biological phenomenon is the sex-ratio game in Sect. 5.2.

Although evolutionary game theory started with consideration of a single species, it was soon extended to two interacting species. This extension was not straightforward, because the crucial mechanism of a (single-species) EGT, that is, negative frequency dependence that stabilizes phenotype frequencies at an equilibrium, is missing if individuals of one species interact with individuals of another species. These games are asymmetric, because the two contestants are in different roles (such asymmetric games also occur within a single species). Such games that can be described by two matrices are called bimatrix games. Representative examples include the Battle of the Sexes (Sect. 6.1) and the Owner–Intruder game (Sect. 6.2). Animal spatial distribution that is evolutionarily stable is called the Ideal Free Distribution (Sect. 7). We discuss first the IFD for a single species and then for two species. The resulting model is described by four matrices, so it is no longer a bimatrix game. The IFD, as an outcome of animal dispersal, is related to the question of under which conditions animal dispersal can evolve (Sect. 8). Section 9 focuses on foraging games. We discuss two models that use EGT. The first model, that uses decision trees, is used to derive the diet selection model of optimal foraging. This model asks what the optimal diet of a generalist predator is in an environment that has two (or more) prey species. We show that this problem can be solved using the so-called agent normal form of an extensive game. We then consider a game between prey individuals that try to avoid their predators and predators that aim to capture prey individuals. The last game we consider in some detail is a signaling game of mate quality, which was developed to help explain the presence of costly ornaments, such as the peacock’s tail.

We conclude with a brief section discussing a few other areas where evolutionary game theory has been applied. However, a large variety of models that use EGT have been developed in the literature, and it is virtually impossible to survey all of them.

## 2 The Hawk–Dove Game: Selection at the Individual Level vs. Selection at the Population Level

One of the first evolutionary games was introduced to understand evolution of aggressiveness among animals (Maynard Smith and Price 1973). Although many species have strong weapons (e.g., teeth or horns), it is a puzzling observation that in many cases antagonistic encounters do not result in a fight. In fact, such encounters often result in a complicated series of behaviors, but without causing serious injuries. For example, in contests between two male red deer, the contestants first approach each other, and provided one does not withdraw, the contest escalates to a roaring contest and then to the so-called parallel walk. Only if this does not lead to the withdrawal of one deer does a fight follow. It was observed (Maynard Smith 1982) that out of 50 encounters, only 14 resulted in a fight. The obvious question is why animals do not always end up in a fight? As it is good for an individual to get the resource (in the case of the deer, the resources are females for mating), Darwinian selection seems to suggest that individuals should fight whenever possible. One possible answer why this is not the case is that such a behavior is for the good of the species, because any species following this aggressive strategy would die out quickly. If so, then we should accept that the unit of selection is not an individual and abandon H. Spencer’s “survival of the fittest” (Spencer 1864).

*V*denotes the value of the contested resource and

*C*is the cost of the fight measured as the damage one individual can cause to his opponent. The payoffs for the row player describe the increase/decrease in the player’s fitness after an encounter with an opponent. The game matrix is and the model predicts that when the cost of the fight is lower than the reward obtained from getting the resource,

*C*<

*V*, all individuals should play the Hawk strategy that is the strict Nash equilibrium (NE) (thus an evolutionarily stable strategy (ESS) ) of the game. When the cost of a fight is larger than the reward obtained from getting the resource,

*C*>

*V*, then

*p*=

*V*∕

*C*(0 <

*p*< 1) is the corresponding monomorphic ESS. In other words, each individual will play Hawk when encountering an opponent with probability

*p*and Dove with probability 1 −

*p*. Thus, the model predicts that aggressiveness in the population decreases with the cost of fighting. In other words, the species that possess strong weapons (e.g., antlers in deer) should solve conflicts with very little fighting.

*p*< 1, the individual fitness and the average fitness in the population are the same and equal to

*p*= 0, i.e., when the level of aggressiveness in the population is zero, all individuals play the Dove strategy, and individual fitness equals

*V*∕2. Thus, if selection operated on a population or a species level, all individuals should be phenotypically Doves who never fight. However, the strategy

*p*= 0 cannot be an equilibrium from an evolutionary point of view, because in a Dove-only population, Hawks will always have a higher fitness (

*V*) than Doves (

*V*∕2) and will invade. In other words, the Dove strategy is not resistant to invasion by Hawkish individuals. Thus, securing all individuals to play the strategy

*D*, which is beneficial from the population point of view, requires some higher organizational level that promotes cooperation between animals (Dugatkin and Reeve 1998, see also Sect. 3).

*p*

^{∗}=

*V*∕

*C*, individual fitness

*V*∕2. However, the ESS cannot be invaded by any other single mutant strategy.

Darwinism assumes that selection operates at the level of an individual, which is then consistent with noncooperative game theory. However, this is not the only possibility. Some biologists (e.g., Gilpin 1975) postulated that selection operates on a larger unit, a group (e.g., a population, a species etc.), maximizing the benefit of this unit. This approach was termed group selection. Alternatively, Dawkins (1976) suggested that selection operates on a gene level. The Hawk–Dove game allows us to show clearly the difference between the group and Darwinian selections.

Group selection vs. individual selection also nicely illustrates the so-called tragedy of the commons (Hardin 1968) (based on an example given by the English economist William Forster Lloyd) that predicts deterioration of the environment, measured by fitness, in an unconstrained situation where each individual maximizes its profit. For example, when a common resource (e.g., fish) is over-harvested, the whole fishery collapses. To maintain a sustainable yield, regulation is needed that prevents over-exploitation (i.e., which does not allow Hawks that would over-exploit the resource to enter). Effectively, such a regulatory body keeps *p* at zero (or close to it), to maximize the benefits for all fishermen. Without such a regulatory body, Hawks would invade and necessarily decrease the profit for all. In fact, as the cost *C* increases (due to scarcity of resources), fitness at the ESS decreases, and when *C* equals Open image in new window , fitness is zero.

### 2.1 Replicator Dynamics for the Hawk–Dove Game

In the previous section, we have assumed that all individuals play the same strategy, either pure or mixed. If the strategy is mixed, each individual randomly chooses one of its elementary strategies on any given encounter according to some given probability. In this monomorphic interpretation of the game, the population mean strategy coincides with the individual strategy. Now we will consider a distinct situation where *n* phenotypes exist in the population. In this polymorphic setting we say that a population is in state \(\mathbf {p}\in \varDelta _n\) (where \(\varDelta _{n}=\{\mathbf {p}\in \mathbb {R}^{n}\;|\;p_{i}\geq 0, p_{1}+\dots +p_{n}=1\}\) is a probability simplex) if *p*_{i} is the proportion of the population using strategy *i*. As opposed to the monomorphic case, in this polymorphic interpretation, the individual strategies and the mean population strategy are different, because the mean strategy characterizes the population, not a single individual.

*x*be the frequency of Hawks in the population. The fitness of a Hawk is

*C*>

*V*, we remark that the interior distribution equilibrium of this equation,

*x*=

*V*∕

*C*, corresponds to the mixed ESS for the underlying game. In this example phenotypes correspond to elementary strategies of the game. It may be that phenotypes also correspond to mixed strategies.

## 3 The Prisoner’s Dilemma and the Evolution of Cooperation

*T*>

*R*>

*P*>

*S*. Thus while Cooperate is Pareto efficient (in the sense that it is impossible to make any of the two players better off without making the other player worse off), Defect row dominates Cooperate and so is the unique ESS, even though mutual cooperation would yield the greater payoff. Real human (and animal) populations, however, involve a lot of cooperation; how is that enforced?

- 1.
Kin selection, that occurs when the donor and recipient of some apparently altruistic act are genetic relatives.

- 2.
Direct reciprocity, requiring repeated encounters between two individuals.

- 3.
Indirect reciprocity, based upon reputation. An altruistic individual gains a good reputation, which means in turn that others are more willing to help that individual.

- 4.
Punishment, as a way to enforce cooperation.

- 5.
Network reciprocity, where there is not random mixing in the population and cooperators are more likely to interact with other cooperators.

- 6.
Multi-level selection, alternatively called group selection, where evolution occurs on more than one level.

### 3.1 Direct Reciprocity

Direct reciprocity requires repeated interaction and can be modeled by the Iterated Prisoner’s Dilemma (IPD). The IPD involves playing the Prisoner’s Dilemma over a (usually large) number of rounds and thus being able to condition choices in later rounds on what the other player played before. This game was popularized by Axelrod (1981, 1984) who held two tournaments where individuals could submit computer programs to play the IPD. The winner of both tournaments was the simplest program submitted, called Tit for Tat (TFT), which simply cooperates on the first move and then copies its opponent’s previous move.

TFT here has three important properties: it is *nice* so it never defects first; it is *retaliatory* so it meets defection with a defection next move; it is *forgiving* so even after previous defections, it meets cooperation with cooperation next move. TFT effectively has a memory of one place, and it was shown in Axelrod and Hamilton (1981) that TFT can resist invasion by any strategy that is not nice if it can resist both Always Defect ALLD and Alternative ALT, which defects (cooperates) on odd (even) moves. However, this does not mean that TFT is an ESS, because nice strategies can invade by drift as they receive identical payoffs to TFT in a TFT population (Bendorf and Swistak 1995). We note that TFT is not the only strategy that can promote cooperation in the IPD; others include Tit for Two Tats (TF2T which defects only after two successive defections of its opponent), Grim (which defects on all moves after its opponent’s first defection), and win stay/lose shift (which changes its choice if and only if its opponent defected on the previous move).

*w*(as in the second computer tournament); the payoffs are then

*R*>

*S*+

*T*is required, since otherwise the right-hand side of inequality (23.5) would be at least 1.

### 3.2 Kin Selection and Hamilton’s Rule

In most evolutionary game theoretical models, the aim of each individual is to maximize its own fitness, irrespective of the fitness of other individuals. However, if individuals are related, then the fitnesses of others should be taken into account.

*r*, which is the probability that they share a copy of a given allele. For example, father and son will have

*r*= 1∕2. One individual acts as a potential donor, the other as a recipient, which receives a benefit

*b*from the donor at the donor’s cost

*c*. The donating individual pays the full cost but also indirectly receives the benefit

*b*multiplied by the above factor

*r*. Thus donation is worthwhile provided that

*Hamilton’s rule*(Hamilton 1964).

Note that this condition is analogous to the condition for cooperation to resist invasion in the IPD above, where a commonly used special class of the PD matrix has payoffs representing cooperation as making a donation and defecting as not. Then TFT resists invasion when *w* > *c*∕*b*.

### 3.3 Indirect Reciprocity and Punishment

The IPD is an example of direct reciprocity. Suppose now we have a population of individuals who play many contests, but these are not in long sequences against a single “opponent” as above? If faced with a series of single-shot games, how can cooperation be achieved?

Such situations are often investigated by the use of public goods games involving experiments with groups of real people, as in the work of Fehr and Gachter (2002). In these experiments individuals play a series of games, each game involving a new group. In each game there were four individuals, each of them receiving an initial endowment of 20 dollars, and each had to choose a level of investment into a common pool. Any money that was invested increased by a factor of 1.6 and was then shared between the four individuals, meaning that the return for each dollar invested was 40 cents to each of the players. In particular the individual making the investment of one dollar only receives 40 cents and so makes a loss of 60 cents. Thus, like the Prisoner’s Dilemma, it is clear that the best strategy is to make no investment but simply to share rewards from the investments of other players. In these experiments, investment levels began reasonably high, but slowly declined, as players saw others cheat.

In later experiments, each game was played over two rounds, an investment round and a punishment round, where players were allowed to punish others. In particular every dollar “invested” in punishment levied a fine of three dollars on the target of the punishment. This led to investments which increased from their initial level, as punishment brought cheating individuals into line. It should be noted that in a population of individuals many, but not all of whom, punish, optimal play for individuals in this case should not be to punish, but to be a second-order free rider who invests but does not punish, and therefore saves the punishment fee. Such a population would collapse down to no investment after some number of rounds. Thus it is clear that the people in the experiments were not behaving completely rationally.

*m*individuals with

*k*cooperators (who invest and punish),

*ℓ*defectors and

*m*−

*k*−

*ℓ*− 1 investors (who do not punish). This game, with this available set of strategies, requires two rounds of punishment as described above. The rewards to our focal individual in this case will be

*V*is the initial level of resources of each individual,

*c*<

*m*is the return on investment (every dollar becomes 1 +

*c*dollars), and

*P*is the punishment multiple (every dollar invested in punishment generates a fine of

*P*dollars). The optimal play for our focal individual is

*V*(1 −

*c*∕

*m*) < (

*m*− 1)

*P*.

We note that there are still issues on how such punishment can emerge in the first place (Sigmund 2007).

## 4 The Rock–Paper–Scissors Game

*a*’s and

*b*’s are positive. For the conventional game played between people

*a*

_{i}=

*b*

_{i}= 1 for

*i*= 1, 2, 3.

*K*is just the sum of the three terms to ensure that

**p**is a probability vector. In addition,

**p**is a globally asymptotically stable equilibrium of the replicator dynamics if and only if

*a*

_{1}

*a*

_{2}

*a*

_{3}>

*b*

_{1}

*b*

_{2}

*b*

_{3}. It is an ESS if and only if

*a*

_{1}−

*b*

_{1},

*a*

_{2}−

*b*

_{2}, and

*a*

_{3}−

*b*

_{3}are all positive, and the largest of their square roots is smaller than the sum of the other two square roots (Hofbauer and Sigmund 1998). Thus if

**p**is an ESS of the RPS game, then it is globally asymptotically stable under the replicator dynamics. However, since the converse is not true, the RPS game provides an example illustrating that while all internal ESSs are global attractors of the replicator dynamics, not all global attractors are ESSs.

We note that the case when *a*_{1}*a*_{2}*a*_{3} = *b*_{1}*b*_{2}*b*_{3} (including the conventional game with *a*_{i} = *b*_{i} = 1) leads to closed orbits of the replicator dynamics, and a stable (but not asymptotically stable) internal equilibrium. This is an example of a *nongeneric* game, where minor perturbations of the parameter values can lead to large changes in the nature of the game solution.

This game is a good representation for a number of real populations. The most well known of these is among the common side-blotched lizard *Uta stansburiana*. This lizard has three types of distinctive throat coloration, which correspond to very different types of behavior. Males with orange throats are very aggressive and have large territories which they defend against intruders. Males with dark blue throats are less aggressive and hold smaller territories. Males with yellow stripes do not have a territory at all but bear a strong resemblance to females and use a sneaky mating strategy. It was observed in Sinervo and Lively (1996) that if the Blue strategy is the most prevalent, Orange can invade; if Yellow is prevalent, Blue can invade; and if Orange is prevalent, then Yellow can invade.

An alternative real scenario is that of *Escherichia coli* bacteria, involving three strains of bacteria (Kerr et al. 2002). One strain produces the antibiotic colicin. This strain is immune to it, as is a second strain, but the third is not. When only the first two strains are present, the second strain outcompetes the first, since it forgoes the cost of colicin production. Similarly the third outcompetes the second, as it forgoes costly immunity, which without the first strain is unnecessary. Finally, the first strain outcompetes the third, as the latter has no immunity to the colicin.

## 5 Non-matrix Games

We have seen that matrix games involve a finite number of strategies with a payoff function that is linear in the strategy of both the focal player and that of the population. This leads to a number of important simplifying results (see Volume I, Chap. 6, “Evolutionary Game Theory”). All of the ESSs of a matrix can be found in a straightforward way using the procedure of Haigh (1975). Further, adding a constant to all entries in a column of a payoff matrix leaves the collection of ESSs (and the trajectories of the replicator dynamics) of the matrix unchanged. Haigh’s procedure can potentially be shortened, using the important Bishop–Cannings theorem (Bishop and Cannings 1976), a consequence of which is that if **p**_{1} is an ESS, no strategy **p**_{2} whose support is either a superset or a subset of the support of **p**_{1} can be an ESS.

However, there are a number of ways that games can involve nonlinear payoff functions. Firstly, playing the field games yield payoffs that are linear in the focal player but not in the population (e.g., see Sects. 7.1 and 9.1). Another way this can happen is to have individual games of the matrix type, but where opponents are not selected with equal probability from the population, for instance, if there is some spatial element. Thirdly, the payoffs can be nonlinear in both components. Here strategies do not refer to a probabilistic mix of pure strategies, but a unique trait, such as the height of a tree as in Kokko (2007) or a volume of sperm; see, e.g., Ball and Parker (2007). This happens in particular in the context of adaptive dynamics (see Volume I, Chap. 6, “Evolutionary Game Theory”).

Alternatively a non-matrix game can involve linear payoffs, but this time with a continuum of strategies (we note that the cases with nonlinear payoffs above can also involve such a continuum, especially the third type). A classical example of this is the war of attrition (Maynard Smith 1974).

### 5.1 The War of Attrition

We consider a Hawk–Dove game, where both individuals play Dove, but that instead of the reward being allocated instantly, they become involved in a potentially long displaying contest where a winner will be decided by one player conceding, and there is a cost proportional to the length of the contest. An individual’s strategy is thus the length of time it is prepared to wait. Pure strategies are all values of *t* on the non-negative part of the real line, and mixed strategies are corresponding probability distributions. These kinds of contests are for example observed in dung flies (Parker and Thompson 1980).

*S*

_{t}(wait until time

*t*) and

*S*

_{s}(wait until time

*s*) for the player that uses strategy

*S*

_{t}is

*f*(

*t*) and

*g*(

*s*) to the

*f*(

*t*) player is

*S*

_{s}is invaded by

*S*

_{t}(i.e.,

*E*(

*S*

_{t},

*S*

_{s}) >

*E*(

*S*

_{s},

*S*

_{s})) for any

*t*>

*s*, or any positive

*t*<

*s*−

*V*∕2. There is a unique ESS which is found by first considering (analogous to the Bishop–Cannings theorem; see Volume I, Chap. 6, “Evolutionary Game Theory”) a probability distribution

*p*(

*s*) that gives equal payoffs to all pure strategies that could be played by an opponent. This is required, since otherwise some potential invading strategies could do better than others, and since

*p*(

*s*) is simply a weighted average of such strategies, it would then be invaded by at least one type of opponent. Payoff of a pure strategy

*S*

_{t}played against a mixed strategy

*S*

_{p(s)}given by a probability distribution

*p*(

*s*) over the time interval is

*t*(assuming that such a derivative exists) gives

*P*(

*t*) is the associated distribution function, so that

*p*(

*t*) =

*P′*(

*t*) for all

*t*≥ 0, then Eq. (23.7) becomes

### 5.2 The Sex-Ratio Game

Why is it that the sex ratio in most animals is close to a half? This was the first problem to be considered using evolutionary game theory (Hamilton 1967), and its consideration, including the essential nature of the solution, dates right back to Darwin (1871). To maximize the overall birth rate of the species, in most animals there should be far more females than males, given that females usually make a much more significant investment in bringing up offspring than males. This, as mentioned before, is the wrong perspective, and we need to consider the problem from the viewpoint of the individual.

Suppose that in a given population, an individual female will have a fixed number of offspring, but that she can allocate the proportion of these that are male. This proportion is thus the strategy of our individual. As each female (irrespective of its strategy) has the same number of offspring, this number does not help us in deciding which strategy is the best. The effect of a given strategy can be measured as the number of grandchildren of the focal female. Assume that the number of individuals in a large population in the next generation is *N*_{1} and in the following generation is *N*_{2}. Further assume that all other females in the population play the strategy *m* and that our focal individual plays strategy *p*.

*N*

_{1}is large, the total number of males in the next generation is

*mN*

_{1}and so the total number of females is (1 −

*m*)

*N*

_{1}. We shall assume that all females (males) are equally likely to be the mother (father) of any particular member of the following generation of

*N*

_{2}individuals. This means that a female offspring will be the mother of

*N*

_{2}∕((1 −

*m*)

*N*

_{1}) of the following generation of

*N*

_{2}individuals, and a male offspring will be the father of

*N*

_{2}∕(

*mN*

_{1}) of these individuals. Thus our focal individual will have the following number of grandchildren

*p*, we maximize

*E*(

*p*,

*m*). For

*m*< 0.5 the best response is

*p*= 1, and for

*m*> 0.5 we obtain

*p*= 0. Thus

*m*= 0.5 is the interior NE at which all values of

*p*obtain the same payoff. This NE satisfies the stability condition in that

*E*(0.5,

*m′*) −

*E*(

*m′*,

*m′*) > 0 for all

*m′*≠0.5 (Broom and Rychtar 2013).

Thus from the individual perspective, it is best to have half your offspring as male. In real populations, it is often the case that relatively few males are the parents of many individuals, for instance, in social groups often only the dominant male fathers offspring. Sometimes other males are actually excluded from the group; lion prides generally consist of a number of females, but only one or two males, for example. From a group perspective, these extra males perform no function, but there is a chance that any male will become the father of many.

## 6 Asymmetric Games

The games we have considered above all involve populations of identical individuals. What if individuals are not identical? Maynard Smith and Parker (1976) considered two main types of difference between individuals. The first type was *correlated asymmetries* where there were real differences between them, for instance, in strength or need for resources, which would mean their probability of success, cost levels, valuation of rewards, set of available strategies, etc., may be different, i.e., the payoffs “correlate” with the type of the player. Examples of such games are the predator–prey games of Sect. 9.2 and the Battle of the Sexes below in Sect. 6.1.

The second type, *uncorrelated asymmetries*, occurred when the individuals were physically identical, but nevertheless occupied different roles; for example, one was the owner of a territory and the other was an intruder, which we shall see in Sect. 6.2. For uncorrelated asymmetries, even though individuals do not have different payoff matrices, it is possible to base their strategy upon the role that they occupy. As we shall see, this completely changes the character of the solutions that we obtain.

We note that the allocation of distinct roles can apply to games in general; for example, there has been significant work on the asymmetric war of attrition (see, e.g., Hammerstein and Parker 1982; Maynard Smith and Parker 1976), involving cases with both correlated and uncorrelated asymmetries.

The ESS was defined for a single population only, and the stability condition of the original definition cannot be easily extended for bimatrix games. This is because bimatrix games assume that individuals of one species interact with individuals of the other species only, so there is no frequency-dependent mechanism that could prevent mutants of one population from invading residents of that population at the two-species NE. In fact, it was shown (Selten 1980) that requiring the stability condition of the ESS definition to hold in bimatrix games restricts the ESSs to strict NEs, i.e., to pairs of pure strategies. Two key assumptions behind Selten’s theorem are that the probability that an individual occupies a given role is not affected by the strategies that it employs, and that payoffs within a given role are linear, as in matrix games. If either of these assumptions are violated, then mixed strategy ESSs can result (see, e.g., Broom and Rychtar 2013; Webb et al. 1999).

There are interior NE in bimatrix games that deserve to be called “stable,” albeit in a weaker sense than was used in the (single-species) ESS definition. For example, some of the NEs are stable with respect to some evolutionary dynamics (e.g., with respect to the replicator dynamics, or the best response dynamics). A static concept that captures such stability that proved useful for bimatrix games is the Nash–Pareto equilibrium (Hofbauer and Sigmund 1998). The Nash-Pareto equilibrium is an NE which satisfies an additional condition that says that it is impossible for both players to increase their fitness by deviating from this equilibrium. For two-species games that cannot be described by a bimatrix (e.g., see Sect. 7.4), this concept of two-species evolutionary stability was generalized by Cressman (2003) (see Volume I, Chap. 6, “Evolutionary Game Theory”) who defined a two-species ESS (*p*^{∗}, *q*^{∗}) as an NE such that, if the population distributions of the two species are slightly perturbed, then an individual in at least one species does better by playing its ESS strategy than by playing the slightly perturbed strategy of this species. We illustrate these concepts in the next section.

### 6.1 The Battle of the Sexes

A classical example of an asymmetric game is the *Battle of the Sexes* (Dawkins 1976), where a population contains females with two strategies, Coy and Fast, and males with two strategies, Faithful and Philanderer. A Coy female needs a period of courtship, whereas a Fast female will mate with a male as soon as they meet. Faithful males are willing to engage in long courtships and after mating will care for the offspring. A Philanderer male will not engage in courtship and so cannot mate with a Coy female and also leaves immediately after mating with a Fast female.

*B*is the fitness gained by having an offspring,

*C*

_{R}is the (potentially shared) cost of raising the offspring, and

*C*

_{C}is the cost of engaging in a courtship. All three of these terms are clearly positive. The above bimatrix is written in the form \((A_{1},A_{2}^{T})\), where matrix

*A*

_{1}is the payoff matrix for males (player 1) and matrix

*A*

_{2}is the payoff matrix for females (player 2), respectively.

*B*(Fig. 23.1A), \(B<\min (C_{R}/2+C_{C}, C_{R})\), a unique ESS (Philanderer, Fast) for sufficiently high

*B*(Fig. 23.1B), when

*B*>

*C*

_{R}. For intermediate

*B*satisfying

*C*

_{R}∕2 +

*C*

_{C}<

*B*<

*C*

_{R}, there is a two-species weak ESS

### 6.2 The Owner–Intruder Game

*V*≥

*C*, then Hawk is the unique ESS (Fig. 23.2A), and that if

*V*<

*C*, then Bourgeois and Anti-Bourgeois (alternatively called Marauder) are the only ESSs (Fig. 23.2B). As we see, there are only pure strategy solutions, as opposed to the case of the Hawk–Dove game, which had a mixed ESS

*V*∕

*C*for

*V*<

*C*, because the interior NE in Fig. 23.2B is not a Nash–Pareto pair as in the upper-left and lower-right regions both arrows are pointing away from the NE. Thus, if both players simultaneously deviate from the Nash equilibrium, their payoffs increase. Consequently, this NE is not a two-species (weak) ESS.

Important recent work on this model and its ramifications for the part that respecting ownership plays has been carried out by Mesterton Gibbons, Sherratt and coworkers (see Mesterton-Gibbons and Sherratt 2014; Sherratt and Mesterton-Gibbons 2015). In particular, why in real populations is the Bourgeois respect for ownership strategy so common and the Anti-Bourgeois strategy so rare? One explanation offered by Maynard Smith (1982) was “infinite regress.” In this argument, immediately after a contest, the winner becomes the owner of the territory, and the loser becomes a potential intruder which could immediately rechallenge the individual that has just displaced it. In an Anti-Bourgeois population, this would result in the new owner conceding and the new intruder (the previous owner) once again being the owner, but then the displaced owner could immediately rechallenge, and the process would continue indefinitely. It is shown in Mesterton-Gibbons and Sherratt (2014) that under certain circumstances, but not always, this allows Bourgeois to be the unique ESS. Sherratt and Mesterton-Gibbons (2015) discuss many issues, such as uncertainty of ownership, asymmetry of resource value, continuous contest investment (as in the war of attrition), and potential signaling of intentions (what they call “secret handshakes,” similar to some of the signals we discuss in Sect. 10) in detail. There are many reasons that can make evolution of Anti-Bourgeois unlikely, and it is probably a combination of these that make it so rare.

### 6.3 Bimatrix Replicator Dynamics

**p**

_{1}∈

*Δ*

_{n}and

**p**

_{2}∈

*Δ*

_{m}are the population mixtures of individuals in role 1 and 2, respectively. For example, for the two-role, two-strategy game, where without loss of generality we can set

*a*

_{11}=

*a*

_{22}=

*b*

_{11}=

*b*

_{22}= 0 (since as for matrix games, adding a constant to all of the payoffs an individual gets against a given strategy does not affect the NEs/ ESSs), we obtain

*x*is the frequency of the first strategy players in the role 1 population and

*y*is the corresponding frequency for role 2. Hofbauer and Sigmund (1998) show that orbits converge to the boundary in all cases except if

*a*

_{12}

*a*

_{21}> 0,

*b*

_{12}

*b*

_{21}> 0, and

*a*

_{12}

*b*

_{12}< 0, which yield closed periodic orbits around the internal equilibrium. Replicator dynamics for the Battle of the Sexes and the Owner–Intruder game are shown in Fig. 23.3.

Note there are some problems with the interpretation of the dynamics of two populations in this way, related to the assumption of exponential growth of populations, since the above dynamics effectively assume that the relative size of the two populations remains constant (Argasinski 2006).

## 7 The Habitat Selection Game

Fretwell and Lucas (1969) introduced the Ideal Free Distribution (IFD) to describe a distribution of animals in a heterogeneous environment consisting of discrete patches *i* = 1, …, *n*. The IFD assumes that animals are free to move between several patches, travel is cost-free, each individual knows perfectly the quality of all patches, and all individuals have the same competitive abilities. Assuming that these patches differ in their basic quality *B*_{i} (i.e., their quality when unoccupied), the IFD model predicts that the best patch will always be occupied.

*B*

_{1}> ⋯ >

*B*

_{n}> 0) and

*m*

_{i}is the animal abundance in patch

*i*. Let

*p*

_{i}=

*m*

_{i}∕(

*m*

_{1}+ ⋯ +

*m*

_{n}) be the proportion of animals in patch

*i*, so that \(\mathbf {p}=(p_1,\dots ,p_n)\) describes the spatial distribution of the population. For a monomorphic population,

*p*

_{i}also specifies the individual strategy as the proportion of the lifetime an average animal spends in patch

*i*. We assume that the payoff in each patch,

*V*

_{i}(

*p*

_{i}), is a decreasing function of animal abundance in that patch, i.e., the patch payoffs are negatively density dependent. Then, fitness of a mutant with strategy \(\tilde {\mathbf {p}}=(\tilde {p}_1,\dots ,\tilde {p}_n)\) in the resident monomorphic population with distribution \(\mathbf {p}=(p_1,\dots ,p_n)\) is

- 1.
There exists a number 1 ≤

*k*≤*n*such that*p*_{1}> 0, …,*p*_{k}> 0 and*p*_{k+1}= ⋯ =*p*_{n}= 0 - 2.
*V*_{1}(*p*_{1}) = ⋯ =*V*_{k}(*p*_{k}) =*V*^{∗}and*V*^{∗}≥*V*_{i}(*p*_{i}) for*i*=*k*+ 1, …,*n*.

### 7.1 Parker’s Matching Principle

*r*

_{i},

*i*= 1, …,

*n*are constant and resources are consumed immediately when they enter the patch and so there is no standing crop. This leads to a particularly simple definition of animal patch payoffs as the ratio of the resource input rate divided by the number of individuals there, i.e.,

*M*=

*m*

_{1}+ ⋯ +

*m*

_{n}is the overall population abundance. The matching principle then says that animals distribute themselves so that their abundance in each patch is proportional to the rate with which resources arrive into the patch,

*p*

_{i}∕

*p*

_{j}=

*r*

_{i}∕

*r*

_{j}. This is nothing other than the IFD for payoff functions (23.12). It is interesting to notice that all patches will be occupied independently of the total population abundance. Indeed, as the consumer density in the

*i*−th patch decreases, payoff

*r*

_{i}∕(

*p*

_{i}

*M*) increases, which attracts some animals, and there cannot be unoccupied patches. There is an important difference between this (nonlinear) payoff function (23.12) and the linear payoff function that we consider in the following Eq. (23.13), because as the local population abundance in a patch decreases, then (23.12) tends to infinity, but (23.13) tends to

*r*

_{i}. This means that in the first case there cannot be unoccupied patches (irrespective of their basic patch quality

*r*

_{i}) because the payoffs in occupied patches are finite, but the payoff in unoccupied patches would be infinite (provided all

*r*

_{i}> 0). This argument does not apply in the case of the logistic payoff (23.13). This concept successfully predicts the distribution of house flies that arrive at a cow pat where they immediately mate (Blanckenhorn et al. 2000; Parker 1978, 1984) or of fish that are fed at two feeders in a tank (Berec et al. 2006; Milinski 1979, 1988).

In the next section, we consider the situation where resources are not consumed immediately upon entering the system.

### 7.2 Patch Payoffs are Linear

*i*(= 1, 2) is a linearly decreasing function of population abundance:

*m*

_{i}is the population density in habitat

*i*,

*r*

_{i}is the intrinsic per capita population growth rate in habitat

*i*, and

*K*

_{i}is its carrying capacity. The total population size in the two-habitat environment is denoted by

*M*(=

*m*

_{1}+

*m*

_{2}), and the proportion of the population in habitat

*i*is

*p*

_{i}=

*m*

_{i}∕

*M*. Payoff (23.13) is often used in population dynamics where it describes the logistic population growth.

*M*, then its fitness in the population with mean strategy \(\mathbf {p}=(p_{1},p_{2})\) is

*i*represents staying in patch

*i*(

*i*= 1, 2). This shows that the Habitat Selection game with a linear payoff can be written for a fixed population size as a matrix game. If the per capita intrinsic population growth rate in habitat 1 is higher than that in habitat 2 (

*r*

_{1}>

*r*

_{2}), the IFD is (Křivan and Sirot 2002)

*V*

_{1}(

*p*

_{1}) =

*V*

_{2}(

*p*

_{2})). Once again, it is important to emphasize here that the IFD concept is different from maximization of the mean animal fitness

*r*

_{1}=

*r*

_{2}. Interestingly, by comparing (23.14) and (23.15), we see that maximizing mean fitness leads to fewer animals than the IFD in the patch with higher basic quality

*r*

_{i}(i.e., in patch 1).

### 7.3 Some Extensions of the Habitat Selection Game

The Habitat Selection game as described makes several assumptions that were relaxed in the literature. One assumption is that patch payoffs are decreasing functions of population abundance. This assumption is important because it guarantees that a unique IFD exists. However, patch payoffs can also be increasing functions of population abundance. In particular, at low population densities, payoffs can increase as more individuals enter a patch, and competition is initially weak. For example, more individuals in a patch can increase the probability of finding a mate. This is called the Allee effect. The IFD for the Allee effect has been studied in the literature (Cressman and Tran 2015; Fretwell and Lucas 1969; Křivan 2014; Morris 2002). It has been shown that for hump-shaped patch payoffs, up to three IFDs can exist for a given overall population abundance. At very low overall population abundances, only the most profitable patch will be occupied. At intermediate population densities, there are two IFDs corresponding to pure strategies where all individuals occupy patch 1 only, or patch 2 only. As population abundance increases, competition becomes more severe, and an interior IFD appears exactly as in the case of negative density-dependent payoff functions. At high overall population abundances, only the interior IFD exists due to strong competition among individuals. It is interesting to note that as the population numbers change, there can be sudden (discontinuous) changes in the population distribution. Such erratic changes in the distribution of deer mice were observed and analyzed by Morris (2002).

Another complication that leads to multiple IFDs is the cost of dispersal. Let us consider a positive migration cost *c* between two patches. An individual currently in patch 1 will migrate to patch 2 only if the payoff there is such that *V*_{2}(*p*_{2}) − *c* ≥ *V*_{1}(*p*_{1}). Similarly, an individual currently in patch 2 will migrate to patch 1 only if its payoff does not decrease by doing so, i.e., *V*_{1}(*p*_{1}) − *c* ≥ *V*_{2}(*p*_{2}). Thus, all distributions (*p*_{1}, *p*_{2}) that satisfy these two inequalities form the set of IFDs (Mariani et al. 2016).

The Habitat Selection game was also extended to situations where individuals perceive space as a continuum (e.g., Cantrell et al. 2007, 2012; Cosner 2005). The movement by diffusion is then combined, or replaced, by a movement along the gradient of animal fitness.

### 7.4 Habitat Selection for Two Species

*M*and

*N*dispersing between two patches. We assume that individuals of these species compete in each patch both intra- and inter-specifically. Following our single-species Habitat Selection game, we assume that individual payoffs are linear functions of species distribution (Křivan and Sirot 2002; Křivan et al. 2008)

*α*

_{i}(respectively

*β*

_{i}) are interspecific competition coefficients,

*r*

_{i}(respectively

*s*

_{i}) are the intrinsic per capita population growth rates, and

*K*

_{i}(respectively

*L*

_{i}) are the environmental carrying capacities. The two-species Habitat Selection game cannot be represented in a bimatrix form (to represent it in a matrix form, we would need four matrices), because the payoff in patch

*i*for a given species depends not only on the distribution (strategy) of its competitors but also on the distribution of its own conspecifics. The equal payoff line for species one (two) are those \((\mathbf {p},\mathbf {q})\in \varDelta _{2}\times \varDelta _{2}\) for which \(V_1(\mathbf {p},\mathbf {q})=V_2(\mathbf {p},\mathbf {q})\) (\(W_1(\mathbf {p},\mathbf {q})=W_2(\mathbf {p},\mathbf {q})\)). Since payoffs are linear functions, these are lines in the coordinates

*p*

_{1}and

*q*

_{1}, but as opposed to the case of bimatrix games in Sects. 6.1 and 6.2, they are neither horizontal nor vertical. If they do not intersect in the unit square, the two species cannot coexist in both patches at an NE. The most interesting case is when the two equal payoff lines intersect inside the unit square. Křivan et al. (2008) showed that the interior intersection is the two-species ESS provided

One of the predictions of the Habitat Selection game for two species is that as competition gets stronger, the two species will spatially segregate (e.g., Křivan and Sirot 2002; Morris 1999). Such spatial segregation was observed in experiments with two bacterial strains in a microhabitat system with nutrient-poor and nutrient-rich patches (Lambert et al. 2011).

## 8 Dispersal and Evolution of Dispersal

Organisms often move from one habitat to another, which is referred to as dispersal. We focus here on dispersal and its relation to the IFD discussed in the previous section.

*n*habitat patches and a population of individuals that disperse between them. In what follows we will assume that the patches are either adjacent (in particular when there are just two patches) or that the travel time between them is negligible when compared to the time individuals spend in these patches. There are two basic questions:

- 1.
When is dispersal an adaptive strategy, i.e., when does individual fitness increase for dispersing animals compared to those who are sedentary?

- 2.
Where should individuals disperse?

*δ*≥ 0) and by a dispersal matrix

*D*. The entries of this matrix (

*D*

_{ij}) describe the transition probabilities that an individual currently in patch

*j*moves to patch

*i*per unit of time. We remark that

*D*

_{ii}is the probability of staying in patch

*i*. Per capita population growth rates in patches are given by

*f*

_{i}(e.g.,

*f*

_{i}can be the logistic growth rate

*V*

_{i}(23.13) in Sect. 7.2). The changes in population numbers are then described by population–dispersal dynamics

*n*patches. Thus, the first term in the above summation describes immigration to patch

*i*from other patches, and the second term describes emigration from patch

*i*to other patches. In addition, we assume that

*D*is irreducible, i.e., there are no isolated patches.

*f*

_{i}are decreasing with

*f*

_{i}(0) > 0 and

*f*

_{i}(

*K*

_{i}) = 0 for some

*K*

_{i}> 0, then model (23.16) has an interior equilibrium which is globally asymptotically stable. However this does not answer the question of whether such an equilibrium is evolutionarily stable, i.e., whether it is resistant to invasion of mutants with the same traits (parameters) as the resident population, but different propensity to disperse

*δ*. The answer to this question depends on the entries of the dispersal matrix. An interior population distribution \(\mathbf {m}^{*}=(m_{1}^{*},\dots ,m_{n}^{*})\) will be the IFD provided patch payoffs in all patches are the same, i.e., \(f_{1}(m_{1}^{*})=\dots =f_{n}(m_{n}^{*}).\) This implies that at the population equilibrium, there is no net dispersal, i.e.,

*δ*= 0. The pattern of equalized immigration and emigration satisfying (23.17) is called “balanced dispersal” (Doncaster et al. 1997; Holt and Barfield 2001; McPeek and Holt 1992). Under balanced dispersal, there is an inverse relation between local population size and its dispersal rate. In other words, individuals at good sites are less likely to disperse than those from poor sites. When dispersal is unbalanced, Hastings (1983) showed that mutants with lower propensity to disperse will outcompete the residents and no dispersal (

*δ*= 0) is the only evolutionarily stable strategy.

*e*of patches can become empty at any time step. A proportion

*p*of migrants survives migration and re-distributes at random (assuming the Poisson distribution) among the patches. These authors derived analytically the evolutionarily stable dispersal strategy that is given by a complicated implicit formula (see formula (3) in Comins et al. 1980). As population abundance increases, the evolutionarily stable dispersal rate converges to a simpler formula

Evolution of mobility in predator–prey systems was also studied by Xu et al. (2014). These authors showed how interaction strength between mobile vs. sessile prey and predators influences the evolution of dispersal.

## 9 Foraging Games

Foraging games describe interactions between prey, their predators, or both. These games assume that either predator or prey behave in order to maximize their fitness. Typically, the prey strategy is to avoid predators while predators try to track their prey. Several models that focus on various aspects of predator–prey interactions were developed in the literature (e.g., Brown and Vincent 1987; Brown et al. 1999, 2001; Vincent and Brown 2005).

An important component of predation is the functional response defined as the per predator rate of prey consumption (Holling 1959). It also serves as a basis for models of optimal foraging (Stephens and Krebs 1986) that aim to predict diet selection of predators in environments with multiple prey types. In this section we start with a model of optimal foraging, and we show how it can be derived using extensive form games (see Volume I, Chap. 6, “Evolutionary Game Theory”). As an example of a predator–prey game, we then discuss predator–prey distribution in a two-patch environment.

### 9.1 Optimal Foraging as an Agent Normal Form of an Extensive Game

*p*

_{1}and prey type 2 with probability

*p*

_{2}during the search time

*τ*

_{s}. For simplicity we will assume that

*p*

_{1}+

*p*

_{2}= 1. Upon an encounter with a prey individual, the predator decides whether to attack the prey (prey type 1 with probability

*q*

_{1}and prey type 2 with probability

*q*

_{2}) or not. When a prey individual is captured, the energy that the predator receives is denoted by

*e*

_{1}or

*e*

_{2}. The predator’s cost is measured by the time lost. This time consists of the search time

*τ*

_{s}and the time needed to handle the prey (

*h*

_{1}for prey type 1 and

*h*

_{2}for prey type 2).

*i*prey are only killed when the path denoted by

*p*

_{i}and then

*q*

_{i}is followed, the functional response to prey

*i*(= 1, 2) is

*x*

_{i}denotes density of prey type

*i*in the environment and the predator meets prey at random,

*p*

_{i}=

*x*

_{i}∕

*x*, where

*x*=

*x*

_{1}+

*x*

_{2}. Setting

*λ*= 1∕(

*τ*

_{s}

*x*) leads to

*q*

_{1},

*q*

_{2}), (0 ≤

*q*

_{i}≤ 1,

*i*= 1, 2).

*Δ*

_{1}= {

*q*

_{1}∣0 ≤

*q*

_{1}≤ 1} and player 2 to node 2 with strategy set

*Δ*

_{2}= {

*q*

_{2}∣0 ≤

*q*

_{2}≤ 1}. Their common payoff

*E*(

*q*

_{1},

*q*

_{2}) is given by (23.18), and we seek the NE of the two-player game. Assuming that prey type 1 is the more profitable for the predator, as its energy content per unit handling time is higher than the profitability of the second prey type (i.e.,

*e*

_{1}∕

*h*

_{1}>

*e*

_{2}∕

*h*

_{2}) we get

*E*(1,

*q*

_{2}) >

*E*(

*q*

_{1},

*q*

_{2}) for all 0 ≤

*q*

_{1}< 1 and 0 ≤

*q*

_{2}≤ 1. Thus, at any NE, player 1 must play

*q*

_{1}= 1. The NE strategy of player 2 is then any best response to

*q*

_{1}= 1 (i.e., any

*q*

_{2}that satisfies \(E(1,q_{2}^{\prime })\leq E(1,q_{2})\) for all \(0\leq q_2^{\prime } \leq 1\)) which yields

*E*) in this model is not a coincidence. Cressman et al. (2014) proved that this is so for all foraging games with a 2-level decision tree. For decision tress with more levels, they showed that the optimal foraging strategy is always an NE of the corresponding agent normal form game and that other, nonoptimal, NE may also appear.

### 9.2 A Predator-Prey Foraging Game

*x*

_{i}denotes the abundance of prey in patch

*i*(= 1, 2), the total abundance of prey is

*x*=

*x*

_{1}+

*x*

_{2}and, similarly, the total abundance of predators is

*y*=

*y*

_{1}+

*y*

_{2}. Let \(\mathbf {u}=(u_{1},u_{2})\) be the distribution of prey and \(\mathbf {v}=(v_{1},v_{2})\) be the distribution of predators. We neglect the travel time between patches so that

*u*

_{1}+

*u*

_{2}=

*v*

_{1}+

*v*

_{2}= 1 (i.e., each animal is either in patch 1 or patch 2). We assume that the prey population grows exponentially at each patch with the per capita population growth rate

*r*

_{i}and it is consumed by predators. The killing rate is given by the functional response. For simplicity we neglect the handling time so that the functional response in patch

*i*is

*f*

_{i}=

*λ*

_{i}

*x*

_{i}, i.e., the per prey per predator killing rate is

*λ*

_{i}. The prey payoff in patch

*i*is given by the per capita prey population growth rate in that patch, i.e.,

*r*

_{i}−

*λ*

_{i}

*v*

_{i}

*y*as there are

*v*

_{i}

*y*predators in patch

*i*. The fitness of a prey individual is

*i*is given by the per capita predator population growth rate

*e*

_{i}

*u*

_{i}

*x*−

*m*

_{i}, where

*e*

_{i}is a coefficient by which the energy gained by feeding on prey is transformed into new predators and

*m*

_{i}is the per capita predator mortality rate in patch

*i*. The fitness of a predator with strategy \(\mathbf {v}=(v_{1},v_{2})\) when the prey use strategy \(\mathbf {u}=(u_{1},u_{2})\) is

*r*

_{1}≥

*r*

_{2}) while for predators patch 1 has a higher mortality rate (

*m*

_{1}>

*m*

_{2}). The corresponding NE is (Křivan 1997)

- (a)
\((u_1^*,v_1^*)\) if \(\displaystyle {x > \frac {m_1-m_2}{e_1 \lambda _1},\;\; y > \frac {r_1-r_2}{\lambda _1}}\),

- (b)
\(\displaystyle {\left (1,1\right ) }\) if \(\displaystyle {x> \frac {m_1-m_2}{e_1 \lambda _1},\;\; y< \frac {r_1-r_2}{\lambda _1}}\),

- (c)
\(\displaystyle {\left (1 ,0\right )}\) if \(\displaystyle {x<\frac {m_1-m_2}{e_1 \lambda _1}},\)

## 10 Signaling Games

Signaling between animals occurs in a number of contexts. This can be signals, often but not necessarily between conspecifics, warning of approaching predators. This situation can be game theoretic, as the signaler runs a potentially higher risk of being targeted by the predator. There are also cases of false signals being given when no predator is approaching to force food to be abandoned which can then be consumed by the signaler (Flower 2011). Alternatively within a group of animals, each individual may need to decide how to divide their time between vigilance and feeding, where each individual benefits from the vigilance of others as well as itself, and this has been modeled game-theoretically (e.g., Brown 1999; McNamara and Houston 1992; Sirot 2012).

Another situation occurs between relatives over items of food, for example, a parent bird feeding its offspring. Young birds beg aggressively for food, and the parent must decide which to feed, if any (it can instead consume the item itself). The most well-known model of this situation is the Sir Philip Sidney game (Maynard Smith 1991) and is a model of cost-free signaling (Bergstrom and Lachmann 1998).

The classic example of a signaling game is between potential mates. Males of differing quality advertise this quality to females, often in a way that is costly, and the females choose who to mate with based upon the strength of the signal. Examples are the tail of the peacock or the elaborate bowers created by bowerbirds. There is obviously a large incentive to cheat, and so how are such signals kept honest? A signal that is not at least partly correlated to quality would be meaningless, and so would eventually be ignored. The solution as developed by Zahavi (1975, 1977), the *handicap principle*, is that these costly signals are easier to bear by higher-quality mates and that evolution leads to a completely honest signal, where each quality level has a unique signal.

### 10.1 Grafen’s Signaling Game

*q*and a single type of female. Assume that a male of quality

*q*gives a signal

*a*=

*A*(

*q*) of this quality, where higher values of

*a*are more costly. It is assumed that there is both a minimum quality level

*q*

_{0}> 0 (there may or may not be a maximum quality level) and a minimum signal level

*a*

_{0}≥ 0 (which can be thought of as giving no signal). When a female receives a signal, she allocates a quality level to the signal

*P*(

*a*). We have a nonlinear asymmetric game with sequential decisions; in particular the nonlinearity makes this game considerably more complicated than asymmetric games such as the Battle of the Sexes of Sect. 6.1. The female pays a cost for misassessing a male of quality

*q*as being of quality

*p*of

*D*(

*q*,

*p*), which is positive for

*p*≠

*q*, with

*D*(

*q*,

*q*) = 0. Assuming that the probability density of males of quality

*q*is

*g*(

*q*), the payoff to the female, which is simply minus the expected cost, is

*A*

^{∗}and

*P*

^{∗}occurs if and only if

*P*

^{∗}(

*A*

^{∗}(

*q*)) =

*q*, for all

*q*. We note that here the female never misassesses a male and so pays zero cost. Clearly any alternative female assessment strategy would do worse. But how can we obtain stability against alternative (cheating) male strategies?

*q*,

*W*(

*a*,

*p*,

*q*), depends upon his true quality, the quality assigned to him by the female and the cost of his signal.

*W*(

*a*,

*p*,

*q*) will be increasing in

*p*and decreasing in

*a*. For stability of the honest signal, we need that the incremental advantage of a higher level of signaling is greater for a high-quality male than for a low-quality one, so that

*q*(note that the ratio is negative, so minus this ratio is positive), i.e., the higher quality the male, the lower the ratio of the marginal cost to the marginal benefit for an increase in the level of advertising. This ensures that completely honest signaling cannot be invaded by cheating, since costs to cheats to copy the signals of better quality males would be explicitly higher than for the better quality males, who could always thus achieve a cost they were willing to pay that the lower quality cheats would not.

*q*

_{0}≤

*q*< 1 and signals of strength

*a*≥

*a*

_{0}, for some

*r*> 0.

*W*(

*a*,

*p*,

*q*). In particular consider the condition from expression (23.23)

*q*. Thus there is an honest signaling solution. This is shown in Grafen (1990a) to be given by

## 11 Conclusion

In this chapter we have covered some of the important evolutionary game models applied to biological situations. We should note that we have left out a number of important theoretical topics as well as areas of application. We briefly touch on a number of those below.

All of the games that we have considered involved either pairwise games, or playing the field games, where individuals effectively play against the whole population. In reality contests will sometimes involve groups of individuals. Such models were developed in Broom et al. (1997), for a recent review see Gokhale and Traulsen (2014). In addition the populations were all both effectively infinite and *well-mixed* in the sense that for any direct contest involving individuals, each pair was equally likely to meet. In reality populations are finite and have (e.g., spatial) structure. The modeling of evolution in finite populations often uses the Moran process (Moran 1958), but more recently games in finite populations have received significant attention (Nowak 2006). These models have been extended to include population structure by considering evolution on graphs (Lieberman et al. 2005), and there has been an explosion of such model applications, especially to consider the evolution of cooperation. Another feature of realistic populations that we have ignored is the state of the individual. A hungry individual may behave differently to one that has recently eaten, and nesting behavior may be different at the start of the breeding season to later on. A theory of state-based models has been developed in Houston and McNamara (1999).

In terms of applications, we have focused on classical biological problems, but game theory has also been applied to medical scenarios more recently. This includes the modeling of epidemics, especially with the intention of developing defense strategies. One important class of models (see, e.g., Nowak and May 1994) considers the evolution of the virulence of a disease as the epidemic spreads. An exciting new line of research has recently been developed which considers the development of cancer as an evolutionary game, where the population of cancer cells evolves in the environment of the individual person or animal (Gatenby et al. 2010). A survey of alternative approaches is considered in Durrett (2014).

## 12 Cross-References

## Notes

### Acknowledgements

This project has received funding from the European Union Horizon 2020 research and innovation programe under the Marie Sklodowska-Curie grant agreement No 690817. VK acknowledges support provided by the Institute of Entomology (RVO:60077344).

## References

- Argasinski K (2006) Dynamic multipopulation and density dependent evolutionary games related to replicator dynamics. A metasimplex concept. Math Biosci 202:88–114Google Scholar
- Axelrod R (1981) The emergence of cooperation among egoists. Am Political Sci Rev 75:306–318Google Scholar
- Axelrod R (1984) The evolution of cooperation. Basic Books, New YorkGoogle Scholar
- Axelrod R, Hamilton WD (1981) The Evolution of cooperation. Science 211:1390–1396Google Scholar
- Ball MA, Parker GA (2007) Sperm competition games: the risk model can generate higher sperm allocation to virgin females. J Evol Biol 20:767–779Google Scholar
- Bendorf J, Swistak P (1995) Types of evolutionary stability and the problem of cooperation. Proc Natl Acad Sci USA 92:3596–3600Google Scholar
- Berec M, Křivan V, Berec L (2003) Are great tits parus major really optimal foragers? Can J Zool 81:780–788Google Scholar
- Berec M, Křivan V, Berec L (2006) Asymmetric competition, body size and foraging tactics: testing an ideal free distribution in two competing fish species. Evol Ecol Res 8:929–942Google Scholar
- Bergstrom C, Lachmann M (1998) Signaling among relatives. III. Talk is cheap. Proc Natl Acad Sci USA 95:5100–5105Google Scholar
- Bishop DT, Cannings C (1976) Models of animal conflict. Adv appl probab 8:616–621Google Scholar
- Blanckenhorn WU, Morf C, Reuter M (2000) Are dung flies ideal-free distributed at their oviposition and mating site? Behaviour 137:233–248Google Scholar
- Broom M, Rychtar J (2013) Game-theoretical models in biology. CRC Press/Taylor & Francis Group, Boca RatonGoogle Scholar
- Broom M, Cannings C, Vickers G (1997) Multi-player matrix games. Bull Austral Math Soc 59:931–952Google Scholar
- Broom M, Luther RM, Ruxton GD (2004) Resistance is useless? – extensions to the game theory of kleptoparasitism. Bull Math Biol 66:1645–1658Google Scholar
- Brown JS (1999) Vigilance, patch use and habitat selection: foraging under predation risk. Evol Ecol Res 1:49–71Google Scholar
- Brown JS, Vincent TL (1987) Predator-prey coevolution as an evolutionary game. Lect Notes Biomath 73:83–101Google Scholar
- Brown JS, Laundré JW, Gurung M (1999) The ecology of fear: optimal foraging, game theory, and trophic interactions. J Mammal 80:385–399Google Scholar
- Brown JS, Kotler BP, Bouskila A (2001) Ecology of fear: Foraging games between predators and prey with pulsed resources. Ann Zool Fennici 38:71–87Google Scholar
- Cantrell RS, Cosner C, DeAngelis DL, Padron V (2007) The ideal free distribution as an evolutionarily stable strategy. J Biol Dyn 1:249–271Google Scholar
- Cantrell RS, Cosner C, Lou Y (2012) Evolutionary stability of ideal free dispersal strategies in patchy environments. J Math Biol 65:943–965. doi:10.1007/s00285-011-0486-5Google Scholar
- Charnov EL (1976) Optimal foraging: attack strategy of a mantid. Am Nat 110:141–151Google Scholar
- Comins H, Hamilton W, May R (1980) Evolutionarily stable dispersal strategies. J Theor Biol 82:205–230Google Scholar
- Cosner C (2005) A dynamic model for the ideal-free distribution as a partial differential equation. Theor Popul Biol 67:101–108Google Scholar
- Cressman R (2003) Evolutionary dynamics and extensive form games. The MIT Press, Cambridge, MAGoogle Scholar
- Cressman R, Křivan V (2006) Migration dynamics for the ideal free distribution. Am Nat 168:384–397Google Scholar
- Cressman R, Tran T (2015) The ideal free distribution and evolutionary stability in habitat selection games with linear fitness and Allee effect. In: Cojocaru MG (ed) Interdisciplinary topics in applied mathematics, modeling and computational science. Springer proceedings in mathematics & statistics, vol 117. Springer, Cham, pp 457–464Google Scholar
- Cressman R, Křivan V, Brown JS, Gáray J (2014) Game-theoretical methods for functional response and optimal foraging behavior. PLOS ONE 9:e88,773Google Scholar
- Darwin C (1871) The descent of man and selection in relation to sex. John Murray, LondonGoogle Scholar
- Dawkins R (1976) The selfish gene. Oxford University Press, OxfordGoogle Scholar
- Doncaster CP, Clobert J, Doligez B, Gustafsson L, Danchin E (1997) Balanced dispersal between spatially varying local populations: an alternative to the source-sink model. Am Nat 150:425–445Google Scholar
- Dugatkin LA, Reeve HK (1998) Game theory & animal behavior. Oxford University Press, New YorkGoogle Scholar
- Durrett R (2014) Spatial evolutionary games with small selection coefficients. Electron J Probab 19:1–64Google Scholar
- Fehr E, Gachter S (2002) Altruistic punishment in humans. Nature 415:137–140Google Scholar
- Flood MM (1952) Some experimental games. Technical report RM-789-1, The RAND corporation, Santa MonicaGoogle Scholar
- Flower T (2011) Fork-tailed drongos use deceptive mimicked alarm calls to steal food. Proc R Soc B 278:1548–1555Google Scholar
- Fretwell DS, Lucas HL (1969) On territorial behavior and other factors influencing habitat distribution in birds. Acta Biotheor 19:16–32Google Scholar
- Gatenby RA, Gillies R, Brown J (2010) The evolutionary dynamics of cancer prevention. Nat Rev Cancer 10:526–527Google Scholar
- Gilpin ME (1975) Group selection in predator-prey communities. Princeton University Press, PrincetonGoogle Scholar
- Gokhale CS, Traulsen A (2014) Evolutionary multiplayer games. Dyn Games Appl 4:468–488Google Scholar
- Grafen A (1990a) Biological signals as handicaps. J Theor Biol 144:517–546Google Scholar
- Grafen A (1990b) Do animals really recognize kin? Anim Behav 39:42–54Google Scholar
- Haigh J (1975) Game theory and evolution. Adv Appl Probab 7:8–11Google Scholar
- Hamilton WD (1964) The genetical evolution of social behavior. J Theor Biol 7:1–52Google Scholar
- Hamilton WD (1967) Extraordinary sex ratios. Science 156:477–488Google Scholar
- Hamilton WD, May RM (1977) Dispersal in stable environments. Nature 269:578–581Google Scholar
- Hammerstein P, Parker GA (1982) The asymmetric war of attrition. J Theor Biol 96:647–682Google Scholar
- Hardin G (1968) The tragedy of the commons. Science 162:1243–1248Google Scholar
- Hastings A (1983) Can spatial variation alone lead to selection for dispersal? Theor Popul Biol 24:244–251Google Scholar
- Hofbauer J, Sigmund K (1998) Evolutionary games and population dynamics. Cambridge University Press, Cambridge, UKGoogle Scholar
- Holling CS (1959) Some characteristics of simple types of predation and parasitism. Can Entomol 91:385–398Google Scholar
- Holt RD, Barfield M (2001) On the relationship between the ideal-free distribution and the evolution of dispersal. In: Danchin JCE, Dhondt A, Nichols J (eds) Dispersal. Oxford University Press, Oxford/New York, pp 83–95Google Scholar
- Houston AI, McNamara JM (1999) Models of adaptive behaviour. Cambridge University Press, Cambridge, UKGoogle Scholar
- Kerr B, Riley M, Feldman M, Bohannan B (2002) Local dispersal promotes biodiversity in a real-life game of rock-paper-scissors. Nature 418:171–174Google Scholar
- Kokko H (2007) Modelling for field biologists and other interesting people. Cambridge University Press, Cambridge, UKGoogle Scholar
- Komorita SS, Sheposh JP, Braver SL (1968) Power, use of power, and cooperative choice in a two-person game. J Personal Soc Psychol 8:134–142Google Scholar
- Krebs JR, Erichsen JT, Webber MI, Charnov EL (1977) Optimal prey selection in the great tit (
*Parus major*). Anim Behav 25:30–38Google Scholar - Křivan V (1997) Dynamic ideal free distribution: effects of optimal patch choice on predator-prey dynamics. Am Nat 149:164–178Google Scholar
- Křivan V (2014) Competition in di- and tri-trophic food web modules. J Theor Biol 343:127–137Google Scholar
- Křivan V, Sirot E (2002) Habitat selection by two competing species in a two-habitat environment. Am Nat 160:214–234Google Scholar
- Křivan V, Cressman R, Schneider C (2008) The ideal free distribution: a review and synthesis of the game theoretic perspective. Theor Popul Biol 73:403–425Google Scholar
- Lambert G, Liao D, Vyawahare S, Austin RH (2011) Anomalous spatial redistribution of competing bacteria under starvation conditions. J Bacteriol 193:1878–1883Google Scholar
- Lieberman E, Hauert C, Nowak MA (2005) Evolutionary dynamics on graphs. Nature 433:312–316Google Scholar
- Lorenz K (1963) Das sogenannte Böse zur Naturgeschichte der Aggression. Verlag Dr. G Borotha-SchoelerGoogle Scholar
- Mariani P, Křivan V, MacKenzie BR, Mullon C (2016) The migration game in habitat network: the case of tuna. Theor Ecol 9:219–232CrossRefGoogle Scholar
- Maynard Smith J (1974) The theory of games and the evolution of animal conflicts. J Theor Biol 47:209–221MathSciNetCrossRefGoogle Scholar
- Maynard Smith J (1982) Evolution and the theory of games. Cambridge University Press, Cambridge, UKCrossRefGoogle Scholar
- Maynard Smith J (1991) Honest signalling: the Philip Sidney Game. Anim Behav 42:1034–1035CrossRefGoogle Scholar
- Maynard Smith J, Parker GA (1976) The logic of asymmetric contests. Anim Behav 24:159–175CrossRefGoogle Scholar
- Maynard Smith J, Price GR (1973) The logic of animal conflict. Nature 246:15–18CrossRefGoogle Scholar
- McNamara JM, Houston AI (1992) Risk-sensitive foraging: a review of the theory. Bull Math Biol 54:355–378CrossRefGoogle Scholar
- McPeek MA, Holt RD (1992) The evolution of dispersal in spatially and temporally varying environments. Am Nat 140:1010–1027CrossRefGoogle Scholar
- Mesterton-Gibbons M, Sherratt TN (2014) Bourgeois versus anti-Bourgeois: a model of infinite regress. Anim Behav 89:171–183CrossRefGoogle Scholar
- Milinski M (1979) An evolutionarily stable feeding strategy in sticklebacks. Zeitschrift für Tierpsychologie 51:36–40CrossRefGoogle Scholar
- Milinski M (1988) Games fish play: making decisions as a social forager. Trends Ecol Evol 3:325–330CrossRefGoogle Scholar
- Moran P (1958) Random processes in genetics. In: Mathematical proceedings of the Cambridge philosophical society, vol 54. Cambridge University Press, Cambridge, UK, pp 60–71Google Scholar
- Morris DW (1999) Has the ghost of competition passed? Evol Ecol Res 1:3–20Google Scholar
- Morris DW (2002) Measuring the Allee effect: positive density dependence in small mammals. Ecology 83:14–20CrossRefGoogle Scholar
- Nowak MA (2006) Five rules for the evolution of cooperation. Science 314:1560–1563CrossRefGoogle Scholar
- Nowak MA, May RM (1994) Superinfection and the evolution of parasite virulence. Proc R Soc Lond B 255:81–89CrossRefGoogle Scholar
- Parker GA (1978) Searching for mates. In: Krebs JR, Davies NB (eds) Behavioural ecology: an evolutionary approach. Blackwell, Oxford, pp 214–244Google Scholar
- Parker GA (1984) Evolutionarily stable strategies. In: Krebs JR, Davies NB (eds) Behavioural ecology: an evolutionary approach. Blackwell, Oxford, pp 30–61Google Scholar
- Parker GA, Thompson EA (1980) Dung fly struggles: a test of the war of attrition. Behav Ecol Sociobiol 7:37–44CrossRefGoogle Scholar
- Poundstone W (1992) Prisoner’s Dilemma. Oxford University Press, New YorkGoogle Scholar
- Selten R (1980) A note on evolutionarily stable strategies in asymmetrical animal conflicts. J Theor Biol 84:93–101MathSciNetCrossRefGoogle Scholar
- Sherratt TN, Mesterton-Gibbons M (2015) The evolution of respect for property. J Evol Biol 28:1185–1202CrossRefGoogle Scholar
- Sigmund K (2007) Punish or perish? Retaliation and collaboration among humans. Trends Ecol Evol 22:593–600CrossRefGoogle Scholar
- Sinervo B, Lively CM (1996) The rock-paper-scissors game and the evolution of alternative male strategies. Nature 380:240–243CrossRefGoogle Scholar
- Sirot E (2012) Negotiation may lead selfish individuals to cooperate: the example of the collective vigilance game. Proc R Soc B 279:2862–2867CrossRefGoogle Scholar
- Spencer H (1864) The Principles of biology. Williams and Norgate, LondonGoogle Scholar
- Stephens DW, Krebs JR (1986) Foraging theory. Princeton University Press, PrincetonGoogle Scholar
- Takeuchi Y (1996) Global dynamical properties of Lotka-Volterra systems. World Scientific Publishing Company, SingaporeCrossRefGoogle Scholar
- Taylor PD, Jonker LB (1978) Evolutionary stable strategies and game dynamics. Math Biosci 40:145–156MathSciNetCrossRefGoogle Scholar
- Vincent TL, Brown JS (2005) Evolutionary game theory, natural selection, and Darwinian dynamics. Cambridge University Press, Cambridge, UKCrossRefGoogle Scholar
- Webb JN, Houston AI, McNamara JM, Székely T (1999) Multiple patterns of parental care. Anim Behav 58:983–993CrossRefGoogle Scholar
- Wynne-Edwards VC (1962) Animal dispersion in relation to social behaviour. Oliver & Boyd, EdinburghGoogle Scholar
- Xu F, Cressman R, Křivan V (2014) Evolution of mobility in predator-prey systems. Discret Contin Dyn Syst Ser B 19:3397–3432MathSciNetCrossRefGoogle Scholar
- Zahavi A (1975) Mate selection–selection for a handicap. J Theor Biol 53:205–214CrossRefGoogle Scholar
- Zahavi A (1977) Cost of honesty–further remarks on handicap principle. J Theor Biol 67:603–605CrossRefGoogle Scholar