Evolutionary Games with Sequential Decisions and Dollar Auctions
 565 Downloads
 1 Citations
Abstract
Conflict occurs throughout the animal world. Such conflicts are often modelled by evolutionary games, where individual animals make a single decision each within the game. These decisions can be sequential, in either order, or simultaneous, and the outcome of the game can depend strongly upon which case is assumed to occur. Real conflicts are generally more complex, however. A fight over a territory, for instance, can involve a succession of different stages and, therefore, choices to be made by the protagonists. In this paper we thus introduce a method of modelling a more complex class of interactions, where each individual can make a sequence of decisions. We show that despite the inherent complexity, under certain assumptions, the resulting game often leads to the case where both animals fight to the fullest extent or where one concedes immediately, thus mirroring the outcomes of simpler single decision games. However, for other cases we see that the outcome is not so simple, and intermediate level contests can occur. This happens principally in cases where the duration of contests is uncertain, and partially governed by external factors which can bring the contest to a sudden end, such as the weather or the appearance of a predator. We thus develop a theory grounded in simple evolutionary models, but extending them in various important ways.
Keywords
Game theory Nash equilibrium Animal conflict Hawk–Dove game Extensive form games1 Introduction
Conflict is ubiquitous throughout the animal world, taking many different forms (see e.g. [22]). Such conflicts can be modelled using evolutionary game theory, where different behaviours within the conflict, for example be aggressive or passive, are represented by the game strategies [4, 19, 29, 31]. A common feature of evolutionary game theoretical models is that contests are often assumed to involve only a single decision by each participant. These decisions can be taken simultaneously, without knowledge of the other player’s choice, or in sequence, so that the second player observes the choice of the first before making its own decision. It is well known that whether the decisions are simultaneous or sequential, and if sequential which order the decisions are made, can fundamentally change the outcome of the game [21].
A classic example of the type of behaviour that we are interested in involves stags, which compete for territories and the associated mating opportunities with females. These contests can be complex, involving observation of the size of opponents, parallel walking, pushing and potentially lethal fighting. The contests are thus not just single events, but go through stages, where the protagonists can make decisions at different points. Such a situation was first modelled by Maynard Smith [29], Maynard Smith and Price [32] using the Hawk–Dove game, one of the most important games in biology. In the classical game, two individuals compete over a resource of value V, with two available strategies, the aggressive Hawk strategy and the passive Dove strategy. If one player plays Hawk and the other Dove, the Hawkplayer gains V and the Dove player 0. If both play Dove, the winner is selected at random, so each receives an average reward of V / 2. If both play Hawk, they fight until one receives an injury of cost C, the other then receiving the reward; the winner is decided at random, so the average reward is \((VC)/2\). Assuming that \(C>V\), if each simultaneously chooses a strategy, for indistinguishable individuals, the unique evolutionarily stable strategy (ESS) is a mixed strategy, where Hawk is played with probability V / C (and Dove with probability \(1V/C\)). For a sequential game, the unique ESS is that the first player plays Hawk and the second Dove, so that the first to play always gains the reward. Here the initial choice by the first player of Hawk commits that player to fight if the other also chooses Hawk, so that the second player knows that a choice of Hawk commits it to obtain the negative reward \((VC)/2\), forcing it to choose Dove. The Hawk–Dove game was extended to involve asymmetries with the owner–intruder game [31] and this has been elaborated more recently in a number of ways (see e.g. [42], for a review).
Real animal conflicts are often more complex than just involving a single decision by each combatant, and animals may make a sequence of decisions affected by the decisions made by themselves and their opponent, and the outcomes of those decisions, earlier in the contest. This is the case in the example of stags competing for a territory considered above. Games with a single decision are often a useful simplification for the more complex case, with the significant advantage of analytical simplicity, but the potential disadvantage of a lack of realism. Thus in the sequential Hawk–Dove game, the first player gains from the unrealistic assumption that its initial aggressive act commits it to fight with full aggression for the whole contest. The models of MestertonGibbons et al. [34, 35] consider a contest for a resource which can feature many stages of challenge and rechallenge. The model is based upon an iterated Hawk–Dove game, which has similar features to the iterated Prisoner’s Dilemma [1], in that optimal behaviour is heavily influenced by both the expectation of an opponent’s future strategy and also the likelihood of further contests.
An alternative situation that can lead to complex contests is animal foraging. Whlist foraging, animals may discover patches of resources, the value of which can vary considerably, and they must balance out the benefit of staying in the patch and continuing to feed, or to leave and seek a potentially richer patch elsewhere. If there is a single individual only and the patch becomes increasingly depleted though time, the individual does not have to consider the choices of an opponent (though the value of patches elsewhere may depend upon some population strategy of others), and it must simply pick the best time to leave [8]. However, if there is more than one individual, the choices of others are important. For a nondepleting patch with no movement costs, this relates to the Ideal Free Distribution [11, 13]. When resources become depleted, then individuals play a game where costs and benefits change through time, and this becomes a generalized war of attrition [3]. This is a development of the classical war of attrition [2] where two individuals compete for a reward by trying to outwait the other individual, see also [18, 31]. A specific example with parasitiod wasps visiting a patch of host larvae is given in [16].
In this paper we introduce a method of analysing an animal conflict over an indivisible valuable resource, as in the Hawk–Dove game, involving a potentially long sequence of decisions. These are binary in form, where individuals choose to persist in the contest (at some additional cost) or concede, concession allowing the opponent to take the resource. The costs may vary, depending upon the stage of the contest, for example if the nature of the contest changes (as in the case of the mentioned stags above). Each possess a level of resources which may prevent them from competing indefinitely. Similarly the contest may eventually stop due to some external influence, for example the arrival of night or a predator, and at that point the resource will be allocated to one of the individuals at random (generally not uniformly at random, however). We analyse our game under a number of scenarios, and also connect it to the dollar auction, a game from economics with some similar features.
2 The Model
Two individuals \(I_{1}\) and \(I_{2}\) compete for a valuable resource, which is worth \(V_{i}\) to individual i for \(i=1, 2\). The contests follow a sequence of alternating decisions, one at each time step, starting with \(I_{1}\), where the individual can remain in the game at a cost, or concede the resource to its opponent. As individuals have to pay a cost any time they decide to remain in the contest, we will also refer to the decision to stay in the contest as an investment and we will denote the cost of the \(j\hbox {th}\) investment of individual \(I_{i}\) by \(c_{i,j}\). We assume that \(c_{i,j}\ge \varepsilon \), for some \(0<\varepsilon <\min (a_{1}V_{1}, a_{2}V_{2})\) and all i, j. We assume that the game may continue for up to \(T\in {\mathbb {N}}\cup \{\infty \}\) time steps.
 1.
If \(I_{1}\) concedes at its \(j\hbox {th}\) decision, then \(I_{1}\) receives payoff \(C_{1,j1}\) and \(I_{2}\) receives \(V_{2}C_{2,j1}\).
 2.
If \(I_{2}\) concedes at its \(j\hbox {th}\) decision, then \(I_{1}\) receives payoff \(V_{1}C_{1,j}\) and \(I_{2}\) receives \(C_{2,j1}\).
 3.
If neither concedes and the game reaches the stopping point after step T, \(I_{1}\) receives payoff \(a_{1}V_{1}C_{1,J_1}\) and \(I_{2}\) receives \(a_{2}V_{2}C_{2,J_2}\).
We are interested in finding the optimal strategy for each player. Assuming that an individual can find itself in the \(I_{1}\) or \(I_{2}\) role, a strategy is then a combination of what to do as \(I_{1}\), and what to do as \(I_{2}\). Each player only has the chance to play if all previous investments of both players have been made, so that a pure strategy is simply the round in each case that an individual would concede, assuming it was reached, as \(I_{1}\) or \(I_{2}\). This can be reduced to a pair of integers (including infinity as an allowable choice), so that \(S_{j_1,j_2}\) represents invest in the first \(j_i1\) rounds and then concede as \(I_{i}\).
We shall consider a number of cases. In Sect. 3 we consider the case where all parameters \(V_{i}, c_{i,j}, a_{i}, R_{i}\) and T are a priori fixed and known to both players. In Sect. 4 we will consider parameters \(V_{i}, c_{i,j}, a_{i}, R_{i}\) fixed and known but will assume that T follows a random distribution, i.e. the stopping time is not a priori fixed, and so unknown to either of the players. In Sect. 5 we will consider the parameters \(V_{i}, a_{i}, R_{i}\) and T known and fixed, but we will allow individuals to choose the cost of staying in the game themselves (under the condition that if the game reaches step T, their total investment must reach an a priori known and fixed value \(C_i\)). Finally, in Sect. 6, we will consider the dollar auction game [43], as a special case of our game and in particular we will solve a variant of the dollar auction game where individuals are assumed to have no memory.
3 Fixed and Known Parameter Values
In this section consider the game \(\varGamma = \langle T, V_i,R_i,a_i, c_{i,j}\rangle \) described above where the values of \(V_{i}, c_{i,j}, a_{i}, R_{i}\) and \(T<\infty \) are a priori fixed and known to both players. Since the \(c_{i,j}\) and T are fixed and known, \(I_{i}\) knows how much it will have to invest to reach the end of the contest, which we denote by \(C_{1}=C_{1,J_1}\) and \(C_{2}=C_{2,J_2}\), for \(I_{1}\) and \(I_{2}\), respectively; here \(J_1\) and \(J_2\) are given by (1) and (2).
We can work from the end of the contests, using the standard backwards induction method (see e.g. [4]), to solve the game. We shall take this approach throughout the paper; when applying this method we shall use the shorthand “working backwards”. Note that, less often, we shall need to work from the start of the game, denoted as “working forwards”.
 1.
to concede, get no reward (and pay the cost \(C_{i,j1}\) after already investing \(j1\) times), or
 2.
to continue, pay the cost \(c_{i,j}\) (plus the cost \(C_{i,j1}\) of already investing \(j1\) times), and expect to get payoff \(B_{i,j}\).
We note that in Eqs. (4) and (5) we have assumed that if the expected future reward for the two choices of continuing and conceding are identical, then the individual concedes. If ties were (sometimes) resolved the other way, then our equations would be slightly different. Often in evolutionary games, we make the generic payoff assumption [4, 39], assuming that as payoffs occur from nature, following some natural variation, such ties never occur. This assumption is reasonable for games where there is a clear end point, which we consider in the rest of this section, Sect. 5 and the first part of Sect. 6.1. There are complications when there is not such an end point, as then the selection of mixed strategies (allowing concession at a given position with a probability between 0 and 1) can generate equalities. In cases with clear roles, such mixtures are unstable [41], so that the assumption is valid for the rest of the paper too, with the exception of Sect. 6.3, where extremely limited memory but infinite resources allows a game without roles and which carries on indefinitely (and where we see that such mixed strategies feature).
3.1 Blocks and Unblocking
Here, we present the main four results for games without blocks. The games in this section are all examples of truly asymmetric finite extensive form games, and thus, following Theorem 4.5.3 of Cressman [10], there is a onetoone correspondence between ESSs and strict Nash equilibria (NE). If we assume generic games and no equalities, then the solutions for each of our main results yields a unique strict NE, which is thus the unique ESS. Without this assumption, they are (perhaps nonstrict) Nash equilibria only. The results are summarised in Table 1.
Theorem 1
The first individual that would reach a concession point must concede immediately, i.e. before making its first investment.
Summary of the results for the case of fixed and known parameter values
Conditions  Outcomes 

R1, R2, P1, P2  \(I_{1}, I_{2}\) invest at every step 
\(R1, R2, P1, \lnot P2\)  \(I_{2}\) concedes immediately 
\(R1, R2, \lnot P1, P2\)  \(I_{1}\) concedes immediately 
\(R1, R2, \lnot P1, \lnot P2\) and (15) and (16) hold for some j  \(I_{1}\) concedes immediately 
\(R1, R2, \lnot P1, \lnot P2\) and neither (15) nor (16) hold for any j  \(I_{2}\) concedes immediately 
\(R1, \lnot R2\)  \(I_{2}\) concedes immediately 
\(\lnot R1, R2\)  \(I_{1}\) concedes immediately 
\(\lnot R1, \lnot R2\) and (11) holds for some j  \(I_{1}\) concedes immediately 
\(\lnot R1, \lnot R2\) and (12) holds for some j  \(I_{2}\) concedes immediately 
Proof
 1.
\(B_{2,j} =0\) (\(I_2\) reached a concession point),
 2.
\(B_{2,j1}>0\) (it was the first concession point for \(I_2\)),
 3.
\(B_{1,j}>0\) (\(I_1\) has not reached a concession point yet).
In general, a game may contain a number of blocks, say at time steps \(T_{b_1}, T_{b_2}, \ldots \). However, the game must stop at \(T_b=\min _j\{T_{b_j}\}\). Any game can be unblocked by considering a game with identical parameters except that \(T_b\) replaces T and (0, 1) (or (1, 0)) replaces \((a_1, a_2)\) if \(T_b\) is odd (or even). Here the outcome (i.e. the optimal strategies and eventual payoffs) of the unblocked game is the same as the original one.
3.2 Solving Games with No Blocks or Unblocked Games
Theorem 2
Note that it is clear that at most one of (11) and (12) can hold.
Proof
The first individual to run out of resources has reached a concession point. Assume that it is individual \(I_2\). As the game is an unblocked game, individual \(I_1\) has not yet reached a block or a point of concession by inability. Working backwards, there is no earlier point of concession by unprofitability.
Indeed, once \(B_{2,j}=0\), then \(A_{1,j}=V_1>0\). Since there is no block, \(V_1>c_{1,j}\) and so \(B_{1,j}=V_1c_{1,j}>0\), i.e. it is not a point of concession by unprofitability for \(I_1\). Also, \(A_{2,j1} = B_{2,j}=0\) and thus \(B_{2,j1} = 0\) and so on. Thus, \(I_2\) reaches a first concession point, and must thus concede before making the first investment by Theorem 1. \(\square \)
Theorem 3
In the unblocked game, if conditions R1 and R2 hold, but at least one of condition P1 and P2 does not, then at the start of the contest, the expected reward of remaining in the contest is nonpositive, conditional on the other player not conceding, for at least one player. Moreover, the player for which this expected payoff becomes positive last should concede immediately.
Proof
We note that here the individual which would make large investments towards the end of the contest is the one that has to concede. Thus an individual that would make large investments at the beginning of the contest might never have to make them, as its opponent concedes immediately. We revisit this idea in Sect. 5.
Theorem 4
If conditions R1, R2 and P1, P2 hold, then each player has positive expected future reward at the start of the contest no matter what the other player does, so both should continue to the end of the contest.
Proof
4 A Variable Stopping Time
Now suppose that T follows a random distribution. In particular, we shall assume that conditional upon not having stopped prior to this point, the probability that the game stops after the \(j\hbox {th}\) investment of \(I_{i}\) is \(\gamma _{i,j}\).
4.1 Bounded Games
 (1)
If there is a first block at time \(T_b\) (i.e. no block at time \(T<T_b\)), then, as we have seen in Sect. 3.1, this is equivalent to a game with a fixed endpoint \(T=T_b\) and \((a_{1}, a_{2})=(0,1)\) or (1,0), and we analyse the game following the method from Sect. 3. We note that in this case theoretically the game could continue indefinitely, if a player mistakenly invests at a block, but optimal play leads to termination of the game.
 (2)
If there is at least one \(\gamma _{i,j}\) which takes value 1, then T is bounded above. If \(\gamma _{I,J}\) is the earliest such \(\gamma \), then we have a fixed endpoint at \(T=2J+I2\).
 (3)
If \(R_{i}\) is finite for \(i=1, 2\), then there will be a point when one player must concede (assuming that the contest reaches this point), so the future values at the point of concession will either be \((V_{1}, 0)\) or \((0, V_{2}\)).
We thus have the following procedure for determining the optimal strategy.
Working backwards, as identified by 1–3) above, we find the expected future reward following Eqs. (21), (22) and (23). If this is ever 0, then an individual would concede, and so the expected future values when concession occurs will be either \((V_{1}, 0)\) or \((0, V_{2}\)). We can thus find the expected future rewards for all positions in the game. Now, working forwards, individuals should play until they reach the first value of \(A_{i,j}\) which takes value no greater than 0, at which point the individual in question should concede.
We note that Theorems 2, 3 and 4 follow from a special case of the above procedure, since as \(\gamma _{i,j}=0\) for all positions in the game, if ever \(A_{1,j}<=0\), then this leads to \(A_{2,k}>0\) and \(A_{1,k}<=0\) for \(k<j\), and a similar result for \(A_{2,j}\).
4.2 Unbounded Contests with Infinite Resources
Now we shall assume that \(\gamma _{i,j} <1\) for all i, j, so that T has no upper bound, and that resources are infinite. This case with unbounded contests is more complicated, as the game could potentially carry on indefinitely. Thus our game becomes an infinite extensive form game and the results from [10] do not extend to this case. In particular we shall make no claim about ESSs here (or in later sections where there are an infinite number of available strategies), and we claim only that solutions are Nash equilibria.
 1.
\(\gamma _{i,j}\) is constant, so that \(\gamma _{i,j}=\gamma \) for all i, j (this corresponds to T having a geometric distribution with parameter \(\gamma \)) and
 2.
costs of investments are constant for each individual, so that \(c_{i,j}=c_{i}\) for all j.
Alternatively, if (27) does not hold, and (28) holds, \(I_{2}\) does not invest (while \(I_{1}\) would invest if given a chance). This means that \(A_{1,j}=V_{1}\) and thus \(I_{1}\) should invest at time \(2j1\). This yields \(A_{2,j1}=0\) and hence that \(I_{2}\) should never invest.
4.2.1 Case 1
4.2.2 Case 2
4.2.3 Case 3
4.2.4 Case 4
So it is clear that one of the individuals must concede, but which should be the first to do so? In the biological case that underlies this paper, this can be solved using the idea of roles as previously discussed. If \(I_{1}\) chooses to (always) invest and \(I_{2}\) to concede immediately, the strategy \(S_{\infty ,1}\), then this is a strict NE. Similarly \(S_{1,\infty }\), \(I_{1}\) conceding immediately with \(I_{2}\) investing conditional upon \(I_{1}\) investing, is also a strict NE.
5 Individuals Can Choose How to Allocate Their Investments
In this game we again consider fixed total costs \(C_{i}\), but further develop the model by allowing individuals to choose how to split their investment, i.e. to choose \(c_{i,j}\) so that \(\sum c_{i,j} = C_i\). Here we set no maximum number of moves T, though there will be an effective maximum as we see below. Should the players pay a little or a large amount initially?
At first sight it would appear that any individual should simply either concede, or pay the smallest cost possible to be able to remain in the contest. What benefit is there in paying more? As we shall see below (see Theorem 6), however, there are situations where an individual should pay more than this minimum value.
For a moment, assume \(R_{i}<C_{i}\), for some \(i=1,2\), i.e. at least one individual, namely \(I_i\), does not have resources to invest to the end of the game. So the end of the game will not be reached and thus both individuals should needs to invest as little as possible, i.e. at any point they should either give up or will have to invest \(\varepsilon \). This turns the game into the game from Sect. 3 and we can solve it through the methods presented there.
If \(R_i<C_i\), then \(I_i\) cannot make more than \(j_i = \text {int}{(R_i/\varepsilon )}\) investments. Firstly assume that \(R_{i}<C_{i}\) for \(i=1,2\). If \(j_{1} \le j_{2}\) then \(I_{1}\) must concede immediately, otherwise \(I_{2}\) must concede immediately. If \(R_{i}<C_{i}\) for precisely one of \(i=1,2\), following arguments analogous to those of Theorem 2, the individual with insufficient resources must concede immediately.
Theorem 5
Theorem 6
Proof
Also, note that if (45) is not satisfied for an individual \(I_i\), then it has to invest at least \(a_iV_i\) to get to the end of the contest.
Note that (44) means that \(I_i\) can achieve (45) in \(j_i\) but not in \(j_i1\) investments (each investment would have to be almost \(V_i\)). So, if \(j_1>j_2\), then \(I_2\) can get to (45) faster than \(I_1\) and at that point \(I_1\) will be forced to concede, following an inductive argument analogous to that from the proof of Theorem 3. It follows that \(I_1\) should concede at the beginning. Consequently, \(I_2\) will never have to invest over \(V_2\) unless \(I_1\) invests over \(V_1\) first. Since \(I_1\) is not going to do that, \(I_1\) should concede at the beginning of the game.
By a similar argument, if \(j_1\le j_2\), then \(I_1\) can get to (45) faster than \(I_2\) and at that point \(I_2\) should concede. Thus, \(I_2\) should concede at the beginning. It remains to identify how much \(I_1\) should invest before \(I_2\) gives up. Note that after an investment of \(c_{1,1}\) by \(I_1\), the game can be seen as the game with changed roles, \(I_2\) starts first and has to invest \(C_2\), \(I_1\) plays second and has to invest \(C_1c_{1,1}\). Consequently, if \(j_1<j_2\), \(I_1\) just needs to invest \(\varepsilon \) (because even after \(I_2\) invests \(V_2\) in the first round, \(I_1\) can get to (45) faster than \(I_2\)). However, when \(j_1=j_2\), then \(I_1\) needs to invest just over \(C_{1}(j_{1}1)V_{1}a_{1}V_{1}\), to make sure that when \(I_2\) invests close to \(V_2\), \(I_1\) still gets to (45) faster then \(I_2\). \(\square \)
Remark 1
If, within the setting of Theorem 6, \(j_1=j_2\), the exact amount of how much \(I_1\) should invest at the beginning may vary. If \(a_i=1/2, C_i\approx V_i\), then \(I_1\) needs to invest a bit over \(V_1/2\). When \(a_i=1/2\) and \(C_i=3/2*V_i\delta \) (for some small \(\delta >0\)), then \(I_1\) needs to invest almost \(V_1\), but when \(C_i=3/2*V_i+\delta \), then \(I_1\) just needs to invest “a little bit” over \(\delta \).
Remark 2
Consider the case when \(a_{i}V_{i} \le C_{i}\) for \(i=1,2\), \(C_1=C_2, a_1=a_2=1/2\) and \(V_1>V_2\). Then the resource is more valuable to individual \(I_1\) and according to Theorem 6, \(I_2\) should concede immediately. Similarly, if \(a_{i}V_{i} \le C_{i}\) for \(i=1,2\), \(C_1=C_2, V_1=V_2\) and \(a_1>a_2\), then \(I_1\) is more likely to win the resource at the end of the game than \(I_2\) and according to Theorem 6, \(I_2\) should concede immediately.
Remark 3
The advantage of a significant early investment can be clearly seen from the following example. Suppose that \(a_{1}=a_{2}=1/2, C_{1}=0.7V, C_{2}=0.6V, \varepsilon =0.11 V\).
The expected reward for reaching the end of the contest is V / 2. Thus any individual whose remaining investment falls below V / 2 should continue to invest rather than concede. Further, if an individual to choose has a remaining investment of greater than V / 2 whilst its opponent has less than this, it knows its opponent will continue to the end, and so must concede.

If it concedes, it of course receives 0.

If it invests less than 0.2V it will still have more than 0.5V to invest. Thus player 2 should keep investing, its minimum investment would take it within 0.5V so the first player would then have to concede. Thus this choice yields player 1 a negative return.

If it invests more than 0.2V, it will then have remaining investment under 0.5V, so that player 2 will then have to concede, thus the expected return for player 1 will be V minus this initial investment.
6 The Dollar Auction
The classical dollar auction was introduced by Shubik [43]. In this game a number of players are invited by an auctioneer to bid for a dollar. In Shubik’s game there were potentially many players, and any bids were allowed (although these had to be in a multiple of 5 cents). The person who won the auction got the dollar minus the value of their winning bid, but crucially the second highest bidder also had to pay the value of their own bid. If bids jump in small amounts, at any stage a player has the potential of winning the “large” prize of a dollar for a small extra outlay. If a player was convinced that their next bid would be the last, then any bid up to (but not including) a dollar is worthwhile. An initial dollar bid is clearly a bad idea, as 0 is obtained if no other player bids, and less than 0 otherwise (this would be a nonoptimal further bid, but we can assume this will happen with a very small probability, using the classical idea of the “trembling hand” [36, 40]). If any player is convinced that their next bid would be the last, then any bid up to (but not including) a dollar is worthwhile; however, for a player that is currently second highest bidder, any bid, including a bid of up to (but not including) a dollar above their previous bid is worthwhile (if their bid will be the last one). If a player was convinced that some other player will keep bidding then it is not worth bidding themselves. There seems no sensible solution to such a game, at first sight.
We shall consider a variant of the dollar auction, where there are only two players, who bid alternately (with one nominated to start), and are only allowed to raise the previous bid by one cent (or bid one cent if it is the start of the auction). We thus have the game as described in Sect. 4 with \(c_{1,1}=1, c_{i,j}=2\) otherwise, \(\gamma _{i,j}=0\), \(V_{1}=V_{2}=100\). We shall first consider two cases a) with infinite resources, \(R_{i}=\infty \) for \(i=1,2\), and b) with finite resources. Then, we will consider the dollar auction with no memory.
6.1 A Dollar Auction with Full Memory and Infinite Resource
We start with the infinite resources case; this corresponds to the original idea from [43]. As we have discussed previously, for a biological population, we can consider this as a game with two distinct roles, where a strategy is a combination of what to do as the first player, and what to do as the second.
The strategy \(S_{j_1,j_2}\) represents invest in the first \(j_11\) (\(j_21\)) rounds and then concede as player \(I_1\) (\(I_2\)). It is clear that \(S_{1,j_2}\) is a Nash equilibrium for any strategy \(j_2\) where \(j_2 > 50\) and \(S_{j_1,1}\) is a Nash equilibrium for any \(j_1\) where \(j_1>51\), since in each case one player concedes immediately, but would need to invest over a dollar to “win” if it changed its strategy. Within a population comprised of individuals of this type, in Shubik’s terminology we would have a (credible) threat, which is enough to force an opponent to concede. This is very similar to the situation from Theorem 4, Sect. 4.2.4. See also [12] for a generalisation of this scenario.
6.2 A Dollar Auction with Full Memory and Finite Resource
Now consider the case with finite resources, \(R_{i}<\infty \) for \(i=1,2\).
Clearly here the game cannot carry on indefinitely and must be terminated by the concession of one of the players. We can reformulate this game in the form of a game with a fixed contest end from Sect. 3, but where the end time is such that \(R_{i}<C_{i}\) for \(i=1,2\), which can never be reached. Thus, following Theorem 2, the first player to run out of resources must concede immediately (and since we have applied Theorem 2, this is a strict NE/ESS). Unless both players have identical wealth, this will be the poorer of the two players. In the unlikely case that the players are equally rich, the first to play must concede immediately if they possess an even number of cents, otherwise the second to play must concede immediately. Note that for any two players with given wealth, swapping their positions in the game does not affect the result, unless their wealth is the same; this thus emphasises the symmetry of the game. As in the previous section, see [12] for a generalisation of this scenario.
6.3 A Dollar Auction with No Memory
 1.
S: A player (either \(I_{1}\) or \(I_2\) selected at random) is about to start the game by potentially investing a cent, or conceding;
 2.
D: A player (either \(I_{1}\) or \(I_{2}\)) is about to make an investment after the game has started, which must be of two cents, given it must go from one down to one up, or to concede;
 3.
U: A player (either \(I_{1}\) or \(I_{2}\)) is a cent up, and waiting to see if its opponent keeps investing.
We need to consider the best choice for a player in either of the two possible situations that it can make a choice, namely in the initial position S or in the D position, when considering the possible choices of the other player from the same positions. An individual’s optimal choices depend only upon the choices that will be made subsequently, and so clearly they cannot depend upon what the other player might do in the initial position. Thus they depend only upon the choice of the other player in position D. Without loss of generality, we can thus restrict ourselves to considering the two choices of \(I_1\) against the possible choice of \(I_2\).
Now, assume that player \(I_2\) employs a strategy \(p_{2,D}\), i.e. concedes at position D with probability \(p_{2,D}\). We are looking for an optimal response of player \(I_1\), i.e. we are looking for optimal values of probabilities to concede at stage S, \(p_{1,S}\), and at stage D, \(p_{1,D}\).
Similarly, if we assume that \(B_{1,U}\ge 2\), we get, by (51), \(B_{1,D}=B_{1,U}2\) and thus, by (52), \(B_{1,U} = 102\frac{2}{p_{2,D}}\). Consequently \(p_{2,D} \ge \frac{1}{50}\).
Also, it follows from (51) that whenever \(B_{1,U}<2\), it is optimal to pick \(p_{1,D}=1\) (i.e. always concede), (b) whenever \(B_{1,U}>2\), it is optimal to pick \(p_{1,D}=0\) (i.e. always invest), and (c) if \(B_{1,U}=2\), then any value of \(p_{1,D}\) yields \(B_{1,D}=0\) (and, from (50), it is optimal to have \(p_{1,S}=0\)).

if \(p_{2,D}<\frac{1}{50}\), then the optimal response is \(p_{1,D}=1\),

if \(p_{2,D}>\frac{1}{50}\), then the optimal response is \(p_{1,D}=0\),

if \(p_{2,D}=\frac{1}{50}\), then the optimal response is any value for \(p_{1,D}\).
Thus an individual should invest at the first step and, then, invest at each subsequent step with probability 49/50. Thus after the first step, each individual concedes following a geometric distribution with parameter \(p=1/50\). Note that this solution is similar to that of the war of attrition [2, 3, 29] where the solution follows an exponential distribution (in each case the stopping time follows a distribution with the lack of memory property, discrete in our case, continuous for the war of attrition, with a mean that yields an expected reward of zero).
7 Discussion
7.1 Our Model and its Conclusions
In this paper we have considered an extension to the modelling of evolutionary contests where contests follow a sequence of decisions, rather that a single instantaneous decision made by each protagonist. This is a natural extension of previous models such as the Hawk–Dove game or the owner–intruder game, which can be regarded as overly simplistic, and not allowing for dynamic interaction between the contestants.
An important conclusion of our work is that, under many circumstances, individuals should fight to the end of the contest, or one should concede immediately. These are thus no more complex than the solutions of the original simple games from, e.g. [29], and provides in our view a strong justification for employing these simple models as approximations of more complex situations.
The exception to the above in our model is when the contest lasts a variable time, governed by random events as well as by strategic decisions of the players, and so an individual may continue to invest in the hope of a short contest, but when it becomes clear that the contest will not be stopped by some early random occurrence and be protracted, it may then concede. We note that in the model of MestertonGibbons et al. [35] there was also a similar possibility of random events terminating the contest. In their model, if termination probability was low, then the classical Bourgeois solution to the owner–intruder model where the intruder always concedes was stable, but other solutions were possible for higher termination probabilities.
An interesting prediction is that if individuals have a choice of how much they invest at a given stage, they should often choose to invest heavily early on, even if this does not immediately help them gain the reward, but means that their remaining investment is sufficiently small that it demonstrates to their opponent that they will fight to the end of the contest, thus forcing that opponent to concede. This at first sight appears to fall foul of the concorde fallacy [46], namely the error of making decisions based upon past already lost investments, when only future returns should matter. However, it works in this case precisely because there is a direct correlation between past investments and required future investments, and choices are made by both players based upon the future values rather than the past ones.
The one example in our paper where there was a stochastic solution was in a special case of the classical dollar auction game discussed in [43], see also [9, 15, 24, 37] and for an overview see [44]. This is a game in economics with sequential investment which occurs in a similar way to our model, and we obtain the stochastic solution by making the individuals have no memories of past events, introduced as being in the spirit of the original problem. As soon as individuals remember the sequence of bids, with a fixed starting point, then the familiar solution of one individual conceding immediately appears.
7.2 Related Models
The main differences between our model and previous works can be summarised as follows. Firstly we introduce sequential games to a repeated evolutionary game scenario. This is different to standard timebased evolutionary contests such as the war of attrition in either continuous or discrete time [2, 3, 7] (also see Axelrod [1] and the vastrelated literature for discrete simultaneous games more generally). Sequential evolutionary games with repeated stages have been relatively underexplored (but see Cressman [10] for important general theoretical work), although they are more common when each player has just a single decision [21]. These models have been extended to consider a whole population where there are many decisions, carried out simultaneously, but where only a subset of the individuals get to make a decision at any given time point, whether any given individual gets to make the choice depends upon its previous play (see e.g. [33]). Such sequential games are more common in classical game theory, for example with the dollar auction [43], which is the closest example to the type of game that we have considered. Below we explore some of the models most similar to ours.
The extended contests considered in our model are of a particular kind, namely contests where each individual has a set of potential exit points, and at each point it decides whether to continue, making a suitable investment, or concede the resource to its opponent. This is thus reminiscent of the classical war of attrition [2, 3], but with discrete stages which are sequential, and so our game does not have this aspect of symmetry of the war of attrition. This leads to a completely different character of solution, where the war of attrition has a probability density function over the stopping times, our individuals either concede with probability 1 or do not concede (although in the contests with an uncertain finish time, the time of the contest follows a probability distribution due to a natural random contest duration). We note that when other asymmetries, e.g. of reward values or costs, or even of perceived role [18, 31] are considered, mixed strategies similarly disappear (they are only maintained through, for example, uncertainty of role).
Whilst simultaneous choices are perhaps natural for the passive waiting game that is assumed in the war of attrition, more active contests might be better modelled using a sequential game as we consider here. We would contend that in many circumstances, such contests would indeed involve a sequence of moves and counter moves, and so often this asymmetry will be more representative of the real situation. This could happen in the fast timescale of actual fights, such as those involved in food stealing [5, 6, 23], but can also be seen in contests over a longer timescale, for example in the classical owner intruder type games, where the dispossessed animal may subsequently rechallenge the resident; in these cases it is the previous loser that has the first move [35]. An alternative type of contest where sequential moves would also be appropriate is the modelling of brood care and desertion, as modelled in Houston and McNamara [21]. Maynard Smith and Harper [30] argued that the standard constant cost situation of the war of attrition is rarely satisfied in biology, and more complex strategies than the simple memoryless “sit and wait” type as in the war of attrition would be more appropriate. This provides some justification for our type of model, where a sequence of distinct choices are made. Note that such complex animals interactions are demonstrated in Hack [17], who considers many tactics of crickets in an “active” war of attrition type fight. Even the classical real example of the war of attrition, between dung flies [38], involved an active contest between the participants, rather than mere waiting.
In the economic literature, the concept of an all pay auction is discussed, which is equivalent to the war of attrition (the dollar auction with two players is an all pay auction). Dekel et al. [12] considered an all pay auction with alternating moves, which featured a minimum investment level of \(\epsilon \), as in our game. They considered different values of the reward \(V_{1}\) and \( V_{2}\), including versions with finite and infinite budgets. This paper is actually a generalization of our game in Sects. 6.1 and 6.2 (our simplified versions were actually relatively straightforward, so we only discussed these briefly). Their work includes results which have similarities to our Theorem 6 for this different case. An alternative model of an all pay auction/ war of attrition is given in Hörner and Sahuguet [20], who consider a game with incomplete information where individuals can choose effort levels, and need to match efforts in sequence. Their model yields similar results to some of ours; one player immediately concedes, so that the stronger player wins at no cost. Note that this is also similar to the standard asymmetric war of attrition which this can be considered a version of, as we have alluded to above (e.g. see [18]). The original modelling work on choosing resource levels to invest in a contest was carried out by Tullock [45] (see [6] for a biological model of this type).
Maskin and Tirole [28] consider an alternating move scenario, again of dollar auction type; in particular related to the dollar auction with no memory. They consider a specific competition game, where payoffs are similar to as in a Cournot competition game (see also [14]). The alternating move system of the above games (and ours) are a special case of that in Libich and Stehlík [26], who consider a specific game (the “time inconsistency game”) but in a general move setting with a series of move points, not necessary alternating between the players (see also [25, 27]).
7.3 Future Work
We note, finally, that a number of other generalisations of the single stage game are possible, not just a sequence of choices of “continue to invest or concede”. It may be that individuals can adjust their level of aggression at any time in a contest, and this may directly affect the time of the end of the contest, and also the costs of both players to remain in the contest in terms of energy usage or risk of injury. More generally individuals might have a range of strategies and they might employ any given sequence of them, as in iterated games such as the iterated prisoner’s dilemma [1] or in the ownership games of [34, 35], with some reward received at every potential decision point. The game we have considered here is thus just one example (albeit, we would argue, an important one) of a game of this type, and there are a number of other interesting cases which can and should be considered as the subject of future research.
Notes
Acknowledgements
The research was supported by Simons Foundation Grant 245400 to JR and by funding from the European Unions Horizon 2020 research and innovation programme under the Marie SkodowskaCurie Grant Agreement No 690817.
References
 1.Axelrod R (1981) The emergence of cooperation among egoists. Am Polit Sci Rev 75(02):306–318CrossRefGoogle Scholar
 2.Bishop D, Cannings C (1976) Models of animal conflict. Adv Appl Probab 8:616–621CrossRefMATHGoogle Scholar
 3.Bishop D, Cannings C (1978) A generalized war of attrition. J Theor Biol 70(1):85–124MathSciNetCrossRefGoogle Scholar
 4.Broom M, Rychtář J (2013) Gametheoretical models in biology. CRC Press, Boca RatonMATHGoogle Scholar
 5.Broom M, Luther R, Ruxton G (2004) Resistance is useless?—Extensions to the game theory of kleptoparasitism. Bull Math Biol 66(6):1645–1658MathSciNetCrossRefMATHGoogle Scholar
 6.Broom M, Johanis M, Rychtář J (2015) The effect of fight cost structure on fighting behaviour. J Math Biol 71(4):979–996MathSciNetCrossRefMATHGoogle Scholar
 7.Cannings C, Whittaker J (1995) The finite horizon war of attrition. Games Econ Behav 11(2):193–236MathSciNetCrossRefMATHGoogle Scholar
 8.Charnov EL (1976) Optimal foraging, the marginal value theorem. Theor Popul Biol 9(2):129–136CrossRefMATHGoogle Scholar
 9.Colman AM (2014) Game theory and experimental games: the study of strategic interaction. Elsevier, AmsterdamGoogle Scholar
 10.Cressman R (2003) Evolutionary dynamics and extensive form games, vol 5. MIT Press, CambridgeMATHGoogle Scholar
 11.Cressman R, Křivan V, Garay J (2004) Ideal free distributions, evolutionary games, and population dynamics in multiplespecies environments. Am Nat 164(4):473–489Google Scholar
 12.Dekel E, Jackson M, Wolinsky A (2007) Jump bidding and budget constraints in allpay auctions and wars of attrition. Discussion Paper: Center for Mathematical Studies in Economics and Management Science, Northwestern UniversityGoogle Scholar
 13.Fretwell SD, Lucas H (1969) On territorial behavior and other factors influencing habitat distribution in birds. Acta Biotheor 19(1):16–36CrossRefGoogle Scholar
 14.Fudenberg D, Tirole J (1991) Game theory. MIT Press, CambridgeMATHGoogle Scholar
 15.Greenberg J (1990) The theory of social situations: an alternative gametheoretic approach. Cambridge University Press, CambridgeCrossRefMATHGoogle Scholar
 16.Haccou P, Glaizot O, Cannings C (2003) Patch leaving strategies and superparasitism: an asymmetric generalized war of attrition. J Theor Biol 225(1):77–89MathSciNetCrossRefGoogle Scholar
 17.Hack MA (1997) The energetic costs of fighting in the house cricket, Acheta domesticus l. Behav Ecol 8(1):28–36MathSciNetCrossRefGoogle Scholar
 18.Hammerstein P, Parker G (1982) The asymmetric war of attrition. J Theor Biol 96:647–682MathSciNetCrossRefGoogle Scholar
 19.Hofbauer J, Sigmund K (1998) Evolutionary games and population dynamics. Cambridge University Press, CambridgeCrossRefMATHGoogle Scholar
 20.Hörner J, Sahuguet N (2011) A war of attrition with endogenous effort levels. Econ Theory 47(1):1–27MathSciNetCrossRefMATHGoogle Scholar
 21.Houston AI, McNamara JM (1999) Models of adaptive behaviour: an approach based on state. Cambridge University Press, CambridgeGoogle Scholar
 22.Huntingford FA (2013) Animal conflict. Springer, BerlinGoogle Scholar
 23.Iyengar E (2008) Kleptoparasitic interactions throughout the animal kingdom and a reevaluation, based on participant mobility, of the conditions promoting the evolution of kleptoparasitism. Biol J Linn Soc 93(4):745–762CrossRefGoogle Scholar
 24.Leininger W (1989) Escalation and cooperation in conflict situations the dollar auction revisited. J Confl Resolut 33(2):231–254CrossRefGoogle Scholar
 25.Leininger W (1991) Patent competition, rent dissipation, and the persistence of monopoly: the role of research budgets. J Econ Theory 53(1):146–172MathSciNetCrossRefMATHGoogle Scholar
 26.Libich J, Stehlík P (2008) Macroeconomic games on time scales. Dyn Syst Appl 5:274–278MathSciNetMATHGoogle Scholar
 27.Libich J, Stehlík P (2010) Incorporating rigidity and commitment in the timing structure of macroeconomic games. Econ Model 27(3):767–781CrossRefGoogle Scholar
 28.Maskin E, Tirole J (1988) A theory of dynamic oligopoly, I: overview and quantity competition with large fixed costs. Econometrica 56(3):549–569MathSciNetCrossRefMATHGoogle Scholar
 29.Maynard Smith J (1982) Evolution and the theory of games. Cambridge University Press, CambridgeMATHGoogle Scholar
 30.Maynard Smith J, Harper D (2003) Animal signals. Oxford University Press, OxfordGoogle Scholar
 31.Maynard Smith J, Parker G (1976) The logic of asymmetric contests. Anim Behav 24:159–175CrossRefGoogle Scholar
 32.Maynard Smith J, Price G (1973) The logic of animal conflict. Nature 246:15–18CrossRefMATHGoogle Scholar
 33.McNamara JM, Szekely T, Webb JN, Houston AI (2000) A dynamic gametheoretic model of parental care. J Theor Biol 205(4):605–623CrossRefGoogle Scholar
 34.MestertonGibbons M, Karabiyik T, Sherratt TN (2014) The iterated Hawk–Dove game revisited: the effect of ownership uncertainty on bourgeois as a pure convention. Dyn Games Appl 4(4):407–431MathSciNetCrossRefMATHGoogle Scholar
 35.MestertonGibbons M, Karabiyik T, Sherratt TN (2016) On the evolution of partial respect for ownership. Dyn Games Appl 6(3):359–395MathSciNetCrossRefMATHGoogle Scholar
 36.Myerson RB (1978) Refinements of the nash equilibrium concept. Int J Game Theory 7(2):73–80MathSciNetCrossRefMATHGoogle Scholar
 37.O’Neill B (1986) International escalation and the dollar auction. J Confl Resolut 30(1):33–50CrossRefGoogle Scholar
 38.Parker G, Thompson EA (1980) Dung fly struggles: a test of the war of attrition. Behav Ecol Sociobiol 7(1):37–44CrossRefGoogle Scholar
 39.Samuelson L (1997) Evolutionary games and equilibrium selection. MIT Press, CambridgeMATHGoogle Scholar
 40.Selten R (1975) Reexamination of the perfectness concept for equilibrium points in extensive games. Int J Game Theory 4(1):25–55MathSciNetCrossRefMATHGoogle Scholar
 41.Selten R (1980) A note on evolutionarily stable strategies in asymmetric animal conflicts. J Theor Biol 84(1):93–101MathSciNetCrossRefGoogle Scholar
 42.Sherratt T, MestertonGibbons M (2015) The evolution of respect for property. J Evol Biol 28(6):1185–1202CrossRefGoogle Scholar
 43.Shubik M (1971) The dollar auction game: a paradox in noncooperative behavior and escalation. J Confl Resolut 15(1):109–111MathSciNetCrossRefGoogle Scholar
 44.Stark RM, Rothkopf MH (1979) Competitive bidding: a comprehensive bibliography. Oper Res 27(2):364–390MathSciNetCrossRefMATHGoogle Scholar
 45.Tullock G (1980) Efficient rentseeking. In: Buchanan J, Tollison R, Tullock G (eds) Toward a theory of the rentseeking society. Texas A&M Press, Texas, pp 269–282Google Scholar
 46.Weatherhead PJ (1979) Do savannah sparrows commit the concorde fallacy? Behav Ecol Sociobiol 5(4):373–381CrossRefGoogle Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.