Evolutionary Games with Sequential Decisions and Dollar Auctions

3.1 Blocks and Unblocking

Here, we present the main four results for games without blocks. The games in this section are all examples of truly asymmetric finite extensive form games, and thus, following Theorem 4.5.3 of Cressman [10], there is a one-to-one correspondence between ESSs and strict Nash equilibria (NE). If we assume generic games and no equalities, then the solutions for each of our main results yields a unique strict NE, which is thus the unique ESS. Without this assumption, they are (perhaps non-strict) Nash equilibria only. The results are summarised in Table 1.

In general, a game may contain a number of blocks, say at time steps \(T_{b_1}, T_{b_2}, \ldots \). However, the game must stop at \(T_b=\min _j\{T_{b_j}\}\). Any game can be unblocked by considering a game with identical parameters except that \(T_b\) replaces T and (0, 1) (or (1, 0)) replaces \((a_1, a_2)\) if \(T_b\) is odd (or even). Here the outcome (i.e. the optimal strategies and eventual payoffs) of the unblocked game is the same as the original one.

3.2 Solving Games with No Blocks or Unblocked Games

Note that it is clear that at most one of (11) and (12) can hold.

We note that here the individual which would make large investments towards the end of the contest is the one that has to concede. Thus an individual that would make large investments at the beginning of the contest might never have to make them, as its opponent concedes immediately. We revisit this idea in Sect. 5.

4.1 Bounded Games

We thus have the following procedure for determining the optimal strategy.

Working backwards, as identified by 1–3) above, we find the expected future reward following Eqs. (21), (22) and (23). If this is ever 0, then an individual would concede, and so the expected future values when concession occurs will be either \((V_{1}, 0)\) or \((0, V_{2}\)). We can thus find the expected future rewards for all positions in the game. Now, working forwards, individuals should play until they reach the first value of \(A_{i,j}\) which takes value no greater than 0, at which point the individual in question should concede.

We note that Theorems 2, 3 and 4 follow from a special case of the above procedure, since as \(\gamma _{i,j}=0\) for all positions in the game, if ever \(A_{1,j}<=0\), then this leads to \(A_{2,k}>0\) and \(A_{1,k}<=0\) for \(k<j\), and a similar result for \(A_{2,j}\).

The results in this section thus follow a similar pattern to those in Sect. 3, and are strict NE/ESSs or NEs under the same conditions (the former if equalities of conditions can be ruled out, the latter otherwise). The bound on the game means that there is a fixed end point which we can work back from, and games will be divided into the same three categories; those where \(I_{1}\) concedes, those where \(I_{2}\) concedes, and those where both players invest until the end of the contest. In Sect. 3, however, the fact that games could only end by the concession of one of the players led to a deterministic process that meant that if \(I_{i}\) would concede, it would do so at the start of the contest. Here this is not so, because individuals doomed to eventually concede can hope that chance intervenes and ends the contest before that concession is necessary. We can see this in the example in Fig. 2.

Open image in new window

4.2 Unbounded Contests with Infinite Resources

Now we shall assume that \(\gamma _{i,j} <1\) for all i, j, so that T has no upper bound, and that resources are infinite. This case with unbounded contests is more complicated, as the game could potentially carry on indefinitely. Thus our game becomes an infinite extensive form game and the results from [10] do not extend to this case. In particular we shall make no claim about ESSs here (or in later sections where there are an infinite number of available strategies), and we claim only that solutions are Nash equilibria.

Alternatively, if (27) does not hold, and (28) holds, \(I_{2}\) does not invest (while \(I_{1}\) would invest if given a chance). This means that \(A_{1,j}=V_{1}\) and thus \(I_{1}\) should invest at time \(2j-1\). This yields \(A_{2,j-1}=0\) and hence that \(I_{2}\) should never invest.

6.1 A Dollar Auction with Full Memory and Infinite Resource

We start with the infinite resources case; this corresponds to the original idea from [43]. As we have discussed previously, for a biological population, we can consider this as a game with two distinct roles, where a strategy is a combination of what to do as the first player, and what to do as the second.

The strategy \(S_{j_1,j_2}\) represents invest in the first \(j_1-1\) (\(j_2-1\)) rounds and then concede as player \(I_1\) (\(I_2\)). It is clear that \(S_{1,j_2}\) is a Nash equilibrium for any strategy \(j_2\) where \(j_2 > 50\) and \(S_{j_1,1}\) is a Nash equilibrium for any \(j_1\) where \(j_1>51\), since in each case one player concedes immediately, but would need to invest over a dollar to “win” if it changed its strategy. Within a population comprised of individuals of this type, in Shubik’s terminology we would have a (credible) threat, which is enough to force an opponent to concede. This is very similar to the situation from Theorem 4, Sect. 4.2.4. See also [12] for a generalisation of this scenario.

6.2 A Dollar Auction with Full Memory and Finite Resource

Now consider the case with finite resources, \(R_{i}<\infty \) for \(i=1,2\).

Clearly here the game cannot carry on indefinitely and must be terminated by the concession of one of the players. We can reformulate this game in the form of a game with a fixed contest end from Sect. 3, but where the end time is such that \(R_{i}<C_{i}\) for \(i=1,2\), which can never be reached. Thus, following Theorem 2, the first player to run out of resources must concede immediately (and since we have applied Theorem 2, this is a strict NE/ESS). Unless both players have identical wealth, this will be the poorer of the two players. In the unlikely case that the players are equally rich, the first to play must concede immediately if they possess an even number of cents, otherwise the second to play must concede immediately. Note that for any two players with given wealth, swapping their positions in the game does not affect the result, unless their wealth is the same; this thus emphasises the symmetry of the game. As in the previous section, see [12] for a generalisation of this scenario.

6.3 A Dollar Auction with No Memory

We need to consider the best choice for a player in either of the two possible situations that it can make a choice, namely in the initial position S or in the D position, when considering the possible choices of the other player from the same positions. An individual’s optimal choices depend only upon the choices that will be made subsequently, and so clearly they cannot depend upon what the other player might do in the initial position. Thus they depend only upon the choice of the other player in position D. Without loss of generality, we can thus restrict ourselves to considering the two choices of \(I_1\) against the possible choice of \(I_2\).

Now, assume that player \(I_2\) employs a strategy \(p_{2,D}\), i.e. concedes at position D with probability \(p_{2,D}\). We are looking for an optimal response of player \(I_1\), i.e. we are looking for optimal values of probabilities to concede at stage S, \(p_{1,S}\), and at stage D, \(p_{1,D}\).

Similarly, if we assume that \(B_{1,U}\ge 2\), we get, by (51), \(B_{1,D}=B_{1,U}-2\) and thus, by (52), \(B_{1,U} = 102-\frac{2}{p_{2,D}}\). Consequently \(p_{2,D} \ge \frac{1}{50}\).

Also, it follows from (51) that whenever \(B_{1,U}<2\), it is optimal to pick \(p_{1,D}=1\) (i.e. always concede), (b) whenever \(B_{1,U}>2\), it is optimal to pick \(p_{1,D}=0\) (i.e. always invest), and (c) if \(B_{1,U}=2\), then any value of \(p_{1,D}\) yields \(B_{1,D}=0\) (and, from (50), it is optimal to have \(p_{1,S}=0\)).

Thus an individual should invest at the first step and, then, invest at each subsequent step with probability 49/50. Thus after the first step, each individual concedes following a geometric distribution with parameter \(p=1/50\). Note that this solution is similar to that of the war of attrition [2, 3, 29] where the solution follows an exponential distribution (in each case the stopping time follows a distribution with the lack of memory property, discrete in our case, continuous for the war of attrition, with a mean that yields an expected reward of zero).

7.1 Our Model and its Conclusions

In this paper we have considered an extension to the modelling of evolutionary contests where contests follow a sequence of decisions, rather that a single instantaneous decision made by each protagonist. This is a natural extension of previous models such as the Hawk–Dove game or the owner–intruder game, which can be regarded as overly simplistic, and not allowing for dynamic interaction between the contestants.

An important conclusion of our work is that, under many circumstances, individuals should fight to the end of the contest, or one should concede immediately. These are thus no more complex than the solutions of the original simple games from, e.g. [29], and provides in our view a strong justification for employing these simple models as approximations of more complex situations.

The exception to the above in our model is when the contest lasts a variable time, governed by random events as well as by strategic decisions of the players, and so an individual may continue to invest in the hope of a short contest, but when it becomes clear that the contest will not be stopped by some early random occurrence and be protracted, it may then concede. We note that in the model of Mesterton-Gibbons et al. [35] there was also a similar possibility of random events terminating the contest. In their model, if termination probability was low, then the classical Bourgeois solution to the owner–intruder model where the intruder always concedes was stable, but other solutions were possible for higher termination probabilities.

An interesting prediction is that if individuals have a choice of how much they invest at a given stage, they should often choose to invest heavily early on, even if this does not immediately help them gain the reward, but means that their remaining investment is sufficiently small that it demonstrates to their opponent that they will fight to the end of the contest, thus forcing that opponent to concede. This at first sight appears to fall foul of the concorde fallacy [46], namely the error of making decisions based upon past already lost investments, when only future returns should matter. However, it works in this case precisely because there is a direct correlation between past investments and required future investments, and choices are made by both players based upon the future values rather than the past ones.

The one example in our paper where there was a stochastic solution was in a special case of the classical dollar auction game discussed in [43], see also [9, 15, 24, 37] and for an overview see [44]. This is a game in economics with sequential investment which occurs in a similar way to our model, and we obtain the stochastic solution by making the individuals have no memories of past events, introduced as being in the spirit of the original problem. As soon as individuals remember the sequence of bids, with a fixed starting point, then the familiar solution of one individual conceding immediately appears.

7.2 Related Models

The main differences between our model and previous works can be summarised as follows. Firstly we introduce sequential games to a repeated evolutionary game scenario. This is different to standard time-based evolutionary contests such as the war of attrition in either continuous or discrete time [2, 3, 7] (also see Axelrod [1] and the vast-related literature for discrete simultaneous games more generally). Sequential evolutionary games with repeated stages have been relatively underexplored (but see Cressman [10] for important general theoretical work), although they are more common when each player has just a single decision [21]. These models have been extended to consider a whole population where there are many decisions, carried out simultaneously, but where only a subset of the individuals get to make a decision at any given time point, whether any given individual gets to make the choice depends upon its previous play (see e.g. [33]). Such sequential games are more common in classical game theory, for example with the dollar auction [43], which is the closest example to the type of game that we have considered. Below we explore some of the models most similar to ours.

The extended contests considered in our model are of a particular kind, namely contests where each individual has a set of potential exit points, and at each point it decides whether to continue, making a suitable investment, or concede the resource to its opponent. This is thus reminiscent of the classical war of attrition [2, 3], but with discrete stages which are sequential, and so our game does not have this aspect of symmetry of the war of attrition. This leads to a completely different character of solution, where the war of attrition has a probability density function over the stopping times, our individuals either concede with probability 1 or do not concede (although in the contests with an uncertain finish time, the time of the contest follows a probability distribution due to a natural random contest duration). We note that when other asymmetries, e.g. of reward values or costs, or even of perceived role [18, 31] are considered, mixed strategies similarly disappear (they are only maintained through, for example, uncertainty of role).

Whilst simultaneous choices are perhaps natural for the passive waiting game that is assumed in the war of attrition, more active contests might be better modelled using a sequential game as we consider here. We would contend that in many circumstances, such contests would indeed involve a sequence of moves and counter moves, and so often this asymmetry will be more representative of the real situation. This could happen in the fast timescale of actual fights, such as those involved in food stealing [5, 6, 23], but can also be seen in contests over a longer timescale, for example in the classical owner intruder type games, where the dispossessed animal may subsequently rechallenge the resident; in these cases it is the previous loser that has the first move [35]. An alternative type of contest where sequential moves would also be appropriate is the modelling of brood care and desertion, as modelled in Houston and McNamara [21]. Maynard Smith and Harper [30] argued that the standard constant cost situation of the war of attrition is rarely satisfied in biology, and more complex strategies than the simple memoryless “sit and wait” type as in the war of attrition would be more appropriate. This provides some justification for our type of model, where a sequence of distinct choices are made. Note that such complex animals interactions are demonstrated in Hack [17], who considers many tactics of crickets in an “active” war of attrition type fight. Even the classical real example of the war of attrition, between dung flies [38], involved an active contest between the participants, rather than mere waiting.

In the economic literature, the concept of an all pay auction is discussed, which is equivalent to the war of attrition (the dollar auction with two players is an all pay auction). Dekel et al. [12] considered an all pay auction with alternating moves, which featured a minimum investment level of \(\epsilon \), as in our game. They considered different values of the reward \(V_{1}\) and \( V_{2}\), including versions with finite and infinite budgets. This paper is actually a generalization of our game in Sects. 6.1 and 6.2 (our simplified versions were actually relatively straightforward, so we only discussed these briefly). Their work includes results which have similarities to our Theorem 6 for this different case. An alternative model of an all pay auction/ war of attrition is given in Hörner and Sahuguet [20], who consider a game with incomplete information where individuals can choose effort levels, and need to match efforts in sequence. Their model yields similar results to some of ours; one player immediately concedes, so that the stronger player wins at no cost. Note that this is also similar to the standard asymmetric war of attrition which this can be considered a version of, as we have alluded to above (e.g. see [18]). The original modelling work on choosing resource levels to invest in a contest was carried out by Tullock [45] (see [6] for a biological model of this type).

Maskin and Tirole [28] consider an alternating move scenario, again of dollar auction type; in particular related to the dollar auction with no memory. They consider a specific competition game, where payoffs are similar to as in a Cournot competition game (see also [14]). The alternating move system of the above games (and ours) are a special case of that in Libich and Stehlík [26], who consider a specific game (the “time inconsistency game”) but in a general move setting with a series of move points, not necessary alternating between the players (see also [25, 27]).

7.3 Future Work

We note, finally, that a number of other generalisations of the single stage game are possible, not just a sequence of choices of “continue to invest or concede”. It may be that individuals can adjust their level of aggression at any time in a contest, and this may directly affect the time of the end of the contest, and also the costs of both players to remain in the contest in terms of energy usage or risk of injury. More generally individuals might have a range of strategies and they might employ any given sequence of them, as in iterated games such as the iterated prisoner’s dilemma [1] or in the ownership games of [34, 35], with some reward received at every potential decision point. The game we have considered here is thus just one example (albeit, we would argue, an important one) of a game of this type, and there are a number of other interesting cases which can and should be considered as the subject of future research.