Introduction

Abstract

Strategic behavior in the human and social world has been increasingly recognized in theory and practice. From a decision-maker’s perspective, it becomes important to consider and accommodate the interdependencies and interactions of human decisions. As a result, game theory has emerged as a fundamental instrument in pure and applied research. In addition, since human beings live in time and decisions generally lead to effects over time, most of the strategic interactions are dynamic rather than static. One particularly complex and fruitful branch of game theory is dynamic games, which investigates interactive decision making over time. Differential (continuous-time dynamic) games were originated by Rufus Isaacs (1951, published in 1965). Discrete-time dynamic games (usually referred to as dynamic games) are multi-stage counterparts of differential games using Bellman’s (1957) discrete-time dynamic programming technique to obtain their solutions. Since then research involving continuous-time and discrete-time dynamic games continue to grow in a large number of fields and studies including economics, engineering, business, biology, mathematics, environmental studies, and social and political sciences. Rather exhaustive collections of differential and dynamic game applications in economics and business can be found in Dockner et al. (2000), Jørgensen and Zaccour (2004) and Long (2010).

Appendix: Numerical Demonstration of Subgame Consistency

To demonstrate the notion of subgame consistency in cooperative dynamic games in a clear way with minimal technical requirement, we consider a simple numerical example with two players in a 4-stage game horizon. The players derive incomes in each stage and there is a state variable x _t , for $t\in \left\{1,2,3,4\right\}$. The players’ incomes and the values of the state variable are affected by the actions of these players.

1.1.1 Non-cooperative Outcome

In a non-cooperative scenario, the players’ incomes in each stage, the values of the state variable and the players’ payoffs are summarized in Table 1.1 below.

Table 1.1 Players' payoffs and stage incomes and the state path under non-cooperation

Note that the payoff of a player at stage t refers to the sum of stage incomes that he will receive from stage t to the last stage of the game (that is stage 4). Now consider the case when the players agree to cooperate and enhance their joint incomes.

1.1.2 Cooperation and Optimality Principle

Consider the case where the players agree to act cooperative under an optimality principle: “Maximize the joint payoff and share the cooperative payoff proportional to their non-cooperative payoffs”. If any side decides to opt out the cooperative plan will be cancelled and the players will revert to playing non-cooperatively. Acting cooperatively in maximizing the joint payoff, the players’ stage cooperative incomes and the values of the state variable are given in Table 1.2 below.

Table 1.2 Players' stage cooperative incomes and the cooperative state path

Summing the stage incomes of player 1 and that of player 2 yields the cooperative joint stage income. The cooperative joint incomes in each stage, the values of the state variable and the maximized joint payoffs are given in Table 1.3 below.

Table 1.3 Cooperative joint incomes in each stage, the cooperative state path and maximized joint payoffs

Let ξ ⁱ(1, x ^*₁ ) denote the payoff that player i will receive in stage 1 under cooperation for $i\in \left\{1,2\right\}$. At initial stage 1, according to the agreed-upon optimality princi ple the players would share the cooperative payoff proportional to their non-cooperative payoffs the cooperative payoff, that is:

$$ \begin{array}{cc}\hfill {\xi}^i\left(1,{x}_1^{*}\right)=\frac{V^i\left(1,{x}_1^{*}\right)}{V^1\left(1,{x}_1^{*}\right)+{V}^2\left(1,{x}_1^{*}\right)}W\left(1,{x}_1^{*}\right)\hfill & \hfill \mathrm{f}\mathrm{o}\mathrm{r}\;i\in \left\{1,2\right\}\hfill \end{array}. $$

Using Tables 1.1 and 1.3, the cooperative payoffs of players 1 and 2 at stage 1 are respectively:

$$ \begin{array}{l}{\xi}^1\left(1,{x}_1^{*}\right)=\frac{350}{350+200}750=477.273\kern0.24em \mathrm{and}\\ {}{\xi}^2\left(1,{x}_1^{*}\right)=\frac{200}{350+200}750=272.727.\end{array} $$

(1.1)

Hence under the optimality principle agreed-upon in the initial stage, player 1 is expected to realize a payoff ${\xi}^1\left(1,{x}_1^{*}\right)=477.273$ and player 2 is expected to realize a payoff ${\xi}^2\left(1,{x}_1^{*}\right)=272.727$. However, according to Table 1.2, the joint payoff maximization scheme would yield a payoff of 510 to player 1 and a payoff of 240 to player 2. A payoff distribution procedure (PDP) has to be designed so that the payoffs according to the optimality principle can be realized.

1.1.3 Notion of Subgame Consistency

In a multi-stage game, a stringent condition for the sustainability of cooperation is the notion of subgame consistency. The idea of subgame consistency is that the specific agreed-upon optimality principle at the initial time must be maintained at subsequent time throughout the game horizon along the optimal state trajectory. Let ξ ⁱ(t, x ^* _t ) denote the payoff that player i will receive in stage t under cooperation for $i\in \left\{1,2\right\}$ and $t\in \left\{1,2,3,4\right\}$. At subsequent stage $t\in \left\{2,3,4\right\}$ after the initial stage, according to the agreed-upon optimality principle: “Maximize the joint payoff and share the cooperative payoff proportional to their non-cooperative payoffs”, player i’s payoff under cooperation in stage t should be

$$ \begin{array}{cc}\hfill {\xi}^i\left(t,{x}_t^{*}\right)=\frac{V^i\left(t,{x}_t^{*}\right)}{V^1\left(t,{x}_t^{*}\right)+{V}^2\left(t,{x}_t^{*}\right)}W\left(t,{x}_t^{*}\right)\hfill & \hfill \begin{array}{cc}\hfill \mathrm{f}\mathrm{o}\mathrm{r}\ i\in \left\{1,2\right\}\hfill & \hfill \mathrm{and}\kern0.3em t\in \left\{2,3,4\right\}\hfill \end{array}\hfill \end{array}. $$

The non-cooperative payoffs of the players along the cooperative path, that is V ⁱ(t, x ^* _t ), for $i\in \left\{1,2\right\}$ and $t\in \left\{2,3,4\right\}$, are given in Table 1.4 below.

Table 1.4 Non-cooperative payoffs of the players along the cooperative state path

Therefore in stage 2 the players would adopt the optimality principle : “Maximize the joint payoff and share the cooperative payoff proportional to their non-cooperative payoffs”. Hence the payoffs to the players in stage 2 have to satisfy:

$$ \begin{array}{cc}\hfill {\xi}^i\left(2,{x}_2^{*}\right)=\frac{V^i\left(2,{x}_2^{*}\right)}{V^1\left(2,{x}_2^{*}\right)+{V}^2\left(2,{x}_2^{*}\right)}W\left(2,{x}_2^{*}\right)\hfill & \hfill \mathrm{f}\mathrm{o}\mathrm{r}\ i\in \left\{1,2\right\}\hfill \end{array}. $$

Using Tables 1.2 and 1.3, the cooperative payoffs of players 1 and 2 at stage 2 are respectively:

$$ \begin{array}{l}{\xi}^1\left(2,{x}_2^{*}\right)=\frac{280}{280+165}570=358.65\kern0.24em \mathrm{and}\\ {}{\xi}^2\left(2,{x}_2^{*}\right)=\frac{165}{280+165}570=211.35.\end{array} $$

(1.2)

Similarly, at stage 3 using the agreed-upon optimality principle the cooperative payoffs of players 1 and 2 are respectively:

$$ \begin{array}{l}{\xi}^1\left(3,{x}_3^{*}\right)=\frac{170}{170+100}370=232.96\kern0.24em \mathrm{and}\\ {}{\xi}^2\left(3,{x}_3^{*}\right)=\frac{100}{170+100}370=137.04.\end{array} $$

(1.3)

Finally at stage 4, using the agreed-upon optimality principle the cooperative payoffs of players 1 and 2 are respectively:

$$ \begin{array}{l}{\xi}^1\left(4,{x}_4^{*}\right)=\frac{75}{75+45}180=112.5\kern0.24em \mathrm{and}\\ {}{\xi}^2\left(4,{x}_4^{*}\right)=\frac{45}{75+45}180=67.5.\end{array} $$

(1.4)

A system of payoffs as in (1.1, 1.2, 1.3 and 1.4) leads a subgame consistent solution. A payoff distribution procedure (PDP) has to be designed so that the payoffs of players 1 and 2 in stage 1 to stage 4 will be realized as (1.1, 1.2, 1.3 and 1.4). Leaving the general theorem for the derivation of PDP to be explained in Chap. 7 we proceed to verify that a PDP with the following cooperative stage incomes would lead to (1.1, 1.2, 1.3 and 1.4):

To verify that the PDP in Table 1.5 would lead to (1.1, 1.2, 1.3 and 1.4) we can readily obtain:

Table 1.5 Payoff Distribution Procedure (PDP) of a subgame consistent solution

$$ \begin{array}{l}{\xi}^1\left(1,{x}_1^{*}\right)=118.623+125.69+120.46+112.5=477.273,\\ {}{\xi}^1\left(2,{x}_2^{*}\right)=125.69+120.46+112.5=358.65,\\ {}{\xi}^1\left(3,{x}_3^{*}\right)=120.46+112.5=232.96,\\ {}{\xi}^1\left(4,{x}_4^{*}\right)=112.5.\end{array} $$

Similarly, from Table 1.5, we obtain ${\xi}^2\left(1,{x}_1^{*}\right)=272.727,{\xi}^2\left(2,{x}_2^{*}\right)=211.35,{\xi}^2\left(3,{x}_3^{*}\right)=137.04\kern0.24em \mathrm{and}\kern0.24em {\xi}^2\left(4,{x}_4^{*}\right)=67.5.$

In order to achieve the PDP in Table 1.5 a transfer payment scheme with stage income transfers received/paid (+/−) in Table 1.6 below has to be adopted.

Table 1.6 Transfer payments for a subgame consistent PDP

Adding the stage transfer payments in Table 1.6 to the players’ cooperative stage incomes in Table 1.2 yields the PDP in Table 1.5 (which leads a subgame consistent solution). Finally, note that player i’s cooperative payoff ξ ⁱ(t, x ^* _t ) is always higher than his non-cooperative payoff V ⁱ(t, x ^* _t ), for $i\in \left\{1,2\right\}$ and $t\in \left\{1,2,3,4\right\}$, and therefore the players would have no incentive to deviate from the above subgame consistent solution at any stage of the game.