On symmetric stochastic games of resource extraction with weakly continuous transitions

Abstract

We study stochastic games of resource extraction, in which the players have identical preferences. The transition probability is either non-atomic or a convex combination of transition probabilities depending on the investment with coefficients also dependent on the investment. Our approach covers the unbounded utility case, which was not examined in this class of games beforehand. We prove the existence of a stationary Markov perfect equilibrium in a non-randomised class of strategies.

1 Introduction

This paper deals with a strategic version of the discrete-time one-sector optimal growth model (see Bhattacharya and Majumdar 2007; Stachurski 2009), which plays a crucial role in both economic dynamics and resource economics. In principle, it can be described as follows. Two agents own a common natural resource and consume certain amount of the available stock in each time period. Their objective is to maximise their individual expected discounted utilities. The next state is usually given either by a deterministic production function or by some stochastic transition probability. The seminal paper of Levhari and Mirman (1980) studies such a strategic optimal growth model assuming that the players have the same logarithmic one-period utilities. Moreover, the next state $s_{t+1}$ evolves according to the Cobb–Douglas production function, i.e., $s_{t+1}=y_t^{\kappa }$, where $\kappa \in (0,1)$ and $y_t$ denotes a joint investment in period t.

Their model has been extended by Sundaram (1989), who considered general utility and production functions. Assuming that the players have identical preferences he proved the existence of a stationary Markov perfect equilibrium in the class of non-randomised strategies. Later Majumdar and Sundaram (1988) and Dutta and Sundaram (1992) reported the existence of a stationary Markov perfect equilibrium in the same class of strategies as Sundaram (1989), but for stochastic resource extraction games with symmetric payoffs. The former work analyses atomless transition probabilities, whereas the latter one embraces the conditions used in Sundaram (1989) and Majumdar and Sundaram (1988). Both papers deal with weakly continuous transition probabilities.

In this paper, we also prove the existence of a stationary Markov perfect equilibrium in the class of non-randomised strategies assuming identical preferences for the players and weak continuity of the transition probabilities. However, there are some essential features which not only distinguish our work from the aforementioned ones but also extend the previous results. Namely, in contrast to Sundaram (1989), Majumdar and Sundaram (1988), Dutta and Sundaram (1992), we deal with unbounded utility functions and we allow the state space to be the $[0,\infty )$ interval. The latter case requires additional work, since we have to show the compactness of certain function spaces. Furthermore, we propose two alternative sets of assumptions. The first set allows to study only games with non-atomic transition probabilities, whereas the second set embraces deterministic transitions and a class of transition probabilities not covered by Dutta and Sundaram (1992). Namely, we assume that the transition probability is a convex combination of transitions depending on the investment with coefficients also depending continuously on the investment. Particularly, our transition probabilities need not satisfy a stochastic dominance condition, whose stronger version plays a fundamental role in the proof of the equilibrium theorem in Dutta and Sundaram (1992).

The paper is organised as follows. The next section is devoted to a description of the game model. In Sect. 3, we provide the assumptions imposed on the transition probability and one-period utility function. Then, we formulate the main theorem and compare in detail our conditions with the ones used by Dutta and Sundaram (1992) and Majumdar and Sundaram (1988). The next section presents examples of transition probabilities that satisfy our assumptions, but not necessarily the ones in Dutta and Sundaram (1992). Section 4 contains compactness results, which are essential in the equilibrium proof. The proof of equilibrium result is given in Sect. 6 and is preceded by a sequence of auxiliary lemmas. Final comments and concluding remarks are put in Sect. 7.

2 The model

Let $\mathbb {N}$ be the set of positive integers and $\mathbb {R}$ be the set of all real numbers. Let $S=[0,\infty )$ and $S_+=(0,\infty ).$ We consider a discounted stochastic game for which:

• (i) S is the state space, i.e., the set of available resource stocks,

• (ii) $A_i(s)=[0, s]$ is the set of actions available to player $i\in P:=\{1,2\}$ in state $s\in S,$

• (iii) $u_i:S \rightarrow \mathbb {R}$ is a non-negative stage utility function for player $i\in P,$

• (iv) for any $s\in S$ and any feasible pair of actions $(a,b)\in A_1(s)\times A_2(s),$ that is, $a+b\le s, q(\cdot |s-a-b)$ is a probability measure on S, 

• (v) $\beta \in (0,1)$ is a discount coefficient.

We assume that $u_1=u_2 =:u$. The function u may be unbounded, but it satisfies the following condition.

• (W1) There exists a continuous increasing function $w:S\rightarrow [1,\infty )$ such that $0\le u (c)\le w( c)$ for all $c\in S.$

• (W2) There exists a constant $\alpha >0$ such that $\alpha \beta <1$ and

  $$\begin{aligned} \int _S w(z)q (\mathrm{d}z|y)\le \alpha w(y)\quad \text{ for } \text{ all } \quad y\in S \end{aligned}$$

In each period the players observe the state $s\in S$ and simultaneously choose their actions $(a,b)\in A_1(s)\times A_2(s)$ that yield the utility vector (u(a), u(b)) provided that the actions are feasible, i.e., $a+b\le s.$ A new state $s'$ is realised from the probability distribution $q(\cdot | s-a-b)$ and the next period begins. The stage utilities are discounted by $\beta .$ Note that the actions available at any state to one player depend on those chosen by the other player. This model is known as a generalised game in the terminology of Debreu (1954), or a coupled constraint game as in Haurie et al. (2012). If the pair of actions (a, b) is infeasible in state s, then one can follow Dutta and Sundaram (1992) and assume that every player receives utility u(s / 2). We restrict attention to strategies generating feasible action pairs during the play. An equilibrium in the symmetric case (where the stage utility functions are identical) will consist of feasible strategies of the players.

A strategy for player $i\in P$ is a sequence of Borel-measurable mappings from the history space to the space of actions available to her/him.¹ The set of strategies for player i is denoted by $\varPi _i$ and its generic element by $\pi _i.$ Let $F_i$ be the set of all Borel-measurable functions $\phi _i: S\rightarrow S$ such that $\phi _i(s) \in A_i(s)=[0,s]$ for each $s\in S.$ A stationary Markov strategy for player $i\in P$ is a constant sequence $(\pi _{it})$ where $\pi _{it}=\phi _i$ for some $\phi _i\in F_i$ and for all $t\in \mathbb {N}.$ Hence, a stationary Markov strategy for player i can be identified with the Borel-measurable mapping $\phi _i\in F_i.$ For any feasible pair $(\pi _1,\pi _2)\in \varPi _1\times \varPi _2,$ an initial state $s\in S$ and $t\in \mathbb {N},$ by $u_i^{(t)}(\pi _1,\pi _2)(s)$ we denote the expected utility for player i in the tth period of the game. The expected discounted utility for player $i\in P$ is

$$\begin{aligned} U_i(s,\pi _1,\pi _2)= \sum _{t=1}^\infty \beta ^{t-1}u^{(t)}_i(\pi _1,\pi _2)(s). \end{aligned}$$

A feasible profile of strategies $(\pi ^*_1,\pi ^*_2)\in \varPi _1\times \varPi _2$ is called a Nash equilibrium if

$$\begin{aligned} U_1(s,\pi ^*_1,\pi ^*_2)\ge U_1(s,\pi _1,\pi ^*_2) \end{aligned}$$

for each $s\in S$ and $\pi _1\in \varPi _1$ such that $(\pi _1,\pi _2^*)$ is feasible, and

$$\begin{aligned} U_2(s,\pi ^*_1,\pi ^*_2)\ge U_2(s,\pi ^*_1,\pi _2) \end{aligned}$$

for each $s\in S$ and $\pi _2\in \varPi _2$ such that $(\pi ^*_1,\pi _2)$ is feasible.

Definition 1

A Stationary Markov Perfect Equilibrium (SMPE) is a Nash equilibrium $(\phi ^*_1,\phi ^*_2)$ that belongs to the class of strategy pairs $F_1\times F_2.$ An $SMPE (\phi ^*_1,\phi ^*_2)$ is symmetric if $\phi ^*_1=\phi ^*_2.$

Remark 1

Conditions (W1)–(W2) are used to guarantee the convergence of the discounted utilities for the players. These assumptions have been already used in dynamic programming and in zero-sum stochastic games, see for instance, Wessels (1977), Hernández-Lerma and Lasserre (1999), Jaśkiewicz and Nowak (2011) and references cited therein. One can observe that under conditions (W1)-(W2), for any feasible pair of strategies $(\pi _1,\pi _2)$ and $t\ge 2$, we have $u^{(t)}_i(\pi _1,\pi _2)(s)\le \alpha ^{t-1}w(s),$ and consequently

$$\begin{aligned} 0\le U_i(s,\pi _1,\pi _2)\le \frac{w(s)}{1-\alpha \beta },\quad i\in P,\quad s\in S. \end{aligned}$$

Hence, the discounted expected utility is well-defined for any pair of feasible strategies for the players.

3 Main result

Let $\Pr (S)$ be the set of all probability measures on the state space S. We recall that a sequence $(\mu _n)$ of probability measures on S converges weakly to some $\mu _0\in \Pr (S)$ ($\mu _n \Rightarrow \mu _0$ for short) if, for any bounded continuous function $v:S\rightarrow \mathbb {R},$ we have $\lim _{n\rightarrow \infty }\int _Sv(s)\mu _n(\mathrm{d}s)= \int _Sv(s)\mu _0(\mathrm{d}s)$; see Billingsley (1968).

We now formulate further assumptions which will be needed in our proofs.

(U) The function $u :S\rightarrow \mathbb {R}$ is non-negative, increasing, strictly concave and continuous at $s=0.$

For the transition probabilities, we accept two alternative sets of conditions (A) or (B1)–(B3).

(A) The transition probability q is weakly continuous on S, that is, if $y_m\rightarrow y_0$ in S, then $q(\cdot |y_m)\Rightarrow q(\cdot |y_0)$ as $m\rightarrow \infty .$ Moreover, for each $y\in S_+,$ the probability measure $q(\cdot |y)$ is non-atomic and $q(\cdot |0)$ has no atoms in $S_+.$

• (B1) Assume that $\lambda _j:S\rightarrow [0,1], j\in J:=\{1,\ldots ,l\},$ are continuous functions such that $\sum _{j=1}^l \lambda _j(y)=1$ for all $y\in S.$ In addition, suppose that there exist transition probabilities $q_j$ from S to $S, j\in J$, such that for each $y\in S$, we have

  $$\begin{aligned} q(\cdot |y)=\sum _{j=1}^l\lambda _j(y)q_j(\cdot |y). \end{aligned}$$

  (1)

  Moreover, for every $j\in J,$ the transition probability $q_j(\cdot |y)$ is weakly continuous on S and $q_j(\{0\}|0)=1.$

• (B2) Every transition probability $q_j(\cdot |y)$ in (1) is either non-atomic for $y\in S_+$ or it satisfies the stochastic dominance condition.²

• (B3) For every $s\in S$ the set $Z^s =\{y\in S:\; q(\{s\}|y)>0\}$ is countable.

We also impose the following assumption.

(C) The function $y\rightarrow \int _Sw(z)q(dz|y)$ is continuous on S.

We now define a special class of stationary strategies of the players. By $F_i^0$, we denote the set of mappings $\phi \in F_i$ such that the function $\varphi (s):= s-\phi (s)$ is non-decreasing, upper semicontinuous and $0\le \phi (s)\le s/2$ for all $s\in S.$ Note that $\varphi$ and thus $\phi$ are continuous from the right. Clearly, $\phi \in F^0_i$ is lower semicontinuous.

We can now state our main result.

Equilibrium Theorem Let either (A) or (B1)–(B3) hold. Assume that (U), (W1)–(W2) and (C) are also satisfied. Then, the game has a symmetric $SMPE (\phi ^*,\phi ^*)\in F^0_1\times F^0_2.$

Remark 2

The result of Dutta and Sundaram (1992) is based on the following assumption.

• (DS) For any $s\in S_+$ and $y_1<y_2,$ we have

  $$\begin{aligned} \lim _{z\nearrow s}Q(z|y_1):=Q(s^-|y_1)\ge Q(s|y_2), \end{aligned}$$

  where $Q(z|y):=q([0,z]|y).$

This assumption is called strong stochastic dominance. Clearly, condition (DS) implies the stochastic dominance property called by Dutta and Sundaram (1992) the weak stochastic dominance. It is not difficult to construct $q(\cdot |y)$ that has the stochastic dominance property and does not satisfy condition (DS). In Examples 4 and 5 we provide transition probabilities q for which conditions (B1)–(B3) hold and (DS) is not satisfied.

Remark 3

The predecessors of our work on symmetric dynamic games of resource extraction are Dutta and Sundaram (1992), Majumdar and Sundaram (1988) and Sundaram (1989). The first two papers deal with stochastic transition probabilities, whereas the last one studies the purely deterministic case. As noted by Dutta and Sundaram (1992), their assumptions cover the ones studied not only by Sundaram (1989) but also by Majumdar and Sundaram (1988), who examined games with atomless transition probabilities. We propose two alternative set of assumptions: (A) or (B1)–(B3). Conditions (A) require $q(\cdot |y)$ to be non-atomic measure on S for any $y\in S_+.$ However, in contrast to the aforementioned papers our novelty is twofold. First, we allow u to be unbounded. As pointed out by Bhattacharya and Majumdar (2007) or Stachurski (2009), such unbounded utilities are commonly used in the theory of economic growth. Second, we get rid of compactness of the resource space S.

More importantly, the alternative set of conditions (B1)–(B3) embraces transition probabilities which were not covered by Dutta and Sundaram (1992). Specifically, our assumptions allow to consider transition probabilities that do not satisfy stochastic dominance property, even its weak version. As mentioned in Remark 2, the strong stochastic dominance was the crux in their proof on existence of a symmetric equilibrium. In particular, conditions (B1)–(B3) enable to analyse transition probabilities which are convex combinations of deterministic transitions with coefficients depending on the investment. Such transition probabilities, as observed in Examples 4–5 in Sect. 4, do not meet the stochastic dominance property. Hence, they do not satisfy the strong version of this condition either.

Our proofs owe much both the techniques developed in Majumdar and Sundaram (1988) and Dutta and Sundaram (1992), and the methods used in the study of multigenerational games; see Balbus et al. (2015b) and references cited therein. In the main body of our proof we exploit assumption (B3) instead of (DS). Assumption (B3) was applied by Balbus et al. (2015b), who studied a different class of non-cooperative games with countably many players called multigenerational games. Here, we only mention that under the (weak) stochastic dominance property condition (B3) implies (DS). However, in general assumptions accepted by Dutta and Sundaram (1992) and in this paper do not coincide. Finally, we wish to mention that our proof, in contrast to Dutta and Sundaram (1992) and Majumdar and Sundaram (1988), does not require an analysis of the generalised game in the sense of Debreu (1954).

Remark 4

Dutta and Sundaram (1992) and Majumdar and Sundaram (1988) also impose other conditions except either strong stochastic dominance or atomless of transition probabilities, respectively. For instance, they assume that for a positive, sufficiently small level of investment, the stock tomorrow is no less than investment today with probability one. This requirement is usually referred to as the Inada condition. Moreover, they also assume that the utility function is differentiable and its right-hand side derivative at zero is $+\infty .$ These two assumptions allow them to obtain an interior symmetric equilibrium. However, their analysis exclude the following utility functions: $u(c)=1-e^{-c}$ or $u(c)=r\ln (1+c),$ where r is arbitrary positive constant. Here, we do not impose the Inada conditions, but then in equilibrium $(\phi ^*,\phi ^*)$ it may happen that $\phi ^*(s)$ is zero or s / 2 for some states.

We do not assume we do not assume other assumptions made by Dutta and Sundaram (1992) either. Specifically, they assume that there is no free production, that strictly positive investment today results in strictly positive stock tomorrow and that there is a maximum sustainable stock.

4 Examples

In this section, we provide some examples of transition probabilities and utility functions for which our assumptions hold. We start with condition (A). A natural transition probability satisfying this assumption is given by the recurrence equation

$$\begin{aligned} s_{t+1}= \bar{f} (y_t,\xi _t), \end{aligned}$$

where $y_t=s_t - a_t$ is the investment in state $s_t$, $(\xi _t)$ is a sequence of i.i.d. random “shocks” having a probability distribution $\pi \in \Pr (S).$ The function $\bar{f}: S\times S\rightarrow S$ is continuous and for any Borel set D in S,  investment $y\in S,$ the transition probability is

$$\begin{aligned} q (D|y)= \int _S 1_D(\bar{f}(y,\omega )) \pi (\mathrm{d}\omega ). \end{aligned}$$

Here, $1_D$ denotes the indicator function of the set D. We now point out three special cases for the above recurrence equation.

Example 1

Let $\bar{f}(y_t,\xi _t)= \xi _tf_1(y_t) +(1-\xi _t)f_2(y_t)$, where $f_i:S \rightarrow S$ is continuous, increasing for $i=1,2$ and $f_1(y)>f_2(y)$ for $y\in S_+.$ Moreover, assume that $f_1(0)=f_2(0)=0.$ In addition, $\pi$ is a non-atomic probability measure on [0, 1].

Example 2

A model with additive shocks. Let $\bar{f} (y_t,\xi _t)= f (y_t) +\xi _t,$ where $f :S\rightarrow S$ is a continuous increasing function. The probability measure $\pi$ is non-atomic with support included in $[0, +\infty ).$

Example 3

A model with multiplicative shocks. Assume that $\bar{f}(y_t,y_t)= f (y_t)\xi _t,$ where f is as in Example 2 and the probability measure $\pi$ is non-atomic with support included in $[0,+\infty ).$

Next we give three examples of transition probabilities satisfying conditions (B1)–(B3) with $l=2.$

Example 4

Let the transition probability be as follows:

$$\begin{aligned} q(\cdot |y)=\lambda _1(y)\delta _{f_1(y)}(\cdot )+ \lambda _2(y)\delta _{f_2(y)}(\cdot ) \end{aligned}$$

and let $f_1, f_2$ be as in Example 1. The symbol $\delta _{f_i(y)}(\cdot )$ denotes the Dirac measure concentrated at the point $f_i(y).$ The functions $\lambda _1,\lambda _2: S\rightarrow [0,1]$ are continuous and $\lambda _1(y)+\lambda _2(y)=1$ for all $y\in S.$ Obviously, this transition probability satisfies conditions (B1) and (B2). In order to see that (B3) holds as well, choose any $s\in S$ and note that $Z^s= \{y\in S: f_i(y)=s \}$ consists of at most two elements. In the pure deterministic case (where $q(\cdot |y) =\delta _{f_1(y)}(\cdot )$) $Z^s$ has at most one point. Furthermore, let $E_q(y)$ we denote the expected value of the resource stock following $y\in S,$ that is,

$$\begin{aligned} E_q(y)= f_1(y)\lambda _1(y)+ f_2(y)\lambda _2(y) =\lambda _2(y)(f_2(y)-f_1(y)) + f_1(y). \end{aligned}$$

Assume that $f_1(y)= y+\sqrt{y}$, $f_2(y)=y,$ for $y\in S.$ Moreover, let $\lambda _1(y)= y$ for $y\in [0,1], \lambda _1(y)= 1/y$ for $y\ge 1$ and $\lambda _2(y)=1-\lambda _1(y)$ for each $y\in S.$ Let $Q(z|y):= q([0,z]|y).$ Note that $Q(z|4)= 3/4$ for $z\in [4,6)$ and $Q(z|5)= 4/5$ for $z\in [5,5+\sqrt{5}).$ Hence, $Q(z|4) <Q(z|5)$ for $z\in [5,6),$ i.e., $q(\cdot |y)$ does not satisfy the stochastic dominance condition. Observe that in this case

$$\begin{aligned} E_q(y)=y(\sqrt{y} + 1) \quad \text{ for }\quad y\in [0,1)\quad \text{ and }\quad E_q(y)= y+\frac{1}{\sqrt{y}} \quad \text{ for }\quad y\ge 1. \end{aligned}$$

Hence, the function $y\rightarrow E_q(y)$ is continuous and increasing, although the stochastic dominance condition fails.

Example 5

Consider now the transition probability from Example 4 with the following functions: $f_1(y)= y$, $f_2(y)=\sqrt{y},$ for $y\in S$ and $\lambda _1(y)=1-\lambda _2(y),$ for $y\in S,$ where $\lambda _2(y)= y$ for $y\in [0,1], \lambda _2(y)= 1-1/y$ for $y\ge 1.$ Observe that $Q(z|9)= 8/9$ for $z\in [3,9)$ and $Q(z|16)= 15/16$ for $z\in [4,16).$ Hence $Q(z|9) <Q(z|16)$ for $z\in [4,9),$ i.e., $q(\cdot |y)$ does not satisfy the stochastic dominance condition. Observe that

$$\begin{aligned} E_q(y)=y\sqrt{y} + y-y^2\quad \text{ for }\quad y\in [0,1)\quad \text{ and }\quad E_q(y)= \sqrt{y}-\frac{1}{\sqrt{y}}+1\quad \text{ for }\quad y\ge 1. \end{aligned}$$

Hence, the expected value $E_q(y)$ is again an increasing and continuous function with respect to $y\in S.$

Example 6

Assume now that

$$\begin{aligned} q(\cdot |y)=\lambda _1(y)q_1(\cdot |y)+ \lambda _2(y)\delta _{f(y)}(\cdot ), \end{aligned}$$

where the functions $\lambda _i$ are as in (B1) and the function $f: S\rightarrow S$ is continuous and increasing with $f(0)=0.$ If $q_1(\cdot |y)$ is non-atomic for $y>0$ and $q(\{0\}|0)=1,$ then this transition probability satisfies assumptions (B1)–(B3). Note that the stochastic dominance condition may fail in this case either.

Finally, we give two examples satisfying conditions (W1)–(W2) and either (A) or (B1)–(B3).

Example 7

A model with Cobb–Douglas production function with constant depreciation. Suppose that the resource stock evolves due to the following recursive equation

$$\begin{aligned} s_{t+1}=y_t^\theta \xi _t+(1-\delta )y_t, \quad t\in \mathbb {N}, \end{aligned}$$

where $\theta \in (0,1)$ and $\delta \in (0,1]$ is a depreciation rate. Suppose that $\bar{z}:=\int _{S}\xi \pi (d\xi )$ is finite and $\pi$ is a non-atomic measure with support included in $[0,+\infty ).$ In addition, let the utility function be given by $u(c)=c^\sigma$ with $\sigma \in (0,1).$ Clearly, assumptions (U) and (A) are satisfied. We claim that (W2) also holds for $w(y)=(r+y)^\sigma$ with arbitrary $r\ge 1.$ Then,

$$\begin{aligned} \int _S w(z) q(\mathrm{d}z|y)=\int _S (r+y^\theta \xi +(1-\delta )y)^\sigma \pi (\mathrm{d}\xi ). \end{aligned}$$

Analogous calculations to the ones on p. 263 in Jaśkiewicz and Nowak (2011) show that (W2) is met with

$$\begin{aligned} \alpha :=\left( 1+\frac{\left( \bar{z}/\delta \right) ^{\frac{1}{1-\theta }} }{r} \right) ^\sigma , \end{aligned}$$

where $r\ge 1$ must be sufficiently large so that $\alpha \beta <1.$ Obviously, (C) also holds true.

Example 8

Let $u(c)=\sqrt{c}$ for each $c\in S$ and

$$\begin{aligned} q(\cdot |y)=\sum _{j=1}^l\lambda _j(y)\delta _{f_j(y)}(\cdot ), \quad y\in S, \end{aligned}$$

where each function $f_j: S\rightarrow S$ is continuous, increasing and $f_j(0)=0$ for every $j=1\in J.$ The coefficient $\lambda _j(y)$ describes a chance that the resource stock will be $f_j(y).$ Moreover, let the functions $\lambda _j, j\in J,$ be as in (B1). Assume that there exist two constants $\rho >0$ and $\theta >1$ such that $\theta \beta <1$ and $f_j(y)\le \rho +\theta y$ for all $j\in J$ and $y\in S.$ For some $r\ge 1$ define $w(y)= r+y, y\in S$. Then,

$$\begin{aligned} \int _Sw(z)q(\mathrm{d}z|y)= \sum _{j=1}^l\lambda _j(y)f_j(w(y))\le \rho +\theta (r+y). \end{aligned}$$

Hence, it follows that

$$\begin{aligned} \frac{\int _Sw(z)q(\mathrm{d}z|y)}{w(y)}\le \frac{\rho +\theta ( r+ y)}{r+y}\le \frac{\rho }{ r} +\theta , \quad y\in S. \end{aligned}$$

If we set $\alpha := \frac{\rho }{r} +\theta$, then (W2) holds for sufficiently large values of r. Note that (W1), (U), (C) hold as well. Moreover, this transition probability satisfies (B1)–(B3).

Other transition probabilities and utility functions satisfying our conditions can be obtained by an adaptation of examples from Sect. 4 in Jaśkiewicz and Nowak (2011) or from Sect. 6 in Jaśkiewicz and Nowak (2018).

5 Basic compactness lemmas

Let X be the vector space of all continuous from the right functions $\phi :S\rightarrow \mathbb {R}$ with bounded variation on every interval $[0,n], n\in \mathbb {N}.$ We assume that X is endowed with the topology of weak convergence. Recall that a sequence $(\varphi _m)$ converges weakly to $\varphi \in X$ iff $\varphi _m(s)\rightarrow \varphi (s)$ as $m\rightarrow \infty$ at any continuity point $s\in S$ of $\varphi .$ The weak convergence of $(\varphi _m)$ to $\varphi$ is denoted by $\varphi _m{\mathop {\rightarrow }\limits ^{\omega }} \varphi$.

Let $\eta :S\rightarrow S$ be a continuous increasing function. We define $X^\eta$ as the set of all non-decreasing functions $\varphi \in X$ such that $0\le \varphi (s)\le \eta (s)$ for all $s\in S.$ Note that each $\varphi \in X^\eta$ is upper semicontinuous. Observe that 0 is a continuity point of every function $\varphi \in X^\eta .$

Proposition 1

The set $X^\eta$ is convex and sequentially compact in X.

Proof

It is obvious that $X^\eta$ is convex. For any $f\in X^\eta$ and $m\in \mathbb { N}$, we define the function $f^m$ as follows: $f^m(s)=f(s)$ for all $s\in [0,m)$ and $f_m(s) = \eta (m)$ for al $s\ge m.$ Then $f^m \in X^\eta$ and can be viewed as a continuous from the right “distribution function” of some non-negative countably additive measure $\nu _m$ such that $\nu _m(S)=\eta (m).$

Consider now an arbitrary sequence $(\varphi _k)$ of functions in $X^\eta .$ We now apply the standard “diagonal method”. By Helly’s selection theorem [see p. 227 in Billingsley (1968)], there exists a subsequence $(n_1(k))$ of (n) such that $(\varphi ^1_{n_{1}(k)})$ converges weakly (as $k\rightarrow \infty$) to some $\gamma ^1\in X^\eta .$ Next, there exists a subsequence $(n_2(k))$ of $(n_1(k))$ such that $(\varphi ^2_{n_{2}(k)})$ converges weakly to some $\gamma ^2\in X^\eta$ and $\gamma ^2(s)=\gamma ^1(s)$ for each $s\in [0,1).$ By induction we infer that for any $r\ge 2$, there exists a subsequence $(n_r(k))$ of $(n_{r-1}(k))$ such that $(\phi ^r_{n_{r}(k)})$ converges weakly to some $\gamma ^r\in X^\eta$ and $\gamma ^r(s)=\gamma ^{r-1}(s)$ for each $s\in [0,r-1).$ Define $\gamma (s):= \gamma ^m(s)$ if $s\in [0,m), m\in \mathbb {N}.$ Then, $\gamma \in X^\eta .$ Consider the “diagonal sequence” defined by $d(k):= n_k(k)$, $k\in \mathbb {N}.$ Then, $(\varphi _{d(k)}){\mathop {\rightarrow }\limits ^{\omega }} \gamma$ as $k\rightarrow \infty .$ Thus $X^\eta$ is sequentially compact. $\square$

Let $Y=X^\eta$ where $\eta (s)=s$ for all $s\in S$ and let $Y^0$ be the subset of all $\varphi \in Y$ such that $\varphi (s)\ge s/2.$ Observe that $F_i^0=K(Y^0)$ where K is the continuous mapping defined by $K(\varphi )(s)= s-\varphi (s), s\in S.$ From Proposition 1, we obtain the following conclusion.

Proposition 2

Y and $F_i^0$ are convex and sequentially compact spaces when endowed with the topology of weak convergence.

Observe that every $\phi \in F^0_i$ is lower semicontinuous and continuous from the right, but it need not be non-decreasing. The function $s\rightarrow s-\phi (s)$ that belongs to $Y^0$ is non-decreasing and upper semicontinuous. This fact will be used frequently in our considerations.

6 Proofs

In this section, $X^\eta$ is considered with $\eta (s)=w(s)/(1-\alpha \beta )$ for all $s\in S.$

Lemma 1

Assume that $f_n {\mathop {\rightarrow }\limits ^{\omega }} f$ in $X^\eta$ and $y_n\rightarrow y_0$ in S as $n\rightarrow \infty .$ Then, $f(y_0)\ge \limsup _{n\rightarrow \infty } f_n(y_n).$

Proof

Let $y>y_0$ be a continuity point of f. Then, there is some $N\in \mathbb {N}$ such that $y_n<y$ for all $n>N.$ Hence, $f_n(y_n)\le f_n(y)$ for $n>N$ and consequently,

$$\begin{aligned} \limsup _{n\rightarrow \infty }f_n(y_n)\le \limsup _{n\rightarrow \infty }f_n(y)=f(y). \end{aligned}$$

Since y can be chosen arbitrarily close to $y_0$ and f is continuous from the right, we deduce that $\limsup _{n\rightarrow \infty }f_n(y_n)\le f(y_0).$ $\square$

Lemma 2

Let either (A), (W1), (C) or (B1), (W1), (C) hold. Assume that $f_n {\mathop {\rightarrow }\limits ^{\omega }} f$ in $X^\eta$ and $y_n\rightarrow y$ in S as $n\rightarrow \infty .$ Then, it follows that

$$\begin{aligned} \limsup _{n\rightarrow \infty } \int _Sf_n(z)q(\mathrm{d}z|y_n) \le \int _Sf (z)q(\mathrm{d}z|y). \end{aligned}$$

Proof

Let $\tilde{f}_n(z):=f_n(z)-w(z)/(1-\alpha \beta )$ for $z\in S.$ Define

$$\begin{aligned} f^*(z)=\sup \{\limsup _{n\rightarrow \infty } \tilde{f}_n(z_n): \ z_n\rightarrow z \} \end{aligned}$$

and observe that by the continuity of w

$$\begin{aligned} f^*(z)=\sup \{\limsup _{n\rightarrow \infty } f_n(z_n): \ z_n\rightarrow z \}-w(z)/(1-\alpha \beta ). \end{aligned}$$

By Lemma 3.2 in Serfozo (1982) and Lemma 1, we have

$$\begin{aligned} \limsup _{n\rightarrow \infty } \int _S \tilde{f}_n(z)q(\mathrm{d}z|y_n) \le \int _S f^* (z) q(\mathrm{d}z|y)\le \int _S (f(z)-w(z)/(1-\alpha \beta )) q(\mathrm{d}z|y). \end{aligned}$$

On the other hand, by (C) we obtain

$$\begin{aligned} \limsup _{n\rightarrow \infty } \int _S \tilde{f}_n(z)q(\mathrm{d}z|y_n)= & {} \limsup _{n\rightarrow \infty } \int _S f_n(z)q(\mathrm{d}z|y_n) - \lim _{n\rightarrow \infty } \int _S w(z)/(1-\alpha \beta )q(\mathrm{d}z|y_n) \\= & {} \limsup _{n\rightarrow \infty } \int _S f_n(z)q(\mathrm{d}z|y_n) - \int _S w(z)/(1-\alpha \beta )q(\mathrm{d}z|y). \end{aligned}$$

Thus, the result follows. $\square$

Lemma 3

Let (A), (W1) and (C) hold. Assume that $f \in X^\eta$ and $y_n \rightarrow y$ in S as $n\rightarrow \infty .$ Then, we have

$$\begin{aligned} \lim _{n\rightarrow \infty }\int _Sf(z)q(\mathrm{d}z|y_n)=\int _Sf(z)q(\mathrm{d}z|y). \end{aligned}$$

Proof

For any $z\in S$ define

$$\begin{aligned} f_*(z)=\inf \{\liminf _{n\rightarrow \infty } f(z_n): \ z_n\rightarrow z \}. \end{aligned}$$

The function $f_*$ is lower semicontinuous and $f_*(z)=f(z)$ for any continuity point $z\in S$ of f (hence, $f_*(0)=f(0)$ as well). Therefore, by (A) we infer

$$\begin{aligned} \int _Sf_*(z)q(\mathrm{d}z|y)=\int _S f(z)q(\mathrm{d}z|y), \quad \text{ for }\ y\in S. \end{aligned}$$

By Lemma 3.2 in Serfozo (1982), it follows that

$$\begin{aligned} \liminf _{n\rightarrow \infty }\int _Sf_n(z)q(\mathrm{d}z|y_n)\ge \int _Sf_*(z)q(\mathrm{d}z|y). \end{aligned}$$

These facts and Lemma 2 yield

$$\begin{aligned} \limsup _{n\rightarrow \infty } \int _Sf_n(z)q(\mathrm{d}z|y_n) \le \int _Sf (z)q(\mathrm{d}z|y )\le \liminf _{n\rightarrow \infty }\int _Sf_n(z)q(\mathrm{d}z|y_n). \end{aligned}$$

Thus, the result follows. $\square$

Lemma 4

Let (B1), (B2), (W1) and (C) hold. Assume that $y_n\searrow y$ in S as $n\rightarrow \infty$ and $f\in X^\eta .$ Then, it follows that

$$\begin{aligned} \lim _{n\rightarrow \infty }\int _S f(z)q(\mathrm{d}z|y_n)=\int _S f(z)q(\mathrm{d}z|y). \end{aligned}$$

Proof

By (B1), we have

$$\begin{aligned} \int _S f(z)q(\mathrm{d}z|y_n)=\sum _{j=1}^l\lambda _j(y_n)\int _S f(z)q_j(\mathrm{d}z|y_n), \end{aligned}$$

where the functions $\lambda _j$ are continuous. If $q_j$ is non-atomic, then by Lemma 3 (with $q:=q_j$), we obtain

$$\begin{aligned} \int _S f(z)q_j(\mathrm{d}z|y_n)\rightarrow \int _S f(z)q_j(\mathrm{d}z|y ). \end{aligned}$$

(2)

Suppose that $q_j$ admits some atoms and satisfies the stochastic dominance condition. Since f is non-decreasing, by Corollary 3.9.1(a) in Topkis (1998) , we have

$$\begin{aligned} \int _S f(z)q_j(\mathrm{d}z|y_n)\ge \int _S f(z)q_j(\mathrm{d}z|y) \end{aligned}$$

(3)

for all $n\in \mathbb {N}.$ Using (2) or (3) and Lemma 2, we obtain that

$$\begin{aligned} \liminf _{n\rightarrow \infty }\int _S f(z)q_j(\mathrm{d}z|y_n)\ge \int _S f(z)q_j(\mathrm{d}z|y ) \ge \limsup _{n\rightarrow \infty }\int _S f(z)q_j(\mathrm{d}z|y_n). \end{aligned}$$

These inequalities finish the proof. $\square$

Let $\phi \in F_2^0$ and $\varPi _1(\phi )$ be the set of all strategies $\pi _1$ for player 1 such that the pair $(\pi _1,\phi )$ is feasible. Define

$$\begin{aligned} V_{\phi }(s):=\sup _{\pi _1\in \varPi _1(\phi )} U_1(s,\pi _1,\phi ), \ s\in S. \end{aligned}$$

In below lemmas we assume that (C), (U), (W1)–(W2) hold. In addition, we assume that (A) or (B1), (B2) are satisfied.

Lemma 5

Let $\varPhi (s)=[0,s-\phi (s)]$ for each $s\in S.$ The function $V_\phi$ is non-decreasing and upper semicontinuous and belongs to the space $X^\eta .$ Moreover, $V_\phi$ is the unique solution to the Bellman equation, i.e.,

$$\begin{aligned} V_{\phi }(s)= & {} \max _{c\in [0,s]}\left( u(c)+\beta \int _SV_{\phi }(z)q(\mathrm{d}z|s-\phi (s)-c)\right) \nonumber \\= & {} \max _{y\in \varPhi (s)}\left( u(s-\phi (s)-y)+\beta \int _SV_{\phi }(z)q(\mathrm{d}z|y)\right) \end{aligned}$$

(4)

for all $s\in S.$

Proof

For any function $V\in X^\eta$ define the dynamic programming operator T as follows:

$$\begin{aligned} TV(s)=\max _{y\in \varPhi (s)}\left( u(s-\phi (s)-y)+\beta \int _SV(z)q(\mathrm{d}z|y)\right) , \quad s\in S. \end{aligned}$$

By Lemma 2, the fact that u is continuous and increasing and the function $s\rightarrow s-\phi (s)$ upper semicontinuous, it follows that the function $(s,y) \rightarrow u(s-\phi (s)-y)+\beta \int _SV(z)q(\mathrm{d}z|y)$ is upper semicontinuous. Since the correspondence $s\rightarrow \varPhi (s)$ has a closed graph, it follows by Theorem 2 in Berge (1963) that TV is upper semicontinuous.

Since $s\rightarrow s-\phi (s)$ is non-decreasing, we observe that $u(s-\phi (s)-y)\le u(s'-\phi (s')-y)$ for $s<s'$ and all $y\in \varPhi (s).$ Hence, $TV(s)\le TV(s'),$ because $\varPhi (s)\subset \varPhi (s').$ Thus, the above considerations and (W1)–(W2) imply that $TV\in X^\eta$. We claim that T is contractive. Indeed, by (W2) for any $V_1, V_2\in X^\eta$ we obtain that

$$\begin{aligned} |TV_1(s)-TV_2(s)|\le & {} \sup _{y\in \varPhi (s)} \beta \int _S |V_1(z)-V_2(z)|q(\mathrm{d}z|y)\\\le & {} \alpha \beta \sup _{s'\in S}\frac{|V_1(s')-V_2(s')|}{w(s')} w(s). \end{aligned}$$

Defining the w-norm of any function $V\in X^\eta$ as follows $\Vert V\Vert _w:=\sup _{s\in S} \frac{|V(s)|}{w(s)},$ it is easily seen that the last display implies the inequality

$$\begin{aligned} \Vert TV_1-TV_2\Vert _w \le \alpha \beta \Vert V_1-V_2\Vert _w. \end{aligned}$$

By the Banach fixed point theorem, there exists a unique function $V_\phi \in X^\eta$ such that $TV_\phi =V_\phi .$ Note that $X^\eta$ is a closed subset of the Banach space of all Borel-measurable functions with the finite w-norm. By standard programming arguments (see Hernández-Lerma and Lasserre 1999), it follows that $V_{\phi }(s)=\sup _{\pi _1\in \varPi _1(\phi )} U_1(s,\pi _1,\phi )$ for all $s\in S,$ which completes the proof. $\square$

Define

$$\begin{aligned} A_{\phi }(s):= \text{ arg } \max _{y\in \varPhi (s)}\left( u(s-\phi (s)-y)+\beta \int _SV_{\phi }(z)q(\mathrm{d}z|y)\right) . \end{aligned}$$

Obviously, the set $A_{\phi }(s)$ is non-empty and compact. For any $s\in S$ we set

$$\begin{aligned} g(\phi )(s):= \max A_{\phi }(s). \end{aligned}$$

Lemma 6

The mapping $s\rightarrow A_\phi (s)$ is ascending, i.e., if $s_1<s_2$ and $y_1\in A_\phi (s_1), y_2\in A_\phi (s_2),$ then $y_1\le y_2.$

Proof

Suppose that $s\rightarrow A_\phi (s)$ is not ascending. Then there exist $s_1 < s_2$ and $y_1 \in A_\phi (s_1), y_2 \in A_\phi (s_2)$ such that $y_1 > y_2.$ Clearly, the set $\mathcal{L}:=\{(s,y):s\in S, \ y\in \varPhi (s)\}$ is a lattice with the usual component-wise order on the plane $\mathbb {R}^2.$ Thus $(s_2,y_1)$ and $(s_1,y_2)$ belong to $\mathcal{L}$. Since u is strictly concave, from the proof of Lemma 2 in Nowak (2006) and the fact that $s_2-\phi (s_2)>s_1-\phi (s_1),$ we conclude the following

$$\begin{aligned} u(s_2-\phi (s_2)-y_1)-u(s_2-\phi (s_2)-y_2) > u(s_1-\phi (s_1)-y_1)-u(s_1-\phi (s_1)-y_2). \end{aligned}$$

(5)

Adding $\beta \int _SV_\phi (z)q(\mathrm{d}z|y_1)-\beta \int _SV_\phi (z)q(\mathrm{d}z|y_2)$ to both sides of (5) and remembering that $y_1 \in A_\phi (s_1)$ and $y_2 \in A_\phi (s_2)$, we obtain

$$\begin{aligned}&0\ge u(s_2-\phi (s_2)-y_1)+\beta \int _SV_\phi (z)q(\mathrm{d}z|y_1) -V_\phi (s_2) >\\&V_\phi (s_1)-u(s_1-\phi (s_1)-y_2)-\beta \int _S V_\phi (z)q(\mathrm{d}z|y_2)\ge 0. \end{aligned}$$

This contradiction implies that the correspondence $s\rightarrow A_\phi (s)$ is ascending. $\square$

Lemma 7

Let $\psi$ be any selector of the correspondence $s\rightarrow A_\phi (s)$, i.e., $\psi (s)\in A_\phi (s)$ for all $s\in S.$ If $\psi$ is continuous at $s_0$, then $A_\phi (s_0)$ is a singleton.

Proof

Clearly, $\psi (0)=0.$ Hence, it is enough to consider $s_0>0.$ Suppose that $y_1$ and $y_2$ belong to $A_\phi (s_0)$ and $y_1 < y_2$. Since $s\rightarrow A_\phi (s)$ is ascending, we conclude that $\psi$ is non-decreasing. Therefore, we have $\lim _{s\rightarrow s_0^-}\psi (s) \le y_1 <y_2\le \lim _{s\rightarrow s_0^+}\psi (s).$ This contradicts our assumption that $\psi$ is continuous at $s_0\in S_+.$ $\square$

Lemma 8

The function $g(\phi )$ is the unique non-decreasing and continuous from the right selector of the correspondence $s\rightarrow A_\phi (s).$

Proof

Clearly, by Lemma 6 the function $g(\phi )$ is non-decreasing. Note that the graph of the correspondence $s\rightarrow A_\phi (s)$ is closed from the right, i.e., for any $s_n\searrow s$ and $y_n\in A_\phi (s_n)$ such that $y_n$ converges to some y,  it follows that $y\in A_\phi (s).$ Therefore, we infer that $g(\phi )$ is continuous from the right. Consequently, $g(\phi )$ is upper semicontinuous. The uniqueness is a consequence of Lemma 7. $\square$

Proof of the equilibrium theorem

Define the operator $L:F^0_2\rightarrow F^0_1=F^0_2$ by

$$\begin{aligned} L\phi (s)=(s-g(\phi )(s))/2, \quad \text{ where } \quad s\in S, \ \phi \in F^0_2. \end{aligned}$$

By Lemma 8, $L\phi \in F^0_1.$ We must show that L is continuous. Suppose that $\phi _n{\mathop {\rightarrow }\limits ^{\omega }} \phi$ as $n\rightarrow \infty .$ By Proposition 1, we can assume without loss of generality that $V_n:=V_{\phi _n} {\mathop {\rightarrow }\limits ^{\omega }} V$ in $X^\eta$ (if necessary take a subsequence). By Proposition 2, we also assume that $\psi _n:=g(\phi _n) {\mathop {\rightarrow }\limits ^{\omega }} \psi$ in Y. Thus, for each $n\in \mathbb {N},$ we have

$$\begin{aligned} V_n(s)= u(s-\phi _n(s)-\psi _n(s))+\beta \int _SV_n(z)q(\mathrm{d}z|\psi _n(s)),\ \text{ for } \text{ all }\ s\in S. \end{aligned}$$

(6)

Let $S_1\subset S$ be the set of all continuity points of the functions V, $\phi$ and $\psi .$ For any $s\in S_1$, $V_n(s)\rightarrow V(s), \phi _n(s)\rightarrow \phi (s)$ and $\psi _n(s)\rightarrow \psi (s)$ as $n\rightarrow \infty .$ By (6), Lemma 2 and assumption (U), we obtain that

$$\begin{aligned} V (s)\le u(s-\phi (s)-\psi (s))+\beta \int _SV(z)q(\mathrm{d}z|\psi (s)). \end{aligned}$$

(7)

Let $s\not \in S_1.$ Since $S_1$ is dense in S and the functions V, $\psi$ and $\phi$ are continuous from the right, we may choose a sequence $(s_m)$ in $S_1$ such that $s_m\searrow s$ as $m\rightarrow \infty .$ Thus, we have

$$\begin{aligned} V (s_m)\le u(s_m-\phi (s_m)-\psi (s_m))+\beta \int _SV(z)q(\mathrm{d}z|\psi (s_m)). \end{aligned}$$

Letting $m\rightarrow \infty$ in the above inequality, we conclude from Lemma 2, the continuity from the right of V and $s\rightarrow s-\phi (s)$ that (7) holds for all $s\in S.$

On the other hand, for any $n\in \mathbb {N}, s\in S$ and $y\in [0,s-\phi _n(s)],$ we have the following inequality

$$\begin{aligned} V_n(s)\ge u(s-\phi _n(s)-y)+\beta \int _S V_n(z)q(\mathrm{d}z|y). \end{aligned}$$

(8)

Let $S_d$ be the countable set of discontinuity points of the function V. Note that $0\not \in S_d.$ By $S_2$ we denote the set of all continuity points of the functions V and $\phi .$ Further define $S_3$ as the set of all $y\in S$ such that $q(S_d|y)=0.$ The set $S_2$ is dense in S and the set $S_3$ is also dense in S either by (A) or (B3). Clearly, by (A) or (B1), the state $0\in S_3.$ Choose any $s\in S_2\cap S_+$ and $y\in S_3\cap [0,s-\phi (s)).$ Then, there exists some $N\in \mathbb {N}$ such that, $y\in [0,s-\phi _n(s)]$ for all $n>N.$ Hence, we have

$$\begin{aligned} V_n(s)\ge u(s-\phi _n(s)-y)+\beta \int _S V_n(z)q(\mathrm{d}z|y), \quad n>N. \end{aligned}$$

By the dominated convergence theorem and the fact that $y\in S_3$, we have

$$\begin{aligned} \lim _{n\rightarrow \infty }\int _SV_n(z)q(\mathrm{d}z|y)=\int _SV (z)q(\mathrm{d}z|y). \end{aligned}$$

Therefore, we can conclude that

$$\begin{aligned} V (s)\ge u(s-\phi (s)-y)+\beta \int _S V (z)q(\mathrm{d}z|y), \end{aligned}$$

(9)

for $y\in [0,s-\phi (s))\cap S_3$ and $s\in S_2\cap S_+.$ Now let us consider $s_0\in S$ and $y_0\in [0,s_0-\phi (s_0)].$ There exist sequences $(s_m)$ and $(y_m)$ such that $s_m\searrow s_0, y_m\searrow y_0$ as $m\rightarrow \infty$ and $s_m\in S_2\cap S_+, y_m\in S_3\cap [0,s_m-\phi (s_m))$ for all $m\in \mathbb {N}.$ Observe that $s_m-\phi (s_m)\ge s_0-\phi (s_0).$ Hence, by (9) we obtain

$$\begin{aligned} V (s_m)\ge u(s_m-\phi (s_m)-y_m)+\beta \int _S V (z)q(\mathrm{d}z|y_m). \end{aligned}$$

Letting m tend to infinity and making use of Lemma 3 in case of assumption (A) or of Lemma 4 in case of assumptions (B), the continuity of u and the continuity from the right of the functions $V, s\rightarrow s-\phi (s)$, we deduce that inequality (9) holds for $s_0\in S$ and $y_0\in [0,s_0-\phi (s_0)].$

By (7) and (9), we obtain for any $s\in S$ that

$$\begin{aligned} V (s)= & {} u(s-\phi (s)-\psi (s))+\beta \int _SV(z)q(\mathrm{d}z|\psi (s))\nonumber \\= & {} \max _{y\in [0,s-\phi (s)]}\left( u(s-\phi (s)-y)+\beta \int _SV(z)q(\mathrm{d}z|y)\right) . \end{aligned}$$

(10)

Since $\psi$ is non-decreasing and upper semicontinuous, from Lemma 8, it follows that $\psi = g(\phi ).$ Thus, the operator L is continuous. By the Schauder–Tychonoff fixed point theorem, there exists $\phi ^*\in F^0_2$ such that $L\phi ^*=\phi ^*.$ This implies that $\phi ^*$ is the best response of player 1 to the strategy $\phi ^*$ chosen by player 2. Since the game is symmetric, it follows that $(\phi ^*,\phi ^*)$ is a SMPE. $\square$

7 Concluding remarks

This section contains a list of conclusions and open problems:

• 1. Our result on the equilibrium existence is also valid for bounded state space $S=[0,\bar{s}],$ with some $\bar{s}>0.$ Then, it is enough to put $w\equiv 1$ in (W1) and (W2). However, in order to avoid upper-endpoint problems in S,  we need to apply the trick used, for instance, by Dutta and Sundaram (1992). They expand the state space to $S^*=[0,s^*]$ with some $s^*>\bar{s}$ and study a set of investment functions defined on $S^*.$ Particularly, Dutta and Sundaram (1992) deal with upper semicontinuous non-decreasing functions on $S^*$ whose values at s do not exceed s and such that the value of such a function at $s^*$ is $s^*.$

• 2. The problem of proving the existence of a Nash equilibrium in a stochastic game of resource extraction with different utility functions for the players seem to be difficult. Partial results were reported by Amir (1996), Balbus and Nowak (2008), Jaśkiewicz and Nowak (2015), where specific structures of transition probabilities were accepted. For example, Amir (1996) analysed so-called “convex transitions”. More precisely, he assumed that the conditional cumulative distribution function induced by the transition probability is convex with respect to investments. He proved the existence of pure stationary Markov perfect equilibria in the class of Lipschitz continuous strategies. The convexity assumption imposed on the transition functions made by Amir (1996) is very restrictive. It holds, for example, if the transition probability is a convex combination of some probability measures on the state space with coefficients depending on joint investments. This class was also examined Balbus and Nowak (2008). However, as argued by Jaśkiewicz and Nowak (2015), this type of transition probabilities makes sense only in the bounded state space case. The most general class of non-symmetric resource extraction games was studied by Jaśkiewicz and Nowak (2015), who considered transition probabilities being a convex combinations of transition probabilities depending on the state space and coefficients depending on a joint investment. Under these conditions they proved the existence of pure stationary Markov perfect equilibria. For further comments and references the reader is referred to Jaśkiewicz and Nowak (2018).

• 3. The problem of an equilibrium existence in non-symmetric stochastic games with weakly continuous transition probabilities is an open non-trivial problem. There are two main reasons. First, the state space is uncountable. Second, the payoff functions for either player i in the auxiliary one-shot game with a continuation vector function $v=(v_1,v_2)$ are neither convex nor concave. More precisely, the function

  $$\begin{aligned} y\rightarrow u_i(c_i)+\beta \int _S v_i(z)q(\mathrm{d}z|y), \quad c=(c_1,c_2) \end{aligned}$$

  is neither convex nor concave for some non-decreasing function $v_i$ on S. Hence, we cannot apply the Nash theorem (see Nash 1951). Furthermore, if we allow to study a general form of the transition probability $q(\cdot |s,c),$ then the auxiliary games are not supermodular in the sense of Milgrom and Roberts (1990) either. Therefore, we cannot apply the techniques from lattice programming, see Topkis (1998). Further examples and a detailed discussion can be found in Jaśkiewicz and Nowak (2015) and Jaśkiewicz and Nowak (2018).

• 4. Finally, we would like to pay attention of researchers to the paper of Amir (1989). It contains some errors that cannot be fixed. Specifically, the limit argument given on p. 1349 is incorrect. Therefore, the problem of equilibrium existence in non-symmetric deterministic dynamic games of resource extraction is still open.