# On symmetric stochastic games of resource extraction with weakly continuous transitions

- 99 Downloads

## Abstract

We study stochastic games of resource extraction, in which the players have identical preferences. The transition probability is either non-atomic or a convex combination of transition probabilities depending on the investment with coefficients also dependent on the investment. Our approach covers the unbounded utility case, which was not examined in this class of games beforehand. We prove the existence of a stationary Markov perfect equilibrium in a non-randomised class of strategies.

### Keywords

Stochastic game Resource extraction game Stationary Markov perfect equilibrium Weakly continuous transition probability### Mathematics Subject Classification

91A15 91A25 91A50 91B62## 1 Introduction

This paper deals with a strategic version of the discrete-time one-sector optimal growth model (see Bhattacharya and Majumdar 2007; Stachurski 2009), which plays a crucial role in both economic dynamics and resource economics. In principle, it can be described as follows. Two agents own a common natural resource and consume certain amount of the available stock in each time period. Their objective is to maximise their individual expected discounted utilities. The next state is usually given either by a deterministic production function or by some stochastic transition probability. The seminal paper of Levhari and Mirman (1980) studies such a strategic optimal growth model assuming that the players have the same logarithmic one-period utilities. Moreover, the next state \(s_{t+1}\) evolves according to the Cobb–Douglas production function, i.e., \(s_{t+1}=y_t^{\kappa }\), where \(\kappa \in (0,1)\) and \(y_t\) denotes a joint investment in period *t*.

Their model has been extended by Sundaram (1989), who considered general utility and production functions. Assuming that the players have identical preferences he proved the existence of a stationary Markov perfect equilibrium in the class of non-randomised strategies. Later Majumdar and Sundaram (1988) and Dutta and Sundaram (1992) reported the existence of a stationary Markov perfect equilibrium in the same class of strategies as Sundaram (1989), but for stochastic resource extraction games with symmetric payoffs. The former work analyses atomless transition probabilities, whereas the latter one embraces the conditions used in Sundaram (1989) and Majumdar and Sundaram (1988). Both papers deal with weakly continuous transition probabilities.

In this paper, we also prove the existence of a stationary Markov perfect equilibrium in the class of non-randomised strategies assuming identical preferences for the players and weak continuity of the transition probabilities. However, there are some essential features which not only distinguish our work from the aforementioned ones but also extend the previous results. Namely, in contrast to Sundaram (1989), Majumdar and Sundaram (1988), Dutta and Sundaram (1992), we deal with unbounded utility functions and we allow the state space to be the \([0,\infty )\) interval. The latter case requires additional work, since we have to show the compactness of certain function spaces. Furthermore, we propose two alternative sets of assumptions. The first set allows to study only games with non-atomic transition probabilities, whereas the second set embraces deterministic transitions and a class of transition probabilities not covered by Dutta and Sundaram (1992). Namely, we assume that the transition probability is a convex combination of transitions depending on the investment with coefficients also depending continuously on the investment. Particularly, our transition probabilities need not satisfy a stochastic dominance condition, whose stronger version plays a fundamental role in the proof of the equilibrium theorem in Dutta and Sundaram (1992).

The paper is organised as follows. The next section is devoted to a description of the game model. In Sect. 3, we provide the assumptions imposed on the transition probability and one-period utility function. Then, we formulate the main theorem and compare in detail our conditions with the ones used by Dutta and Sundaram (1992) and Majumdar and Sundaram (1988). The next section presents examples of transition probabilities that satisfy our assumptions, but not necessarily the ones in Dutta and Sundaram (1992). Section 4 contains compactness results, which are essential in the equilibrium proof. The proof of equilibrium result is given in Sect. 6 and is preceded by a sequence of auxiliary lemmas. Final comments and concluding remarks are put in Sect. 7.

## 2 The model

- (i)
*S*is the*state space,*i.e., the set of available resource stocks, - (ii)
\(A_i(s)=[0, s]\) is the

*set of actions*available to player \(i\in P:=\{1,2\}\) in state \(s\in S,\) - (iii)
\(u_i:S \rightarrow \mathbb {R}\) is a non-negative

*stage utility function*for player \(i\in P,\) - (iv)
for any \(s\in S\) and any feasible pair of actions \((a,b)\in A_1(s)\times A_2(s),\) that is, \(a+b\le s, q(\cdot |s-a-b)\) is a probability measure on

*S*, - (v)
\(\beta \in (0,1)\) is a

*discount coefficient*.

*u*may be unbounded, but it satisfies the following condition.

- (W1)
There exists a continuous increasing function \(w:S\rightarrow [1,\infty )\) such that \(0\le u (c)\le w( c)\) for all \(c\in S.\)

- (W2)There exists a constant \(\alpha >0\) such that \( \alpha \beta <1 \) and$$\begin{aligned} \int _S w(z)q (\mathrm{d}z|y)\le \alpha w(y)\quad \text{ for } \text{ all } \quad y\in S \end{aligned}$$

*u*(

*a*),

*u*(

*b*)) provided that the actions are feasible, i.e., \(a+b\le s.\) A new state \(s'\) is realised from the probability distribution \(q(\cdot | s-a-b)\) and the next period begins. The stage utilities are discounted by \(\beta .\) Note that the actions available at any state to one player depend on those chosen by the other player. This model is known as a generalised game in the terminology of Debreu (1954), or a coupled constraint game as in Haurie et al. (2012). If the pair of actions (

*a*,

*b*) is infeasible in state

*s*, then one can follow Dutta and Sundaram (1992) and assume that every player receives utility

*u*(

*s*/ 2). We restrict attention to strategies generating feasible action pairs during the play. An equilibrium in the symmetric case (where the stage utility functions are identical) will consist of feasible strategies of the players.

*strategy*for player \(i\in P\) is a sequence of Borel-measurable mappings from the history space to the space of actions available to her/him.

^{1}The set of strategies for player

*i*is denoted by \(\varPi _i\) and its generic element by \(\pi _i.\) Let \(F_i\) be the set of all Borel-measurable functions \(\phi _i: S\rightarrow S\) such that \(\phi _i(s) \in A_i(s)=[0,s]\) for each \(s\in S.\) A

*stationary Markov strategy*for player \(i\in P\) is a constant sequence \((\pi _{it})\) where \(\pi _{it}=\phi _i\) for some \(\phi _i\in F_i\) and for all \(t\in \mathbb {N}.\) Hence, a stationary Markov strategy for player

*i*can be identified with the Borel-measurable mapping \(\phi _i\in F_i.\) For any feasible pair \((\pi _1,\pi _2)\in \varPi _1\times \varPi _2,\) an initial state \(s\in S\) and \(t\in \mathbb {N},\) by \(u_i^{(t)}(\pi _1,\pi _2)(s)\) we denote the

*expected utility*for player

*i*in the

*t*th period of the game. The

*expected discounted*utility for player \(i\in P\) is

*Nash equilibrium*if

### Definition 1

A Stationary Markov Perfect Equilibrium (SMPE) is a Nash equilibrium \((\phi ^*_1,\phi ^*_2)\) that belongs to the class of strategy pairs \(F_1\times F_2.\) An \(SMPE (\phi ^*_1,\phi ^*_2)\) is *symmetric* if \(\phi ^*_1=\phi ^*_2.\)

### Remark 1

## 3 Main result

Let \(\Pr (S)\) be the set of all probability measures on the state space *S*. We recall that a sequence \((\mu _n)\) of probability measures on *S* *converges weakly* to some \(\mu _0\in \Pr (S)\) (\(\mu _n \Rightarrow \mu _0\) for short) if, for any bounded continuous function \(v:S\rightarrow \mathbb {R},\) we have \(\lim _{n\rightarrow \infty }\int _Sv(s)\mu _n(\mathrm{d}s)= \int _Sv(s)\mu _0(\mathrm{d}s)\); see Billingsley (1968).

We now formulate further assumptions which will be needed in our proofs.

(U) The function \(u :S\rightarrow \mathbb {R}\) is non-negative, increasing, strictly concave and continuous at \(s=0.\)

For the transition probabilities, we accept two alternative sets of conditions (A) or (B1)–(B3).

*q*is weakly continuous on

*S*, that is, if \(y_m\rightarrow y_0\) in

*S*, then \(q(\cdot |y_m)\Rightarrow q(\cdot |y_0)\) as \(m\rightarrow \infty .\) Moreover, for each \(y\in S_+,\) the probability measure \(q(\cdot |y)\) is non-atomic and \(q(\cdot |0)\) has no atoms in \(S_+.\)

- (B1)Assume that \(\lambda _j:S\rightarrow [0,1], j\in J:=\{1,\ldots ,l\},\) are continuous functions such that \(\sum _{j=1}^l \lambda _j(y)=1\) for all \(y\in S.\) In addition, suppose that there exist transition probabilities \(q_j\) from
*S*to \(S, j\in J\), such that for each \(y\in S\), we haveMoreover, for every \(j\in J,\) the transition probability \(q_j(\cdot |y)\) is weakly continuous on$$\begin{aligned} q(\cdot |y)=\sum _{j=1}^l\lambda _j(y)q_j(\cdot |y). \end{aligned}$$(1)*S*and \(q_j(\{0\}|0)=1.\) - (B2)
Every transition probability \(q_j(\cdot |y)\) in (1) is either non-atomic for \(y\in S_+\) or it satisfies the

*stochastic dominance*condition.^{2} - (B3)
For every \(s\in S\) the set \(Z^s =\{y\in S:\; q(\{s\}|y)>0\}\) is countable.

(C) The function \(y\rightarrow \int _Sw(z)q(dz|y)\) is continuous on *S*.

We now define a special class of stationary strategies of the players. By \(F_i^0\), we denote the set of mappings \(\phi \in F_i\) such that the function \(\varphi (s):= s-\phi (s)\) is non-decreasing, upper semicontinuous and \(0\le \phi (s)\le s/2\) for all \(s\in S.\) Note that \(\varphi \) and thus \(\phi \) are continuous from the right. Clearly, \(\phi \in F^0_i\) is lower semicontinuous.

We can now state our main result.

**Equilibrium Theorem ** *Let either (A) or (B1)–(B3) hold. Assume that (U), (W1)–(W2) and (C) are also satisfied. Then, the game has a symmetric * \(SMPE (\phi ^*,\phi ^*)\in F^0_1\times F^0_2.\)

### Remark 2

- (DS)For any \(s\in S_+\) and \(y_1<y_2, \) we havewhere \(Q(z|y):=q([0,z]|y).\)$$\begin{aligned} \lim _{z\nearrow s}Q(z|y_1):=Q(s^-|y_1)\ge Q(s|y_2), \end{aligned}$$

*q*for which conditions (B1)–(B3) hold and (DS) is not satisfied.

### Remark 3

The predecessors of our work on symmetric dynamic games of resource extraction are Dutta and Sundaram (1992), Majumdar and Sundaram (1988) and Sundaram (1989). The first two papers deal with stochastic transition probabilities, whereas the last one studies the purely deterministic case. As noted by Dutta and Sundaram (1992), their assumptions cover the ones studied not only by Sundaram (1989) but also by Majumdar and Sundaram (1988), who examined games with atomless transition probabilities. We propose two alternative set of assumptions: (A) or (B1)–(B3). Conditions (A) require \(q(\cdot |y)\) to be non-atomic measure on *S* for any \(y\in S_+.\) However, in contrast to the aforementioned papers our novelty is twofold. First, we allow *u* to be unbounded. As pointed out by Bhattacharya and Majumdar (2007) or Stachurski (2009), such unbounded utilities are commonly used in the theory of economic growth. Second, we get rid of compactness of the resource space *S*.

More importantly, the alternative set of conditions (B1)–(B3) embraces transition probabilities which were not covered by Dutta and Sundaram (1992). Specifically, our assumptions allow to consider transition probabilities that do not satisfy stochastic dominance property, even its weak version. As mentioned in Remark 2, the strong stochastic dominance was the crux in their proof on existence of a symmetric equilibrium. In particular, conditions (B1)–(B3) enable to analyse transition probabilities which are convex combinations of deterministic transitions with coefficients depending on the investment. Such transition probabilities, as observed in Examples 4–5 in Sect. 4, do not meet the stochastic dominance property. Hence, they do not satisfy the strong version of this condition either.

Our proofs owe much both the techniques developed in Majumdar and Sundaram (1988) and Dutta and Sundaram (1992), and the methods used in the study of multigenerational games; see Balbus et al. (2015b) and references cited therein. In the main body of our proof we exploit assumption (B3) instead of (DS). Assumption (B3) was applied by Balbus et al. (2015b), who studied a different class of non-cooperative games with countably many players called multigenerational games. Here, we only mention that under the (weak) stochastic dominance property condition (B3) implies (DS). However, in general assumptions accepted by Dutta and Sundaram (1992) and in this paper do not coincide. Finally, we wish to mention that our proof, in contrast to Dutta and Sundaram (1992) and Majumdar and Sundaram (1988), does not require an analysis of the generalised game in the sense of Debreu (1954).

### Remark 4

Dutta and Sundaram (1992) and Majumdar and Sundaram (1988) also impose other conditions except either strong stochastic dominance or atomless of transition probabilities, respectively. For instance, they assume that for a positive, sufficiently small level of investment, the stock tomorrow is no less than investment today with probability one. This requirement is usually referred to as the Inada condition. Moreover, they also assume that the utility function is differentiable and its right-hand side derivative at zero is \(+\infty .\) These two assumptions allow them to obtain an interior symmetric equilibrium. However, their analysis exclude the following utility functions: \(u(c)=1-e^{-c}\) or \(u(c)=r\ln (1+c),\) where *r* is arbitrary positive constant. Here, we do not impose the Inada conditions, but then in equilibrium \((\phi ^*,\phi ^*)\) it may happen that \(\phi ^*(s)\) is zero or *s* / 2 for some states.

We do not assume we do not assume other assumptions made by Dutta and Sundaram (1992) either. Specifically, they assume that there is no free production, that strictly positive investment today results in strictly positive stock tomorrow and that there is a maximum sustainable stock.

## 4 Examples

*D*in

*S*, investment \(y\in S,\) the transition probability is

*D*. We now point out three special cases for the above recurrence equation.

### Example 1

Let \(\bar{f}(y_t,\xi _t)= \xi _tf_1(y_t) +(1-\xi _t)f_2(y_t)\), where \(f_i:S \rightarrow S\) is continuous, increasing for \(i=1,2\) and \(f_1(y)>f_2(y)\) for \(y\in S_+.\) Moreover, assume that \(f_1(0)=f_2(0)=0.\) In addition, \(\pi \) is a non-atomic probability measure on [0, 1].

### Example 2

A model with *additive shocks*. Let \(\bar{f} (y_t,\xi _t)= f (y_t) +\xi _t,\) where \(f :S\rightarrow S\) is a continuous increasing function. The probability measure \(\pi \) is non-atomic with support included in \([0, +\infty ).\)

### Example 3

A model with *multiplicative shocks*. Assume that \(\bar{f}(y_t,y_t)= f (y_t)\xi _t,\) where *f* is as in Example 2 and the probability measure \(\pi \) is non-atomic with support included in \([0,+\infty ).\)

Next we give three examples of transition probabilities satisfying conditions (B1)–(B3) with \(l=2.\)

### Example 4

### Example 5

### Example 6

Finally, we give two examples satisfying conditions (W1)–(W2) and either (A) or (B1)–(B3).

### Example 7

*constant depreciation.*Suppose that the resource stock evolves due to the following recursive equation

### Example 8

*r*. Note that (W1), (U), (C) hold as well. Moreover, this transition probability satisfies (B1)–(B3).

Other transition probabilities and utility functions satisfying our conditions can be obtained by an adaptation of examples from Sect. 4 in Jaśkiewicz and Nowak (2011) or from Sect. 6 in Jaśkiewicz and Nowak (2018).

## 5 Basic compactness lemmas

Let *X* be the vector space of all continuous from the right functions \(\phi :S\rightarrow \mathbb {R}\) with bounded variation on every interval \([0,n], n\in \mathbb {N}.\) We assume that *X* is endowed with the *topology of weak convergence*. Recall that a sequence \((\varphi _m)\) converges weakly to \(\varphi \in X\) iff \(\varphi _m(s)\rightarrow \varphi (s)\) as \(m\rightarrow \infty \) at any continuity point \(s\in S\) of \(\varphi .\) The weak convergence of \((\varphi _m)\) to \(\varphi \) is denoted by \(\varphi _m{\mathop {\rightarrow }\limits ^{\omega }} \varphi \).

Let \(\eta :S\rightarrow S\) be a continuous increasing function. We define \(X^\eta \) as the set of all non-decreasing functions \(\varphi \in X\) such that \(0\le \varphi (s)\le \eta (s)\) for all \(s\in S.\) Note that each \(\varphi \in X^\eta \) is upper semicontinuous. Observe that 0 is a continuity point of every function \(\varphi \in X^\eta .\)

### Proposition 1

The set \(X^\eta \) is convex and sequentially compact in *X*.

### Proof

It is obvious that \(X^\eta \) is convex. For any \(f\in X^\eta \) and \(m\in \mathbb { N}\), we define the function \(f^m\) as follows: \(f^m(s)=f(s)\) for all \(s\in [0,m)\) and \(f_m(s) = \eta (m)\) for al \(s\ge m.\) Then \(f^m \in X^\eta \) and can be viewed as a continuous from the right “distribution function” of some non-negative countably additive measure \(\nu _m\) such that \(\nu _m(S)=\eta (m).\)

Consider now an arbitrary sequence \((\varphi _k)\) of functions in \(X^\eta .\) We now apply the standard “diagonal method”. By Helly’s selection theorem [see p. 227 in Billingsley (1968)], there exists a subsequence \((n_1(k))\) of (*n*) such that \((\varphi ^1_{n_{1}(k)})\) converges weakly (as \(k\rightarrow \infty \)) to some \(\gamma ^1\in X^\eta .\) Next, there exists a subsequence \((n_2(k))\) of \((n_1(k))\) such that \((\varphi ^2_{n_{2}(k)})\) converges weakly to some \(\gamma ^2\in X^\eta \) and \(\gamma ^2(s)=\gamma ^1(s)\) for each \(s\in [0,1).\) By induction we infer that for any \(r\ge 2\), there exists a subsequence \((n_r(k))\) of \((n_{r-1}(k))\) such that \((\phi ^r_{n_{r}(k)})\) converges weakly to some \(\gamma ^r\in X^\eta \) and \(\gamma ^r(s)=\gamma ^{r-1}(s)\) for each \(s\in [0,r-1).\) Define \(\gamma (s):= \gamma ^m(s)\) if \(s\in [0,m), m\in \mathbb {N}.\) Then, \(\gamma \in X^\eta .\) Consider the “diagonal sequence” defined by \(d(k):= n_k(k)\), \(k\in \mathbb {N}.\) Then, \((\varphi _{d(k)}){\mathop {\rightarrow }\limits ^{\omega }} \gamma \) as \(k\rightarrow \infty .\) Thus \(X^\eta \) is sequentially compact. \(\square \)

Let \(Y=X^\eta \) where \(\eta (s)=s\) for all \(s\in S\) and let \(Y^0\) be the subset of all \(\varphi \in Y\) such that \(\varphi (s)\ge s/2.\) Observe that \(F_i^0=K(Y^0)\) where *K* is the continuous mapping defined by \(K(\varphi )(s)= s-\varphi (s), s\in S.\) From Proposition 1, we obtain the following conclusion.

### Proposition 2

*Y* and \(F_i^0\) are convex and sequentially compact spaces when endowed with the topology of weak convergence.

Observe that every \(\phi \in F^0_i\) is lower semicontinuous and continuous from the right, but it need not be non-decreasing. The function \(s\rightarrow s-\phi (s)\) that belongs to \(Y^0\) is non-decreasing and upper semicontinuous. This fact will be used frequently in our considerations.

## 6 Proofs

In this section, \(X^\eta \) is considered with \(\eta (s)=w(s)/(1-\alpha \beta ) \) for all \(s\in S.\)

### Lemma 1

Assume that \(f_n {\mathop {\rightarrow }\limits ^{\omega }} f\) in \(X^\eta \) and \(y_n\rightarrow y_0\) in *S* as \(n\rightarrow \infty .\) Then, \(f(y_0)\ge \limsup _{n\rightarrow \infty } f_n(y_n).\)

### Proof

*f*. Then, there is some \(N\in \mathbb {N}\) such that \(y_n<y\) for all \(n>N.\) Hence, \(f_n(y_n)\le f_n(y)\) for \(n>N\) and consequently,

*y*can be chosen arbitrarily close to \(y_0\) and

*f*is continuous from the right, we deduce that \(\limsup _{n\rightarrow \infty }f_n(y_n)\le f(y_0).\) \(\square \)

### Lemma 2

*S*as \(n\rightarrow \infty .\) Then, it follows that

### Proof

*w*

### Lemma 3

*S*as \(n\rightarrow \infty .\) Then, we have

### Proof

*f*(hence, \(f_*(0)=f(0)\) as well). Therefore, by (A) we infer

### Lemma 4

*S*as \(n\rightarrow \infty \) and \(f\in X^\eta .\) Then, it follows that

### Proof

*f*is non-decreasing, by Corollary 3.9.1(a) in Topkis (1998) , we have

### Lemma 5

### Proof

*T*as follows:

*u*is continuous and increasing and the function \(s\rightarrow s-\phi (s)\) upper semicontinuous, it follows that the function \((s,y) \rightarrow u(s-\phi (s)-y)+\beta \int _SV(z)q(\mathrm{d}z|y)\) is upper semicontinuous. Since the correspondence \(s\rightarrow \varPhi (s)\) has a closed graph, it follows by Theorem 2 in Berge (1963) that

*TV*is upper semicontinuous.

*T*is contractive. Indeed, by (W2) for any \(V_1, V_2\in X^\eta \) we obtain that

*w*-norm of any function \(V\in X^\eta \) as follows \(\Vert V\Vert _w:=\sup _{s\in S} \frac{|V(s)|}{w(s)},\) it is easily seen that the last display implies the inequality

*w*-norm. By standard programming arguments (see Hernández-Lerma and Lasserre 1999), it follows that \(V_{\phi }(s)=\sup _{\pi _1\in \varPi _1(\phi )} U_1(s,\pi _1,\phi )\) for all \(s\in S,\) which completes the proof. \(\square \)

### Lemma 6

The mapping \(s\rightarrow A_\phi (s)\) is ascending, i.e., if \(s_1<s_2\) and \(y_1\in A_\phi (s_1), y_2\in A_\phi (s_2),\) then \(y_1\le y_2.\)

### Proof

*u*is strictly concave, from the proof of Lemma 2 in Nowak (2006) and the fact that \(s_2-\phi (s_2)>s_1-\phi (s_1),\) we conclude the following

### Lemma 7

Let \(\psi \) be any selector of the correspondence \(s\rightarrow A_\phi (s)\), i.e., \(\psi (s)\in A_\phi (s)\) for all \(s\in S.\) If \(\psi \) is continuous at \(s_0\), then \(A_\phi (s_0)\) is a singleton.

### Proof

Clearly, \(\psi (0)=0.\) Hence, it is enough to consider \(s_0>0.\) Suppose that \(y_1\) and \(y_2\) belong to \(A_\phi (s_0)\) and \(y_1 < y_2\). Since \(s\rightarrow A_\phi (s)\) is ascending, we conclude that \(\psi \) is non-decreasing. Therefore, we have \(\lim _{s\rightarrow s_0^-}\psi (s) \le y_1 <y_2\le \lim _{s\rightarrow s_0^+}\psi (s).\) This contradicts our assumption that \(\psi \) is continuous at \(s_0\in S_+.\) \(\square \)

### Lemma 8

The function \(g(\phi )\) is the unique non-decreasing and continuous from the right selector of the correspondence \(s\rightarrow A_\phi (s).\)

### Proof

Clearly, by Lemma 6 the function \(g(\phi )\) is non-decreasing. Note that the graph of the correspondence \(s\rightarrow A_\phi (s)\) is closed from the right, i.e., for any \(s_n\searrow s\) and \(y_n\in A_\phi (s_n)\) such that \(y_n\) converges to some *y*, it follows that \(y\in A_\phi (s).\) Therefore, we infer that \(g(\phi )\) is continuous from the right. Consequently, \(g(\phi )\) is upper semicontinuous. The uniqueness is a consequence of Lemma 7. \(\square \)

### Proof of the equilibrium theorem

*L*is continuous. Suppose that \(\phi _n{\mathop {\rightarrow }\limits ^{\omega }} \phi \) as \(n\rightarrow \infty .\) By Proposition 1, we can assume without loss of generality that \(V_n:=V_{\phi _n} {\mathop {\rightarrow }\limits ^{\omega }} V\) in \(X^\eta \) (if necessary take a subsequence). By Proposition 2, we also assume that \(\psi _n:=g(\phi _n) {\mathop {\rightarrow }\limits ^{\omega }} \psi \) in

*Y*. Thus, for each \(n\in \mathbb {N},\) we have

*V*, \(\phi \) and \(\psi .\) For any \(s\in S_1\), \(V_n(s)\rightarrow V(s), \phi _n(s)\rightarrow \phi (s)\) and \(\psi _n(s)\rightarrow \psi (s)\) as \(n\rightarrow \infty .\) By (6), Lemma 2 and assumption (U), we obtain that

*S*and the functions

*V*, \(\psi \) and \(\phi \) are continuous from the right, we may choose a sequence \((s_m)\) in \(S_1\) such that \(s_m\searrow s\) as \(m\rightarrow \infty .\) Thus, we have

*V*and \(s\rightarrow s-\phi (s)\) that (7) holds for all \(s\in S.\)

*V*. Note that \(0\not \in S_d.\) By \(S_2\) we denote the set of all continuity points of the functions

*V*and \(\phi .\) Further define \(S_3\) as the set of all \(y\in S\) such that \(q(S_d|y)=0.\) The set \(S_2\) is dense in

*S*and the set \(S_3\) is also dense in

*S*either by (A) or (B3). Clearly, by (A) or (B1), the state \(0\in S_3.\) Choose any \(s\in S_2\cap S_+\) and \(y\in S_3\cap [0,s-\phi (s)).\) Then, there exists some \(N\in \mathbb {N}\) such that, \(y\in [0,s-\phi _n(s)]\) for all \(n>N.\) Hence, we have

*m*tend to infinity and making use of Lemma 3 in case of assumption (A) or of Lemma 4 in case of assumptions (B), the continuity of

*u*and the continuity from the right of the functions \(V, s\rightarrow s-\phi (s)\), we deduce that inequality (9) holds for \(s_0\in S\) and \(y_0\in [0,s_0-\phi (s_0)].\)

*L*is continuous. By the Schauder–Tychonoff fixed point theorem, there exists \(\phi ^*\in F^0_2\) such that \(L\phi ^*=\phi ^*.\) This implies that \(\phi ^*\) is the best response of player 1 to the strategy \(\phi ^*\) chosen by player 2. Since the game is symmetric, it follows that \((\phi ^*,\phi ^*)\) is a

*SMPE*. \(\square \)

## 7 Concluding remarks

- 1.
Our result on the equilibrium existence is also valid for bounded state space \(S=[0,\bar{s}],\) with some \(\bar{s}>0.\) Then, it is enough to put \(w\equiv 1\) in (W1) and (W2). However, in order to avoid upper-endpoint problems in

*S*, we need to apply the trick used, for instance, by Dutta and Sundaram (1992). They expand the state space to \(S^*=[0,s^*]\) with some \(s^*>\bar{s}\) and study a set of investment functions defined on \(S^*.\) Particularly, Dutta and Sundaram (1992) deal with upper semicontinuous non-decreasing functions on \(S^*\) whose values at*s*do not exceed*s*and such that the value of such a function at \(s^*\) is \(s^*.\) - 2.
The problem of proving the existence of a Nash equilibrium in a stochastic game of resource extraction with different utility functions for the players seem to be difficult. Partial results were reported by Amir (1996), Balbus and Nowak (2008), Jaśkiewicz and Nowak (2015), where specific structures of transition probabilities were accepted. For example, Amir (1996) analysed so-called “convex transitions”. More precisely, he assumed that the conditional cumulative distribution function induced by the transition probability is convex with respect to investments. He proved the existence of pure stationary Markov perfect equilibria in the class of Lipschitz continuous strategies. The convexity assumption imposed on the transition functions made by Amir (1996) is very restrictive. It holds, for example, if the transition probability is a convex combination of some probability measures on the state space with coefficients depending on joint investments. This class was also examined Balbus and Nowak (2008). However, as argued by Jaśkiewicz and Nowak (2015), this type of transition probabilities makes sense only in the bounded state space case. The most general class of non-symmetric resource extraction games was studied by Jaśkiewicz and Nowak (2015), who considered transition probabilities being a convex combinations of transition probabilities depending on the state space and coefficients depending on a joint investment. Under these conditions they proved the existence of pure stationary Markov perfect equilibria. For further comments and references the reader is referred to Jaśkiewicz and Nowak (2018).

- 3.The problem of an equilibrium existence in non-symmetric stochastic games with weakly continuous transition probabilities is an open non-trivial problem. There are two main reasons. First, the state space is uncountable. Second, the payoff functions for either player
*i*in the auxiliary one-shot game with a continuation vector function \(v=(v_1,v_2)\) are neither convex nor concave. More precisely, the functionis neither convex nor concave for some non-decreasing function \(v_i\) on$$\begin{aligned} y\rightarrow u_i(c_i)+\beta \int _S v_i(z)q(\mathrm{d}z|y), \quad c=(c_1,c_2) \end{aligned}$$*S*. Hence, we cannot apply the Nash theorem (see Nash 1951). Furthermore, if we allow to study a general form of the transition probability \(q(\cdot |s,c),\) then the auxiliary games are not supermodular in the sense of Milgrom and Roberts (1990) either. Therefore, we cannot apply the techniques from lattice programming, see Topkis (1998). Further examples and a detailed discussion can be found in Jaśkiewicz and Nowak (2015) and Jaśkiewicz and Nowak (2018). - 4.
Finally, we would like to pay attention of researchers to the paper of Amir (1989). It contains some errors that cannot be fixed. Specifically, the limit argument given on p. 1349 is incorrect. Therefore, the problem of equilibrium existence in non-symmetric deterministic dynamic games of resource extraction is still open.

## Footnotes

## Notes

### Acknowledgements

We thank anonymous referees for helpful comments. The authors acknowledge the financial support from the National Science Centre, Poland: Grant 2016/23/B/ST1/00425.

### References

- Amir R (1989) A lattice–theoretic approach to a class of dynamic games. Comput Math Appl 17:1345–1349CrossRefGoogle Scholar
- Amir R (1996) Continuous stochastic games of capital accumulation with convex transitions. Games Econ Behav 15:111–131CrossRefGoogle Scholar
- Balbus Ł, Nowak AS (2008) Existence of perfect equilibria in a class of multigenerational stochastic games of capital accumulation. Automatica 44:1471–1479CrossRefGoogle Scholar
- Balbus Ł, Jaśkiewicz A, Nowak AS (2015b) Stochastic bequest games. Games Econ Behav 90:247–256CrossRefGoogle Scholar
- Berge C (1963) Topological spaces. Oliver and Boyd Ltd, EdinburghGoogle Scholar
- Bhattacharya R, Majumdar M (2007) Random dynamical systems: theory and applications. Cambridge University Press, CambridgeCrossRefGoogle Scholar
- Billingsley P (1968) Convergence of probability measures. Wiley, New YorkGoogle Scholar
- Debreu G (1954) A social equilibrium existence theorem. Proc Natl Acad Sci USA 38:931–938Google Scholar
- Dutta PK, Sundaram R (1992) Markovian equilibrium in a class of stochastic games: existence theorems for discounted and undiscounted models. Econ Theory 2:197–214CrossRefGoogle Scholar
- Haurie A, Krawczyk JB, Zaccour G (2012) Games and dynamic games. World Scientific, SingaporeCrossRefGoogle Scholar
- Hernández-Lerma O, Lasserre JB (1999) Further topics on discrete-time Markov control processes. Springer, New YorkCrossRefGoogle Scholar
- Jaśkiewicz A, Nowak AS (2011) Stochastic games with unbounded payoffs: applications to robust control in economics. Dyn Games Appl 1:253–279CrossRefGoogle Scholar
- Jaśkiewicz A, Nowak AS (2015) Stochastic games of resource extraction. Automatica 54:310–316CrossRefGoogle Scholar
- Jaśkiewicz A, Nowak AS (2018) Non-zero-sum stochastic games. In: Başar T, Zaccour G (eds) Handbook of dynamic game theory. Birkhäuser, BaselGoogle Scholar
- Levhari D, Mirman LJ (1980) The great fish war: an example using a dynamic Cournot–Nash solution. ACA Trans 11:322–344Google Scholar
- Majumdar MK, Sundaram RK (1988) Symmetric stochastic games of resource extraction. The existence of non-randomized stationary equilibrium. In: Raghavan TES et al (eds) Stochastic games and related topics. Kluwer Academic Publishers, Dordrecht, pp 175–190Google Scholar
- Milgrom P, Roberts J (1990) Rationalizability, learning, and equilibrium in games with strategic complementarities. Econometrica 58:1255–1277CrossRefGoogle Scholar
- Nash JF (1951) Non-cooperative games. Ann Math 54:286–295CrossRefGoogle Scholar
- Nowak AS (2006) On perfect equilibria in stochastic models of growth with intergenerational altruism. Econ Theory 28:73–83CrossRefGoogle Scholar
- Serfozo R (1982) Convergence of Lebesgue integrals with varying measures. Sankhya Indian J Stat (Ser A) 44:380–402Google Scholar
- Stachurski J (2009) Economic dynamics: theory and computation. MIT Press, CambridgeGoogle Scholar
- Sundaram RK (1989) Perfect equilibrium in a class of symmetric dynamic games. J Econ Theory 47:153–177 Corrigendum: J Econ Theory 49:385–187CrossRefGoogle Scholar
- Topkis D (1998) Supermodularity and complementarity. Princeton University Press, PrincetonGoogle Scholar
- Wessels J (1977) Markov programming by successive approximations with respect to weighted supremum norms. J Math Anal Appl 58:326–335CrossRefGoogle Scholar

## Copyright information

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.