On recursive utilities with non-affine aggregator and conditional certainty equivalent

Abstract

In this paper, we consider the problem of the existence and the uniqueness of a recursive utility function defined on intertemporal lotteries. The purpose of this paper is to provide the results of the existence and the uniqueness of a recursive utility function. The utility function is obtained as the limit of iterations on a nonlinear operator and is independent on initial starting points, with iterations converging at an exponential rate. We also find the maximum utility and an optimal strategy by means of iterations of the Bellman operator.

1 Introduction

Dynamic models in economics often assume the utility functions defined over sequences of random consumptions are represented by a time-additive expected overall utility which discounts future temporal utilities at a constant rate. The existing literature clearly shows that this standard utility assumption is restrictive in numerous economic situations.¹ To name just few limitations, first in the standard case, the elasticity of intertemporal substitution (EIS) is equal to the inverse of the risk aversion coefficient. As a result, the standard utility formulation cannot explain many important puzzles in the literature (e.g., the equity premium puzzle postulated by Mehra and Prescott (1985) in the literature on asset pricing). Additionally, there is strong evidence that some decision makers prefer to know the realization of uncertainty as quickly as possible, while others prefer to know the realization of uncertainty at a later date. Again, this situation cannot be captured within the standard utility framework [see Kreps and Porteus (1978), Chew and Epstein (1989) as well as Klibanoff and Ozdenoren (2007)]. Finally, it is worth mentioning that the standard utility formulation is incompatible with confirmed important “paradoxes” in the experimental economics literature (e.g., the Allais paradox and the Ellsberg paradox).

Principle of optimality in dynamic programming together with the Bellman equation assure overall utility from today onward which can be expressed as a linear transformation of today’s temporal utility and the overall utility from future periods. In the standard discounted utility model, Koopmans (1960) formulated the set of axioms on the aggregator that connected the today’s temporal utility with the utility from future (continuation) periods. Similar class of recursive utilities provide Asheim et al. (2012) by proposing another axioms of sustainable recursive preferences. The set of deterministic recursive utilities, defined as such, included the standard discounted utility model as a special case. The ideas by Koopmans (1960) were extended by Kreps and Porteus (1978) to models with uncertainty with the finite time horizon and by Epstein and Zin (1989) to models with the infinite time horizon, in each case defining recursive utilities on a set of lotteries. Whereas Kreps and Porteus (1978) and Klibanoff and Ozdenoren (2007) parametrized a utility from future periods by means of the expected value, Epstein and Zin (1989) used the most general concept of Conditional Certainty Equivalent (henceforth, CCE for short).

A large body of the literature has already established extensions of the standard discounted utility model toward nonadditive aggregators under both deterministic and stochastic settings. Deterministic utility functions based on Koopmans equations can be found in numerous papers in the literature, including works by Boyd-III (2006), Bich et al. (2018), Duran (2000), Le-Van and Vailakis (2005), Martins-da-Rocha and Vailakis (2010), Jaśkiewicz et al. (2014), among others, while the utility function based on Epstein and Zin (1989) equations can be found in the work of Weil (1993), Skiadas (2015), Bäuerle and Jaskiewicz (2018), Marinacci and Montrucchio (2010), Ozaki and Streufert (1996), and Bloise and Vailakis (2018).

There are three fundamental questions associated with the specification of a recursive utility. First, whether the (recursive utility) function exists and is unique? Second, whether the optimal value is a fixed point of the corresponding Bellman operator and third, whether the optimal value function is a global attractor, i.e., whether a sequence of iterations defined on the Bellman operator uniformly converges to the recursive utility function regardless of the starting point? In this paper, we consider all three aforementioned questions using a nonlinear aggregator and a sub-homogeneous CCE. Clearly, apart from deterministic models, the standard expected utility operator is homogenous, and a lot of quasilinear models [as in Chew (1983), for example] are sub-homogenous. Moreover, a measure of risk sensitivity postulated in Weil (1993) is a special case in our formulation of CCE. Finally, it bears mentioning that the sub-homogeneity here encompasses a large class of aggregators including Kreps and Porteus (1978), Klibanoff and Ozdenoren (2007) as well as most of “Thompson aggregators” used by Marinacci and Montrucchio (2010).

From a technical perspective, we reduce the question of finding a recursive utility to finding a fixed point of an appropriately defined nonlinear operator. To the study of the set of fixed points of this operator, we apply a key theorem in Guo et al. (2004) on the cone of nonnegative functions. This theorem gives sufficient conditions for the existence and the uniqueness of fixed points, as well as provides the results on the global convergence of iterations.²

This approach is new in the literature. For example, some authors (e.g., Ozaki and Streufert 1996) use Knaster–Tarski Theorem (see Tarski 1955). In Jaśkiewicz et al. (2014), the authors use Matkowski Theorem (see Matkowski 1975). Becker and Rincón-Zapatero (2016) and Bloise and Vailakis (2018) used the geometric fixed point theorem by Krasnoselski and Zabreiko (1984). Indirect methods via iterations on extreme selections have been proposed in Le-Van and Vailakis (2005). The approach proposed in this paper is different. Finally, in this paper, we provide specific examples where neither Banach Fixed Point Theorem, nor its extension in Matkowski’s Theorem (Matkowski 1975), can be applied.

The rest of the paper is organized as follows: Sect. 2 contains preliminary information on cones, as well as a statement of Guo–Cho–Zhu Theorem. Section 3 contains a description of the model and the fundamental assumptions. The main results are included in Sects. 4 and 5. In Sect. 4, the existence, the uniqueness, and the global convergence results are proven. In Sect. 5, by means of Bellman equations, we obtain similar results for the optimal value of the recursive utility formulation. The proofs of all Lemmas and Propositions can be found in “Appendix”.

2 Preliminaries

2.1 Fixed point theorems on solid normal cones

Let $(V,||\cdot ||)$ be a Banach space with $\mathbf {0} \in V$ as its zero vector.

Definition 1

A subset $C\subset V$ is said to be a cone, if the following axioms are satisfied:

• If $v\in C$ and $t\in \mathbb {R}_+$, then $t v\in C$.

• If $v\in C$ and $-v\in C$, then $v=\mathbf {0}$.

Each cone generates a relation of partial ordering $\le _{C}$ in the following way: $v\le _{C} w$, iff $w-v\in C$.

Definition 2

A cone $C\subset V$ is said to be solid, if its interior³$\mathrm{int}(C)$ is nonempty. A cone C is said to be normal, if for all $v\in C$ and $w\in C$⁴

$$\begin{aligned} \text{ if } \mathbf {0}\le _C v\le _C w, \text{ then } ||v||\le ||w||. \end{aligned}$$

Definition 3

Let $(V,||\cdot ||)$ be a Banach space. Let $V_0\subset V$, and let $T:V_0\rightarrow V_0$ be an operator. Let $v^*\in V_0$ be a fixed point of T. The fixed point $v^*$ is said to have a global attracting property (or global attractivity) on $V_0$, if for all $v_0\in V_0$, it holds $\lim \nolimits _{n\rightarrow \infty }||T^nv_0-v^*||=0$, where $T^n:=T\circ \cdots \circ T$.

Let us introduce a few definitions.

Definition 4

Let $(V,||\cdot ||)$ be a Banach space, and let C be a cone generating an order on V. A set $V_0$ is said to be a countably chain complete subset of V if each countable chain of elements of $V_0$ (i.e., totally ordered subset of $V_0$) has a supremum and an infimum in $V_0$.

Definition 5

Let $(V,||\cdot ||)$ be a Banach space, and let C be a cone generating an order on V. Let $V_0$ be a countably chain complete subset of V. An operator $T:V_0\mapsto V_0$ is monotonically sup-preserving (monotonically inf-preserving) if for any countable chain $\tilde{V}$ of elements of $V_0$, the equality $T(\sup \tilde{V})=\sup T(\tilde{V})$ ($T(\sup \tilde{V})=\sup T(\tilde{V})$) holds.

We introduce the following fixed point theorem that we appeal to in Sects. 4 and 5 to prove the existence, the uniqueness, and the global attractivity of a recursive utility function (Theorem 2). Moreover, we prove the global attractivity of the optimal value function (Theorems 3 and 4).

Theorem 1

[Theorem 3.1.7. in Guo et al. (2004)]

Let $(V,||\cdot ||)$ be a Banach space. Assume $C\subset V$ is a solid and normal cone generating a partial ordering $\le _{C}$. Let $T:\mathrm{int}(C)\rightarrow \mathrm{int}(C)$ be an increasing operator⁵ such that there exists $r\in ]0,1[$ such that for all $v\in \mathrm{int}(C)$, $t\in ]0,1]$ it holds

$$\begin{aligned} T(tv)\ge _{C}t^{r}T(v). \end{aligned}$$

Then, T has a unique fixed point $v^{*}\in \mathrm{int}(C)$, with the global attracting property and the estimation rate of convergence:

$$\begin{aligned} ||T^{n}v_{0}-v^{*}||\le M\left( 1-\alpha ^{r^{n}}\right) \text{ for } \text{ all } n\in \mathbb {N}. \end{aligned}$$

Here, $M=2||v_{0}||$, $\alpha =\frac{t_{0}}{s_{0}}$, and $t_{0}$ and $s_{0}$ are chosen in the following way:

$$\begin{aligned} 0<t_{0}<1<s_{0}, \text{ and } t_{0}^{1-r}v_{0}\le _{C}T(v_{0})\le _{C}s_{0}^{1-r}v_{0}. \end{aligned}$$

It should be noted that the operator T in Theorem 1 maps an open set into itself. (Hence, a continuous extension of T to the closure of C can have additional fixed points.) For example, $T:\mathbb {R} _{+}\rightarrow \mathbb {R}_{+}$ with $T(x)=\sqrt{x}$ has two fixed points, but only one in the interior. Hence, neither Banach Contraction Theorem nor its extensions (like Matkowski 1975) are applicable in this case.

2.2 Basic notations and definitions in a space of functions and measures

In this section, we introduce some notations used throughout the paper. As usual, put $\mathbb {R}_+:=[0,\infty [$, $\mathbb { R}_{++}:=]0,\infty [$, and let $\mathbf {0}$ and $\mathbf {1}$ be zero and unit constant functions, respectively. Moreover, we introduce the following notation.

• Let $\varTheta$ be a metric space.

  • $\mathcal {B}(\varTheta )$ is the collection of Borel subsets of $\varTheta$.

  • $B(\varTheta )$ is the set of all Borel measurable, bounded, and real-valued functions on $\varTheta$. Clearly, $B(\varTheta )$ is a Banach space equipped with the sup-norm $||\cdot ||_{\varTheta }$, i.e.,

    $$\begin{aligned} ||v||_{\varTheta }:=\sup \limits _{\theta \in \varTheta }|v(\theta )|, \end{aligned}$$

    and by $\rightrightarrows$, we denote the convergence in the topology induced by this norm (i.e., uniform convergence on $\varTheta$).

  • By $\left[ \cdot \right] _{\varTheta }$, we denote the infimum operator on $B(\varTheta )$, i.e.,

    $$\begin{aligned} \left[ v\right] _{\varTheta }=\inf \limits _{\theta \in \varTheta }v(\theta ). \end{aligned}$$

• If $\varTheta$ is a Polish space, then $\varDelta (\varTheta )$ denotes the set of all Borel probability measures on $\varTheta$.

We now introduce the notion of a generalized mean-valued operator referred as Conditional Certainty Equivalent (CCE).

Definition 6

(Conditional Certainty Equivalent (CCE)) Let $\mathcal {F}$ be a set such that $\mathcal {F}\supset B(\varTheta )$. An operator $\mathcal {M}: \mathcal {F}\mapsto \mathcal {F}$ is said to be a Conditional Certainty Equivalent (CCE) if the following conditions hold:

• (i) Monotonicity: for each $f_1\in \mathcal {F}$ and $f_2\in \mathcal {F}$

  $$\begin{aligned} \text{ if } f_1(\cdot )\le f_2(\cdot ), \text{ then } \mathcal {M}(f_1)(\cdot )\le \mathcal {M}(f_2)(\cdot ). \end{aligned}$$

• (ii) Constant preserving property: for each $\gamma \in \mathbb { R}$, $\mathcal {M}(\gamma )(\cdot )\equiv \gamma$.

For each $f\in \mathcal {F}$ and $\theta \in \varTheta$, denote $\mathcal {M} _{\theta }(f):=\mathcal {M}(f)(\theta )$.

3 The model

3.1 Description of the model

Consider a dynamical system specified by the following objects (X, $\varGamma$, $\varOmega$, u, q, W, $\mathcal {M})$ where:

• $X\subset \mathbb {R}$ denotes the space of all possible capital levels. Suppose that $X=[0,\bar{x}]$, where $\bar{x}\in \mathbb {R}_+$ or $X=[0,\infty [$.

• For each $x\in X$, let $\varGamma (x):=[0,x]$ be the set of feasible investment levels when the current capital level is x.

• $\varOmega :=[\underline{\omega },\overline{\omega }]$ (here $0<\underline{ \omega }\le \overline{\omega }$) is the space of random shocks endowed with a Borel probability measure $\rho$.

• $q:X\times \varOmega \rightarrow \mathbb {R}$ denotes a random production function.

• $u: X \rightarrow \mathbb {R}_+$ is a temporal utility function (one-period utility).

• $W:\mathbb {R}_+\times \mathbb {R}_+\rightarrow \mathbb {R}_+$ is called an aggregator function.

• $\mathcal {M}:B(X)\rightarrow B(X)$ is a Conditional Certainty Equivalent (CCE) depending on q in the following way: if $f_1\in B(X)$, $f_2\in B(X)$, $y\in X$, and $f_1(q(y,\omega ))=f_2(q(y,\omega ))$ for $\rho -$a.a. $\omega \in \varOmega$, then

  $$\begin{aligned} \mathcal {M}_y(f_1)=\mathcal {M}_y(f_2). \end{aligned}$$

In other words, we consider a model in which the decision maker chooses a consumption level in each period $n\in \mathbb {N}$. If $x_{1}\in X$ is an initial capital level, then the agent chooses a level of the investment $y_{1}$ from the set $\varGamma (x_{1}):=[0,x_{1}],$ which implies the leftover $c_{1}:=x_{1}-y_{1}$ is called a consumption level. Then, the temporal utility for this agent $u(c_{1})$ is generated, and the next capital level is produced by the formula $x_{2}=q(y_{1},\omega _{1})$, where $\omega _{1}$ is a random shock unobservable in state 1, whose distribution is $\rho$. In state $x_{2},$ again, two things happen: the agent selects an investment level $y_{2}\in [0,x_{2}]$ and a consumption level $c_{2}:=x_{2}-y_{2}$, and the utility $u(c_{2})$ is incurred. The capital level $y_{2}$ determines the next capital level according to the formula $x_{3}=q(y_{2},\omega _{2})$. Here, $\omega _{1}$ and $\omega _{2}$ are equal in law, and both $\omega _{1}$ and $\omega _{2}$ are independent random variables. Recursively, let $(x_{n},y_{n})_{n\in \mathbb {N}}$ be a history on the capital investment system. At the same time, the sequence of independent random variables $(\omega _{n})_{n\in \mathbb {N}}$ having the distribution $\rho$ is generated.

Let H be the set of all feasible histories. Mathematically, H is the set of all sequences $h:=(x_{n},y_{n})_{n=1}^{\infty }\in Gr(\varGamma )^{\infty }$. Endow H with the natural Borel product $\sigma -$ algebra on $(Gr(\varGamma ))^{\infty }$. For $n>1$, we denote $H_n:=Gr(\varGamma )^{n-1}$ as the set of all feasible histories before step n. A policy is a sequence of jointly Borel measurable⁶ mappings such that $\sigma _1:X\rightarrow \varDelta (Y)$, $\sigma _1(\varGamma (x)|x)=1$ for each $x\in X$, and for $n>1$, $\sigma _n:H_n\times X\rightarrow \varDelta (Y)$ is such that for each $(h_n,x)\in H_n\times X$, it holds $\sigma _n( \varGamma (x_n)|h_n,x_n)=1$. Let $\varSigma$ denote the set of all policies. A policy $\sigma$ is said to be pure if for each $n\in \mathbb {N}$, $h_n\in H_n$, $x\in X$ there is $y\in Y$ such that $\sigma _n(\{y\}|h_n,x)=1$. A Markov policy is such that $\sigma _n(\cdot |h_n,x)=f_n(\cdot )$, where $f_n:X\rightarrow \varDelta (Y)$ is a Borel measurable function. The Markov policy is stationary if $\sigma _n=f$$(n\in \mathbb {N})$ for some Borel measurable function $f:X\rightarrow \varDelta (Y)$. The stationary Markov policy is identified with f. Let $x\in X$ be an initial state, and let $(\sigma _n)_{n\in \mathbb {N}}$ be an arbitrary policy. By Ionescu–Tulcea Theorem (see Neveu 1965), the production function q, the initial capital $x\in X$, and the policy $\sigma$ induce a unique probability measure $P_{x}^{\sigma }$ on H.

For each $\sigma \in \varSigma$ and $n>1$, let $\sigma ^{n}:H_{n}\rightarrow \varSigma$ be defined as $\sigma ^n:=\left( \sigma _{n+\tau }\right) _{\tau =0}^{\infty }$. Here, $\sigma ^{n}$ is called a $n-$th shift policy, i.e., the policy from the period n onward. Observe that $\sigma$ is a Markov policy if and only if for each $n\ge 1$, the $\sigma ^n$ does not depend on $H_{n}$ (i.e., is a “constant” strategy).

3.2 Construction of a recursive utility function: basic assumptions and a literature review

The purpose of this section is a construction of a recursive utility function. We start with a list of assumptions on the temporal utility, the aggregator, the production function, and CCE that are sufficient for the existence and the global attracting property of any recursive utility function.

Assumption 1

(Measurability and sub-homogeneity of CCE) $\mathcal {M}$ obeys the following conditions:

• (i) Measurability: let $k\in \mathbb {N}$ and $Z\in \mathcal {B}(\mathbb { R}^k)$, and suppose that $f:X\times Z\rightarrow \mathbb {R}$ is a jointly measurable function. Then,

  $$\begin{aligned} (y,z)\in X\times Z\rightarrow \mathcal {M}_y(f(\cdot ,z)), \end{aligned}$$

  is jointly measurable.

• (ii) Sub-homogeneity: for each $y\in X$, the operator $\mathcal {M}_{y}$ is sub-homogenous, i.e.,

  $$\begin{aligned} \text{ if } v\in B(X) \text{ and } t\in [0,1]\text{, } \text{ then } \mathcal {M}_{y}(t v)\ge t\mathcal {M}_y(v). \end{aligned}$$

For the construction of a recursive utility function, the measurability assumption of CCE is crucial. Following Hansen and Sargent (1995), for any initial state $x\in X$ and $\sigma \in \varSigma$, we define an overall utility for the agent as an approach of overall utilities in n-stage models. More precisely, the $n-$ stage overall utility is defined recursively as follows:

$$\begin{aligned} J_{1}(x,\sigma ):=\int \limits _{\varGamma (x)}W(u(x-y),0)\sigma _{1}(\mathrm{d}y|x), \end{aligned}$$

and for each $n>1$

$$\begin{aligned} J_{n}(x,\sigma ):=\int \limits _{\varGamma (x)}W\left( u(x-y),\mathcal {M} _y(J_{n-1}(\cdot ,\sigma ^{2}(x,y)))\right) \sigma _1(\mathrm{d}y|x). \end{aligned}$$

Then, the overall utility (if exists) is defined as follows:

$$\begin{aligned} J(x,\sigma )=\lim \limits _{n\rightarrow \infty }J_{n}(x,\sigma ). \end{aligned}$$

(1)

Observe that if $\sigma \in \varSigma$, then $(x^{\prime },x,y)\in X\times H_{2}\rightarrow J_{1}(x^{\prime },\sigma ^{2}(x,y))$ must be jointly measurable. Hence,

$$\begin{aligned} (y^{\prime },x,y)\in X\times H_{2}\rightarrow \mathcal {M}_{y^{\prime }}(J_{1}(\cdot ,\sigma ^{2}(x,y))) \end{aligned}$$

must be jointly measurable by (Measurability) Assumption 1. Consequently, $J_{2}$ is jointly measurable. Then, by induction, we can conclude the joint measurability of any $J_{n}$ and hence also of its supremum over $n\in \mathbb {N}$. Since W is nonnegative value, increasing in the second argument, and $\mathcal {M}_{y}(\cdot )$ is increasing for each $y\in X$, it is easy to verify that $J_{1}\le J_{2}\le \cdots J_{n}\le \cdots$. As a result, the limit in (1) always exists (although it may be infinite).

Observe that if $W(v_{1},v_{2})=v_{1}+\beta v_{2},$ and if

$$\begin{aligned} \mathcal {M}_{y}(v)=E_{y}(v):=\int _{\varOmega }v(q(y,\cdot ))\mathrm{d}\rho (\cdot ) \end{aligned}$$

(2)

then, J is the standard $\beta -$ discounted utility function.

Assumption 2

(Temporal utility) Assume $u:X\mapsto \mathbb {R}$ is a strictly increasing, bounded, and continuous function such that $u(0)\ge 0$.

Assumption 3

(Aggregator) Assume that the aggregator W obeys the following conditions:

• W is increasing in both arguments.

• W is jointly continuous.

• $W(v_1,v_2)=0$ if and only if $v_1=v_2=0$.

• There exists a constant $r\in ]0,1[$ such that for all $v_1\ge u(0)$, $v_2>0$ and $t\in ]0,1[$, it holds

  $$\begin{aligned} W(v_1,t v_2)\ge t^{r}W(v_1,v_2). \end{aligned}$$

  (3)

Assumption 4

(Production function) Suppose that the production function q obeys the following conditions:

• For each $\omega \in \varOmega$, $q(\cdot ,\omega )$ is a strictly increasing and continuous function such that $q(0,\omega )=0$.

• There exists an increasing function $K:\varOmega \rightarrow \mathbb {R}_{++}$ such that $q(x,\omega )> x$ for all $x\in ]0, K(\omega )[$ and $q(x,\omega )\le x$ for all $x\in [ K(\omega ),\infty [$.

The following assumption is needed for proving that there exists an optimal policy in the finite horizon model. Moreover, with this assumption the optimal value of the function J in the infinite time horizon model can be approximated by the n-stage optimal value function for sufficiently large n.

Assumption 5

(Continuity of CCE) Assume that for each $y\in X$, $\mathcal {M}_{y}$ satisfies:

• (i) $\mathcal {M}_{y}$ is a monotonically sup-preserving operator.

• (ii) The operator $y\in X\mapsto \mathcal {M}_{y}(v)$ is continuous whenever v is continuous.

A few comments are in order. Assumption 2 is standard. Assumption 3 is satisfied (under smoothness of W) whenever the elasticity of the aggregator with respect to its second argument has a supremum strictly less than one. More precisely, let $v_{1}\ge u(0)$, $v_{2}>0$, and $t\in ]0,1[$. Let us define an auxiliary function $f:]0,1]\rightarrow \mathbb {R}$ as follows:

$$\begin{aligned} f(t):=\ln \left( W(v_{1},tv_{2})\right) -r\ln (t). \end{aligned}$$

Then, we can obtain inequality (3) whenever f is an increasing function with each fixed pair $(v_{1},v_{2})$. Let

$$\begin{aligned} \mathcal {E}_{W}:=\sup \limits _{v_{1}\ge u(0),v_{2}>0}\frac{\frac{\partial W}{ \partial v_{2}}(v_{1},v_{2})}{W(v_{1},v_{2})}v_{2}. \end{aligned}$$

Observe,

$$\begin{aligned} f^{\prime }(t)=\frac{1}{t}\left( \frac{\frac{\partial W}{\partial v_{2}} (v_{1},tv_{2})tv_{2}}{W(v_{1},tv_{2})}-r\right) \le \frac{1}{t}(\mathcal {E} _{W}-r)\le 0 \end{aligned}$$

whenever $\mathcal {E}_{W}\le r$. Hence, if $\mathcal {E}_{W}<1$, then Assumption 3 is satisfied with r sufficiently close to 1. Assumption 4 means the production function is generally disturbed by a random noise. Further, at each step, the noise is independent of noises at each other steps. A classic example is the Cobb–Douglas production function in case of deterministic model, i.e., $\varOmega$ is a singleton. A random production function satisfying Assumption 4 can be found in many papers on the growth models. The standard expectation operator is a classic example of CCE. The expectation operator is homogenous and additionally obeys Assumption 5. More generally, CCE so called a quasilinear mean can be found in the papers by Chew (1983) and Dekel (1986). Similar (homogeneity) property has CCE based on maxmin utility considered by Gilboa and Schmeidler (1989), or on maxmax utility considered by Saponara (2018), as well as on Choquet Integral⁷ considered by Schmeidler (1989), Qu (2017), or Asano and Kojima (2019).

Example 1

The quasilinear mean has the following form:

$$\begin{aligned} \mathcal {M}_{y}(v)=\phi ^{-1}\left( E_{y}(\phi \circ v)\right) \end{aligned}$$

(4)

where $E_y(\cdot )$ is defined in Eq. (2) and $\phi$ is strictly monotone and continuous. Observe the quasilinear mean in Eq. (4) satisfies (Measurability) Assumptions 1 and 5, but the sub-homogeneity holds only in some special cases of $\phi$.

The following proposition yields a list of properties of $\phi$ that are sufficient for sub-homogeneity of the quasilinear mean in (4).

Proposition 1

Suppose that $\mathcal {M}$ has a form as in (4). Let $\eta >0$ and suppose that $\phi :[0,\eta ]\rightarrow \mathbb {R} _+$ is continuous, strictly monotone, and is differentiable on $]0,\eta [$ with a continuous and bounded derivative. Assume additionally $\phi ^{\prime }(x)\ne 0$ for $x\ne 0$, and at least one of the following alternatives are true:

• (i) $\phi$ is strictly increasing, and the function $\phi ^{\prime }(\phi ^{-1}(\cdot ))\phi ^{-1}(\cdot )$ is concave.

• (ii) Or $\phi$ is strictly decreasing, and the function $\phi ^{\prime }(\phi ^{-1}(\cdot ))\phi ^{-1}(\cdot )$ is convex.

Then, for each $y\in X$, $\mathcal {M}_{y}(\cdot )$ is sub-homogenous and consequently satisfies Assumption 1.

By Proposition 1, it is easy to verify the class of CCE in Eq. (4) with $\phi (t)=t^{\gamma }$ ($\gamma >0$) satisfies Assumption 1. The class of CCE with $\phi (t)=e^{-\gamma t}$ ($\gamma >0$, and $t>0$) called a risk measure also satisfies Assumption 1. Indeed, $\phi$ is strictly decreasing, and $\phi ^{-1}(\tau )=-\frac{1}{\gamma }\ln (\gamma \tau )$ for $\tau \in ]0,\frac{1}{\gamma }]$. Note further that

$$\begin{aligned} \phi ^{\prime }\left( \phi ^{-1}(t)\right) \phi ^{-1}(t)=\gamma t\ln (\gamma t) \end{aligned}$$

is convex. Hence, by Proposition 1 it follows that the risk measure satisfies Assumption 1.

Example 2

It should be noted that the aggregator W in Eq. (1) is not uniquely determined. The “natural” aggregator W does not obey Assumption 3, but there exists another aggregator which does obey this assumption and yields the same output as W in Eq. (1). For example, consider the case $W(v_{1},v_{2})=v_{1}+\beta v_{2}$. Assume u is bounded above by $\bar{u}$ and $u(0):=\delta >0$. Then, $\mathcal {E}_{W}=1$ and Assumption 3 is not satisfied. On the other hand, $J(x,\sigma )\le \frac{\bar{u}}{1-\beta }:=\bar{U}$ for each $x\in X$ and $\sigma \in \varSigma$. Put

$$\begin{aligned} \tilde{W}(v_{1},v_{2}):=W(v_{1},\min (v_{2},\bar{U})). \end{aligned}$$

(5)

Hence, both W and $\tilde{W}$ yield the same output in Eq. (1). Then, $\mathcal {E}_{\tilde{W}}=\frac{\beta \bar{U}}{\epsilon +\beta \bar{U}} <1$. Therefore, Assumption 3 is satisfied. Similar transformation works for other aggregators where $\bar{U}$ is the least fixed point of $W(\bar{u},\cdot )$.

Finally, let us comment on Assumption 3. Observe that the paper by Marinacci and Montrucchio (2010) includes a few aggregators which satisfy Assumption 3.

Example 3

Consider a class of Thompson aggregators⁸ in the following form:

$$\begin{aligned} W(v_{1},v_{2})=\left( v_{1}^{\xi }+\beta v_{2}^{\eta }\right) ^{\frac{1}{p}}, \end{aligned}$$

(6)

where $\xi >0$, $0<\eta <p$, and $\beta \in ]0,1[$. Indeed, for each $v_{1}>0$, $v_{2}>0$, and $t\in ]0,1[$, we have

$$\begin{aligned} \frac{\frac{\partial }{\partial v_{2}}W(v_{1},v_{2})}{W(v_{1},v_{2})}v_{2}= \frac{\eta }{p}\frac{\beta v_{2}^{\eta }}{v_{1}^{\xi }+\beta v_{2}^{\eta }} \le \frac{\eta }{p}<1. \end{aligned}$$

As a result, this aggregator satisfies Assumption 3.

The next aggregator is a modification of the aggregator by Koopmans et al. (1964).

Example 4

Consider the aggregator in the following form:

$$\begin{aligned} W(v_{1},y_{2})=\frac{1}{\theta }\log \left( 1+v_{1}^{\xi }+\beta v_{2}^{\eta }\right) , \end{aligned}$$

with $\theta >0$, $\xi >0$, and $\eta \in ]0,1[$. We have in this case

$$\begin{aligned} \frac{\frac{\partial }{\partial v_{2}}W(v_{1},v_{2})}{W(v_{1},v_{2})} v_{2}= & {} \eta \frac{\beta v_{2}^{\eta }}{(1+v_{1}^{\xi }+\beta v_{2}^{\eta })\log (1+v_{1}^{\xi }+\beta v_{2}^{\eta })}\nonumber \\ {}\le & {} \eta \sup _{x>0}\frac{x}{ (1+x)\log (1+x)}<1. \end{aligned}$$

As a result, Assumption 3 is satisfied also in this case.

The following proposition states that Thompson aggregators in fact obey Assumption 3 whenever $u(0)=\delta >0$.

Proposition 2

Assume W is Thompson, $u(0)=\delta >0$ and $||u||_{X}=\bar{u}$. Let $\bar{U}$ be the least fixed point of $W(\bar{u},\cdot )$ and let us define $\tilde{W}$ as in Eq. (5). Then, $\tilde{W}$ obeys Assumption 3.

Another example shows that Assumption 3 does not imply Thompson property. Namely, in this example, the concavity at 0 is violated.

Example 5

Let $\psi :[0,\infty [\rightarrow [0,\infty [$ be defined as follows: $\psi (x)=\root 4 \of {x}$ if $x\in [0,1[$ and $\psi (x)=\sqrt{x}$ otherwise. Consider the aggregator of the form $W(v_{1},v_{2})=\psi (v_{1}+v_{2})$. Clearly, such W satisfies Assumption 3 since for each $t\in ]0,1[$

$$\begin{aligned} \psi (v_{1}+tv_{2})\ge \psi (t(v_{1}+v_{2}))\ge t^{\frac{1}{2}}\psi (v_{1}+v_{2}). \end{aligned}$$

Observe, however, W is not concave at 0. Put $v_{1}=0.9$, $v_{2}=1.1$, and $t=1/11$. Then, $v_{1}+v_{2}=2$ and $v_{1}+tv_{2}=1$. Hence, $W(v_{1},tv_{2})=1,$ but

$$\begin{aligned} tW(v_{1},v_{2})+(1-t)W(v_{1},0)=\frac{1}{11}\sqrt{2}+\frac{10}{11}\root 4 \of { 0.9}\approx 1.04>1=W(v_{1},tv_{2}), \end{aligned}$$

which contradicts the concavity at 0. We can obtain similar results for the aggregator $\psi (v_{1}+\beta v_{2})$ where $\beta \in ]0,1[$.

4 Existence and global attracting property of a recursive utility function

In this section, we prove the existence and the global attractivity of a recursive utility function.

First, we need to construct a universal set for desired functions. For each $h\in H,$ and $n\in \mathbb {N}$, let $h_{n}$ be the natural projection of h on $H_{n}$. Define

$$\begin{aligned} \mathbf {U}:= & {} \{U:X\times \varSigma \rightarrow \mathbb {R}:U \text{ is } \text{ bounded, } \text{ and } \text{ for } \text{ each } \sigma \in \varSigma ,n\in \mathbb {N}\\&U(x_{n},\sigma ^{n}(h_{n})) \text{ is } \text{ jointly } \text{ measurable } \text{ in } (h_{n},x_{n})\in H_{n}\times X\}. \end{aligned}$$

Clearly, $\mathbf {U}$ endowed with the natural sup-norm topology

$$\begin{aligned} ||U||_{X\times \varSigma }:=\sup \limits _{(x,\sigma )\in X\times \varSigma }|U(x,\sigma )| \end{aligned}$$

is a Banach space. (More precisely, it is a closed subspace of the Banach space of bounded functions defined on $X\times \varSigma$).

Put $\mathbf {U}_{+}$ as the set of nonnegative functions from $\mathbf {U}$. Then, $\mathbf {U}_{+}$ is a normal cone. Moreover, it induces the standard component-wise order, that is $U_{1}\le U_{2}$ iff $U_{1}(x,\sigma )\le U_{2}(x,\sigma )$ for each $(x,\sigma )\in X\times \varSigma$.

Clearly, $\mathbf {U}_+$ is a solid cone, and

$$\begin{aligned} \mathrm{int}(\mathbf {U}_+):=\left\{ U\in \mathbf {U}_+:\inf \limits _{(x,\sigma )\in X\times \varSigma }U(x,\sigma )>0 \right\} . \end{aligned}$$

(7)

According to Koopmans (1960), Epstein and Zin (1989), or Marinacci and Montrucchio (2010), among others, we introduce the following definition:

Definition 7

$U^{*}\in \mathbf {U}_{+}$ is said to be a recursive utility function if for any policy $\sigma \in \varSigma$, and any initial state $x\in X,$ the following holds:

$$\begin{aligned} U^{*}(x,\sigma )=\int _{\varGamma (x)}W\left( u(x-y),\mathcal {M}_{y}(U^{*}(\cdot ,\sigma ^{2}(x,y)))\right) \sigma _{1}(\mathrm{d}y|x). \end{aligned}$$

Observe a recursive utility function $U^{*}$ (if it exists) is a fixed point of the following operator:

$$\begin{aligned} T_{W}(U)(x,\sigma ):=\int _{\varGamma (x)}W\left( u(x-y),\mathcal {M}_{y}(U(\cdot ,\sigma ^{2}(x,y)))\right) \sigma _{1}(\mathrm{d}y|x). \end{aligned}$$

For all $\delta \ge 0,$ put

$$\begin{aligned} T_{W}^{\delta }(U)(x,\sigma ):=\int _{\varGamma (x)}W\left( u_{\delta }(x-y), \mathcal {M}_{y}(U(\cdot ,\sigma ^{2}(x,y)))\right) \sigma _{1}(\mathrm{d}y|x), \end{aligned}$$

where $u_{\delta }(x-y)=\max \left( u(x-y),\delta \right)$. Obviously, $u_{0}\equiv u$. If $u(0)>0$, then $T_{W}^{\delta }\equiv T_{W}$ for $\delta <u(0)$. The purpose of this section is to construct a recursive utility function and find its relationship with the function J defined in (1). In the first main result below, we state the existence and the global attracting property of a bounded recursive utility function.

Theorem 2

Suppose Assumptions 1, 2, 3, and 4 are satisfied. Then, there exists a unique recursive utility function such that $U^{*}\in \mathrm{int}(\mathbf {U}_{+})$. Moreover,

• (i) $U^*$ is globally attractive on $\mathrm{int}(\mathbf {U}_+)$, i.e.,

  $$\begin{aligned} \lim _{n\rightarrow \infty }||U^*-T^n_W(U)||_{X\times \varSigma }=0 \end{aligned}$$

  whenever $U\in \mathrm{int}(\mathbf {U}_+)$.

• (ii) The truncation error satisfies:

  $$\begin{aligned} ||T^n_W(U)-U^*||_{X\times \varSigma }\le M\left( 1-\alpha ^{r^n}\right) \text{ for } \text{ all } n\in \mathbb {N} \end{aligned}$$

  whenever $U\in \mathrm{int}(\mathbf {U}_+)$. Here, $M=2||U||_{X\times \varSigma }$, $\alpha = \frac{t_0}{s_0}$, and $t_0$ and $s_0$ are chosen in the following way:

  $$\begin{aligned} 0<t_0<1<s_0, \text{ and } \text{ it } \text{ holds } t_0^{1-r}U(\cdot )\le T_W(U)(\cdot )\le s_0^{1-r} U(\cdot ). \end{aligned}$$

• (iii) J is a recursive utility function. Moreover, $J(x,\sigma )\le U^*(x,\sigma )$ for each $x\in X$ and $\sigma \in \varSigma$, and $J(x,\sigma )=U^*(x,\sigma )$ whenever $x>0$ and $\sigma \in \varSigma _{x,\delta }$.

To prove Theorem 2, we need to prove some auxiliary results, namely Lemma 1 and Lemma 2. Proofs of both Lemmas are in the “Appendix”.

Lemma 1

Let Assumptions 1, 2, 3, and 4 be satisfied. Then, for each $\delta \ge 0$,

• (i) $T_W^{\delta }$ maps both $\mathbf {U}_+$ and $\mathrm{int}(\mathbf {U}_+)$ into itself.

• (ii) $T_W^{\delta }(\cdot )$ is an increasing operator, and for each $U\in \mathbf {U}_+$ the function $T_W^{\delta }(U)$ is increasing in $\delta$.

• (iii) If $\delta >0$, then $T_W^{\delta }(\mathbf {0})(x,\sigma )\ge W(\delta ,0)>0$.

• (iv) J is well defined, and if U is any fixed point of $T_W^{\delta }$, then $J\le U.$

For each $x>0$ and $\delta >0$, let

$$\begin{aligned} \varSigma _{x,\delta }:=\left\{ \sigma \in \varSigma : P^{\sigma }_{x}\left( \left\{ h:=(x_n,y_n)_{n\in \mathbb {N}}:\inf _{n\in \mathbb {N} }u(x_n-y_n)\ge \delta \right\} \right) =1\right\} . \end{aligned}$$

The next lemma shows that if a capital level $x\in X$ is strictly positive, and then, we can guarantee the our temporal utility is never below some fixed value.

Lemma 2

Let $x>0$ and suppose that Assumptions 1, 2, 3, and 4 hold. Then, there exists $\delta >0$ such that $\varSigma _{x,\delta }\ne \emptyset$.

Now, we prove Theorem 2.

Proof of Theorem 2

Let us prove (i) and (ii) together. Moreover, we prove similar results for all operators $T_{W}^{\delta }$$(\delta \ge 0)$. Put $t\in ]0,1[$, $U\in \mathrm{int}(\mathbf {U}_{+})$, and $(x,\sigma )\in X\times \varSigma$. From the sub-homogeneity of $\mathcal {M}_{y}$ (Assumption 1) and from Assumption 3, it holds

$$\begin{aligned} T_{W}^{\delta }(tU)(x,\sigma )&\ge t^{r}\int _{\varGamma (x)}W\left( u_{\delta }(x-y),\mathcal {M}_{y}(U(\cdot ,\sigma ^{2}(x,y)))\right) \sigma _{1}(\mathrm{d}y|x)\\&=t^{r}T_{W}^{\delta }(U)(x,\sigma ). \end{aligned}$$

As a result, by Theorem 1, there is a unique $U^{\delta }\in \mathrm{int}(\mathbf {U}_{+})$ such that $U^{\delta }$ is a fixed point of $T_{W}^{\delta }$. In particular, we put $U^{*}:=U^{0}$, which satisfies (i) and (ii).

Proof of (iii). For all $n\in \mathbb {N}$, $x\in X$, $\sigma \in \varSigma$, and $\delta \ge 0$ define $J^{\delta }_n(x,\sigma ):=(T^{\delta }_W)^n(\mathbf {0} )(x,\sigma )$ (the nth composition of $\mathbf {0}$-function). By Lemma 1, $J^{\delta }_n\in \mathrm{int}(\mathbf {U}_+)$ whenever $\delta >0$. By Theorem 1,

$$\begin{aligned} \lim \limits _{n\rightarrow \infty }||J^{\delta }_n-U^{\delta }||_{X\times \varSigma }=0, \end{aligned}$$

(8)

and $U^{\delta }$ satisfies (i) and (ii) of this theorem.

By Lemma 1 (iv), it holds $J\le U^{*}$. Hence, J is always bounded. We show that $J(x,\sigma )\ge U^{*}(x,\sigma )$ if $x>0$, and $\sigma \in \varSigma _{x,\delta }$ where $\delta >0$ is defined in such a way $\varSigma _{x,\delta }\ne \emptyset$. Observe that by Lemma 2, such $\delta$ exists. We need to show that $J_{n}(x,\sigma )=(T_{W}^{\delta })^{n}(\mathbf {0})(x,\sigma )$ for any $n\ge 1$, $x>0$, and $\sigma \in \varSigma _{x,\delta }$. We prove this fact by induction on n. By the definition of $J_{1}$, $u_{\delta }$, and the definition of $\varSigma _{x,\delta }$, we have

$$\begin{aligned} J_{1}(x,\sigma )=\int _{\varGamma (x)}W(u_{\delta }(x-y),0)\sigma _{1}(\mathrm{d}y|x)=T_{W}^{\delta }(\mathbf {0})(x,\sigma ). \end{aligned}$$

Suppose that

$$\begin{aligned} J_{k}(x^{\prime },\sigma ^{\prime })=(T_{W}^{\delta })^{k}(\mathbf {0} )(x^{\prime },\sigma ^{\prime }) \end{aligned}$$

(9)

for some integer k and each $x^{\prime }\in X$, $\sigma ^{\prime }\in \varSigma _{x^{\prime },\delta }$. Since $\sigma \in \varSigma _{x,\delta }$, $\sigma (x,y)\in \varSigma _{x^{\prime },\delta }$ for $P_{2}-$ a.a. $(x,y,x^{\prime })\in H_{2}\times X$, where $P_{2}$ is the marginal of $P_{x}^{\sigma }$ on $H_{2}\times X$. Hence, by (9) we have

$$\begin{aligned} J_{k+1}(x,\sigma )= & {} \int _{\varGamma (x)}W(u(x-y),\mathcal {M}_y(J_k(\cdot , \sigma ^2(x,y))))\sigma _1(\mathrm{d}y|x)\nonumber \\ {}= & {} (T^{\delta }_W)^{k+1}(\mathbf {0})(x,\sigma ). \end{aligned}$$

As a result, for each k it holds $J_{k}(x,\sigma )=(T^{\delta }_W)^{k}(\mathbf {0})(x,\sigma )$. Hence, by (8), $J(x,\sigma )$ coincides with $U^{\delta }$ for $x>0$ and $\sigma \in \varSigma _{x,\delta }$. As a result, by the point (i) of this theorem $J(x,\sigma )\ge U^*(x,\sigma )$ for such $\sigma$. $\square$

5 Bellman equation and the existence of an optimal program

In this section, we study the maximization problem of the utility J. The so-called J-optimal value function is defined as follows

$$\begin{aligned} x\in X \mapsto \sup _{\sigma \in \varSigma }J(x,\sigma ). \end{aligned}$$

(10)

With the aforementioned optimization problem is associated a dual maximization problem of $U^*$, where $U^*$ a unique recursive utility from $\mathrm{int}(\mathbf {U})$ which exists by Theorem 2. The so-called U-optimal value in the dual problem is defined as follows

$$\begin{aligned} x\in X\mapsto \sup \limits _{\sigma \in \varSigma }U^*(x,\sigma ). \end{aligned}$$

(11)

In this section, we show that both U-optimal and J-optimal value functions satisfy the corresponding Bellman equation, i.e.,

$$\begin{aligned} v(x)=\sup \limits _{y\in \varGamma (x)}W(u(x-y),\mathcal {M}_{y}(v)), \end{aligned}$$

whenever v is the U-optimal or the J-optimal value function.

Let

$$\begin{aligned} BP(v)(x)=\sup \limits _{y\in \varGamma (x)}W(u(x-y),\mathcal {M}_{y}(v)). \end{aligned}$$

The main results of this section are established in Theorems 3 and 4. In Theorem 3, we consider some properties of the U-optimal value function, and we show that it obeys the Bellman equation and establish the existence of a U-optimal policy. In turn, in Theorem 4, we then consider some properties of J-optimal value function, and we show that it satisfies the Bellman equation.

In the remaining part of this paper, we use the following notation $B_{+}:=B_{+}(X)$ and $B_{+}^{o}:=\mathrm{int}(B_{+}(X))$ for short.

Theorem 3

Under Assumptions 1, 2, 3, 4, BP maps $B_{+}^{o}$ into itself and it is an increasing operator. Moreover,

• (i) There exists a function $\tilde{v}$ that is a unique fixed point of BP such that $\tilde{v}\in B_{+}^{o}$, and such that for each $v\in B_{+}^{o}$

  $$\begin{aligned} \lim \limits _{n\rightarrow \infty }||BP^{n}(v)-\tilde{v}||_{X}=0, \end{aligned}$$

  with the truncation error satisfying

  $$\begin{aligned} ||BP^{n}(v)-\tilde{v}||_{X}\le M\left( 1-\alpha ^{r^{n}}\right) \text{ for } \text{ all } n\in \mathbb {N}, \end{aligned}$$

  where $M=2||v||_{X}$, $\alpha =\frac{t_{0}}{s_{0}}$, and $t_{0}$ and $s_{0}$ are chosen in the following way:

  $$\begin{aligned} 0<t_{0}<1<s_{0}, \text{ and } \text{ it } \text{ holds } t_{0}^{1-r}v(\cdot )\le BP(v)(\cdot )\le s_{0}^{1-r}v(\cdot ). \end{aligned}$$

• (ii) $\tilde{v}$ is increasing. Moreover, if additionally Assumption 5 holds, $\tilde{v}$ is continuous.

• (iii) If Assumption 5 holds, then for each $x\in X$

  $$\begin{aligned} \tilde{v}(x)=\sup \limits _{\sigma \in \varSigma }U^*(x, \sigma ), \end{aligned}$$

  and there exists a stationary and pure U-optimal policy $\tilde{\sigma } \in \varSigma$ satisfying

  $$\begin{aligned} \tilde{v}(x)=W(x-\tilde{\sigma }(x),\mathcal {M}_{\tilde{\sigma }(x)}(\tilde{v} ))=\sup \limits _{y\in \varGamma (x)}W(u(x-y),\mathcal {M}_{y}(\tilde{v})). \end{aligned}$$

  (12)

Corollary 1

Suppose that Assumptions 1, 2, 3, 4, and 5 hold. Furthermore, suppose that $u\in B_+^o$. Then, the set of U-optimal and J-optimal policies are nonempty and coincide. As a result,

$$\begin{aligned} \sup \limits _{\sigma \in \varSigma }J(x,\sigma )=\sup \limits _{\sigma \in \varSigma }U^*(x, \sigma )\quad \text{ and } x\in X. \end{aligned}$$

Moreover,

$$\begin{aligned} \tilde{v}(x)=W(u(x-\tilde{\sigma }(x)),\mathcal {M}_{\tilde{\sigma }(x)}(\tilde{ v}))=BP(\tilde{v})(x). \end{aligned}$$

(13)

Let us first introduce the following notation. For each sequence $(x_{n})_{n\in \mathbb {N}},$ let the sequence $x^{(n)}:=\left( x_{k}^{(n)}\right) _{k\in \mathbb {N}}$ be defined in the following way: $x_{k}^{(n)}=x_{k}$ for $k\le n$ and $x_{k}^{(n)}=0$ for $k\ge n+1$. In turn, the sequence $x^{(-n)}$ is defined in the following way: $x_{k}^{(-n)}=x_{n-k+1}$ if $k\le n$ and $x_{k}^{(-n)}=0$ for $k\ge n+1$. Now, consider the $n-$period horizon model. Each policy $\pi$ in the n-step model is identified with $\sigma ^{(n)}$ for some policy $\sigma \in \varSigma$. Observe if $\pi$ is a policy in the n-step model, then $\pi$ can be embedded into $\varSigma$ in the canonical way. Hence, we can write $J_{n}(x,\pi )=J(x,\sigma ^{(n)})$.

Let $v_{n}^{*}(\cdot )$ be the value function in the n-step model. That is

$$\begin{aligned} v_{n}^{*}(x)=\sup \limits _{\pi \in \varSigma ^{n}}J_{n}(x,\pi ). \end{aligned}$$

Here, $\varSigma ^{n}$ denotes the set of policies in the $n-$ step game. Observe $\pi \in \varSigma ^{n}$ if and only if $\pi =\sigma ^{(n)}$ for some $\sigma \in \varSigma$.

In Theorem 4, we shall study some properties of the optimal value of J, and we argue that under additional Assumption 5 the optimal values of $U^{*}$ and J coincide with $X\setminus \{0\}$. We also establish some results of an existence of optimal policies in the finite step model, and we show that the J-optimal policy obeys the Bellman equations. Unfortunately, Theorem 4 does not establish the existence of a J-optimal policy, since the optimal function need not be upper semicontinuous at $x=0$.

Theorem 4

Let Assumptions 1, 2, 3, 4, and 5 be satisfied. Put $v_0^*=\mathbf {0}$. Then,

• (i) For all $n\in \mathbb {N}$, $v_n^*(\cdot )$ is an increasing and continuous function. Furthermore, there exists a sequence of Borel measurable functions $(\hat{\sigma }_k)_{k\in \mathbb {N}}$ that are selections of $\varGamma$ such that $\hat{\sigma }^{(-n)}$ is a Markov optimal policy in the n-step model, for each $x\in X$ it holds

  $$\begin{aligned} v_{n+1}^*(x)=BP(v_n^*)(x), \end{aligned}$$

  (14)

  and for $n\ge 1$, and $v_1^*(x)\le v_2^*(x)\le \cdots v_n^*(x)\le \cdots$. As a result, there exists a limit

  $$\begin{aligned} v^{*}(x):=\lim \limits _{n\rightarrow \infty }v_n(x). \end{aligned}$$

  (15)

• (ii) Let v be an arbitrary fixed point of BP. Then, $v^{*}(\cdot )\le v(\cdot )$.

• (iii) For each $x\in X$

  $$\begin{aligned} v^*(x)=\lim \limits _{n\rightarrow \infty }v_n^*(x)=\sup \limits _{\sigma \in \varSigma }J(x, \sigma ). \end{aligned}$$

  (16)

• (iv) For each $x\in X\setminus \{0\}$, it holds $\tilde{v}(x)= v^*(x)$; as a result, $v^*$ is continuous on $X\setminus \{0\}$.

The following corollary establishes the existence of the J-optimal policy. The following corollary is immediate from Theorem 3(iv).

Corollary 2

If $\tilde{\sigma }$ be a stationary J-optimal policy, then it is also a U-optimal policy.

Remark 1

Observe that a U-optimal policy need not be J-optimal. On the other hand, if $(x_n)_{n\in \mathbb {N}}$ is a sequence of capital levels generated by the policy $\tilde{\sigma }$, and if

$$\begin{aligned} \inf _{n\in \mathbb {N}}u(x_n-y_n)>0 \text{ for } P_{x}^{\tilde{\sigma }}- \text{ almost } \text{ all } \text{ histories } h=(x_n,y_n)_{n\in \mathbb {N}} \end{aligned}$$

then by Theorem 2 (iii), the utility $U(x,\tilde{ \sigma })$ coincides with $J(x,\tilde{\sigma })$, and consequently $\tilde{ \sigma }$ is a J-optimal policy. In particular, all the theses hold if $u\in B_+^o$.

The following remark justifies some problems with the uniqueness of the solution of Bellman equation.

Remark 2

It should be noted that $BP^{n}(\mathbf {1})(\cdot )\rightrightarrows \tilde{v} (\cdot )$, while (unless $u(0)>0$) $BP^{n}(\mathbf {0})(\cdot )\rightarrow v^{*}(\cdot )$ pointwise only. The reason is neither of the function $BP^{n}(\mathbf {0})(\cdot )$ is an element of $B_{+}^{o}$. Hence, theorem by Guo et al. (Theorem 1) does not work with $\mathbf {0}$ as a starting point.

Theorem 4 is applicable for many aggregators known in the literature. Sometimes, a trick is needed.

Example 6

Koopmans et al. (1964) study the aggregator

$$\begin{aligned} W(v_{1},v_{2})=\theta ^{-1}\ln \left( 1+v_{1}^{\rho }+\beta v_{2}\right) , \end{aligned}$$

with $\theta >0$ (small enough), and $\rho$, $\beta >0$. Observe $W(0,\cdot )$ has exactly one strictly positive fixed point $\kappa$. Put $W^{\kappa }(v_{1},v_{2}):=W(v_{1},\max (v_{2},\kappa ))$.⁹ Observe

$$\begin{aligned} \mathcal {E}_{W^{\kappa }}\le \frac{\beta \kappa }{\left( 1+\beta \kappa \right) \ln \left( 1+\beta \kappa \right) }<1. \end{aligned}$$

By Lemma 2, for any $x_{0}\in X$ a ”constant” policy $y^{*}(x_{0})$ is feasible and $v^{*}(x_{0})\ge J(x_{0},y^{*}(x_{0}))\ge \kappa$. Let $J^{\kappa }$ be defined as J where W is substituted by $W^{\kappa }$. Obviously, $J^{\kappa }(\cdot )\ge J(\cdot )$. Then, $v^{*}(x)=\sup \nolimits _{\sigma \in \varSigma }J^{\kappa }(x,\sigma )$. Hence, all theses of Theorem 4 hold.

Sometimes, the trick from the previous remark is impossible. The next theorem establishes asymptotic properties of the optimal value function with varying lower bound of the temporal utility. For any $\delta >0$, consider the dynamical system (X,  $\varGamma ,$$\varOmega ,$$u^{\delta },$q,  W,  $\mathcal {M})$ called a $\delta$-model, i.e., the dynamical system where u is substituted by $u^{\delta }(\cdot )$. Let $v_n^{\delta }$ and $v^{\delta }$ denote the optimal values in the n and, respectively, in the infinite step model. This theorem extends the applicability toward all Thompson aggregators without restrictions that $u(0)>0$.

Theorem 5

(Asymptotic properties) Let assumptions of Theorem 4 be satisfied for any $\delta$-model with $\delta >0$. Then,

$$\begin{aligned} v^*(x)=\sup \limits _{n\in \mathbb {N}}\inf \limits _{\delta >0}v_n^{\delta }(x). \end{aligned}$$

(17)

For proving Theorems 3 and 4, we applied Theorem 1. Alternatively, we could try to apply Knaster–Tarski Theorem (Tarski 1955). But in the next remark, we indicate some problems with the applicability of Knaster–Tarski Theorem in this context.

Example 7

Let $\epsilon >0$ and $N>0$ be such that $\epsilon <N$. Let us define

$$\begin{aligned} \mathbf {U}^{\epsilon ,N}:=\left\{ U\in \mathbf {U}_{+}:U(x,\sigma )\in \left[ \epsilon ,N\right] \text{ for } \text{ each } (x,\sigma )\in X\times \varSigma \right\} . \end{aligned}$$

Notice, for small enough $\epsilon$ and large enough N,  the operator $T_{W}$ maps $\mathbf {U}^{\epsilon ,N}$ into itself. Also, note that $\mathbf { U}^{\epsilon ,N}$ is not a complete lattice under the pointwise partial orders (because of the measurability requirement). Because of this fact, we cannot use directly Knaster–Tarski Theorem. On the other hand, we can easily prove that the sequence of iterations $T_{W}^{n}(N)$ is decreasing in n and consequently the limit $T_{W}^{\infty }(N)$ exists and belongs to $\mathbf {U} ^{\epsilon ,N}$. At this stage, we cannot conclude that $T_{W}^{\infty }(N)$ is a fixed point of $T_{W}$ as $T_{W}$ need not be an inf-preserving operator.¹⁰ For justifying this fact, consider the following example. Let $X=[0,1]$, $\varOmega =\left\{ 1-\frac{1}{k}:k\in \mathbb {N}\setminus \{0\}\right\}$, $\rho$ be a measure supported on whole $\varOmega$, $q(y,\omega ):=\frac{1+\omega }{2} \sqrt{y}$, and

$$\begin{aligned} \mathcal {M}_{y}(f):=\inf \limits _{\{\omega \in \varOmega \}}f(q(y,\omega )). \end{aligned}$$

We show that $\mathcal {M}$ is not inf-preserving. It is easy to see that $\mathcal {M}$ is a CCE satisfying Assumption 1. But taking $f_{n}(x)=1-x^{n}$ and $y=1$, we have $\mathcal {M}_{y}(f_{n})=1$ for each $n\in \mathbb {N}$ while for $f(x)=1-\mathbf {1}_{\{1\}}(x)$, we have $\mathcal {M} _{y}(f)=0$.¹¹

Finally, we prove Theorems 3, 4, and next Theorem 5.

Proof of Theorem 3

We shall prove all the points separately. Clearly, BP is an increasing operator on $B_+$.

Proof of (i). First, we show that BP maps $B^o_+$ into itself. Indeed, if $v\in B^o_+$, then $||v||_X<\infty$, $[v]_X>0$, and for each $(x,y)\in Gr(\varGamma )$

$$\begin{aligned} BP(v)(x)\ge W(u(x-y),\mathcal {M}_y(v))\ge W(0,[v]_X)>0. \end{aligned}$$

Moreover,

$$\begin{aligned} BP(v)\le W\left( ||u||_{X},||v||_{X}\right) <\infty . \end{aligned}$$

Observe that, by Assumptions 2, the function $BP(v)(\cdot )$ must be increasing and hence Borel measurable. Therefore, $BP(v)\in B_{+}^{o}$. So, BP maps $B_{+}^{o}$ into itself. It is easy to see that $BP(\cdot )$ is an increasing operator. By Assumption 3 and (the sub-homogeneity) Assumption 1, we conclude $BP(tv)(\cdot )\ge t^{r}BP(v)(\cdot )$ for each $v\in B_{+}^{o}$ and $t\in ]0,1[$. Then, as a result of Theorem 1, the proof of (i) is complete.

Proof of (ii). It is clear that $\tilde{v}$ is increasing. Indeed, in the point (i) of this theorem, we have already established that the range of $BP(\cdot )$ is included in the set of increasing functions.

Now, suppose Assumption 5 holds. We show that $\tilde{v}$ is continuous. First, we need to demonstrate that $B(v)(\cdot )$ is a continuous function whenever v is. Indeed, if v is continuous, then from Assumption 5 the function $y\in X\mapsto \mathcal {M}_{y}(v)$ is continuous. Therefore, $(x,y)\in Gr(\varGamma )\mapsto W(u(x-y),\mathcal {M} _{y}(v))$ is jointly continuous. Hence, by Berge Maximum Theorem [Theorem 17.31 in Aliprantis and Border (2006)], it follows that $BP(v)(\cdot )$ is continuous. Consequently, $BP(\mathbf {1})(\cdot )$ is continuous, and hence, for each $n\in \mathbb {N}$, $BP^{n}(\mathbf {1})(\cdot )$ is continuous. By the point (i) of this theorem, it follows that $BP^{n}(\mathbf {1})(\cdot )\rightrightarrows \tilde{v}(\cdot )$. Hence, $\tilde{v}(\cdot )$ must be continuous.

Proof of (iii). From the point (ii) of this theorem, Assumptions 2, 3, 5, and Measurable Maximum Theorem [see Theorem 18.19 in Aliprantis and Border (2006)], there exists a measurable function $\tilde{\sigma }:X\mapsto Y$ (a selection of $\varGamma$) such that $\tilde{\sigma }$ and $\tilde{v}$ satisfy Eq. (12). We shall show that $\tilde{\sigma }$ is U-optimal. Consider another policy $\sigma \in \varSigma$. Let $h=(x_{n},y_{n})_{n\in \mathbb {N}}$ be a history generated by $\sigma$ such that $x_{1}=x$. Then,

$$\begin{aligned} \tilde{v}(x_{\tau })\ge W(u(x_{\tau }-y_{\tau }),\mathcal {M}_{y_{\tau }}(\tilde{v})) \end{aligned}$$

for each $\tau \in \mathbb {N}$. Hence, by the definition of CCE and Assumption 3

$$\begin{aligned} \tilde{v}(x)\ge T_{W}^{n}(\tilde{v})(x,\sigma ) \end{aligned}$$

for each $n\in \mathbb {N}$. Note that $\tilde{v}$ may be embedded into $\mathrm{int}(\mathbf {U}_{+})$. Indeed, $\tilde{v}$ is Borel measurable as a function of the initial state, and $\tilde{v}$ does not depend on $\sigma$. Hence, taking the limit $n\rightarrow \infty$ and applying Theorem 2 we have $\tilde{v}(x)\ge U^{*}(x,\sigma )$. On the other hand,

$$\begin{aligned} \tilde{v}(x_{\tau })=W(u(x_{\tau }-\tilde{\sigma }(x_{\tau })),\mathcal {M}_{ \tilde{\sigma }(x_{\tau })}(\tilde{v})) \end{aligned}$$

for each $\tau \in \mathbb {N}$ and for each $n\in \mathbb {N}$, and it holds

$$\begin{aligned} \tilde{v}(x)=T_{W}^{n}(\tilde{v})(x,\tilde{\sigma }). \end{aligned}$$

Hence, taking the limit $n\rightarrow \infty$ and applying Theorem 2 we have $\tilde{v}(x)=U^{*}(x,\tilde{\sigma })$. As a result, $U^{*}(x,\tilde{\sigma })\ge U^{*}(x,\sigma )$ for each $\sigma \in \varSigma$. $\square$

Proof of Theorem 4

Proof of (i). For each $n\in \mathbb {N}$, we construct a sequence of measurable functions $(\hat{ \sigma }_n)_{n\in \mathbb {N}}$ that are selections of $\varGamma$ and satisfy the following conditions:

• $\hat{\sigma }_1(x)=0$ for each $x\in X$, and $\hat{\sigma }_n(\cdot )$ is a Borel measurable selection of $\varGamma$ satisfying:

  $$\begin{aligned} W(u(x-\hat{\sigma }_n(x)),\mathcal {M}_{\hat{\sigma }_n(x)}(v^*_{n-1}))=\max \limits _{y\in \varGamma (x)}W(u(x-y),\mathcal {M}_y(v^*_{n-1})). \end{aligned}$$

  (18)

• For each $n\in \mathbb {N}$, $\hat{\sigma }^{(-n)}=(\hat{ \sigma }_n,\hat{\sigma }_{n-1},\ldots ,\hat{\sigma }_1)$ is an optimal policy in the n-step model.

For $n=1$, $\hat{\sigma }_1$ and $v_0^*$ are both continuous functions, and Eq. (18) holds. Moreover, $\hat{ \sigma }^{(-1)}(\cdot )=\hat{\sigma }_1(\cdot )$ must be optimal in the 1-step model. Suppose that $v_{k-1}^*$ is continuous, $\hat{\sigma }_k$ is measurable for all $k\le n$, Eq. (18) holds for n, and $\hat{\sigma }^{(-n)}$ is an optimal policy in the n step model. We shall show that all these theses hold for $n+1$. Since $v^*_n$ is continuous, by Assumptions 3 and 5, $(x,y)\in Gr(\varGamma )\mapsto W(u(x-y),\mathcal {M} _{y}(v_n^*))$ is jointly continuous. As a result, by Measurable Maximum Theorem [see Theorem 18.19 in Aliprantis and Border (2006)] there is a measurable $\hat{\sigma }_{n+1}$—a solution of the maximization problem in ( 18) with $n=k+1$. By the induction hypothesis, $v_n^*(x)=J_n(x,\hat{\sigma }^{(-n)})$. Take another policy $\pi$ in the $n+1$ step model. Then, for each initial state x and $y\in \varGamma (x)$, $\pi ^2(x,y)$ is a policy in the n step model, and by the induction hypothesis $v_{n}^*(x^{\prime })=J_n(x^{\prime },\hat{\sigma }^{(-n)})\ge J_n(x^{\prime }, \pi ^{2}(x,y))$ for each state $x^{\prime }\in X$. Hence, by the definition of $\hat{\sigma }_{n+1}$ and (18) it follows that

$$\begin{aligned} J_{n+1}(x,\hat{\sigma }^{(-n-1)})&=W(u(x-\hat{\sigma }_{n+1}(x)),\mathcal {M}_{ \hat{\sigma }_{n+1}(x)}(v^*_{n})) \\&\ge \int _{\varGamma (x)} W(u(x-y),\mathcal {M}_{y}(v^*_{n}))\pi (\mathrm{d}y|x) \\&\ge \int _{\varGamma (x)} W(u(x-y),\mathcal {M}_{y}(J_n(\cdot ,\pi ^{(2)}(x,y)))) \pi (\mathrm{d}y|x)\nonumber \\ {}&=J_{n+1}(x,\pi ). \end{aligned}$$

Since $\pi$ is arbitrary, $v_{n+1}^*(x)=J_{n+1}(x,\hat{\sigma } ^{(-n-1)})$. As a result, Eq. (14) holds, and $\hat{\sigma } ^{(-n)}$ is an optimal policy in the n-step model for each $n\in \mathbb {N }$.

Observe that $(v_n^*)_{n\in \mathbb {N}}$ is an increasing sequence. Indeed, $v_n^*(\cdot )=BP^n(\mathbf {0})(\cdot )$, BP is an increasing operator, and $\mathbf {0}\le BP(\mathbf {0})(\cdot )$. Therefore, using the induction method we easily conclude that $v_n^*$ is increasing in n, and therefore, the limit in (15) must exist.

Proof of (ii). If v is a fixed point of BP in $B_+$, then $v=BP^n(v)$, and as we have already established in part (i) of this theorem $v^*_n=BP^n(\mathbf {0})$ for each $n\in \mathbb {N}$. Since BP is increasing and $v\ge \mathbf {0}$, for each $x\in X$ it holds $v_n(x)\le v(x)$. Consequently, $v^{*}(x)=\lim \nolimits _{n\rightarrow \infty }v_n^*(x)\le v(x)$.

Proof of (iii). Put an arbitrary policy $\sigma \in \varSigma$. Then, by the definition of $v^{*}$ and $v_n^*$ we have

$$\begin{aligned} v^{*}(x)=\lim \limits _{n\rightarrow \infty } v_n^*(x)\ge \lim \limits _{n\rightarrow \infty } J_n(x,\sigma ^{(n)})=J(x,\sigma ). \end{aligned}$$

Hence, the inequality $v^{*}(x)\ge \sup \nolimits _{\sigma \in \varSigma }J(x,\sigma )$ holds. To show the opposite inequality, observe that

$$\begin{aligned} \sup \limits _{\sigma \in \varSigma }J(x,\sigma )\ge J_n(x,\hat{\sigma }^{(-n)})= v_n^*(x)\rightarrow v^{*}(x) ( \text{ as } n\rightarrow \infty ). \end{aligned}$$

Hence, $\sup \limits _{\sigma \in \varSigma }J(x,\sigma )\ge v^{*}(x)$, and Eq. (16) holds.

Proof of (iv). Without loss of generality suppose that $u(0)=0$. For otherwise, all the theses are established in Corollary 1.

First, we show that $v^*$ is a fixed point of BP. By Assumption 5, it holds $\mathcal {M}_{y}(v_n^*)\rightarrow \mathcal {M}_y(v^*)$ for each $y\in X$ as $n\rightarrow \infty$. Let $y\in \varGamma (x)$ be arbitrary. Then, taking the limit with $n\rightarrow \infty$, and using equality (14), we obtain

$$\begin{aligned} v^{*}(x)\ge W(u(x-y),\mathcal {M}_{y}(v^{*})) \text{ for } \text{ each } y\in \varGamma (x), \end{aligned}$$

and consequently

$$\begin{aligned} v^{*}(x)\ge BP(v^{*})(x) \end{aligned}$$

(19)

for each $x\in X$. To show that $v^{*}$ satisfies the opposite inequality, observe for each $x>0$ and $\sigma \in \varSigma$, it holds

$$\begin{aligned} J(x,\sigma )\le \sup \limits _{y\in \varGamma (x)} W(u(x-y),\mathcal {M} _y(J(\cdot ,\sigma ^{2}(x,y)))) \le BP(v^*)(x), \end{aligned}$$

where the second inequality follows from the definition of BP and the point (iii) of this theorem. Hence, if we take the supremum over $\sigma \in \varSigma$, then we obtain $v^*(x)\le BP(v^*)(x)$. This, together with inequality (19) implies that $v^*$ is a fixed point of BP.

Now, we need to show that $v^{*}(x)=\tilde{v}(x)$ for each $x>0$. We shall show that $\inf \limits _{x>0}v^{*}(x)>0$. By Lemma 2, for each $x>0$ there is $\delta >0$ and $\sigma _{x}\in \varSigma _{\delta }$ such that

$$\begin{aligned} J(x,\sigma _{x})=U^{*}(x,\sigma _{x})\ge [U^{*}]_{X\times \varSigma }>0. \end{aligned}$$

Hence, by the point (iii) of this theorem it follows $\inf \limits _{x>0}v^{*}(x)>0$.

As a result, the function is defined as follows $v^{**}(x):=v^{*}(x)$ for $x>0$, and $v^{**}(0)=\lim \limits _{x\downarrow 0}v^{*}(x)$ is in the interior of $B_{+}$. We show that $v^{**}$ is a fixed point of BP. By Assumption 5 it holds $\mathcal {M}_{y}(v^{*})=\mathcal {M} _{y}(v^{**})$, for all $y>0$. Hence, for such y it holds

$$\begin{aligned} W(u(x-y),\mathcal {M}_y(v^*))=W(u(x-y),\mathcal {M}_y(v^{**})). \end{aligned}$$

As a result,

$$\begin{aligned} \sup \limits _{y\in ]0,x]}W(u(x-y),\mathcal {M}_y(v^*))=\sup \limits _{y\in ]0,x]}W(u(x-y),\mathcal {M}_y(v^{**})). \end{aligned}$$

Since $\mathcal {M}_0(v^*)=v^*(0)=0$

$$\begin{aligned} W(u(x),\mathcal {M}_0(v^*))<W(u(x),v^{**}(0)) =\lim \limits _{y\downarrow 0}W(u(x-y),\mathcal {M}_y(v^{*})). \end{aligned}$$

Therefore, a maximum of $y\in \varGamma (x)\mapsto W(u(x-y),\mathcal {M}_y(v^*))$ (if exists) is never evaluated at $y=0.$ Hence, $v^{**}=BP(v^*)=BP(v^{**})$, and $v^{**}$ is a fixed point of BP. By the point (i) of Theorem 3, $\tilde{v}$ is a unique fixed point of BP in $B_+^o$. Hence, $\tilde{v}\equiv v^{**}$. As a result, $\tilde{v}(x)=v^*(x)$ for $x>0$.

Therefore, by the point (ii) of Theorem 3, it follows that $v^*$ is continuous on $X\setminus \{0\}$.

$\square$

Proof of Theorem 5

For proving (17), we only need to show

$$\begin{aligned} \lim \limits _{\delta \rightarrow 0}||v^{\delta }_n-v^*||_X=0\quad \text{ for } \text{ every } n\in \mathbb {N}. \end{aligned}$$

(20)

Obviously, Eq. (20) is satisfied for $n=1$. Suppose it is true for some n. Let $\epsilon >0$. Then, for sufficiently small $\delta >0$, we have $||u^{\delta }-u||_X\le \epsilon$, $||v_n^{\delta }-v^*_n||_X\le \epsilon$. Applying Assumption 5, Theorem 4 (iv), and Dini Theorem, we may assume

$$\begin{aligned} \sup \limits _{y\in \varGamma (x)}|\mathcal {M}_y(v^{\delta }_n)-\mathcal {M} _y(v_n^*)|\le \epsilon \end{aligned}$$

for sufficiently small $\delta >0$. By the uniform continuity of W, we have (20) for $n+1$. $\square$

6 Final conclusions

The paper contains the proof of the existence and the uniqueness of a recursive utility in the model with a nonlinear aggregator and CCE. Moreover, we consider the problem of optimization of a recursive utility by means of Bellman equations. The transition probability satisfies assumptions known of growth models. The results which follow from this paper extend the results known from Jaśkiewicz et al. (2014). Also, many Thompson’s aggregators from Marinacci and Montrucchio (2010) and Becker and Rincón-Zapatero (2016) satisfy the conditions of my paper. Moreover, this paper contains an example of an aggregator which satisfies its conditions, yet is not a Thompson’s aggregator. Since CCE is nonlinear, the results in this paper expand the deterministic models as in Bich et al. (2018), Martins-da-Rocha and Vailakis (2010), Le-Van and Vailakis (2005), and Becker and Rincón-Zapatero (2016). However, let us note that, as opposed to Bich et al. (2018), Jaśkiewicz et al. (2014), and Le-Van and Vailakis (2005), Assumption 3 requires that the temporal utility u is bounded below by 0. The main results from this paper contain an iterative algorithm used to calculate recursive utility. For our results, we make extensive use of Guo–Cho–Zhu Theorem (Guo et al. 2004), which distinguishes this paper from other papers in the literature, which used standard Banach Fixed Point Theorems (e.g., in Marinacci and Montrucchio (2010) or in Martins-da-Rocha and Vailakis (2010)] or the extension from Matkowski (1975) (see also Jaśkiewicz et al. 2014), as well as papers that apply Krasnoselski and Zabreiko (1984) (see Becker and Rincón-Zapatero 2016; Bloise and Vailakis 2018).

The applications of various aggregators and CCE are possible in various theoretical problems in financial market (see asset pricing in Talarini Jr. (2000), or sovereign debt paradox in Martins-da-Rocha and Vailakis (2017)), Pareto optimal allocations (see Anderson 2005) or the managing global environment (see Asheim et al. 2012).

Appendix

Proof of Proposition 1

We only show that $\mathcal {M}_{y}$ satisfies (ii) in Assumption 1. Let $v\in \mathrm{int}(B_{+}(X))$, $y\in X$, and define $f:]0,1[\rightarrow \mathbb {R}_{+}$ by

$$\begin{aligned} f(t):=\mathcal {M}_{y}(tv)=\phi ^{-1}(E_{y}(\phi (tv))). \end{aligned}$$

Clearly, $f(t)>0$ for each $t\in ]0,1[$. Since f is differentiable, the sub-homogeneity is equivalent to $\frac{f^{\prime }(t)t}{f(t)}\le 1$ for all $t\in ]0,1[$.

Let $A:=\phi (tv)$. Observe that

$$\begin{aligned} \frac{f^{\prime }(t)t}{f(t)}=\frac{E_{y}(\phi ^{\prime }(\phi ^{-1}(A))\phi ^{-1}(A))}{\phi ^{\prime }(\phi ^{-1}(E_{y}(A)))\phi ^{-1}(E_{y}(A))}. \end{aligned}$$

(21)

We only prove (i) (i.e., $\phi$ is strictly increasing, and $\phi ^{\prime }\left( \phi ^{-1}(\cdot )\right) \phi ^{-1}(\cdot )$ is concave). The proof of (ii) is similar. By Jensen inequality, we have

$$\begin{aligned} E_{y}(\phi ^{\prime }(\phi ^{-1}(A))\phi ^{-1}(A))\le \phi ^{\prime }(\phi ^{-1}(E_{y}(A)))\phi ^{-1}(E_{y}(A)). \end{aligned}$$

(22)

Since $\phi$ is strictly increasing, $\phi ^{\prime }$ is strictly positive as well as $\phi ^{-1}$. Hence, (22) together with (21) yields $\frac{f^{\prime }(t)t}{f(t)}\le 1$.

$\square$

Proof of Proposition 2

We only prove that $W(v_1,t v_2)\ge t^r W(v_1,v_2)$ for all $v_1\ge \epsilon$, $v_2\in [0,\bar{U}]$, $t\in [0,1]$, and some $r\in ]0,1[$. Since W is Thompson, we only find r such that

$$\begin{aligned} t W(v_1,v_2)+(1-t)W(v_1,0)\ge t^r W(v_1,v_2) \end{aligned}$$

(23)

or equivalently

$$\begin{aligned} (1-t)\delta \ge t^r-t\quad \text{ where } \quad \delta =\sup \limits _{v_1\ge \epsilon } \frac{W(v_1,0)}{W(v_1,\bar{U})}. \end{aligned}$$

Hence, Eq. (23) is satisfied for

$$\begin{aligned} r>\sup \limits _{t\in ]0,1[}\frac{\ln (\delta +t(1-\delta ))}{\ln (t)}. \end{aligned}$$

Since the right-hand side above is less than 1, the proof is complete. $\square$

Proof of Lemma 1

Let $U\in \mathbf {U}_+$ and $\delta \ge 0$.

Proof of (i). We show that $T_{W}^{\delta }(U)\in \mathbf {U}_{+}$. For each $n>1$, let $h_{n}\in H_{n}$ and $x_{n}\in X$. First, we show that $T_{W}^{\delta }(U)(x_{n},\sigma ^{n}(h_{n}))$ is jointly measurable in $(h_{n},x_{n})$. As $U\in \mathbf {U}_{+}$, it follows $(h_{n+1},x^{\prime })\in H_{n+1}\times X\rightarrow U(x^{\prime },\sigma ^{n+1}(h_{n+1}))$ is jointly measurable. Hence, by Assumption 1, $\zeta (h_{n+1},y^{ \prime }):=\mathcal {M}_{y^{\prime }}(U(\cdot ,\sigma ^{n+1}(h_{n+1}))$ is jointly measurable in $H_{n+1}\times X$. Observe further

$$\begin{aligned} T_{W}^{\delta }(U)(x_{n},\sigma ^{n}(h_{n}))=\int _{\varGamma (x_{n})}W(u_{\delta }(x_{n}-y_{n}),\zeta (h_{n+1},y^{\prime }))\sigma _{n}(\mathrm{d}y_{n}|h_{n},x_{n}). \end{aligned}$$

Hence, $T_{W}^{\delta }(U)(x_{n},\sigma ^{n}(h_{n}))$ must be jointly measurable on $H_{n}\times X$. Therefore, $T_{W}^{\delta }$ maps $\mathbf {U} _{+}$ into itself. To show that $T_{W}^{\delta }$ maps $\mathrm{int}(\mathbf {U}_{+})$ into itself, observe that by (7) and Assumption 3 it holds $T_{W}(U)(x,\sigma )\ge W(0,[U]_{X\times \varSigma })>0$.

Proof of (ii). The proof is easy by Assumptions 3 and the definition of CCE.

Proof of (iii). By (ii), it holds $T_W^{\delta }(U)(x,\sigma )\ge W\left( \delta ,[U]_{X\times \varSigma }\right) >0$. In particular, if $U=\mathbf {0}$, then $T_W^{\delta }(\mathbf {0})(x,\sigma )\ge W(\delta ,0)>0$.

Proof of (iv). From part (i) of this theorem, it follows that all $J_n$ are well defined. From part (ii) of this theorem, $J_n$ increases in n. Indeed, $J_1\ge \mathbf {0}$, and suppose $J_{k+1}\ge J_{k}$ for some integer k. Then, $J_{k+2}=T_W(J_{k+1})\ge T_W(J_k)=J_{k+1}$. Consequently, $J_n$ increases in n and hence converges pointwise to J. We show that J is no greater than any other fixed point of $T_W$. Let U be any fixed point of $T_W$. Observe that $U\ge \mathbf {0}$, and suppose $U\ge J_k$. Then, by (ii) and the induction hypothesis, $U=T_W(U)\ge T_W(J_k)=J_{k+1}$, and consequently, passing to the limit, we get that U is greater than J. $\square$

Proof of Lemma 2

Step 1. First suppose $x<K(\underline{\omega })$. Then, by Assumption 4, $q(x,\underline{\omega })>x>q(0,\underline{\omega })=0$. By Darboux Theorem, there is $y^{*}(x)\in ]0,x[$ such that $q(y^{*}(x),\underline{\omega })=x$. Define $\sigma _{1}(x_{1})=y^{*}(x_{1})$, and for each $\tau >1$ and history $h:=(x_{n},y_{n})_{n\in \mathbb {N}}\in H$, $\sigma _{\tau }(\cdot |h_{\tau },x_{\tau })\equiv y^{*}(x_{1})$. We show that such $\sigma$ is feasible. We show more:

$$\begin{aligned} P_{x}^{\sigma }\left( h=(x_{n},y_{n})_{n\in \mathbb {N}}:\inf \limits _{n\in \mathbb {N}}x_{n}\ge x_{1}\right) =1. \end{aligned}$$

(24)

Since $x_{1}=x<K(\underline{\omega })$, by Assumption 4, we have

$$\begin{aligned} x_{2}=q(y^{*}(x_{1}),\omega _{1})\ge q(y^{*}(x_{1}),\underline{ \omega })=x_{1}=x. \end{aligned}$$

Suppose $x_{k}\ge x_{1}$$P_{x}^{\sigma }-$ a.s. Then, repeating the reasoning above $x_{k+1}\ge x_{1}$ as desired. Therefore, (24) holds, and such $\sigma$ is feasible. Hence, $\inf \limits _{n\in \mathbb {N} }u(x_{n}-y^{*}(x))>0$$P_{x}^{\sigma }$-a.s. Hence, $\varSigma _{x,\delta }\ne \emptyset$ with $\delta =u(x-y^{*}(x))>0$.

Step 2. Now, assume that $x\ge K(\underline{\omega })$. Put any $x_0\in ]0,K(\underline{\omega })[$. Then, define $\sigma _1(x_1)=y^*(x_0)$, and for $\tau >1$, $\sigma _{\tau }(\cdot |h_{\tau -1},x_{\tau })\equiv y^*(x_0)$. As in Step 1, we show that such $\sigma$ is feasible, and $\sigma \in \varSigma _{x,\delta }$ with $\delta =u(x_0-y^*(x_0))$. $\square$

Proof of Corollary 1

By Theorem 2 (iii), it follows that for each $(x,\sigma )\in X\times \varSigma$ it holds $J(x,\sigma )= U^*(x,\sigma )$. Hence, the sets of J- and U-optimal policies and values must coincide. By Theorem 3 (iii), the set of J-optimal (and U-optimal) policies is nonempty, and Eq. (13) holds. $\square$