A stochastic approximation method for approximating the efficient frontier of chance-constrained nonlinear programs

Abstract

We propose a stochastic approximation method for approximating the efficient frontier of chance-constrained nonlinear programs. Our approach is based on a bi-objective viewpoint of chance-constrained programs that seeks solutions on the efficient frontier of optimal objective value versus risk of constraints violation. To this end, we construct a reformulated problem whose objective is to minimize the probability of constraints violation subject to deterministic convex constraints (which includes a bound on the objective function value). We adapt existing smoothing-based approaches for chance-constrained problems to derive a convergent sequence of smooth approximations of our reformulated problem, and apply a projected stochastic subgradient algorithm to solve it. In contrast with exterior sampling-based approaches (such as sample average approximation) that approximate the original chance-constrained program with one having finite support, our proposal converges to stationary solutions of a smooth approximation of the original problem, thereby avoiding poor local solutions that may be an artefact of a fixed sample. Our proposal also includes a tailored implementation of the smoothing-based approach that chooses key algorithmic parameters based on problem data. Computational results on four test problems from the literature indicate that our proposed approach can efficiently determine good approximations of the efficient frontier.

Appendices

Algorithm outlines

1.1 Implementation of the scenario approximation algorithm

Algorithm 7 details our implementation of the tuned scenario approximation algorithm. Instead of solving each scenario approximation problem instance using an off-the-shelf solver by imposing all of the sampled constraints at once, we adopt a cutting-plane approach that iteratively adds some of the violated scenario constraints at a candidate solution to the list of enforced constraints in a bid to reduce the overall computational effort (cf. Algorithm 2.6 in Bienstock et al. [9]). We use $M = 50$ iterations and $R = 20$ replicates for our computational experiments. Settings for sample sizes $N_i$, constraints added per iteration $N_c$, and number of samples $N_{MC}$ are problem dependent and provided below. Line 19 in Algorithm 7, which estimates the risk level of a candidate solution, may be replaced, if possible, by one that computes the true analytical risk level or a numerical estimate of it [59].

• $N_i = \lceil 10^{a_i}\rceil$, $i = 1,\ldots ,50$, where: for Case study 1: $a_i = 1 + \frac{5}{49}(i-1)$; for Case study 2: $a_i = 1 + \frac{4}{49}(i-1)$; for Case study 3: $a_i = 1 + \frac{\log _{10}(50000)-1}{49}(i-1)$; for Case study 4: $a_i = 1 + \frac{4}{49}(i-1)$; and for Case study 5: $a_i = 1 + \frac{4}{49}(i-1)$ (the upper bounds on $N_i$ were determined based on memory requirements)

• $N_c$: Case study 1: 1000; Case study 2: 100000; Case study 3: 10; Case study 4: 5; and Case study 5: 10 (these values were tuned for good performance)

• $N_{MC}$: See Sect. 5.1 of the paper (we use the same Monte Carlo samples that were used by our proposed method to estimate risk levels).

1.2 Solving (CCP) for a fixed risk level

Algorithm 8 adapts Algorithm 2 to solve (CCP) for a given risk level $\hat{\alpha } \in (0,1)$. An initial point $\bar{x}^0$ and an upper bound on the optimal objective value $\nu _{up}$ can be obtained in a manner similar to Algorithm 2, whereas an initial lower bound on the optimal objective value $\nu _{low}$ can be obtained either using lower bounding techniques (see [41, 43]), or by trial and error. Note that the sequence of approximations in line 9 of Algorithm 8 need not be solved until termination if Algorithm 3 determines that $\bar{\alpha }^i < \hat{\alpha }$ before its termination criteria have been satisfied.

1.3 Implementation of the sigmoidal approximation algorithm

We used the following ‘tuned’ settings to solve each iteration of the sigmoidal approximation problem (see Section 4 of Cao and Zavala [13]) using IPOPT: tol $= 10^{-4}$, max_iter = 10000, hessian_approximation = limited_memory, jac_c_constant=yes, and max_cpu_time=3600 s. We terminated the loop of Algorithm SigVar-Alg of Cao and Zavala [13] when the objective improved by less than $0.01\%$ relative to the previous iteration. In what follows, we use the notation of Algorithm SigVar-Alg of Cao and Zavala [13]. For our ‘adaptive $\gamma$’ setting, we use the proposal of Cao and Zavala [13] to specify $\gamma$ when the solution of the CVaR problem corresponds to $t_c(\alpha ) < 0$. When $t_c(\alpha ) = 0$ at the CVaR solution returned by Gurobi, we try to estimate a good value of $t_c(\alpha )$ by looking at the true distribution of $g(x_c(\alpha ),\xi )$.

1.4 Tailored projection step for Case study 2

Because Algorithm 2 requires projecting onto the sets $X_{\nu } := \left\{ x \in \varDelta _N : {x}^\text {T} \varSigma x \le \nu \right\}$ many times for Case study 2 with different values of $\nu$, we develop a tailored projection routine that is computationally more efficient than using a general-purpose NLP solver to solve these quadratically-constrained quadratic programs. To project a point $y \in {\mathbb {R}}^N$ onto $X_{\nu }$ for some $\nu > 0$, note that the KKT conditions for the projection problem ${\mathop {\min }\nolimits _{x \in X_{\nu }}} 0.5\Vert x-y\Vert ^2$ yield:

$$\begin{aligned} x_i&= \frac{y_i-\mu +\pi _i}{1+2\sigma ^2_i\lambda }, \quad x_i \ge 0, \quad \pi _i \ge 0, \quad x_i \pi _i = 0, \quad \forall i \in \{1,\ldots ,N\}, \\ \lambda&\ge 0, \quad \sum _{i=1}^{N} x_i = 1, \quad \sum _{i=1}^{N} \alpha ^2_i x_i^2 \le \nu , \quad \lambda \left( \sum _{i=1}^{N} \alpha ^2_i (x_i)^2 - \nu \right) = 0, \end{aligned}$$

where $\mu$, $\lambda$, and $(\pi _1,\ldots ,\pi _N)$ denote KKT multipliers. Note that the inequalities in the first row can be simplified as $x_i = \max \left\{ 0,\frac{y_i-\mu }{1+2\sigma ^2_i\lambda }\right\}$, $\forall i \in \{1,\ldots ,N\}$. We first check whether $\lambda = 0$ satisfies the above system of equations (i.e., if the quadratic constraint is inactive at the solution) by setting x to be the projection of y onto the unit simplex. If $\lambda = 0$ is infeasible, we solve the following system of equations by using binary search over $\lambda$:

$$\begin{aligned} x_i&= \max \left\{ 0,\frac{y_i-\mu }{1+2\sigma ^2_i\lambda }\right\} , \,\, \forall i \in \{1,\ldots ,N\}, \quad \sum _{i=1}^{N} x_i = 1, \quad \sum _{i=1}^{N} \alpha ^2_i x_i^2 = \nu , \end{aligned}$$

where for each fixed $\lambda > 0$, the first two sets of equations in x and $\mu$ are solved by adapting the algorithm of Condat [18] for projection onto the unit simplex. Once solved, if the solution (say $x^*$) satisfies $\sum _{i=1}^{N} \alpha ^2_i (x^*_i)^2 < \nu$, then we conclude the current $\lambda$ is too large, and if $\sum _{i=1}^{N} \alpha ^2_i (x^*_i)^2 > \nu$ then we conclude the current $\lambda$ is too small.

Proof of Proposition 6

We first sketch an outline of the proof (note that our proof will follow a different order for reasons that will become evident). We will establish that (see Chapters 4 and 5 of Rockafellar and Wets [54] for definitions of technical terms)

$$\begin{aligned} \underset{k \rightarrow \infty }{\limsup _{x \rightarrow \bar{x}}}\, \partial {\hat{p}}_k(x) + N_{X_{\nu }}(x)&= \underset{k \rightarrow \infty }{\limsup _{x \rightarrow \bar{x}}}\, \partial {\hat{p}}_k(x) + \underset{k \rightarrow \infty }{\limsup _{x \rightarrow \bar{x}}}\, N_{X_{\nu }}(x). \end{aligned}$$

(3)

Then, because of the outer semicontinuity of the normal cone mapping, it suffices to prove that the outer limit of $\partial {\hat{p}}_k(x)$ is a subset of $\{\nabla p(\bar{x})\}$. To demonstrate this, we will first show that $\partial {\hat{p}}_k(x) = - \partial \int _{-\infty }^{\infty } F(x,\eta ) d\phi \left( \eta ;\tau _c^{k-1}\right) d\eta$, where F is the cumulative distribution function of $\max \left[ \bar{g}(x,\xi )\right]$. Then, we will split this integral into two parts - one accounting for tail contributions (which we will show vanishes when we take the outer limit), and the other accounting for the contributions of F near $(x,\eta ) = (\bar{x},0)$ that we will show satisfies the desired outer semicontinuity property.

Since ${\hat{p}}_k(x)$ can be rewritten as ${\hat{p}}_k(x) = {\mathbb {E}}\big [{\phi \left( \max \left[ \bar{g}(x,\xi )\right] ;\tau _c^{k-1}\right) }\big ]$ by Assumptions 4B and 6, we have

$$\begin{aligned} {\hat{p}}_k(x) = {\mathbb {E}}\Big [{\phi \left( \max \left[ \bar{g}(x,\xi )\right] ;\tau _c^{k-1}\right) }\Big ]&= \int _{-\infty }^{+\infty } \phi \left( \max \left[ \bar{g}(x,\xi )\right] ;\tau _c^{k-1}\right) d{\mathbb {P}}\\&= \int _{-\infty }^{+\infty } \phi \left( \eta ;\tau _c^{k-1}\right) dF(x,\eta ) \\&= \lim _{\eta \rightarrow +\infty } \phi \left( \eta ;\tau _c^{k-1}\right) \\&\quad - \int _{-\infty }^{+\infty } F(x,\eta ) d\phi \left( \eta ;\tau _c^{k-1}\right) d\eta , \end{aligned}$$

where the second line is to be interpreted as a Lebesgue–Stieljes integral, and the final step follows by integrating by parts and Assumption 4. This yields (see Proposition 5 and Assumption 6 for the existence of these quantities)

$$\begin{aligned} \partial {\hat{p}}_k(x)&\subset - \partial \int _{|\eta | \ge \varepsilon _k} F(x,\eta ) d\phi \left( \eta ;\tau _c^{k-1}\right) d\eta - \partial \int _{-\varepsilon _k}^{\varepsilon _k} F(x,\eta ) d\phi \left( \eta ;\tau _c^{k-1}\right) d\eta \end{aligned}$$

(4)

by the properties of the Clarke generalized gradient, where $\{\varepsilon _k\}$ is defined in Assumption 6.

Let $\{x_k\}$ be any sequence of points in $X_{\nu }$ converging to $\bar{x} \in X_{\nu }$. Suppose $v_k \in - \partial \int _{|\eta | \ge \varepsilon _k} F(x_k,\eta ) d\phi \left( \eta ;\tau _c^{k-1}\right) d\eta$ with $v_k \rightarrow v \in {\mathbb {R}}^n$. We would like to show that $v = 0$. Note that by an abuse of notation (where we actually take norms of the integrable selections that define $v_k$)

$$\begin{aligned} \lim _{k \rightarrow \infty } \Vert v_k\Vert&= \lim _{k \rightarrow \infty } \Vert - \partial \int _{|\eta | \ge \varepsilon _k} F(x_k,\eta ) d\phi \left( \eta ;\tau _c^{k-1}\right) d\eta \Vert \nonumber \\&= \lim _{k \rightarrow \infty } \Vert \int _{|\eta | \ge \varepsilon _k} \partial _x F(x_k,\eta ) d\phi \left( \eta ;\tau _c^{k-1}\right) d\eta \Vert \nonumber \\&\le \lim _{k \rightarrow \infty } \int _{|\eta | \ge \varepsilon _k} \Vert \partial _x F(x_k,\eta )\Vert d\phi \left( \eta ;\tau _c^{k-1}\right) d\eta \nonumber \\&\le \lim _{k \rightarrow \infty } \int _{|\eta | \ge \varepsilon _k} L_F(\eta ) d\phi \left( \eta ;\tau _c^{k-1}\right) d\eta = 0, \end{aligned}$$

(5)

where the first step follows from Theorem 2.7.2 of Clarke [17] (whose assumptions are satisfied by virtue of Assumption 6B), the second inequality follows from Proposition 2.1.2 of Clarke [17], and the final equality follows from Assumption 6C.

The above arguments establish that $\limsup \nolimits _{\begin{array}{c} x \rightarrow \bar{x} \\ k \rightarrow \infty \end{array}}\, - \partial \int _{|\eta | \ge \varepsilon _k} F(x,\eta ) d\phi \left( \eta ;\tau _c^{k-1}\right) d\eta = \{0\}$. Consequently, we have $\limsup \nolimits _{\begin{array}{c} x \rightarrow \bar{x} \\ k \rightarrow \infty \end{array}}\, \partial {\hat{p}}_k(x) \subset \limsup \nolimits _{\begin{array}{c} x \rightarrow \bar{x} \\ k \rightarrow \infty \end{array}}\, - \partial \int _{-\varepsilon _k}^{\varepsilon _k} F(x,\eta ) d\phi \left( \eta ;\tau _c^{k-1}\right) d\eta$ from Eq. (4). We now consider the outer limit $\limsup \nolimits _{\begin{array}{c} x \rightarrow \bar{x} \\ k \rightarrow \infty \end{array}}\, - \partial \int _{-\varepsilon _k}^{\varepsilon _k} F(x,\eta ) d\phi \left( \eta ;\tau _c^{k-1}\right) d\eta$. Suppose $w_k \in - \partial \int _{-\varepsilon _k}^{\varepsilon _k} F(x_k,\eta ) d\phi \left( \eta ;\tau _c^{k-1}\right) d\eta$ with $w_k \rightarrow w \in {\mathbb {R}}^n$. We wish to show that $w = \nabla p(\bar{x})$. Invoking Theorem 2.7.2 of Clarke [17] once again, we have that

$$\begin{aligned}&\quad \!\qquad \underset{k \rightarrow \infty }{\limsup }\, - \partial \int _{-\varepsilon _k}^{\varepsilon _k} F(x_k,\eta ) d\phi \left( \eta ;\tau _c^{k-1}\right) d\eta \nonumber \\&\quad \subset \underset{k \rightarrow \infty }{\limsup }\, - \int _{-\varepsilon _k}^{\varepsilon _k} \nabla _x F(x_k,\eta ) d\phi \left( \eta ;\tau _c^{k-1}\right) d\eta \nonumber \\&\quad = \, \underset{k \rightarrow \infty }{\limsup }\, - \left( \int _{-\varepsilon _k}^{\varepsilon _k} \nabla _x F(x_k,\eta ) d\hat{\phi }\left( \eta \right) d\eta \right) \left( \int _{-\varepsilon _k}^{\varepsilon _k} d\phi \left( z;\tau _c^{k-1}\right) dz\right) \nonumber \\&\quad = \, \left( \underset{k \rightarrow \infty }{\limsup }\, - \int _{-\varepsilon _k}^{\varepsilon _k} \nabla _x F(x_k,\eta ) d\hat{\phi }\left( \eta \right) d\eta \right) \,\, \left( \lim _{k \rightarrow \infty } \int _{-\varepsilon _k}^{\varepsilon _k} d\phi \left( z;\tau _c^{k-1}\right) dz\right) \nonumber \\&\quad = \, \underset{k \rightarrow \infty }{\limsup }\, - \int _{-\varepsilon _k}^{\varepsilon _k} \nabla _x F(x_k,\eta ) d\hat{\phi }_k\left( \eta \right) d\eta , \end{aligned}$$

(6)

where for each $k \in {\mathbb {N}}$ large enough, $\hat{\phi }_k:{\mathbb {R}}\rightarrow {\mathbb {R}}$ is defined as $\hat{\phi }_k\left( y\right) = \dfrac{\phi \left( y;\tau _c^{k-1}\right) }{\int _{-\varepsilon _k}^{\varepsilon _k} d\phi \left( z;\tau _c^{k-1}\right) dz}$, the first step follows (by an abuse of notation) from the fact that $\varepsilon _k < \theta$ for k large enough (see Assumption 6B), and the third and fourth steps follow from Assumption 6C. Noting that⁴

$$\begin{aligned} \int _{-\varepsilon _k}^{\varepsilon _k} \nabla _x F(x_k,\eta ) d\hat{\phi }_k(\eta ) d\eta&= {\begin{bmatrix} \dfrac{\partial F}{\partial x_1}(x_k,\omega _{1,k}), \ldots , \dfrac{\partial F}{\partial x_n}(x_k,\omega _{n,k}) \end{bmatrix}}^\text {T} \int _{-\varepsilon _k}^{\varepsilon _k} d\hat{\phi }_k(\eta ) d\eta \\&= {\begin{bmatrix} \dfrac{\partial F}{\partial x_1}(x_k,\omega _{1,k}), \ldots , \dfrac{\partial F}{\partial x_n}(x_k,\omega _{n,k}) \end{bmatrix}}^\text {T} \end{aligned}$$

for some constants $\omega _{i,k} \in (-\varepsilon _k,\varepsilon _k)$, $i = 1,\ldots ,n$, by virtue of Assumption 6B and the first mean value theorem for definite integrals, we have

$$\begin{aligned}&\quad \!\qquad \underset{k \rightarrow \infty }{\limsup }\, - \partial \int _{-\varepsilon _k}^{\varepsilon _k} F(x_k,\eta ) d\phi \left( \eta ;\tau _c^{k-1}\right) d\eta \\&\quad \subset \underset{k \rightarrow \infty }{\limsup }\, \left\{ - {\begin{bmatrix} \dfrac{\partial F}{\partial x_1}(x_k,\omega _{1,k}), \ldots , \dfrac{\partial F}{\partial x_n}(x_k,\omega _{n,k}) \end{bmatrix}}^\text {T}\right\} \\&\quad = \{-\nabla _x F(\bar{x},0)\} = \{\nabla p(\bar{x})\}, \end{aligned}$$

where the first equality above follows from Assumption 6B, and the second equality follows from the fact that $p(x) = 1 - F(x,0)$ for each $x \in X_{\nu }$. Reconciling our progress with Eq. (4), we obtain

$$\begin{aligned} \underset{k \rightarrow \infty }{\limsup _{x \rightarrow \bar{x}}}\, \partial {\hat{p}}_k(x) \subset \{\nabla p(\bar{x})\}. \end{aligned}$$

To establish the desirable equality in Eq. (3), it suffices to show that $\partial {\hat{p}}_k(x_k) \subset C$ for k large enough, where $C \subset {\mathbb {R}}^n$ is independent of k. From Eq. (5), we have that the first term in the right-hand side of Eq. (4) is contained in a bounded set that is independent of $k \in {\mathbb {N}}$. From Eq. (6), we have that any element of the second term in the right-hand side of Eq. (4) is bounded above in norm by $\max \nolimits _{\begin{array}{c} x \in {\text {cl}}\left( B_{\delta }(\bar{x})\right) \\ \eta \in [-0.5\theta ,0.5\theta ] \end{array}} \Vert \nabla _x F(x,\eta )\Vert$ for k large enough for any $\delta > 0$. The above arguments in conjunction with Eq. (4) establish Eq. (3). The desired result then follows from the outer semicontinuity of the normal cone mapping, see Proposition 6.6 of Rockafellar and Wets [54]. $\square$

Recourse formulation

As noted in Sect. 1 of the paper, Problem (CCP) can also be used to model chance-constrained programs with static recourse decisions, e.g., by defining g through the solution of the following auxiliary optimization problem for each $(x,\xi ) \in X \times \varXi$:

$$\begin{aligned} \begin{array}{rl} g(x,\xi ) := \, \underset{y, \eta }{\min } \,\, &{} \eta \\ \,\, \text {s.t.} &{} T(x,\xi ) + W(\xi ) y + \eta e \ge 0, \\ &{} y \in {\mathbb {R}}^{n_y}_+, \,\, \eta \ge -1, \end{array} \end{aligned}$$

where $T: {\mathbb {R}}^n \times {\mathbb {R}}^d \rightarrow {\mathbb {R}}^{m_r}$ is continuously differentiable, and $W:{\mathbb {R}}^d \rightarrow {\mathbb {R}}^{m_r \times n_y}$. Note that g can be recast in the form of a joint chance constraint by appealing to linear programming duality, viz.,

$$\begin{aligned} g(x,\xi ) = \underset{(v,w) \in \mathrm {EXT}(V(\xi ))}{\max } -{v}^\text {T} T(x,\xi ) - w, \end{aligned}$$

where $\mathrm {EXT}(V(\xi ))$ denotes the set of extreme points of the polytope

$$\begin{aligned} V(\xi ) = \left\{ (v,w) \in {\mathbb {R}}^{m_r}_+ \times {\mathbb {R}}_+ : {v}^\text {T}W(\xi ) \le 0, \, {v}^\text {T} e + w = 1 \right\} . \end{aligned}$$

Explicitly reformulating the recourse constraints into the above explicit form is not generally practical since Algorithms 4 and 5 rely on stepping through each of the constraint functions, and the cardinality of the set of extreme points of $V(\xi )$ may be huge. Therefore, we consider the case when the approximations ${\hat{p}}_k$ are constructed using a single smoothing function, i.e., ${\hat{p}}_k(x) = {\mathbb {E}}\big [{\phi _k(g(x,\xi ))}\big ]$, where $\{\phi _k\}$ is a sequence of smooth scalar functions that approximate the step function.

Throughout this section, we assume that g that can be reformulated as $g(x,\xi ) := \underset{j \in J}{\max } \,\, h_j(x,\xi )$, where J is a (potentially large) finite index set, and $h_j:{\mathbb {R}}^n \times R^d \rightarrow {\mathbb {R}}$ are continuously differentiable functions that are Lipschitz continuous and have Lipschitz continuous gradients in the sense of Assumption 3. We let $L^{'}_h(\xi )$ denote the Lipschitz constant of $\nabla h(\cdot ,\xi )$ on $X_{\nu }$ with ${\mathbb {E}}\big [{\left( L^{'}_h(\xi )\right) ^2}\big ] < +\infty$. While the theory in this section is also applicable to the ordinary joint chance constrained setting, it is usually advantageous to consider individual smoothing functions for each chance constraint function in that case whenever feasible.

We first characterize the Clarke generalized gradient of the approximation ${\hat{p}}_k$ under this setup.

Proposition 11

Suppose Assumptions 4 and 5 and the above assumptions on $g(x,\xi ) = \max \left[ h(x,\xi )\right]$ and $\phi _k$ hold. Then $\partial {\hat{p}}_k(x) = {\mathbb {E}}\big [{d\phi _k\left( \max \left[ h(x,\xi )\right] \right) \times \partial _x \max \left[ h(x,\xi )\right] }\big ]$.

Proof

Note that for any $(x,\xi ) \in X \times \varXi$, we have $\partial _x \phi _k(g(x,\xi )) = d\phi _k(g(x,\xi )) \, \partial _x \max \left[ h(x,\xi )\right]$, see Theorem 2.3.9 of Clarke [17]. A justification for swapping the derivative and expectation operators then follows from Proposition 5 of the paper and the fact that ${\hat{p}}_k(x) = {\mathbb {E}}\big [{\phi _k(g(x,\xi ))}\big ] = {\mathbb {E}}\big [{\underset{j}{\max }\left[ \phi _k\left( g_j(x,\xi )\right) \right] }\big ]$ since $\phi _k$ is monotonically nondecreasing on ${\mathbb {R}}$. $\square$

The following result establishes that the approximation ${\hat{p}}_k$ continues to enjoy the weak convexity property for the above setup under mild assumptions.

Proposition 12

Suppose Assumptions 1, 4, and 5 hold. Additionally, suppose $g(x,\xi ) = \max \left[ h(x,\xi )\right]$ satisfies the above conditions and $\phi _k$ is a scalar smoothing function. Then ${\hat{p}}_k(\cdot )$ is $\bar{L}_k$-weakly convex for some constant $\bar{L}_k$ that depends on $L^{'}_h(\xi )$, $L^{'}_{\phi ,k}$, $M^{'}_{\phi ,k}$, and the diameter of $X_{\nu }$.

Proof

First, note that $g(\cdot ,\xi ) := \max \left[ h(\cdot ,\xi )\right]$ is $L^{'}_h(\xi )$-weakly convex on $X_{\nu }$ for each $\xi \in \varXi$, see Lemma 4.2 of Drusvyatskiy and Paquette [23]. In particular, this implies that for any $y, z \in X_{\nu }$,

$$\begin{aligned} g(z,\xi ) = \max \left[ h(z,\xi )\right]&\ge g(y,\xi ) + {s}^\text {T}_y(\xi ) (z - y) - \dfrac{L^{'}_h(\xi )}{2} \Vert z - y\Vert ^2 \end{aligned}$$

for any $s_y(\xi ) \in \partial \max \left[ h(y,\xi )\right]$. The Lipschitz continuity of $d\phi _k(\cdot )$ implies that for any scalars v and w:

$$\begin{aligned} \phi _k(w) \ge \phi _k(v) + d\phi _k(v) (w - v) - \dfrac{L^{'}_{\phi ,k}}{2} (w - v)^2. \end{aligned}$$

From the monotonicity Assumption 4B and by recursively applying the above result, we have for any $\xi \in \varXi$ and $y, z \in X_{\nu }$:

$$\begin{aligned} \phi _k\left( g(z,\xi )\right)&\ge \phi _k\left( g(y,\xi ) + {s}^\text {T}_y(\xi ) (z - y) - \dfrac{L^{'}_h(\xi )}{2} \Vert z - y\Vert ^2 \right) \\&\ge \phi _k\left( g(y,\xi ) + {s}^\text {T}_y(\xi ) (z - y) \right) -\dfrac{L^{'}_h(\xi )}{2} d\phi _k\left( g(y,\xi ) + {s}^\text {T}_y (z - y) \right) \Vert z - y\Vert ^2 \\&\qquad -\dfrac{L^{'}_{\phi ,k}\left( L^{'}_h(\xi )\right) ^2}{8} \Vert z - y\Vert ^4 \\&\ge \phi _k\left( g(y,\xi ) \right) + {\left( d\phi _k\left( g(y,\xi ) \right) s_y(\xi ) \right) }^\text {T} (z - y) - \dfrac{L^{'}_{\phi ,k}}{2}\left( {s}^\text {T}_y(\xi ) (z - y) \right) ^2 \\&\qquad -\dfrac{L^{'}_h(\xi ) M^{'}_{\phi ,k}}{2} \Vert z - y\Vert ^2 - \dfrac{L^{'}_{\phi ,k}\left( L^{'}_h(\xi )\right) ^2}{8} \Vert z - y\Vert ^4 \\&\ge \phi _k\left( g(y,\xi ) \right) + {\left( d\phi _k\left( g(y,\xi ) \right) s_y(\xi ) \right) }^\text {T} (z - y) \\&\quad - \dfrac{\Vert z - y\Vert ^2}{2} \Biggl [L^{'}_{\phi ,k}\Vert s_y(\xi )\Vert ^2 + L^{'}_h(\xi ) M^{'}_{\phi ,k} + \dfrac{L^{'}_{\phi ,k} \left( L^{'}_h(\xi )\right) ^2 \text {diam}\left( X_{\nu }\right) ^2}{4} \Biggr ]. \end{aligned}$$

Taking expectation on both sides and noting that $\Vert s_y(\xi )\Vert \le L^{'}_h(\xi )$, we get

$$\begin{aligned} {\mathbb {E}}\big [{\phi _k\left( g(z,\xi )\right) }\big ]&\ge {\mathbb {E}}\big [{\phi _k\left( g(y,\xi ) \right) }\big ] + {\mathbb {E}}\big [{{\left( d\phi _k\left( g(y,\xi ) \right) s_y \right) }^\text {T} (z - y)}\big ] \\&\quad -\dfrac{\Vert z - y\Vert ^2}{2} \left[ L^{'}_{\phi ,k} {\mathbb {E}}\big [{\left( L^{'}_h(\xi ) \right) ^2}\big ] + {\mathbb {E}}\big [{L^{'}_h(\xi )}\big ] M^{'}_{\phi ,k} + \dfrac{L^{'}_{\phi ,k} {\mathbb {E}}\big [{\left( L^{'}_h(\xi )\right) ^2}\big ] \text {diam}\left( X_{\nu }\right) ^2}{4} \right] . \end{aligned}$$

Therefore, ${\hat{p}}_k$ is $\bar{L}_k$-weakly convex on $X_{\nu }$ with

$$\begin{aligned} \bar{L}_k := L^{'}_{\phi ,k} {\mathbb {E}}\big [{\left( L^{'}_h(\xi ) \right) ^2}\big ] + {\mathbb {E}}\big [{L^{'}_h(\xi )}\big ] M^{'}_{\phi ,k} + \dfrac{1}{4} L^{'}_{\phi ,k} {\mathbb {E}}\big [{\left( L^{'}_h(\xi )\right) ^2}\big ] \text {diam}\left( X_{\nu }\right) ^2. \end{aligned}$$

$\square$

We now outline our proposal for estimating the weak convexity parameter $\bar{L}_k$ of ${\hat{p}}_k$ for the above setting. First, we note that estimate of the constants $L^{'}_{\phi ,k}$ and $M^{'}_{\phi ,k}$ can be obtained from Proposition 7. Next, we propose to replace the unknown constant $\text {diam}\left( X_{\nu }\right)$ with the diameter 2r of the sampling ball in Algorithm 5. Finally, we note that estimating the Lipschitz constant $L^{'}_h(\xi )$ for any $\xi \in \varXi$ is tricky since it would involve looking at all of the $|J|$ constraints in general, just the thing we wanted to avoid! To circumvent this, we propose to replace $L^{'}_h(\xi )$ in $\bar{L}_k$ with an estimate of the local Lipschitz constant of our choice of the B-subdifferential element of $g(\cdot ,\xi )$ through sampling (similar to the proposal in Algorithm 5). When g is defined through the solution of the recourse formulation considered, we can estimate a B-subdifferential element by appealing to linear programming duality and use this as a proxy for the ‘gradient’ of g. Note that a crude estimate of the step length does not affect the theoretical convergence guarantee of the stochastic subgradient method of Davis and Drusvyatskiy [21].

Fig. 7

Enclosure of the trajectories of the efficient frontier for Case study 1 generated by ten replicates of the proposed approach

Fig. 8

Enclosure of the trajectories of the efficient frontier for Case study 2 generated by ten replicates of the proposed approach

Fig. 9

Enclosure of the trajectories of the efficient frontier for Case study 3 generated by ten replicates of the proposed approach

Fig. 10

Enclosure of the trajectories of the efficient frontier for Case study 4 generated by ten replicates of the proposed approach

Fig. 11

Comparison of the efficient frontiers for Case study 5

Fig. 12

Enclosure of the trajectories of the efficient frontier for Case study 5 generated by ten replicates of the proposed approach

Additional computational results

We present results of our replication experiments for the four case studies in the paper, and also consider a variant of Case study 3 that has a known analytical solution to benchmark our proposed approach.

Figure 7 presents the enclosure of the trajectories of the EF generated by ten replicates of the proposed approach when applied to Case study 1. For each value of the objective bound $\nu$, we plot the smallest and largest risk level determined by Algorithm 3 at that bound over the different replicates. We find that the risk levels returned by the proposed algorithm do not vary significantly across the different replicates, with the maximum difference in the risk levels across the 26 points on the EF being a factor of 1.48. Figure 8 presents the corresponding plot for Case study 2. We again find that the risk levels returned by the proposed algorithm do not vary significantly across the different replicates, with the maximum difference in the risk levels across the 20 points on the EF being a factor of 1.006. Figure 9 presents the corresponding plot for Case study 3. The maximum difference in the risk levels at the 31 points on the EF for this case is a factor of 1.22 across the ten replicates. Figure 10 presents the corresponding plot for Case study 4. The maximum difference in the risk levels at the 26 points on the EF for this case is a factor of 1.25 across the ten replicates.

Case study 5

We consider Case study 3 when the random variables $\xi _{ij} \sim {\mathcal {P}}:= \mathcal {N}(0,1)$ are i.i.d., the number of variables $n = 100$, the number of constraints $m = 100$, and bound $U = 100$. The EF can be computed analytically in this case, see Section 5.1.1 of Hong et al. [34]. Figure 11 compares a typical EF obtained using our approach against the analytical EF and the solutions generated by the tuned scenario approximation algorithm. Our proposed approach is able to converge to the analytical EF, whereas the scenario approximation method is only able to determine a suboptimal EF. Our proposal takes 2332 s on average (and a maximum of 2379 s) to approximate the EF using 32 points, whereas it took the scenario approximation a total of 16, 007 s to generate its 1000 points in Fig. 11. We note that about $60\%$ of the reported times for our method is spent in generating random numbers because the random variable $\xi$ is high-dimensional.

Figure 12 presents an enclosure of the trajectories of the EF generated by the proposed approach over ten replicates for this case. We find that the risk levels returned by the proposed algorithm do not vary significantly across the different replicates, with the maximum difference in the risk levels across the 32 points on the EF being a factor of 1.001.