Construction and Efficient Implementation of Adaptive Objective-Based Designs of Experiments

Abstract

This work is devoted to the development of a new and efficient procedure for the construction of adaptive model-based designs of experiments. This work couples kriging theory with design of experiments optimization techniques and its originality relies on two main ingredients: (i) the definition of a general criterion in the objective function that allows one to take into account the analyst’s choices in order to explore critical regions, and (ii) an efficient numerical implementation including a stabilization step to avoid numerical problems due to kriging matrix inversion and a computational cost reduction strategy that allows for the model’s use in industrial applications. After a full description of these key points, the resulting efficient algorithm is applied to extend over a territory in France a network of probes for environmental monitoring. This study leads to a final design of experiments where the new probes are located in undersampled regions of high population density. Moreover, it can be performed with an affordable computational cost, which is not the case with classical approaches based on an optimization over the whole domain.

Appendix: Asymptotical Behavior of the Condition Number

For sake of simplicity in the proofs, the following notations are introduced: $\forall(i,j), \ C^{j}_{i}=C(\|x_{j}-x_{i}\|_{2})$ and $C^{\star}_{i}=C(\|x^{\star}-x_{i}\|_{2})$. Moreover, $\varSigma^{X^{k}}$ is replaced by Σ _k . Finally, in order to separate the influence of the number of points k and of the distance associated with the new point, the perturbation term is written as $\tilde{V}=(k+1) \tilde{U}$.

1.1 A.1 Proof of Proposition 2

Σ _k+1 can be written as

$$\begin{aligned} \varSigma_{k+1}=\left [ \begin{array}{c@{\quad}c} \varSigma_k & \mathcal{C}^{X^k}(x^{\star})\\ \mathcal{C}^{X^k}(x^{\star})^T & C(0)\\ \end{array} \right ]. \end{aligned}$$

(31)

Introducing the Schur complement $\alpha_{k}=C(0)-\mathcal {C}^{X^{k}}(x^{\star})^{T} \varSigma_{k}^{-1} \mathcal{C}^{X^{k}}(x^{\star})$ and according to a classical result of matrix calculus, $\varSigma_{k+1}^{-1}$ is given by

$$\begin{aligned} \varSigma_{k+1}^{-1}=\left [ \begin{array}{c@{\quad}c} \varSigma_k^{-1} (I_k+\mathcal{C}^{X^k}(x^{\star}) \alpha_k^{-1} \mathcal{C}^{X^k}(x^{\star})^T \varSigma_k^{-1} )& -\varSigma_k^{-1} \mathcal{C}^{X^k}(x^{\star}) \alpha_k^{-1}\\ -\alpha_k^{-1} \mathcal{C}^{X^k}(x^{\star})^T \varSigma_k^{-1}&\alpha _k^{-1}\\ \end{array} \right ]. \end{aligned}$$

(32)

Combining Eqs. (31), (32), and (24), we find

$$\begin{aligned} &\operatorname {Cond}^2_F\bigl(\varSigma_{k+1}^{1/2} \bigr)\\ &\quad = \bigl(C(0)+\operatorname {Tr}(\varSigma_k ) \bigr) \bigl(\operatorname {Tr}\bigl( \varSigma_k^{-1} \bigr)+\operatorname {Tr}\bigl(\varSigma_k^{-1} \mathcal {C}^{X^k}\bigl(x^{\star}\bigr) \ \alpha_k^{-1} \ \mathcal{C}^{X^k}\bigl(x^{\star}\bigr)^T \varSigma_k^{-1} \bigr)+ \alpha_k^{-1} \bigr), \\ &\quad =\frac{k+1}{k} \operatorname {Tr}(\varSigma_k ) \bigl(\operatorname {Tr}\bigl( \varSigma _k^{-1} \bigr)+\alpha_k^{-1} \bigl(\operatorname {Tr}\bigl(\varSigma_k^{-1} \mathcal {C}^{X^k}\bigl(x^{\star}\bigr) \ \mathcal{C}^{X^k} \bigl(x^{\star}\bigr)^T \varSigma_k^{-1} \bigr)+ 1 \bigr) \bigr). \end{aligned}$$

(33)

Noticing that $\beta_{k}=\varSigma_{k}^{-1} \mathcal{C}^{X^{k}}(x^{\star})$ and $\operatorname {Tr}(\varSigma_{k} )=k$ C(0), the previous expression leads to the expected result.

1.2 A.2 Proof of Proposition 3

The proof relies on the asymptotical decomposition of $\tilde{U}$ for large l. Let us first notice that for any semi-variogram models of $\mathcal{V}$, there exists $K_{\mathcal{V}}$ such that, for any x _i and $\mathcal{L}\geq K_{\mathcal{V}}$

$$\begin{aligned} C^{\star}_i=C^1_i+ \sum_{n=K_{\mathcal{V}}}^{\mathcal{L}} a^{\mathcal {V}}_{n,i} \frac{1}{2^{nl}} + o \biggl(\frac{1}{2^{\mathcal{L}l}} \biggr), \end{aligned}$$

(34)

where $\{a^{\mathcal{V}}_{n,i}\}_{n\in\{K_{\mathcal{V}},\ldots,\mathcal {L}\}}$ is a set of real numbers. Coming back to the expression of the perturbation term (Eq. (26)), we first focus on the asymptotical behaviour of α _k .

From Eq. (34), α _k can be written

$$\begin{aligned} \alpha_k=C(0)- \Biggl(A_0+\sum _{n=K_{\mathcal{V}}}^{\mathcal{L}} A_n + A_{\mathcal{L}+1} \Biggr)^T \varSigma_k^{-1} \Biggl(A_0+ \sum_{n=K_{\mathcal{V}}}^{\mathcal{L}} A_n + A_{\mathcal{L}+1} \Biggr), \end{aligned}$$

(35)

where

$$\begin{aligned} A_0 =& \bigl(C^1_i,\ldots,C^1_k \bigr)^T, \end{aligned}$$

(36)

$$\begin{aligned} A_n =& \biggl(a^{\mathcal{V}}_{n,1} \frac{1}{2^{nl}}, \ldots, a^{\mathcal {V}}_{n,k} \frac{1}{2^{nl}} \biggr)^T, \end{aligned}$$

(37)

and with $\|A_{\mathcal{L}+1}\|_{2} = o (\frac{1}{2^{\mathcal {L}l}} )$. Equation (35) can be reformulated as

$$\begin{aligned} \alpha_k= \bigl(C(0)-A_0^T \varSigma^{-1}_k A_0 \bigr) - \sum _{(i,j)\in\{0,K_{\mathcal{V}},\ldots,\mathcal{L}+1\}, \ i+j \geq K_{\mathcal{V}}} A_i^T \varSigma^{-1}_k A_j. \end{aligned}$$

(38)

Noticing that $\varSigma^{-1}_{k} A_{0}$ corresponds to the weights computed from a simple kriging at x ₁, $\varSigma^{-1}_{k} A_{0}= (1,0,\ldots ,0 )^{T}$ since kriging interpolation is exact at any point of the DoE {x _i } _i=1,…,k . Therefore,

$$\begin{aligned} A_0^T \varSigma^{-1}_k A_0=C_1^1=C(0), \end{aligned}$$

(39)

and the first term of Eq. (38) is equal to zero. As for the second term, since α _k ≠0 (invertibility of Σ _k+1), there exists an integer $p_{\mathcal{V}} \geq K_{\mathcal{V}}$ ² such that

$$\begin{aligned} \sum_{(i,j)\in\{K_\mathcal{V},\ldots,\mathcal{L}+1\}, \ i+j \geq K_\mathcal{V}} A_i^T \varSigma^{-1}_k A_j=g_{p_\mathcal{V}} \frac {1}{2^{p_\mathcal{V}l}} + o \biggl(\frac{1}{2^{p_\mathcal{V}l}} \biggr), \end{aligned}$$

(40)

with $g_{p_{\mathcal{V}}}$, where ≠0. $p_{\mathcal{V}} > K_{\mathcal{V}}$ when the first term of the sum vanishes (see the example of a Gaussian semi-variogram at the end of the section). Therefore, the previous expression leads to the final estimation of α _k that reads

$$\begin{aligned} \alpha_k=-g_{p_{\mathcal{V}}} \frac{1}{2^{p_\mathcal{V}l}} + o \biggl(\frac{1}{2^{p_{\mathcal{V}}l}} \biggr). \end{aligned}$$

(41)

From Eq. (41) and noticing after a matricial calculus that, for large l, $1+\operatorname {Tr}(\beta_{k} \beta_{k}^{T})=1+o(1)$, it is straightforward that

$$\begin{aligned} \tilde{U} \sim2^{p_\mathcal{V}l}, \end{aligned}$$

(42)

that concludes the proof.

1.3 A.3 Proof of Proposition 4

Similarly to the proof of Proposition 3, the key point is the asymptotical decomposition of α _k . The difference is that in Eq. (38), the first term is no more equal to 0 since a simple kriging with error variance is no more exact for the points of the DoE {x _i } _i=1,…,k . As a result, for large enough l, α _k ∼1.

1.4 A.4 Proof of Proposition 5

Similarly to Proposition 3, the proof relies of the asymptotical decomposition of α _k for large l. According to Fig. 2, it is straightforward that

$$\begin{aligned} \big\|x_1-x^{\star}\big\|_2 =& \frac{\sqrt{2}}{2^l}, \end{aligned}$$

(43)

$$\begin{aligned} \big\|x_2-x^{\star}\big\|_2 =& \big\|x_3-x^{\star}\big\|_2 = \sqrt{\frac {1}{2^{2l}}+\biggl(1-\frac{1}{2^l}\biggr)^2}, \end{aligned}$$

(44)

$$\begin{aligned} \big\|x_4-x^{\star}\big\|_2 =& \sqrt{2} \biggl(1- \frac{1}{2^l}\biggr). \end{aligned}$$

(45)

Let us first focus on Gaussian semi-variograms:

According to the asymptotical development (i.e., for small distance h) of the semi-variogram models, it comes in this case

$$\begin{aligned} C^{\star}_1 =& c \biggl(1-\frac{2}{a^2 2^{2l}} \biggr)+o \biggl(\frac {1}{2^{2l}} \biggr), \end{aligned}$$

(46)

$$\begin{aligned} C^{\star}_2 =& C^{\star}_3 =c\ e^{-\frac{1}{a^2}} \biggl(1+\frac{2}{a^2 2^l}+\frac{2}{2^{2l}}\biggl( \frac{1}{a^4}-\frac{1}{a^2}\biggr) \biggr)+o \biggl(\frac {1}{2^{2l}} \biggr), \end{aligned}$$

(47)

$$\begin{aligned} C^{\star}_4 =& c\ e^{-\frac{2}{a^2}} \biggl(1+ \frac{4}{a^2 2^l}+\frac {2}{2^{2l}}\biggl(\frac{4}{a^4}-\frac{1}{a^2} \biggr) \biggr)+o \biggl(\frac {1}{2^{2l}} \biggr). \end{aligned}$$

(48)

Note that the previous expressions are of Eq. (34) with $(K_{\mathcal{V}},\mathcal{L} )=(1,2)$. According to Eq. (39)

$$\begin{aligned} \alpha_k=-\sum _{(i,j)\in\{0,1,2,3\}, \ i+j \geq1} A_i^T \varSigma^{-1}_4 A_j, \end{aligned}$$

(49)

where

$$\begin{aligned} A_0 =& \bigl(c,c e^{-\frac{1}{a^2}},c e^{-\frac{1}{a^2}},c e^{-\frac {2}{a^2}} \bigr)^T, \end{aligned}$$

(50)

$$\begin{aligned} A_1 =& \biggl(0,c e^{-\frac{1}{a^2}}\frac{2}{a^2 2^l},c e^{-\frac {1}{a^2}}\frac{2}{a^2 2^l},c e^{- \frac{2}{a^2}}\frac{4}{a^2 2^l} \biggr)^T, \end{aligned}$$

(51)

$$\begin{aligned} A_2 =& \biggl(-c \frac{2}{a^4 2^{2l}},c e^{-\frac{1}{a^2}} \frac {2}{2^{2l}}\biggl(\frac{1}{a^4}-\frac{1}{a^2}\biggr),c e^{-\frac{1}{a^2}}\frac{2}{2^{2l}} \biggl(\frac{1}{a^4}-\frac{1}{a^2} \biggr), \\ &\hphantom{\biggl(} c e^{-\frac{2}{a^2}}\frac{2}{2^{2l}}\biggl(\frac {4}{a^4}- \frac{1}{a^2}\biggr) \biggr)^T, \end{aligned}$$

(52)

and with $\|A_{3}\|_{2} = o (\frac{1}{2^{2l}} )$. Splitting the sum in Eq. (49) with respect to the power of $\frac {1}{2^{l}}$, it remains to evaluate each term. This is performed in what follows.

Term

$\sum_{(i,j)\in\{0,1,2,3\}, \ i+j=1} A_{i}^{T} \varSigma^{-1}_{4} A_{j}$:

Since $\varSigma^{-1}_{4} A_{0}= (1,0,\ldots,0 )^{T}$, it comes using the expression of A ₁

$$\begin{aligned} A_1^T \varSigma^{-1}_4 A_0=0. \end{aligned}$$

(53)

Moreover, by definition the vector $(\lambda_{0},\lambda_{1},\lambda _{2},\lambda_{3} )^{T}=\varSigma^{-1}_{4} A_{1}$ satisfies: $c \lambda_{0}+ c e^{-\frac{1}{a^{2}}} \lambda_{1}+c e^{-\frac{1}{a^{2}}} \lambda_{2} + e^{-\frac{2}{a^{2}}}\lambda_{3}=0$. Therefore,

$$\begin{aligned} A_0^T \varSigma^{-1}_4 A_1=0. \end{aligned}$$

(54)

Term

$\sum_{(i,j)\in\{0,1,2,3\}, \ i+j=2} A_{i}^{T} \varSigma^{-1}_{4} A_{j}$:

With the same kind of calculation, one can find

$$\begin{aligned} A_0^T \varSigma^{-1}_4 A_2=A_2^T \varSigma^{-1}_4 A_0=-c \frac{2}{a^2 2^{2l}}. \end{aligned}$$

(55)

For the term $A_{1}^{T} \varSigma^{-1}_{4} A_{1}$, a direct calculation using the software Mathematica, for example, shows that $A_{1}^{T} \varSigma^{-1}_{4} A_{1} \neq c \frac{4}{a^{2} 2^{2l}}$ which implies that

$$\begin{aligned} \sum_{(i,j)\in\{0,1,2,3\}, \ i+j=2} A_i^T \varSigma^{-1}_4 A_j = K \frac {1}{2^{2l}}, \end{aligned}$$

(56)

where K does not depend on l. Combining Eqs. (53), (54), and (56), we finally find

$$\begin{aligned} \alpha_k=-\frac{K}{2^{2l}} + o \biggl( \frac{1}{2^{2l}} \biggr). \end{aligned}$$

(57)

As for exponential semi-variograms, the proof mimics the previous one and is therefore not fully described in the sequel. The starting point is again the asymptotical development of the exponential semi-variogram model, which gives

$$\begin{aligned} C^{\star}_1 =& c \biggl(1-\frac{\sqrt{2}}{a 2^{l}} \biggr)+o \biggl(\frac {1}{2^l} \biggr), \end{aligned}$$

(58)

$$\begin{aligned} C^{\star}_2 =& C^{\star}_3 =c \ e^{-\frac{1}{a}} \biggl(1+\frac{1}{a 2^l} \biggr)+o \biggl(\frac{1}{2^l} \biggr), \end{aligned}$$

(59)

$$\begin{aligned} C^{\star}_4 =& c \ e^{-\frac{\sqrt{2}}{a}} \biggl(1+ \frac{\sqrt{2}}{a 2^l} \biggr)+o \biggl(\frac{1}{2^{l}} \biggr). \end{aligned}$$

(60)

Note that in this case, $(K_{\mathcal{V}},\mathcal{L} )=(1,1)$. It leads to

$$\begin{aligned} \alpha_k=-\sum_{(i,j)\in\{0,1,2\}, \ i+j \geq1} A_i^T \varSigma^{-1}_4 A_j \end{aligned}$$

(61)

where

$$\begin{aligned} \begin{array}{l} A_0= \bigl(c,c e^{-\frac{1}{a}},c e^{-\frac{1}{a}},c e^{-\frac{\sqrt {2}}{a}} \bigr)^T,\\ A_1= \biggl(-c \displaystyle\frac{\sqrt{2}}{a 2^{l}},c e^{-\frac{1}{a}} \frac{2}{a 2^l},c e^{-\frac{1}{a}} \frac{2}{a 2^l},c e^{-\frac{\sqrt{2}}{a}} \frac {\sqrt{2}}{a 2^l} \biggr)^T, \end{array} \end{aligned}$$

(62)

and with $\|A_{2}\|_{2} = o (\frac{1}{2^{l}} )$. By direct calculation, one can find

$$\begin{aligned} \sum_{(i,j)\in\{0,1,2\}, \ i+j = 1} A_i^T \varSigma^{-1} A_j =& - \frac {2 \sqrt{2} c}{a} \frac{1}{2^l}, \end{aligned}$$

(63)

therefore,

$$\begin{aligned} \alpha_k=\frac{2 \sqrt{2} c}{a} \frac{1}{2^l} + o \biggl(\frac {1}{2^{l}} \biggr). \end{aligned}$$

(64)

That concludes the proof.