Performance study of gradient-enhanced Kriging

Abstract

The use of surrogate models for approximating computationally expensive simulations has been on the rise for the last two decades. Kriging-based surrogate models are popular for approximating deterministic computer models. In this work, the performance of Kriging is investigated when gradient information is introduced for the approximation of computationally expensive black-box simulations. This approach, known as gradient-enhanced Kriging, is applied to various benchmark functions of varying dimensionality (2D-20D). As expected, results from the benchmark problems show that additional gradient information can significantly enhance the accuracy of Kriging. Gradient-enhanced Kriging provides a better approximation even when gradient information is only partially available. Further comparison between gradient-enhanced Kriging and an alternative formulation of gradient-enhanced Kriging, called indirect gradient-enhanced Kriging, highlights various advantages of directly employing gradient information, such as improved surrogate model accuracy, better conditioning of the correlation matrix, etc. Finally, gradient-enhanced Kriging is used to model 6- and 10-variable fluid–structure interaction problems from bio-mechanics to identify the arterial wall’s stiffness.

Appendices

Appendix A: Analytical expressions for likelihood gradients

1.1 Gaussian correlation function

Derivative of correlation function with respect to $\theta _k$:

$$\begin{aligned} \frac{\partial }{\partial \theta _{k}} \left( \psi (d_{u(v)})\right) = -10^{\theta _{u(v)}}d_{u(v)}^2{\rm{log}}(10){\rm{exp}}\left( -\sum _{m=1}^{k} \theta _{m} d_{m}^2 \right) \end{aligned}$$

(22)

Derivatives of cross-correlation functions with respect to $\theta _k$:

$$\begin{array}{l}\frac{\partial }{\partial \theta _{k}} \left( \frac{\partial \varvec{\Psi }^{(i,j)}}{\partial x_{v}^{(i)}}\right) \nonumber = {\left\{ \begin{array}{ll} 2 d_v 10^{\theta _{v}} {\rm{log}}(10) \varvec{\Psi }^{(i,j)} \left[ 1-10^{\theta _{k}}d_{k}^2 \right] &\quad {\text {if}}\,\, v = k \\ 2 d_v 10^{\theta _{v}} {\rm{log}}(10) \varvec{\Psi }^{(i,j)} \left[ -10^{\theta _{k}}d_{k}^2 \right] &\quad {\text {if}}\,\, v \not = k \end{array}\right. } \end{array}$$

(23)

$$\begin{array}{l}\frac{\partial }{\partial \theta _{k }} \left( \frac{\partial ^{2}\varvec{\Psi }^{(i,j)}}{\partial x_{u}^{(i)}\partial x_{v}^{(j)}} \right) = {\left\{ \begin{array}{ll} -4 d_u d_v 10^{\theta _{u}} 10^{\theta _{v}} {\rm{log}}(10) \varvec{\Psi }^{(i,j)} \left[ 1-10^{\theta _k}d_k^2 \right] &\quad {\text {if}}\,\, u|v = k \\ 4 d_u d_v d_k^210^{\theta _{u}} 10^{\theta _{v}} 10^{\theta _k} {\rm{log}}(10) \varvec{\Psi }^{(i,j)} &\quad {\text {otherwise}} \\ \end{array}\right. } \end{array}$$

(24)

$$\begin{array}{l}\frac{\partial }{\partial \theta _{k }} \left( \frac{\partial ^{2}\varvec{\Psi }^{(i,j)}}{\partial x_{u=v}^{(i)}\partial x_{u=v}^{(j)}}\right) = {\left\{ \begin{array}{ll} {\rm{log}}(10) \varvec{\Psi }^{(i,j)} \Bigl [2(10^{\theta }) + 4 (10^{3\theta }) d^4 - 10 (10^{2\theta }) d^2 \Bigr ] &\quad {\text {if}}\,\, (u = v) = k \\ -{\rm{log}}(10) \varvec{\Psi }^{(i,j)} 10^{\theta _k}d_k^2 \left[ 2(10^{\theta }) - 4 (10^{2\theta }) d^2 \right] &\quad {\text {if}}\,\, (u=v) \not = k \\ \end{array}\right. } \end{array}$$

(25)

1.2 Matérn $\frac{3}{2}$ correlation function

Derivative of correlation function with respect to $\theta _k$:

$$\begin{aligned} \frac{\partial }{\partial \theta _{k }} \left( \psi _{\nu = 3/2} (d_{u(v)})\right) = -1.5 (10^{\theta _{u(v)}}) {\rm{log}}(10) d_{u(v)}^2 {\rm{exp}}\left( -\sqrt{3}a\right) \end{aligned}$$

(26)

Derivatives of cross-correlation functions with respect to $\theta _k$:

$$\begin{array}{l} \frac{\partial }{\partial \theta _{k }} \left( \frac{\partial \varvec{\Psi }^{(i,j)}}{\partial x_{v}^{(i)}}\right) = {\left\{ \begin{array}{ll} 3 (10^{\theta _{v}}) d_{v} {\rm{log}}(10) {\rm{exp}}\left( -\sqrt{3}a\right) \left[ 1 - \frac{\sqrt{3} 10^{\theta _{k }} d_{k}^2}{2a}\right] &\quad {\text {if}}\,\, v = k \\ 3 (10^{\theta _{v}}) d_{v} {\rm{log}}(10) {\rm{exp}}\left( -\sqrt{3}a\right) \left[ \frac{-\sqrt{3} 10^{\theta _{k }} d_{k}^2}{2a}\right] &\quad {\text {if}}\,\, v \not = k \end{array}\right. } \end{array}$$

(27)

$$\begin{array}{l} \frac{\partial }{\partial \theta _{k }} \left( \frac{\partial ^{2}\varvec{\Psi }^{(i,j)}}{\partial x_{u}^{(i)}\partial x_{v}^{(j)}} \right) = {\left\{ \begin{array}{ll} V_1 \left[ \frac{-10^{\theta _{k }}d_{k}^2}{2a^3} + \frac{1}{a} -\frac{\sqrt{3} 10^{\theta _{k }} d_{k}^2}{2a^2}\right] &\quad {\text {if}}\,\, u|v = k \\ V_1 \left[ \frac{-10^{\theta _{k }} d_{k}^2}{2a^3} - \frac{\sqrt{3} 10^{\theta _{k }} d_{k}^2}{2a^2}\right] &\quad {\text {otherwise}} \\ \end{array}\right. } \end{array}$$

(28)

$$\begin{array}{l}\frac{\partial }{\partial \theta _{k }} \left( \frac{\partial ^{2}\varvec{\Psi }^{(i,j)}}{\partial x_{u=v}^{(i)}\partial x_{u=v}^{(j)}}\right) = {\left\{ \begin{array}{ll} \frac{- V_1 10^{\theta _{k }}d_{k}^2 (1+\sqrt{3}a)}{2a^3} + \frac{2V_1}{a} &{} \\ + V_2 \left[ 1 - \frac{\sqrt{3} 10^{\theta _{k }} d_{k}^2}{2a} \right] &\quad {\text {if}}\,\, (u = v) = k \\ \frac{- V_1 10^{\theta _{k }}d_{k}^2 (1+\sqrt{3}a)}{2a^3} - \frac{V_2 \sqrt{3} 10^{\theta _{k }} d_{k}^2}{2a} &\quad {\text {if}}\,\, (u=v) \not = k \\ \end{array}\right. } \end{array}$$

(29)

where

$$\begin{aligned} V_1= -3\sqrt{3}10^{\theta _{u}} 10^{\theta _{v}} d_{u} d_{v} {\rm{log}}(10) {\rm{exp}}\left( -\sqrt{3}a\right) \quad \& \quad V_2 = 3 (10^{\theta _{u=v}}) {\rm{log}}(10) {\rm{exp}}\left( -\sqrt{3}a\right) \end{aligned}$$

(30)

1.3 Matérn $\frac{5}{2}$ correlation function

Derivative of correlation function with respect to $\theta _k$:

$$\begin{aligned} \frac{\partial }{\partial \theta _{k }} \left( \psi _{\nu = 5/2} (d_{u(v)})\right) = \frac{-\left( 5+5\sqrt{5} a\right) 10^{\theta } {\rm{log}}(10) d_{u(v)}^2 {\rm{exp}}\left( -\sqrt{5}a\right) }{6} \end{aligned}$$

(31)

Derivatives of cross-correlation functions with respect to $\theta _k$:

$$\begin{array}{l}\frac{\partial }{\partial \theta _{k }} \left( \frac{\partial \varvec{\Psi }^{(i,j)}}{\partial x_{v}^{(i)}}\right) = {\left\{ \begin{array}{ll} 10^{\theta _v} d_v C_2 \left[ C_1 + \left( \frac{-25}{6}\right) 10^{\theta _{k }}d_{k}^2\right] &\quad {\text {if}}\,\, v = k \\ 10^{\theta _v} 10^{\theta _k} d_v d_k^2 \left( \frac{-25C_2}{6}\right) &\quad {\text {if}}\,\, v \not = k \end{array}\right. } \end{array}$$

(32)

$$\begin{array}{l}\frac{\partial }{\partial \theta _{k }} \left( \frac{\partial ^{2}\varvec{\Psi }^{(i,j)}}{\partial x_{u}^{(i)}\partial x_{v}^{(j)}} \right) = {\left\{ \begin{array}{ll} \frac{-25C_2 \left( 1 - \frac{\sqrt{5} 10^{\theta _{k }}d_{k}^2 }{2a} \right) 10^{\theta _u} 10^{\theta _v} d_u d_v }{3} &\quad {\text {if}}\,\, u|v = k \\ \frac{C_2 25 \sqrt{5} 10^{\theta _u} 10^{\theta _v} 10^{\theta _k} d_u d_v d_{k}^2}{6a} &\quad {\text {otherwise}} \\ \end{array}\right. } \end{array}$$

(33)

$$\begin{array}{l} \frac{\partial }{\partial \theta _{k }} \left( \frac{\partial ^{2}\varvec{\Psi }^{(i,j)}}{\partial x_{u=v}^{(i)}\partial x_{u=v}^{(j)}}\right) = {\left\{ \begin{array}{ll} V_3 + V_4 &\quad {\text {if}}\,\, (u = v) = k \\ V_3 &\quad {\text {if}}\,\, (u=v) \not = k \end{array}\right. } \end{array}$$

(34)

where

$$\begin{aligned}V_3 = \left[ \left( \frac{25\sqrt{5}}{6a}\right) (10^{\theta _{u=v}})^2(d_{u=v})^2 - \left( \frac{25}{6}\right) 10^{\theta _{u=v}} \right] C_2 10^{\theta _{k}}d_{k}^2 \end{aligned}$$

(35)

$$\begin{aligned}V_4 = \left( \frac{-50C_2 (10^{\theta })^2d^2}{3}\right) + C_1C_2 10^{\theta }; \qquad C_1 = \left( \frac{5\sqrt{5}}{3} a + \frac{5}{3}\right);\qquad C_2 = {\rm{log}}(10){\rm{exp}}\left( -\sqrt{5}a\right) \end{aligned}$$

(36)

Appendix B: Problem description (fluid structure interaction problem)

In 2008, the World Health Organization reported that cardiovascular diseases are the leading cause of death in the world. Aortic stiffening has been linked to many (patho)physiological mechanisms and conditions. In all of them, the stiffness and mechanical properties of the aortic wall are altered. Consequently, non-invasive measurements of the arterial stiffness are needed and assessing the arterial stiffness should be part of the routine clinical diagnosis and follow-up procedures [19]. In this example, the stiffness distribution along the length of an artery is identified using a simplified numerical model. Previously, unconstrained quasi-Newton optimization with line search and a discrete adjoint solver for the calculation of the gradient were applied for this purpose [5].

The numerical model is one-dimensional in an axisymmetric $(r,\phi ,z)$ coordinate system, as depicted in Fig. 23. It consists of $k-1$ elastic segments, each with its own stiffness. Inside the artery, there is an incompressible blood flow. Furthermore, the interaction between this blood flow and the elastic wall is taken into account.

Fig. 23

The one-dimensional and axisymmetric model for blood flow in an artery with the prescribed velocity at the inlet (left) and the Windkessel model at the outlet (right). The segments, radius $r$, wall thickness $h$ and length $\ell$ are indicated

The blood flow rate at a point can be measured as a function of time using non-invasive techniques. So, the flow rate at the inlet is prescribed as a function of the time $t$ with a period corresponding to one heart beat $t_b$. A Windkessel model relates this velocity with the outlet pressure [35]. This Windkessel model (See Fig. 23) represents the remainder of the circulation, downstream from the artery. The capacitor $C$ represents the compliance of the arterial system, while the resistors $R_p$ and $R_d$ model the proximal and distal viscous resistance, respectively.

The goal is to adjust the stiffness parameters of this fluid–structure interaction model so that the displacement of the arterial wall as a function of time matches the displacement data from a non-invasive measurement. The elasticity modulus $E_i$ of each segment $i$ ($i\in \{1,\ldots ,k-1\}$) is modified by the corresponding parameter $x_i$ which varies from −1 to 1.

$$\begin{aligned} E_i=E_o\left( 1+\frac{1}{2}x_i\right) \end{aligned}$$

(37)

As the Windkessel model also has a significant impact on the wall displacement, the value of $C$ is modified by the parameter $x_k$ which varies from −1 to 1.

$$\begin{aligned} C=C_o\left( 1+\frac{1}{2}x_k\right) ^{-1} \end{aligned}$$

(38)

The parameter $x_k$ will be identified, together with the parameters $x_i$ ($i\in \{1,\ldots ,k-1\}$). All fixed parameters are listed in Table 13.

Table 13 The parameters of the fluid–structure interaction model and the Windkessel model [35]

The governing flow equations and the structural equations, which are formulated, discretized and linearized in reference [5], are solved separately. Consequently, coupling iterations using the IQN-ILS algorithm [4] need to be performed between the flow equations and the structural equations to obtain the solution of the coupled problem. A cost function $y({\varvec{{x}}})$ is defined as the sum over all time steps and all segments of the squared difference between the radius in the simulation and in the measurement. This measurement, which would normally be obtained from a non-invasive medical imaging technique such as ultrasound, is mimicked by a simulation with the same model. It is then assumed that the parameter values in this “measurement simulation” have been forgotten and their values are calculated using the parameter identification. The vector ${\varvec{{x}}}$ contains the $k$ parameters which are defined in 37 and 38. The state vector ${\varvec{{s}}}$ contains the radius in all segments and all time steps. The parameter identification can thus be reformulated as a minimization problem

$$\begin{aligned} \min _{{\varvec{{x}}},{\varvec{{s}}}}y({\varvec{{x}}},{\varvec{{s}}}) \end{aligned}$$

(39)

subject to the governing equations as constraints. As the state vector ${\varvec{{s}}}$ depends on the parameters ${\varvec{{x}}}$, the total derivative of the cost function $y({\varvec{{x}}},{\varvec{{s}}})=y({\varvec{{x}}},{\varvec{{s}}}({\varvec{{x}}}))$ with respect to the parameters is obtained with the chain rule.

$$\begin{aligned} {\frac{{\mathrm {d}}y}{{\mathrm {d}}{\varvec{{x}}}}}={\frac{\partial y}{\partial {\varvec{{x}}}}}+{\frac{\partial y}{\partial {\varvec{{s}}}}}{\frac{{\mathrm {d}}{\varvec{{s}}}}{{\mathrm {d}}{\varvec{{x}}}}} \end{aligned}$$

(40)

To avoid the direct calculation of ${{\mathrm {d}}{\varvec{{s}}}/{\mathrm {d}}{\varvec{{x}}}}$, the adjoint equations of this unsteady fluid–structure interaction problem are derived and solved, which involves backward time steps. In each of these steps, the adjoint flow equations and adjoint structural equations are coupled using the IQN-ILS algorithm [4], similarly to the forward equations.

The stiffness off each segment is identified by constructing a surrogate model for $y({\varvec{{x}}})$, followed by a search for its minimum and the corresponding values of ${\varvec{{x}}}$.

Appendix C: Surrogate model accuracy

See Tables 14, 15, 16, 17.

Table 14 Prediction of derivatives

Table 15 Efficiency of GEK and Matérn class of correlation functions (prediction of derivatives)

Table 16 Efficiency of GEK (10D FSI function)

Table 17 Prediction of derivatives (FSI functions)