# Sensitivity and calibration of turbulence model in the presence of epistemic uncertainties

- 74 Downloads

## Abstract

The solution of Reynolds-averaged Navier–Stokes equations employs an appropriate set of equations for the turbulence modelling. The closure coefficients of the turbulence model were calibrated using empiricism and arguments of dimensional analysis. These coefficients are considered universal, but there is no guarantee this property applies to test cases other than those used in the calibration process. This work aims at revisiting the calibration of the closure coefficients of the original Spalart–Allmaras turbulence model using machine learning, adaptive design of experiments and accessing a high-performance computing facility. The automated calibration procedure is carried out once for a transonic, wall-bounded flow around the RAE 2822 aerofoil. It was found that: (a) an optimal set of closure coefficients exists that minimises numerical deviations from experimental data; (b) the improved prediction accuracy of the calibrated turbulence model is consistent across different flow solvers; and (c) the calibrated turbulence model outperforms slightly the standard model in analysing complex flow features around additional test cases (ONERA M6 wing, axisymmetric transonic bump, forced sinusoidal motion of NACA 0012 aerofoil). A by-product of this study is a fully calibrated turbulence model that leverages on current state-of-the-art computational techniques, overcoming inherent limitations of the manual fine-tuning process.

## Keywords

Uncertainty quantification Calibration Turbulence model closure coefficients Machine-learning Sobol indices Adaptive design of experiments## 1 Introduction

A deterministic computational fluid dynamics (CFD) analysis gives a single solution for a certain set of input parameters, e.g. geometry, free-stream flow conditions, etc. In practice, these parameters may be uncertain and the associated variability may have a significant impact on the final results. For this reason, stochastic CFD analyses are needed to assess the uncertainty in the solution and to achieve a certain level of robustness or reliability in the final aerodynamic design [24]. The point-collocation nonintrusive polynomial chaos technique is the method of choice to propagate the uncertainty in CFD analyses, as exemplified in [8] (and references therein). This technique requires less deterministic CFD analyses than Monte Carlo techniques by assuming a polynomial chaos expansion of low degree for the uncertain output variables.

Today, it is apparent that CFD workflows contain considerable uncertainty, often not quantified [18]. Numerical uncertainties in the results come from: (a) physical modelling errors and uncertainties, for example, in accurate predictions of turbulent flows; (b) numerical errors arising from mesh and discretisation inadequacies; and (c) aleatory uncertainties derived from natural variability, and epistemic uncertainties due to the lack of knowledge in the parameters of a specific fluid problem. The work presented in the current paper addresses the last type of uncertainty above-mentioned, which calls for turbulence modelling uncertainty quantification, sensitivity analysis and parameter calibration.

Uncertainty in the closure coefficients of a turbulence model is an important source of error in Reynolds-averaged Navier–Stokes (RANS) analyses, but no reliable estimator for this error component exists. It is important to stress that RANS equations rely on several assumptions, e.g. the Boussinesq approximation, which assumes that the turbulent shear stress depends linearly on the mean rate of strain rate. Due to this, it is very unlikely that truly universal coefficients exist. There is no consensus on the best values of these coefficients, as suggested by the wide range of values proposed in the open literature [1]. Current efforts to address these concerns use Bayesian approaches. For example, Ref. [6] described a stochastic error estimate of turbulence models based on variability in the model coefficients. In a sensitivity analysis, it was found that Von Kármán constant, *k*, has the largest impact on uncertainty in \(u^+\) in the log layer of a flat plate boundary layer. This conclusion was suggested analysing results from several turbulence models, including Spalart–Allmaras [21] and Wilcox \(k-\omega\) models. In [12], a Bayesian inference framework was used to quantify the uncertainty in Spalart–Allmaras model due to the uncertainty in the closure coefficients. For a flat plate and a backward-facing step problem, the coefficients *k*, \(c_{v1}\), and \(c_{b1}\) were found to contribute most to the uncertainty in Spalart–Allmaras model for the chosen output quantities of interest.

References [10, 11] proposed methodologies to estimate the closure coefficients of the Wilcox \(k-\omega\) turbulent model for steady and unsteady flow scenarios, respectively. In [11], similar in scope to the one proposed herein, a gradient-based optimization strategy was employed to find the values of the model coefficients providing the best fit between experimental observations and simulated data. Differently from [11], the methodology developed in this work exploits machine learning techniques to build accurate surrogate models that: (a) are used to perform a global sensitivity analysis, providing interesting insights about the problem under study; and (b) are coupled with a global optimization strategy, mitigating the impact of the choice of the initial point on the optimizer’s behaviour and the risk to find sub-optimal solutions due to local minima.

Reference [14] quantified the uncertainty and sensitivity of three turbulence models (Spalart–Allmaras, Wilcox \(k-\omega\), and Menter shear–stress transport models) due to uncertainty in the values of closure coefficients for transonic, wall-bounded flows. The analysis was carried out using point-collocation nonintrusive polynomial chaos technique. The test cases were for the flow around an asymmetric bump at 0\(^{\circ }\) angle of attack and for Case 6 of the RAE 2822 aerofoil at a prescribed normal force coefficient [2]. For the aerofoil case, the angle of attack was adjusted for each (baseline) turbulence model to match the target normal force coefficient. The same angle of attack was then used in all subsequent simulations where the closure coefficients were modified for uncertainty quantification. Observe that this approach fails to meet the prescribed normal force coefficient for any variation of the closure coefficients from their baseline values. Furthermore, no indications were given on the best values of the closure coefficients for each turbulence model that improved the agreement with experimental data.

The aim of this study is to revisit the calibration of the standard values of the closure coefficients commonly employed in Spalart–Allmaras turbulence model. The work is structured around three technical objectives. The first objective is to exploit current state-of-the-art machine-learning techniques to assess the sensitivity of the output quantities of interest on the uncertainty in turbulence model closure coefficients. The second objective is to calibrate the closure coefficients of Spalart-Allmaras turbulence model by minimising the deviation of numerical results from available experimental data for transonic flows around an aerofoil (Case 6 of RAE 2822). The third objective evaluates the generality of the calibrated Spalart–Allmaras turbulence model on different flow solvers and the expected improvements in prediction accuracy for a variety of transonic flows (Case 9 of RAE 2822, ONERA M6, axisymmetric transonic bump, forced sinusoidal motion of NACA 0012 aerofoil).

The uncertainty quantification, the sensitivity analyses, and the calibration of the turbulence model closure coefficients suffer from the curse of dimensionality [3]. In this respect, the reader is invited to reflect upon the work by Sørensen [20]:The \(\gamma\) transition model has only been calibrated for classical boundary layer flows. Application to other types of wall-bounded flows is possible, but might require a modification of the underlying correlations.

^{1}

To overcome the large amount of computations, a strategy based on surrogate models is employed. This requires setting up and running a design of experiments (DOE) plan to acquire the relevant information on the system behaviour. A surrogate model that mimics the dependence between the turbulence model closure coefficients and the output quantities of interest is then built and employed to perform the sensitivity analysis and the model calibration. The key aspect of this strategy is to minimise the number of deterministic CFD simulations while maintaining an accurate representation of the system behaviour. In this study, these aspects are encapsulated in an adaptive DOE (ADOE) algorithm that: (a) identifies the regions of the design space that are more difficult to model due to strong non-linearities or scarcity of data, for example; (b) distributes iteratively the design points in those areas of the design space; and (c) selects automatically the surrogate model that best fits the results obtained from the DOE plan. All these features are supported by the machine-learning framework described in [4], where the robustness, efficiency and accuracy of the proposed ADOE algorithm were found to be superior to traditional DOE techniques.Determining the empirical correlations by numerical optimization, along with debugging the model, demands a very large amount of computations, and it is the hope that other researchers can confirm the present expressions by implementation in other flow solvers.

The paper continues in Sect. 2 with a description of the flow solver and the machine-learning approach used to calibrate the closure coefficients of Spalart–Allmaras turbulence model. Then, the sensitivity and calibration of Spalart–Allmaras turbulence model is presented in Sect. 3. The application of the calibrated turbulence model to cases different from the calibration case is discussed in Sect. 4. Finally, conclusions and future recommendations are given in Sect.on 5.

## 2 Methodology

The computational framework consists of two software tools. The flow solver in Sect. 2.1 was used for the flow predictions. The uncertainty quantification, the sensitivity analysis and the optimisation of the closure coefficients were carried out with the software described in Sect. 2.2.

### 2.1 Flow solver

The flow solver employed in this study is DLR-Tau [17], a finite volume based CFD flow solver used by several aerospace industries across Europe. The DLR-Tau solver uses an edge-based vertex-centred scheme, where the convective terms are computed via several first and second-order schemes, including central and upwind types. The viscous terms are computed with a second-order central scheme. Time integration is performed either with various explicit Runge–Kutta schemes or the Lower-Upper Symmetric Gauss-Seidel (LU-SGS) implicit approximate factorisation scheme. For time accurate computations, the dual time stepping approach of Jameson [9] is employed. Convergence is improved with a multi-grid acceleration technique based on agglomerated coarse grids. Several models for turbulence closure are available.

Spalart–Allmaras closure coefficients and epistemic intervals

Parameter | Standard value | Lower bound | Upper bound |
---|---|---|---|

\(\sigma\) | \(0.666\bar{6}\) | 0.6000 | 1.4000 |

\(k\times 10^{1}\) | 4.1000 | 3.6000 | 4.2000 |

\(c_{v1}\) | 7.1000 | 6.9000 | 7.5000 |

\(c_{w3}\) | 2.0000 | 1.5000 | 2.7500 |

\(c_{t3}\) | 1.2000 | 1.0000 | 2.0000 |

\(c_{t4}\times 10^{1}\) | 5.0000 | 3.0000 | 7.0000 |

\(c_{b1}\times 10^{1}\) | 1.3550 | 1.2893 | 1.4000 |

\(c_{b2}\times 10^{1}\) | 6.2200 | 6.0983 | 7.0000 |

\(c_{w2}\times 10^{1}\) | 3.0000 | 0.5500 | 3.5250 |

The choice for using DLR-Tau was made to demonstrate that uncertainty in closure coefficients has been overlooked in the past, even for an industrial-grade software tool. This situation may have arisen for convenience, by removing additional difficulties from the multifaced complexities of CFD algorithmic implementation, or negligence, by treating CFD as an established technique. The present work carries out an investigation into an intrinsic weakness of turbulence modelling, and creates a preliminary background knowledge for a robust engineering design.

### 2.2 Machine-learning Framework

The machine-learning framework is provided by the software platform Noesis Optimus.^{2} The framework consists of an iterative ADOE technique that analyses available data, generally produced by previous iterations or previous DOE runs, to distribute the design points of the next iteration in areas of the parameter space considered of interest. The choice of the location of new sample points is driven by two competing factors. The first factor, denoted space-learning, tends to cover uniformly the design space. No information about the response of the model is, therefore, needed. The second factor, denoted feature-learning, aims at improving the accuracy of the surrogates by identifying critical areas of the design space, such as non-linearities and discontinuities. The reader is referred to [4] for more details on our implementation of space-learning and feature-learning factors, the associated algorithms, and the relevant benchmark cases (analytical and industrially-relevant).

- 1.
for each type of model, a leave-1-out cross-validation is applied;

- 2.
the corresponding values of the coefficient of determination (denoted as \(R_{\text {PRESS}}^2\)) is calculated based on the cross-validation residuals;

- 3.
the best model is identified as the one characterized by the smallest value of the \(R_{\text {PRESS}}^2\) metric.

## 3 Transonic wall-bounded test case

Uncertainty quantification and sensitivity analysis for the aerofoil test case are reported in this section. The section continues with the calibration of the closure coefficients, discusses the implementation across different flow solvers, and investigates the influence of the spatial discretisation on the results.

### 3.1 Description

Navier–Stokes calculations for Case 6 of the RAE 2822 aerofoil [2] were performed with Spalart–Allmaras turbulence model. The experimental data for this case are for \(M\,=\,0.729\) and \(Re\,=\,6.5\times 10^6\) at a prescribed normal force coefficient \(C_{\text {N}}\,=\,0.743\). For all calculations, the angle of attack, \(\alpha\), was adjusted to match this value of \(C_{\text {N}}\). This is a countermeasure taken to minimise the possibility of an inaccurate measurement of the angle of attack, favouring the use of the numerical integration of the measured pressure coefficient data as a more reliable reference condition. Ensuring the accuracy of experimental data is at the centre of much research, and the interested reader is referred to the concluding remarks for recent initiatives in this direction.

The computational grid adopted for the flow simulations, shown in Fig. 1a, is available from the NPARC Alliance Validation Archive web site.^{3} The C-grid, denoted hereafter the coarse grid, consists of a single-block with \(369\times 65\) points. The far-field boundary is placed at about 20 chords from the aerofoil, and the distance of the first grid points off the aerofoil surface is about \({10}^{-5}\) chord.

For all steady calculations, the explicit time stepping and the fourth order Runge–Kutta scheme were used. To accelerate the convergence to a steady state, a local time-stepping, implicit residual smoothing and a full multigrid method were used. The discretisation of the convective and diffusive fluxes of both RANS and Spalart–Allmaras equations is based on the second order Roe’s flux difference splitting scheme. Venkatakrishnan’s flux limiter was used for all simulations reported in this paper. A no-slip boundary condition was applied on the aerofoil surface, and far-field boundary conditions were applied to the far-field boundaries. The ratio of eddy viscosity to molecular viscosity 0.5 is prescribed at the far-field boundaries, whereas at smooth walls, zero turbulence condition is enforced. The CFL number was set to 1.2 and the number of multigrid (MG) levels to 3. Simulations were run for 4000 MG cycles to compute the steady state solution. With this setup, the overall residuals of Navier–Stokes and Spalart–Allmaras equations decreased by about five orders of magnitude, and all force and moment components converged within 2000 MG cycles.

### 3.2 Generation of the response surface model

The computational framework described in Sect. 2 was used to generate a response surface model between the input parameters and the system outputs. The input parameters include nine uncertain closure coefficients of Spalart–Allmaras turbulence model, which are described by the epistemic intervals in Table 1, and the angle of attack for matching the prescribed normal force coefficient. The system outputs that were monitored consist of three quantities: the lift and drag coefficients, and the sum of squared errors (SSE) between the pressure coefficient distribution from experimental data and that from numerical results. For uncertainty quantification and sensitivity analysis, and for the calibration of the closure coefficients, SSE is used as the output quantity of interest.

- 1.
the ADOE algorithm was initialised with 1025 sample points selected by a two-level full factorial approach including the central point;

- 2.
these results were then used to initialize the ADOE strategy and to compute sequentially (in batches of 25 analyses) an additional set of 1500 CFD simulations. At the end of this step, the machine-learning framework identified automatically the best surrogate models to link the ten input variables to the system outputs based on a table containing 2525 design points;

- 3.
for each set of values of the closure coefficients included in this table, the surrogate models were interrogated to find the angle of attack that matched the target \(C_{\text {N}}\,=\,0.743\). By removing the dependence on the angle of attack, the 1024 sample points of the full factorial plan were reduced to 512. Hence, this step reduced the size of the table to 2013 sample points by adjusting the angle of attack to match the target \(C_{\text {N}}\);

- 4.
finally, results at the previous step were validated by running a set of deterministic CFD simulations. Due to problems occurred on the local network connectivity, only 1980 experiments out of 2013 were successful and used to build the final surrogate models employed in the sensitivity analysis and calibration of the turbulence closure coefficients.

- 1.
map the value assigned to the turbulence coefficients within the input file required to run the CFD analysis;

- 2.
submit the job with the instructions to launch the DLR-Tau flow solver to the resource manager of the HPC facility Iridis4, monitor its status and retrieve the output files upon job completion;

- 3.
parse the output files to extract the values of the drag, lift, pitch moment, and pressure coefficients;

- 4.
reorder the list of the calculated pressure coefficients to match those available from the experimental dataset;

- 5.
calculate the mismatch between simulated and measured pressure coefficients in terms of the SSE metric.

### 3.3 Global sensitivity analysis

The global sensitivity analysis of the SSE between measured and simulated pressure coefficient on the uncertainty of the turbulence closure coefficients was analysed using the surrogate model generated with 1980 sample points. The relative contribution of the input parameters to the total variability of SSE was quantified by relying on the variance-based Sobol indices [19]. This information is a useful measure as it allows identifying the closure coefficients with the largest influence on the output variability. This, in turn, informs the calibration process by retaining only those coefficients with a Sobol index larger than a user predefined threshold.

Table 2 reports the values of the first order Sobol indices estimated via Monte Carlo integration performed with 10,000 random points evaluated on the surrogate model. In agreement with [6], the Von Kármán constant, *k*, is the parameter that has the greatest impact on the variability of the system output. The mismatch between experimental data and numerical simulations is also influenced by the value of \(c_{v1}\) and, to a lesser extent, by \(c_{b1}\) and \(\sigma\). The sensitivities with respect to the remaining five parameters (\(c_{w2}\), \(c_{b2}\), \(c_{t3}\), \(c_{w3}\), \(c_{t4}\)) are virtually null. Being the value of the residual approximately equal to 0.01 (not reported in Table 2), no significant sensitivity can be attributed to the mutual interaction between the different input parameters.

*k*, \(c_{v1}\), \(c_{b1}\), \(\sigma\)) are therefore active.

First order Sobol indices of Spalart–Allmaras closure coefficients with respect to *SSE*

### 3.4 Calibration of turbulence model closure coefficients

A two-steps approach was adopted to calibrate the closure coefficients of Spalart–Allmaras turbulence model.

*k*, \(c_{v1}\), \(\sigma\), \(c_{b1}\)). These findings agree well with the set of optimised closure coefficients obtained through one of the deterministic CFD analyses executed during the DOE plan. Discrepancies on the remaining five parameters (\(c_{w2}\) through \(c_{t4})\) are non-influential, being their Sobol indices virtually null.

Optimal values of closure coefficients and corresponding SSE based on genetic algorithm evaluated on surrogate model and best experiment found during the DOE plan

Parameter | Optimisation 1 | Optimisation 2 | Optimisation 3 | Optimal DOE point |
---|---|---|---|---|

SSE = 0.140 | SSE = 0.140 | SSE = 0.140 | SSE = 0.139 | |

\(k\times 10^{1}\) | 3.600 | 3.600 | 3.600 | 3.610 |

\(c_{v1}\) | 7.500 | 7.500 | 7.500 | 7.431 |

\(\sigma\) | 0.997 | 1.003 | 1.009 | 1.163 |

\(c_{b1}\times 10^{1}\) | 1.400 | 1.400 | 1.400 | 1.380 |

\(c_{w2}\times 10^{1}\) | 3.000 | 3.000 | 3.000 | 3.260 |

\(c_{b2}\times 10^{1}\) | 6.220 | 6.220 | 6.220 | 6.260 |

\(c_{t3}\) | 1.200 | 1.200 | 1.200 | 1.963 |

\(c_{w3}\) | 2.000 | 2.000 | 2.000 | 1.815 |

\(c_{t4}\times 10^{1}\) | 5.000 | 5.000 | 5.000 | 3.040 |

Spalart–Allmaras closure coefficients for the standard and calibrated versions

### 3.5 Improved prediction accuracy

### 3.6 Concluding remarks

A finer grid was generated to investigate the influence of the spatial discretisation on the prediction capability for Case 6. The grid of Fig. 5a, denoted hereafter the medium grid, consists of a single-block with \(865\times 161\) points. The far-field boundary is placed at 50 chords from the aerofoil, and the distance of the first grid points off the aerofoil surface is about \({5\times 10}^{-6}\) chord.

Case 6 of RAE 2822 aerofoil: impact of spatial discretisation on aerodynamic coefficients for standard and calibrated Spalart–Allmaras turbulence models

Exp data | Coarse grid | Medium grid | |||
---|---|---|---|---|---|

Standard | Calibrated | Standard | Calibrated | ||

\(\alpha\) (deg) | 2.29 | 2.51 | 2.37 | 2.39 | 2.27 |

\(C_{\text {D}}\) (counts) | 127 | 150 | 137 | 142 | 124 |

\(C_{\text {m}}\) | \(-\,0.095\) | \(-\,0.091\) | \(-\,0.096\) | \(-\,0.092\) | \(-\,0.096\) |

With the above paragraphs as background, one can identify different scenarios for the applicability of the current work. In a research scenario, the work can be used to determine the robustness of a turbulence model, and whether efforts should be addressed to improve the model in some way. In a design scenario, there is an interest to reduce the uncertainty arising from the turbulence modelling, or else to design a factor of safety around it. In a commercial scenario, the work could be used in a cost analysis, whereby uncertainty in drag coefficient will affect projected fuel costs.

## 4 Additional test cases

The prediction capability of the calibrated turbulence model was assessed on additional test cases which were not used in the calibration process. These four cases include three steady-state problems, and one unsteady problem for forced sinusoidal oscillations in pitch.

### 4.1 RAE 2822 Aerofoil, Case 9

The first additional test case involves Case 9 of the RAE 2822 aerofoil. The experimental measurements suffered from wind tunnel interference effects, which cannot be estimated a posteriori. Hence, the analyses presented herein are performed for a fixed normal force coefficient, as done for Case 6 in Sect. 3. Case 9 is run at \(M\,=\,0.73\), \(Re\,=\,6.5\times 10^6\), and \(C_{\text {N}}\,=\,0.803\), and this results in little separation downstream of the shock position. Both the coarse and medium grids were used with the standard and calibrated Spalart–Allmaras turbulence models.

### 4.2 ONERA M6 wing

The ONERA M6 wing is a swept wing with no twist, built with the symmetric ONERA D aerofoil. The computational grid is available from the validation web site of the NASA CFD code CFL3D.^{4} The grid with \(288\times 64\times 48\) cells consists of one zone wrapped as a C-grid about the wing leading edge. Symmetry boundary condition was used on one side of the domain. The wing span features 256 cells in the chord-wise direction and 48 cells in the span-wise direction. The minimum wall distance of the first grid points off the wing surface is about \(2.5\times {10}^{-6}\) chords at the leading edge and about \(5\times {10}^{-6}\) chords at the trailing edge. The grid is non-dimensionalised by the span, and the mean aerodynamic chord is \(c\,=\,0.54\,b\).

Calculations were performed with Spalart–Allmaras turbulence model. Experimental data are available for \(M\,=\,0.84\) and Reynolds number, based on the mean aerodynamic chord, \(Re\,=\,12.7\times 10^6\) at a prescribed angle of attack \(\alpha \,=\,3.06^{\circ }\) [16].

As for the aerofoil case, an explicit time stepping and the fourth order Runge–Kutta scheme were used. To accelerate the convergence to a steady state, a local time-stepping and implicit residual smoothing were used. The discretisation of the convective and diffusive fluxes of the RANS equations was based on the second order Roe’s flux difference splitting scheme, and the first order accurate scheme was used for Spalart–Allmaras fluxes. Venkatakrishnan’s flux limiter was used. A no-slip boundary condition was set at the wing surface, and far-field boundary conditions were applied to the far-field boundaries. The CFL number was set to 1.2 and simulations were run for 20,000 iterations to compute the steady state solution. With this setup, the overall residuals of Navier-Stokes and Spalart–Allmaras equations decreased by about five orders of magnitude, and all force and moment components converged within 15,000 iterations.

### 4.3 Axisymmetric transonic bump

^{5}The experiment, which used a circular-arc bump, was performed in the NASA-Ames wind tunnel at Mach 0.875 and Reynolds number 2.8 million. The grid with dimensions of \(360\times 160\) cells (available from the NASA website) was used in this study, see Fig. 8.

### 4.4 NACA 0012 aerofoil forced sinusoidal motion

This test case concerns transonic flow predictions for the NACA 0012 aerofoil undergoing a forced sinusoidal motion in pitch around one-quarter of the chord. The flow conditions for AGARD CT5 are Mach number 0.755 and Reynolds number 5.5 million. The motion is characterised by the reduced frequency \(k\,=\,0.0814\), mean angle of attack \(\alpha _0\,=\,0.016^{\circ }\), and amplitude \(\alpha _A\,=\,2.51^{\circ }\).

The unstructured grid, which was also used in a previous work [7], consists of about 15.3 thousand mesh elements, see Fig. 10a. The first grid layer on the wall was placed at \(5\times 10^{-6}\) (for a chord of one) to ensure that \(y^+\) was smaller than 1. An implicit dual-time stepping scheme was used, with a target residual drop of three orders at each physical time step.

## 5 Conclusions

Uncertainty in the closure coefficients of a turbulence model is an important source of error in Reynolds-averaged Navier–Stokes simulations. This requires turbulence modelling uncertainty quantification, sensitivity analysis and parameter calibration. The work detailed in this study addressed these aspects using state-of-the-art computational techniques, including a machine-learning software platform with an adaptive design of experiments algorithm, a modern flow solver, and a high-performance computing facility. The original Spalart–Allmaras turbulence model was analysed. The key elements of this work are: (a) uncertainty quantification and sensitivity analysis required only about 1,000 CPU hours to explore a ten-dimensional design space; (b) only a selected number of closure coefficients have a large impact on the uncertainty of the output quantities of interest; this is not unexpected because Spalart–Allmaras turbulence model has a nested structure, with the outer versions not altering the lower ones; (c) the optimal values of the closure coefficients, which were implemented in a calibrated version of Spalart–Allmaras model, were chosen to minimise the sum of square error of the pressure coefficient between experimental data and numerical results; (d) the closure coefficients with the largest Sobol indices are determined accurately as they have the largest impact on the output quantities of interest. It was found that the calibrated Spalart–Allmaras turbulence model slightly outperforms the standard version for transonic, wall-bounded flows around the RAE 2822 aerofoil. The expected prediction accuracy holds for a variety of other test cases (Case 9 of RAE 2822, ONERA M6, axisymmetric transonic bump, forced sinusoidal motion of NACA 0012 aerofoil) other than that used in the calibration process, as well as across different flow solvers.

Part of the problem with improving the simulation accuracy of the various effects that occur at challenging flow conditions has been the inability to access unbiased, data-rich experimental data-banks. There certainly are alternative computational approaches, e.g. Direct Numerical Simulation, that are being developed and play an important role in turbulence modelling development and validation. However, these approaches are limited in the types of geometries and flow conditions to be simulated. Recently, the European Commission through the Horizon 2020 programme has recognised these challenging issues, and supported the “Holistic Optical Metrology for Aero-elastic Research” (HOMER, project ID: 769237) project. Among the basic science research directions, the project aims at providing space and time-accurate experimental measurements of fluid and structure for a range of aeronautical flows, and at supporting the development and enhancement of numerical methods following a similar approach to that presented herein. Considering a variety of flow conditions and geometries is necessary to extend the approach here discussed to a large subset of the turbulence model closure coefficients.

## Footnotes

- 1.
https://www.sharcnet.ca/Software/Ansys/17.0/en-us/help/flu_th/flu_th_sec_turb_intermittency_over.html (retrieved 2019/03/20 10:20:18).

- 2.
https://www.noesissolutions.com/our-products/optimus (retrieved 2019/03/20 10:20:18).

- 3.
https://www.grc.nasa.gov/WWW/wind/valid/raetaf/raetaf.htm (retrieved 2019/03/20 10:20:18).

- 4.
http://cfl3d.larc.nasa.gov/ (retrieved 2019/03/20 10:20:18).

- 5.
https://turbmodels.larc.nasa.gov/axibump_val.html (retrieved 2019/03/20 10:20:18).

## Notes

### Acknowledgements

The authors acknowledge the use of the IRIDIS High Performance Computing Facility, and associated support services at the University of Southampton, in the completion of this work. Dr Da Ronch acknowledges the financial support from Horizon 2020 of the European Commission under the topic “MG-1.3-2017—Maintaining industrial leadership in aeronautics” (Project ID: 769237). Furthermore, Dr Da Ronch and Dr Drofelnik gratefully acknowledge the financial support from the Engineering and Physical Sciences Research Council (Grant number: EP/P006795/1).

## References

- 1.Bailey, S.C.C., Vallikivi, M., Hultmark, M., Smits, A.J.: Estimating the value of von Karmans constant in turbulent pipe flow. J. Fluid Mech.
**749**, 79–98 (2014). https://doi.org/10.1017/jfm.2014.208 CrossRefGoogle Scholar - 2.Cook, P.H., McDonald, M.A., Firmin, M.C.P.: Aerofoil RAE 2822—pressure distributions, boundary layer and wake measurements. AGARD Report AR 138, Appendix A-2D Configurations (1979)Google Scholar
- 3.Da Ronch, A., Ghoreyshi, M., Badcock, K.J.: On the generation of flight dynamics aerodynamic tables by computational fluid dynamics. Progress Aerosp. Sci.
**47**(8), 597–620 (2011). https://doi.org/10.1016/j.paerosci.2011.09.001 CrossRefGoogle Scholar - 4.Da Ronch, A., Panzeri, M., Abd Bari, M.A., d’Ippolito, R., Franciolini, M.: Adaptive design of experiments for efficient and accurate estimation of aerodynamic loads. Aircr. Eng. Aerosp. Technol.
**89**(4), 558–569 (2017). https://doi.org/10.1108/AEAT-10-2016-0173 CrossRefGoogle Scholar - 5.Economon, T.D., Palacios, F., Copeland, S.R., Lukaczyk, T.W., Alonso, J.J.: SU2: an open-source suite for multiphysics simulation and design. AIAA J.
**54**(3), 828–846 (2015). https://doi.org/10.2514/1.J053813 CrossRefGoogle Scholar - 6.Edeling, W.N., Cinnella, P., Dwight, R.P.: Predictive rans simulations via bayesian model-scenario averaging. J. Comput. Phys.
**275**, 65–91 (2014). https://doi.org/10.1016/j.jcp.2014.06.052 MathSciNetCrossRefzbMATHGoogle Scholar - 7.Franciolini, M., Da Ronch, A., Drofelnik, J., Raveh, D.: Efficient infinite-swept wing solver for steady and unsteady compressible flows. Aerosp. Sci. Technol.
**72**, 217–229 (2017). https://doi.org/10.1016/j.ast.2017.10.034 CrossRefGoogle Scholar - 8.Hosder, S., Walters, R.W., Balch, M.: Point-collocation nonintrusive polynomial chaos method for stochastic computational fluid dynamics. AIAA J.
**48**(12), 2721–2730 (2010). https://doi.org/10.2514/1.39389 CrossRefGoogle Scholar - 9.Jameson, A., Schmidt, W., Turkel, E.: Numerical solutions of the Euler equations by finite volume methods using Runge–Kutta time—stepping schemes. In: 14th Fluid and Plasma Dynamics Conference, Fluid Dynamics and Co-located Conferences, Palo Alto, CA (1981). https://doi.org/10.2514/6.1981-1259
- 10.Li, Z., Hoagg, J.B., Martin, A., Bailey, S.C.C.: Retrospective cost adaptive Reynolds-averaged navier-stokes \(k-\omega\) model for data-driven unsteady turbulent simulations. J. Comput. Phys.
**357**, 353–374 (2018). https://doi.org/10.1016/j.jcp.2017.11.037 MathSciNetCrossRefzbMATHGoogle Scholar - 11.Li, Z., Zhang, H., Bailey, S.C.C., Hoagg, J.B., Martin, A.: A data-driven adaptive reynolds-averaged navier-stokes \(k-\omega\) model for turbulent flow. J. Comput. Phys.
**345**, 111–131 (2017). https://doi.org/10.1016/j.jcp.2017.05.009 MathSciNetCrossRefzbMATHGoogle Scholar - 12.Papadimitriou, D.I., Papadimitriou, C.: Bayesian uncertainty quantification of turbulence models based on high-order adjoint. Comput. Fluids
**120**, 82–97 (2015). https://doi.org/10.1016/j.compfluid.2015.07.019 MathSciNetCrossRefzbMATHGoogle Scholar - 13.Schaefer, J., Hosder, S., West, T., Rumsey, C., Carlson, J.R., Kleb, W.: Uncertainty quantification of turbulence model closure coefficients for transonic wall-bounded flows. AIAA J.
**55**, 195–213 (2017). https://doi.org/10.2514/1.J054902 CrossRefGoogle Scholar - 14.Schaefer, J.A.: Uncertainty quantification of turbulence model closure coefficients for transonic wall—bounded flows. M.Sc. Thesis, Missouri University of Science and Technology (2015)Google Scholar
- 15.Schittkowski, K.: NLPQL: a Fortran subroutine for solving constrained nonlinear programming problems. Ann. Oper. Res.
**5**(2), 485–500 (1986). https://doi.org/10.1007/BF02022087 MathSciNetCrossRefGoogle Scholar - 16.Schmitt, V., Charpin, F.: Pressure distributions on the ONERA-M6–wing at transonic Mach numbers. AGARD Report AR 138, Appendix B—3D Configurations (1979)Google Scholar
- 17.Schwamborn, D., Gerhold, T., Heinrich, R.: The DLR TAU-code: recent applications in research and industry. In: Proceedings of the European Conference on Computational Fluid Dynamics (ECCOMAS) (2006)Google Scholar
- 18.Slotnick, J., Khodadoust, A., Alonso, J., Darmofal, D., Gropp, W., Lurie, E., Mavriplis, D.: CFD vision 2030 study: a path to revolutionary computational aerosciences. NASA/CR-20140218178 (2014)Google Scholar
- 19.Sobol, I.M.: Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Math. Comput. Simul.
**55**(1–3), 271–280 (2001). https://doi.org/10.1016/S0378-4754(00)00270-6 MathSciNetCrossRefzbMATHGoogle Scholar - 20.Sørensen, N.N.: CFD modelling of laminar-turbulent transition for airfoils and rotors using the \(\gamma -\widetilde{{Re}}_{\vartheta }\). Wind Energy
**12**, 715–733 (2009). https://doi.org/10.1002/we.325 CrossRefGoogle Scholar - 21.Spalart, P.R., Allmaras, S.R.: A one-equation turbulence model for aerodynamic flows. Rec. Aérosp.
**275**, 5–21 (1994)Google Scholar - 22.Storn, R., Price, K.: Differential evolution—a simple and efcient heuristic for global optimization over continuous spaces. J. Glob. Optim.
**11**(4), 341–359 (1997). https://doi.org/10.1023/a:1008202821328 CrossRefzbMATHGoogle Scholar - 23.Tijdeman, H., Seebass, R.: Transonic flow past oscillating airfoils. Annu. Rev. Fluid Mech.
**12**, 181–222 (1980)CrossRefzbMATHGoogle Scholar - 24.Yang, G., Da Ronch, A., Drofelnik, J., Xie, Z.T.: Sensitivity assessment of optimal solution in aerodynamic design optimisation using SU2. Aerosp. Sci. Technol.
**81**, 362–374 (2018). https://doi.org/10.1016/j.ast.2018.08.012 CrossRefGoogle Scholar

## Copyright information

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.