# Comparative study of HDMRs and other popular metamodeling techniques for high dimensional problems

- 294 Downloads

## Abstract

The efficiency of optimization for the high dimensional problem has been improved by the metamodeling techniques in multidisciplinary in the past decades. In this study, comparative studies are implemented for high dimensional problems on the accuracy of four popular metamodeling methods, Kriging (KRG), radial basis function (RBF), least square support vector regression (LSSVR) and cut-high dimensional model representation (cut-HDMR) methods. Besides, HDMR methods with different basis functions are considered, including KRG-HDMR, RBF-HDMR and SVR-HDMR. Four factors that might influence the quality of metamodeling methods involving parameter interaction of problems, sample sizes, noise level and sampling strategies are considered. The results show that the LSSVR with Gaussian kernel, using Latin hypercube sampling (LHS) strategy, constructs more accurate metamodels than the KRG. The RBF with Gaussian basis function performs poor in the group. Generally, cut-HDMR methods perform much better than the other metamodeling methods when handling the function with weak parameter interaction, but not better when handling the function with strong parameter interaction.

## Keywords

Metamodel High dimensional problem HDMR Parameter interaction## Nomenclature

*R*Correlation function

**x**Sample point

*θ*Correlation coefficient

*x*Vector of sample point

- \( \widehat{y} \)
Prediction of sample point

*β*Weigh coefficient

*ϕ*Predefined polynomial basis function

*z*Realization of stochastic process

*Var*Variance

*ψ*Basis functions

*J*Cost function

**β**A vector of weights

*γ*Penalty parameter

*e*Error variable

*y**E*valuation of sample point*D*A set of sample point

*φ*Nonlinear mapping

*b*Model offset

*α*Lagrange multiplier

*K*Kernel function

*σ*Tuning parameter

*R*Design domain

**p**Vector consisting linear variable terms

*E, F*Coefficients

*N*Standard Gaussian distribution

*η*Random number sampled

**c**Center points

*δ*Side length

**g**Unit vector

- w
Best function values

## Subscripts

*j,k*Point index

*n*Dimension

*t*Point index

*m*Number of sample point

## Notes

### Acknowledgments

This work has been supported by National Key Research and Development Program of China 2017YFB0203701 and Project of the Program of National Natural Science Foundation of China under the Grant Numbers 11572120.

## References

- Backlund PB, Shahan DW, Seepersad CC (2012) A comparative study of the scalability of alternative metamodelling techniques. Eng Optim 7(44):767–786CrossRefGoogle Scholar
- Bai YC, Han X, Jiang C et al (2012) Comparative study of metamodeling techniques for reliability analysis using evidence theory. Adv Eng Softw 53:61–71CrossRefGoogle Scholar
- Bratley P, Fox BL (1988) Algorithm 659: Implementing Sobol's quasirandom sequence generator. ACM Transactions on Mathematical Software (TOMS) 14(1):88–100zbMATHCrossRefGoogle Scholar
- Chen L, Li E, Wang H et al (2016) Time-based reflow soldering optimization by using adaptive Kriging-HDMR method. Soldering & Surface Mount Technology 28(2):101–113CrossRefGoogle Scholar
- Clarke SM, Griebsch JH, Simpson TW (2005) Analysis of support vector regression for approximation of complex engineering analyses. J Mech Des 127(6):1077–1087CrossRefGoogle Scholar
- Díaz-Manríquez A, Toscano G, Coello CAC (2017) Comparison of metamodeling techniques in evolutionary algorithms. Soft Comput 21(19):5647–5663Google Scholar
- Faure H (1992) Good permutations for extreme discrepancy. Journal of Number Theory 42(1):47–56MathSciNetzbMATHCrossRefGoogle Scholar
- Galanti S, Jung A (1997) Low-discrepancy sequences: Monte Carlo simulation of option prices. The Journal of Derivatives 5(5):63–83CrossRefGoogle Scholar
- Halton JH (1960) On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals. Numer Math 2(1):84–90MathSciNetzbMATHCrossRefGoogle Scholar
- Hardy RL (1971) Multiquadric equations of topography and other irregular surfaces. J Geophys Res 76(8):1905–1915CrossRefGoogle Scholar
- Jin R, Chen W, Simpson TW (2001) Comparative studies of metamodelling techniques under multiple modelling criteria. Struct Multidiscip Optim 23(1):1–13CrossRefGoogle Scholar
- Jones DR (2001) Direct global optimization algorithm direct global optimization algorithm. Encyclopedia of optimization. Springer, Boston, MA, p 431–440Google Scholar
- Kalagnanam JR, Diwekar UM (1997) An efficient sampling technique for off-line quality control. Technometrics 39(3):308–319zbMATHCrossRefGoogle Scholar
- Kim BS, Lee YB, Choi DH (2009) Comparison study on the accuracy of metamodeling technique for non-convex functions. J Mech Sci Technol 23(4):1175–1181Google Scholar
- Koivunen AC, Kostinski AB (1998) The Feasibility of data whitening to improve performance of weather radar. J Appl Meteorol 38(6):741–749CrossRefGoogle Scholar
- Kostinski AB, Koivunen AC (2000) On the condition number of Gaussian sample-covariance matrices. IEEE Transactions on Geoscience & Remote Sensing 38(1):329–332CrossRefGoogle Scholar
- Lee Y, Oh S, Choi DH (2008) Design optimization using support vector regression. J Mech Sci Technol 22(2):213CrossRefGoogle Scholar
- Li E, Wang H, Li G (2012) High dimensional model representation (HDMR) coupled intelligent sampling strategy for nonlinear problems. Comput Phys Commun 183(9):1947–1955MathSciNetCrossRefGoogle Scholar
- Liu H, Xu S, Wang X (2016) Sampling strategies and metamodeling techniques for engineering design: comparison and application”//ASME Turbo Expo 2016: Turbomachinery Technical Conference and Exposition. American Society of Mechanical Engineers V02CT45A019-V02CT45A019Google Scholar
- McKay MD, Beckman RJ, Conover WJ (1979) Comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 12(2):239–245MathSciNetzbMATHGoogle Scholar
- Paiva RM, Carvalho ARD, Crawford C et al (2010) Comparison of surrogate models in a multidisciplinary optimization framework for wing design. AIAA J 48(5):995–1006CrossRefGoogle Scholar
- Rabitz H, Aliş ÖF (1999) General foundations of high-dimensional model representations. J Math Chem 25(2-3):197–233MathSciNetzbMATHCrossRefGoogle Scholar
- Sacks J, Welch WJ, Mitchell TJ et al (1989) Design and analysis of computer experiments. Stat Sci 4:409–423MathSciNetzbMATHCrossRefGoogle Scholar
- Shan S, Wang GG (2010a) Metamodeling for high dimensional simulation-based design problems. J Mech Des 132(5):051009CrossRefGoogle Scholar
- Shan S, Wang GG (2010b) Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions. Struct Multidiscip Optim 41(2):219–241MathSciNetzbMATHCrossRefGoogle Scholar
- Sobol IM (1976) Uniformly distributed sequences with an additional uniform property. USSR Comput Math Math Phys 16(5):236–242zbMATHCrossRefGoogle Scholar
- Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300CrossRefGoogle Scholar
- Toshimitsu H, Andrea S (1996) Importance measure in global sensitivity analysis of nonlinear models. Reliab Eng Syst Saf 52(1):1–17CrossRefGoogle Scholar
- Viana FAC et al (2004) Special section on multidisciplinary design optimization: Metamodeling in multidisciplinary design optimization: How far have we really come? AIAA J 52(4):670–690CrossRefGoogle Scholar
- Wang H, Tang L, Li GY (2011) Adaptive MLS-HDMR metamodeling techniques for high dimensional problems. Expert Syst Appl 38(11):14117–14126Google Scholar
- Zhou XJ, Jiang T (2016) Metamodel selection based on stepwise regression. Struct Multidiscip Optim 54(3):641–657MathSciNetCrossRefGoogle Scholar
- Zimmermann R (2015) On the condition number anomaly of Gaussian correlation matrices. Linear Algebra Appl 466:512–526MathSciNetzbMATHCrossRefGoogle Scholar