Constrained mixed-integer Gaussian mixture Bayesian optimization and its applications in designing fractal and auxetic metamaterials

  • Anh Tran
  • Minh Tran
  • Yan WangEmail author
Research Paper


Bayesian optimization (BO) is a global optimization method that has the potential for design optimization. However, in classical BO algorithm, the variables are considered as continuous. In real-world engineering problems, both continuous and discrete variables are present. In this work, an efficient approach to incorporate discrete variables to BO is proposed. In the proposed constrained mixed-integer BO method, the sample set is decomposed into smaller clusters during sequential sampling, where each cluster corresponds to a unique ordered set of discrete variables, and a Gaussian process regression (GP) metamodel is constructed for each cluster. The model prediction is formed as the Gaussian mixture model, where the weights are computed based on the pair-wise Wasserstein distance between clusters and gradually converge to an independent GP as the optimization process advances. The definition of neighborhood can be flexibly and manually defined to account for independence between clusters, such as in the case of categorical variables. Theoretical results are provided in concert with two numerical and engineering examples, and two examples for metamaterial developments, including one fractal and one auxetic metamaterials, where the effective properties depend on both the geometry and the bulk material properties.


Bayesian optimization Gaussian process Constrained Mixed-integer Metamaterials 



Authors thank Prof. Hongyuan Zha (Georgia Tech) for numerous helpful conversations about Bayesian optimization. The authors are grateful to two anonymous reviewers for their constructive feedback.

Funding information

The research was supported in part by the National Science Foundation under grant number CMMI-1306996. Also, this research was supported in part through research cyberinfrastructure resources and services provided by the Partnership for an Advanced Computing Environment (PACE) at the Georgia Institute of Technology, Atlanta, GA, USA.


  1. Barnsley MF (2014) Fractals everywhere. Academic Press, New YorkzbMATHGoogle Scholar
  2. Bianchi L, Dorigo M, Gambardella LM, Gutjahr WJ (2009) A survey on metaheuristics for stochastic combinatorial optimization. Nat Comput 8(2):239–287MathSciNetCrossRefGoogle Scholar
  3. Bower AF (2011) Applied mechanics of solids. CRC Press, Boca RatonGoogle Scholar
  4. Cagnina LC, Esquivel SC, Coello CAC (2008) Solving engineering optimization problems with the simple constrained particle swarm optimizer. Informatica 32(3):319–326zbMATHGoogle Scholar
  5. Cho Y, Shin JH, Costa A, Kim TA, Kunin V, Li J, Lee S, Yang S, Han HN, Choi IS et al (2014) Engineering the shape and structure of materials by fractal cut. Proc Natl Acad Sci 111(49):17390–17395CrossRefGoogle Scholar
  6. Datta D, Figueira JR (2011) A real-integer-discrete-coded particle swarm optimization for design problems. Appl Soft Comput 11(4):3625–3633CrossRefGoogle Scholar
  7. Davis E, Ierapetritou M (2009) A kriging based method for the solution of mixed-integer nonlinear programs containing black-box functions. J Glob Optim 43(2-3):191–205MathSciNetCrossRefGoogle Scholar
  8. Deb K, Goyal M (1996) A combined genetic adaptive search (geneAS) for engineering design. Comput Sci Inf 26:30–45Google Scholar
  9. Digabel SL, Wild SM (2015) A taxonomy of constraints in simulation-based optimization. arXiv:1505.07881
  10. Gandomi AH, Yang XS (2011) Benchmark problems in structural optimization. In: Computational optimization, methods and algorithms. Springer, pp 259–281Google Scholar
  11. Gardner JR, Kusner MJ, Xu ZE, Weinberger KQ, Cunningham JP (2014) Bayesian optimization with inequality constraints. In: ICML, pp 937–945Google Scholar
  12. Gelbart M, Snoek J, Adams R (2014) Bayesian optimization with unknown constraints. arXiv:1403.5607
  13. Givens CR, Shortt RM et al (1984) A class of Wasserstein metrics for probability distributions. Mich Math J 31(2):231–240MathSciNetCrossRefGoogle Scholar
  14. Gramacy RB, Lee H (2008a) Gaussian processes and limiting linear models. Comput Stat Data Anal 53 (1):123–136MathSciNetCrossRefGoogle Scholar
  15. Gramacy RB, Lee HKH (2008b) Bayesian treed Gaussian process models with an application to computer modeling. J Am Stat Assoc 103(483):1119–1130MathSciNetCrossRefGoogle Scholar
  16. Gramacy RB, Taddy M et al (2010) Categorical inputs, sensitivity analysis, optimization and importance tempering with tgp version 2, an R package for treed Gaussian process models. J Stat Softw 33(6):1–48CrossRefGoogle Scholar
  17. Hansen N, Müller SD, Koumoutsakos P (2003) Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES). Evol Comput 11(1):1–18CrossRefGoogle Scholar
  18. Hemker T, Fowler KR, Farthing MW, von Stryk O (2008) A mixed-integer simulation-based optimization approach with surrogate functions in water resources management. Optim Eng 9(4):341–360MathSciNetCrossRefGoogle Scholar
  19. Hernández-Lobato JM, Gelbart M, Hoffman M, Adams R, Ghahramani Z (2015) Predictive entropy search for Bayesian optimization with unknown constraints. In: International conference on machine learning, pp 1699–1707Google Scholar
  20. Hernández-Lobato JM, Gelbart M, Adams R, Hoffman MW, Ghahramani Z (2016) A general framework for constrained Bayesian optimization using information-based search. Journal of Machine Learning ResearchGoogle Scholar
  21. Huang D, Allen TT, Notz WI, Zeng N (2006) Global optimization of stochastic black-box systems via sequential kriging meta-models. J Glob Optim 34(3):441–466MathSciNetCrossRefGoogle Scholar
  22. Jang HL, Cho H, Choi KK, Cho S (2014) Reliability-based design optimization of fluid–solid interaction problems. Proc Inst Mech Eng Part C: J Mech Eng Sci 228(10):1724–1742CrossRefGoogle Scholar
  23. Kim K, Lee M, Lee S, Jang G (2017a) Optimal design and experimental verification of fluid dynamic bearings with high load capacity applied to an integrated motor propulsor in unmanned underwater vehicles. Tribol Int 114:221–233CrossRefGoogle Scholar
  24. Kim Y, Lee S, Yee K, Rhee DH (2017b) High-to-low initial sample ratio of hierarchical kriging for film hole array optimization. Journal of Propulsion and PowerGoogle Scholar
  25. Li M, Li G, Azarm S (2008) A kriging metamodel assisted multi-objective genetic algorithm for design optimization. J Mech Des 130(3):031401CrossRefGoogle Scholar
  26. Li X, Gong C, Gu L, Jing Z, Fang H, Gao R (2018) A reliability-based optimization method using sequential surrogate model and Monte Carlo simulation. Struct Multidiscip Optim:1–22Google Scholar
  27. Lin Y, Zhang HH (2006a) Component selection and smoothing in multivariate nonparametric regression. Ann Stat 34(5):2272–2297MathSciNetCrossRefGoogle Scholar
  28. Lin Y, Zhang HH (2006b) Component selection and smoothing in smoothing spline analysis of variance models. Ann Stat 34(5):2272–2297CrossRefGoogle Scholar
  29. Liu J, Song WP, Han ZH, Zhang Y (2017) Efficient aerodynamic shape optimization of transonic wings using a parallel infilling strategy and surrogate models. Struct Multidiscip Optim 55(3):925– 943CrossRefGoogle Scholar
  30. Martins JR, Lambe AB (2013) Multidisciplinary design optimization: a survey of architectures. AIAA J 51(9):2049–2075CrossRefGoogle Scholar
  31. Meza LR, Das S, Greer JR (2014) Strong, lightweight, and recoverable three-dimensional ceramic nanolattices. Science 345(6202):1322–1326CrossRefGoogle Scholar
  32. Mockus J (1975) On Bayesian methods for seeking the extremum. In: Optimization techniques IFIP technical conference. Springer, pp 400–404Google Scholar
  33. Mockus JB, Mockus LJ (1991) Bayesian approach to global optimization and application to multiobjective and constrained problems. J Optim Theory Appl 70(1):157–172MathSciNetCrossRefGoogle Scholar
  34. Müller J, Shoemaker CA, Piché R (2013) SO-MI: a surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems. Comput Oper Res 40(5):1383–1400MathSciNetCrossRefGoogle Scholar
  35. Müller J (2016) MISO: mixed-integer surrogate optimization framework. Optim Eng 17(1):177–203MathSciNetCrossRefGoogle Scholar
  36. Müller J, Shoemaker CA, Piché R (2014) SO-I: a surrogate model algorithm for expensive nonlinear integer programming problems including global optimization applications. J Glob Optim 59(4):865–889MathSciNetCrossRefGoogle Scholar
  37. Nguyen-Tuong D, Peters J (2008) Local Gaussian process regression for real-time model-based robot control. In: 2008. IROS 2008. IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp 380–385Google Scholar
  38. Nguyen-Tuong D, Peters J, Seeger M (2009a) Local Gaussian process regression for real time online model learning. In: Advances in neural information processing systems, pp. 1193–1200Google Scholar
  39. Nguyen-Tuong D, Seeger M, Peters J (2009b) Model learning with local Gaussian process regression. Adv Robot 23(15):2015–2034CrossRefGoogle Scholar
  40. Nguyen-Tuong D, Seeger M, Peters J (2010) Real-time local Gaussian process model learning. In: From motor learning to interaction learning in robots. Springer, Berlin, pp 193–207Google Scholar
  41. Nielsen HB, Lophaven SN, Søndergaard J (2002) DACE, a MATLAB kriging toolbox, vol 2. Citeseer, PrincetonGoogle Scholar
  42. Qian PZG, Wu H, Wu CJ (2008) Gaussian process models for computer experiments with qualitative and quantitative factors. Technometrics 50(3):383–396MathSciNetCrossRefGoogle Scholar
  43. Queipo NV, Haftka RT, Shyy W, Goel T, Vaidyanathan R, Tucker PK (2005) Surrogate-based analysis and optimization. Prog Aerosp Sci 41(1):1–28CrossRefGoogle Scholar
  44. Rao SS (2009) Engineering optimization: theory and practice. Wiley, New YorkCrossRefGoogle Scholar
  45. Ravindran A, Reklaitis GV, Ragsdell KM (2006) Engineering optimization: methods and applications. Wiley, New YorkCrossRefGoogle Scholar
  46. Rehman SU, Langelaar M (2017) Expected improvement based infill sampling for global robust optimization of constrained problems. Optim Eng 18(3):723–753MathSciNetCrossRefGoogle Scholar
  47. Shahriari B, Swersky K, Wang Z, Adams R, de Freitas N (2016) Taking the human out of the loop: a review of Bayesian optimization. Proc IEEE 104(1):148–175CrossRefGoogle Scholar
  48. Shahzad M, Kamran A, Siddiqui MZ, Farhan M (2015) Mechanical characterization and FE modelling of a hyperelastic material. Mater Res 18(5):918–924CrossRefGoogle Scholar
  49. Simpson TW, Mauery TM, Korte JJ, Mistree F (2001) Kriging models for global approximation in simulation-based multidisciplinary design optimization. AIAA J 39(12):2233–2241CrossRefGoogle Scholar
  50. Sóbester A., Forrester AI, Toal DJ, Tresidder E, Tucker S (2014) Engineering design applications of surrogate-assisted optimization techniques. Optim Eng 15(1):243–265CrossRefGoogle Scholar
  51. Song C, Song W, Yang X (2017) Gradient-enhanced hierarchical kriging model for aerodynamic design optimization. J Aerosp Eng 30(6):04017072CrossRefGoogle Scholar
  52. Srinivas N, Krause A, Kakade SM, Seeger M (2009) Gaussian process optimization in the bandit setting: no regret and experimental design. arXiv:0912.3995
  53. Srinivas N, Krause A, Kakade SM, Seeger MW (2012) Information-theoretic regret bounds for Gaussian process optimization in the bandit setting. IEEE Trans Inf Theory 58(5):3250–3265MathSciNetCrossRefGoogle Scholar
  54. van Stein B, Wang H, Kowalczyk W, Bäck T, Emmerich M (2015) Optimally weighted cluster kriging for big data regression. In: International symposium on intelligent data analysis. Springer, pp 310–321Google Scholar
  55. Storlie C, Bondell HD, Reich BJ, Zhang HH (2011) Surface estimation, variable selection, and the nonparametric oracle property. Stat Sin 21(2):679MathSciNetCrossRefGoogle Scholar
  56. Swiler LP, Hough PD, Qian P, Xu X, Storlie C, Lee H (2014) Surrogate models for mixed discrete-continuous variables. In: Constraint programming and decision making. Springer, pp 181–202Google Scholar
  57. Tang Y, Yin J (2017) Design of cut unit geometry in hierarchical kirigami-based auxetic metamaterials for high stretchability and compressibility. Extreme Mech Lett 12:77–85CrossRefGoogle Scholar
  58. Tran A, He L, Wang Y (2018) An efficient first-principles saddle point searching method based on distributed kriging metamodels. ASCE-ASME J Risk Uncertain Eng Syst Part B: Mech Eng 4(1):011006CrossRefGoogle Scholar
  59. Viana FA, Simpson TW, Balabanov V, Toropov V (2014) Special section on multidisciplinary design optimization: metamodeling in multidisciplinary design optimization: How far have we really come? AIAA J 52(4):670–690CrossRefGoogle Scholar
  60. Zhang Y, Hu S, Wu J, Zhang Y, Chen L (2014) Multi-objective optimization of double suction centrifugal pump using kriging metamodels. Adv Eng Softw 74:16–26CrossRefGoogle Scholar
  61. Zhou Q, Qian PZ, Zhou S (2011) A simple approach to emulation for computer models with qualitative and quantitative factors. Technometrics 53(3):266–273MathSciNetCrossRefGoogle Scholar
  62. Zhou Q, Wang Y, Choi SK, Jiang P, Shao X, Hu J (2017) A sequential multi-fidelity metamodeling approach for data regression. Knowledge-Based SystemsGoogle Scholar
  63. Zhou Q, Wang Y, Choi S-K, Jiang P, Shao X, Hu J, Shu L (2018) A robust optimization approach based on multi-fidelity metamodel. Struct Multidiscip Optim 57(2):1–23MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.George W. Woodruff School of Mechanical EngineeringGeorgia Institute of TechnologyAtlantaUSA

Personalised recommendations