Journal of Global Optimization

, Volume 71, Issue 4, pp 957–985 | Cite as

Surrogate-based feasibility analysis for black-box stochastic simulations with heteroscedastic noise

  • Zilong Wang
  • Marianthi IerapetritouEmail author


Feasibility analysis has been developed to evaluate and quantify the capability that a process can remain feasible under uncertainty of model inputs and parameters. It can be conducted during the design stage when the objective is to get a robust design which can tolerate a certain amount of variations in the process conditions. Also, it can be used after a design is fixed when the objective is to characterize its feasible region. In this work, we have extended the usage of feasibility analysis to the cases in which inherent stochasticity is existent in the model outputs. With a surrogate-based adaptive sampling framework, we have developed and compared three algorithms that are promising to make accurate predictions on the feasible regions with a limited sampling budget. Both the advantages and limitations are discussed based on the results from five benchmark problems. Finally, we apply such methods to a pharmaceutical manufacturing process and demonstrate its potential application in characterizing the design space of the process.


Feasibility analysis Surrogate modeling Stochastic Kriging Adaptive sampling Stochastic simulation 



The authors would like to acknowledge financial support from FDA (DHHS - FDA - 1 U01 FD005295-01) as well as National Science Foundation Engineering Research Center on Structured Organic Particulate Systems (NSF-ECC 0540855).


  1. 1.
    Halemane, K.P., Grossmann, I.E.: Optimal process design under uncertainty. AIChE J. 29(3), 425–433 (1983)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Swaney, R.E., Grossmann, I.E.: An index for operational flexibility in chemical process design. Part I: formulation and theory. AIChE J. 31(4), 621–630 (1985)CrossRefGoogle Scholar
  3. 3.
    Swaney, R.E., Grossmann, I.E.: An index for operational flexibility in chemical process design. Part II: computational algorithms. AIChE J. 31(4), 631–641 (1985)CrossRefGoogle Scholar
  4. 4.
    Grossmann, I.E., Floudas, C.A.: Active constraint strategy for flexibility analysis in chemical processes. Comput. Chem. Eng. 11(6), 675–693 (1987)CrossRefGoogle Scholar
  5. 5.
    Floudas, C.A., Gümüş, Z.H., Ierapetritou, M.G.: Global optimization in design under uncertainty: feasibility test and flexibility index problems. Ind. Eng. Chem. Res. 40(20), 4267–4282 (2001)CrossRefGoogle Scholar
  6. 6.
    Pistikopoulos, E., Mazzuchi, T.: A novel flexibility analysis approach for processes with stochastic parameters. Comput. Chem. Eng. 14(9), 991–1000 (1990)CrossRefGoogle Scholar
  7. 7.
    Dimitriadis, V.D., Pistikopoulos, E.N.: Flexibility analysis of dynamic systems. Ind. Eng. Chem. Res. 34(12), 4451–4462 (1995)CrossRefGoogle Scholar
  8. 8.
    Adi, V.S.K., Chang, C.-T.: A mathematical programming formulation for temporal flexibility analysis. Comput. Chem. Eng. 57, 151–158 (2013)CrossRefGoogle Scholar
  9. 9.
    Ierapetritou, M.G.: New approach for quantifying process feasibility: convex and 1D quasiconvex regions. AIChE J. 47(6), 1407–1417 (2001)CrossRefGoogle Scholar
  10. 10.
    Goyal, V., Ierapetritou, M.G.: Determination of operability limits using simplicial approximation. AIChE J. 48(12), 2902–2909 (2002)CrossRefGoogle Scholar
  11. 11.
    Banerjee, I., Ierapetritou, M.G.: Feasibility evaluation of nonconvex systems using shape reconstruction techniques. Ind. Eng. Chem. Res. 44(10), 3638–3647 (2005)CrossRefGoogle Scholar
  12. 12.
    Adi, V.S.K., Laxmidewi, R., Chang, C.-T.: An effective computation strategy for assessing operational flexibility of high-dimensional systems with complicated feasible regions. Chem. Eng. Sci. 147, 137–149 (2016)CrossRefGoogle Scholar
  13. 13.
    Michalewicz, Z., Schoenauer, M.: Evolutionary algorithms for constrained parameter optimization problems. Evol. Comput. 4(1), 1–32 (1996)CrossRefGoogle Scholar
  14. 14.
    Banerjee, I., Pal, S., Maiti, S.: Computationally efficient black-box modeling for feasibility analysis. Comput. Chem. Eng. 34(9), 1515–1521 (2010)CrossRefGoogle Scholar
  15. 15.
    Boukouvala, F., Muzzio, F.J., Ierapetritou, M.G.: Design space of pharmaceutical processes using data-driven-based methods. J. Pharm. Innov. 5(3), 119–137 (2010)CrossRefGoogle Scholar
  16. 16.
    Rogers, A., Ierapetritou, M.: Feasibility and flexibility analysis of black-box processes. Part 1: surrogate-based feasibility analysis. Chem. Eng. Sci. 137, 986–1004 (2015)CrossRefGoogle Scholar
  17. 17.
    Rogers, A., Ierapetritou, M.: Feasibility and flexibility analysis of black-box processes. Part 2: surrogate-based flexibility analysis. Chem. Eng. Sci. 137, 1005–1013 (2015)CrossRefGoogle Scholar
  18. 18.
    Wang, Z., Ierapetritou, M.: A novel feasibility analysis method for blackbox processes using a radial basis function adaptive sampling approach. AIChE J. 63(2), 532–550 (2017)CrossRefGoogle Scholar
  19. 19.
    Chen, X., Zhou, Q.: Sequential design strategies for mean response surface metamodeling via stochastic Kriging with adaptive exploration and exploitation. Eur. J. Oper. Res. 262(2), 575–585 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13(4), 455–492 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Shao, X., Shi, L.: Enhancing feasibility determination with Kriging metamodels. In: 2016 IEEE International Conference on Automation Science and Engineering (CASE), pp. 476–481. IEEE (2016)Google Scholar
  22. 22.
    Wang, Z., Wang, P.: An integrated performance measure approach for system reliability analysis. J. Mech. Des. 137(2), 021406 (2015)CrossRefGoogle Scholar
  23. 23.
    Sadoughi, M.K., Hu, C., MacKenzie, C.A., Eshghi, A.T., Lee, S.: Sequential exploration-exploitation with dynamic trade-off for efficient reliability analysis of complex engineered systems. Struct. Multidiscipl. Optim. 57(1), 235–250 (2018)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Grossmann, I.E., Calfa, B.A., Garcia-Herreros, P.: Evolution of concepts and models for quantifying resiliency and flexibility of chemical processes. Comput. Chem. Eng. 70, 22–34 (2014)CrossRefGoogle Scholar
  25. 25.
    Birjandi, M.R.S., Shahraki, F., Razzaghi, K.: Hydrogen network retrofit via flexibility analysis: the steady-state flexibility index. Chem. Eng. Res. Des. 117, 83–94 (2017)CrossRefGoogle Scholar
  26. 26.
    Garcia-Munoz, S., Luciani, C.V., Vaidyaraman, S., Seibert, K.D.: Definition of design spaces using mechanistic models and geometric projections of probability maps. Org. Process Res. Dev. 19(8), 1012–1023 (2015)CrossRefGoogle Scholar
  27. 27.
    Wang, Z., Escotet-Espinoza, M.S., Ierapetritou, M.: Process analysis and optimization of continuous pharmaceutical manufacturing using flowsheet models. Comput. Chem. Eng. 107, 77–91 (2017)CrossRefGoogle Scholar
  28. 28.
    Sharifzadeh, M., Meghdari, M., Rashtchian, D.: Multi-objective design and operation of solid oxide fuel cell (SOFC) triple combined-cycle power generation systems: integrating energy efficiency and operational safety. Appl. Energy 185, 345–361 (2017)CrossRefGoogle Scholar
  29. 29.
    Sahay, N., Ierapetritou, M.: Flexibility assessment and risk management in supply chains. AIChE J. 61(12), 4166–4178 (2015)CrossRefGoogle Scholar
  30. 30.
    Wang, H., Mastragostino, R., Swartz, C.L.: Flexibility analysis of process supply chain networks. Comput. Chem. Eng. 84, 409–421 (2016)CrossRefGoogle Scholar
  31. 31.
    Zhang, Q., Grossmann, I.E., Lima, R.M.: On the relation between flexibility analysis and robust optimization for linear systems. AIChE J. 62(9), 3109–3123 (2016)CrossRefGoogle Scholar
  32. 32.
    Boukouvala, F., Ierapetritou, M.G.: Derivativefree optimization for expensive constrained problems using a novel expected improvement objective function. AIChE J. 60(7), 2462–2474 (2014)CrossRefGoogle Scholar
  33. 33.
    Amaran, S., Sahinidis, N.V., Sharda, B., Bury, S.J.: Simulation optimization: a review of algorithms and applications. 4OR 12(4), 301–333 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Picheny, V., Ginsbourger, D., Richet, Y., Caplin, G.: Quantile-based optimization of noisy computer experiments with tunable precision. Technometrics 55(1), 2–13 (2013)MathSciNetCrossRefGoogle Scholar
  35. 35.
    Ankenman, B., Nelson, B.L., Staum, J.: Stochastic Kriging for simulation metamodeling. Oper. Res. 58(2), 371–382 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Cressie, N.: Statistics for Spatial Data. Wiley, Hoboken (2015)zbMATHGoogle Scholar
  37. 37.
    Kleijnen, J.P.: Kriging metamodeling in simulation: a review. Eur. J. Oper. Res. 192(3), 707–716 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Martin, J.D., Simpson, T.W.: Use of Kriging models to approximate deterministic computer models. AIAA J. 43(4), 853–863 (2005)CrossRefGoogle Scholar
  39. 39.
    Jalali, H., Van Nieuwenhuyse, I., Picheny, V.: Comparison of Kriging-based algorithms for simulation optimization with heterogeneous noise. Eur. J. Oper. Res. 261(1), 279–301 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Žilinskas, A., Zhigljavsky, A.: Stochastic global optimization: a review on the occasion of 25 years of Informatica. Informatica 27(2), 229–256 (2016)CrossRefzbMATHGoogle Scholar
  41. 41.
    Močkus, J.: On Bayesian methods for seeking the extremum. In: Optimization Techniques IFIP Technical Conference, pp. 400–404. Springer (1975)Google Scholar
  42. 42.
    Zhilinskas, A.: Single-step Bayesian search method for an extremum of functions of a single variable. Cybern. Syst. Anal. 11(1), 160–166 (1975)CrossRefGoogle Scholar
  43. 43.
    Schonlau, M., Welch, W.J., Jones, D.: Global optimization with nonparametric function fitting. In: Proceedings of the ASA, Section on Physical and Engineering Sciences, pp. 183–186 (1996)Google Scholar
  44. 44.
    Picheny, V., Wagner, T., Ginsbourger, D.: A benchmark of Kriging-based infill criteria for noisy optimization. Struct. Multidiscipl. Optim. 48(3), 607–626 (2013)CrossRefGoogle Scholar
  45. 45.
    Huang, D., Allen, T.T., Notz, W.I., Zeng, N.: Global optimization of stochastic black-box systems via sequential Kriging meta-models. J. Global Optim. 34(3), 441–466 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  46. 46.
    Ranjan, P., Bingham, D., Michailidis, G.: Sequential experiment design for contour estimation from complex computer codes. Technometrics 50(4), 527–541 (2008)MathSciNetCrossRefGoogle Scholar
  47. 47.
    Lophaven, S., Nielsen, H., Sondergaard, J., Toolbox, D.-A.M.K.: Technical University of Denmark. Technical Report IMM-TR2002-12 (2002)Google Scholar
  48. 48.
    Al-Asady, R.B., Dhenge, R.M., Hounslow, M.J., Salman, A.D.: Roller compactor: determining the nip angle and powder compaction progress by indentation of the pre-compacted body. Powder Technol. 300, 107–119 (2016)CrossRefGoogle Scholar
  49. 49.
    Hsu, S.-H., Reklaitis, G.V., Venkatasubramanian, V.: Modeling and control of roller compaction for pharmaceutical manufacturing. Part I: process dynamics and control framework. J. Pharm. Innov. 5(1–2), 14–23 (2010)CrossRefGoogle Scholar
  50. 50.
    Singh, R., Ierapetritou, M., Ramachandran, R.: An engineering study on the enhanced control and operation of continuous manufacturing of pharmaceutical tablets via roller compaction. Int. J. Pharm. 438(1), 307–326 (2012)CrossRefGoogle Scholar
  51. 51.
    Fu, M.C., Glover, F.W., April, J.: Simulation optimization: a review, new developments, and applications. In: Proceedings of the 2005 Winter Simulation Conference, p. 13. IEEE (2005)Google Scholar
  52. 52.
    Rulliere, D., Faleh, A., Planchet, F., Youssef, W.: Exploring or reducing noise? Struct. Multidiscipl. Optim. 47(6), 921–936 (2013)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Chemical and Biochemical EngineeringRutgers – The State University of New JerseyPiscatawayUSA

Personalised recommendations