Trade Space Exploration: Assessing the Benefits of Putting Designers “Back-in-the-Loop” during Engineering Optimization

  • Timothy W. Simpson
  • Dan Carlsen
  • Matthew Malone
  • Joshua Kollat


Trade space exploration is a promising decision-making paradigm that provides a visual and more intuitive means for formulating, adjusting, and ultimately solving engineering design optimization problems. This is achieved by combining multi-dimensional data visualization techniques with visual steering commands to allow designers to “steer” the optimization process while searching for the best, or Pareto optimal, designs. After introducing the trade space exploration paradigm and visual steering capabilities that we developed, we compare the performance of different combinations of visual steering commands implemented by two users to a multi-objective genetic algorithm executed “blindly” on the same problem with no human intervention. The results indicate that the visual steering commands—regardless of the order and combination in which they are invoked—provide a 4–7× increase in the number of Pareto solutions obtained for a given number of function evaluations when the human is “in-the-loop” during the optimization process. As such, this study provides empirical evidence of the benefits of interactive visualization-based strategies to support engineering design optimization and decision-making. Future work is also discussed.


Function Evaluation Pareto Front Differential Evolution Algorithm Pareto Solution User Trial 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



We thank Dr. Scott Ferguson for his assistance with the vehicle model and the results from the exhaustive MOGA. This work has been supported by the National Science Foundation under Grant No. CMMI-0620948. Any opinions, findings, and conclusions or recommendations presented in this chapter are those of the authors and do not necessarily reflect the views of the National Science Foundation.


  1. Balling R (1999) Design by shopping: a new paradigm? In: Proceedings of the third world congress of structural and multidisciplinary optimization (WCSMO-3), University at Buffalo, Buffalo, New York, pp 295–297Google Scholar
  2. Becker RA, Cleveland WS (1987) Brushing scatterplots. Technometrics 29(1):127–142MathSciNetCrossRefGoogle Scholar
  3. Buja A, McDonald JA, Michalak J, Stuetzle W (1991) Interactive data visualization using focusing and linking. In: Proceedings of the IEEE Conference on Visualization '91, IEEE Computer Society Press, San Diego, CA, pp 156–163Google Scholar
  4. Donndelinger J, Ferguson S, Lewis K (2006) Exploring mass trade-offs in preliminary vehicle design using pareto sets. 11th AIAA/ISSMO symposium on multidisciplinary analysis and optimization, Portsmouth, VA, AIAA-2006-7056Google Scholar
  5. Dym CL, Wood WH, Scott MJ (2006) On the legitimacy of pairwise comparisons. Decision Making in Engineering Design, ASME, 135–143Google Scholar
  6. Ferguson S, Gurnani A, Donndelinger J, Lewis, K (2005a) A study of convergence and mapping in multiobjective optimization problems. ASME design engineering technical conferences & Computers and information in engineering conference, Long Beach, CA, ASME, Paper No. DETC2005/CIE-84852Google Scholar
  7. Ferguson S, Gurnani A, Donndelinger J, Lewis K (2005b) An approach to feasibility assessment in preliminary design. ASME design engineering technical conferences—design automation conference, Long Beach, CA, ASME, Paper No. DETC2005/CIE-84853Google Scholar
  8. Kesavadas T, Sudhir A (2000) Computational steering in simulation of manufacturing systems. In: Proceedings of the 2000 IEEE International Conference on Robotics and Automation. IEEE, San Francisco, CA, pp 2654–2658Google Scholar
  9. Kollat JB, Reed PM (2005a) Comparing state-of-the-art evolutionary multi-objective algorithms for long-term groundwater monitoring design. Adv Water Resour 29(6):792–807CrossRefGoogle Scholar
  10. Kollat JB, Reed PM (2005b) The value of online adaptive search: aperformance comparison of NSGAII, e-NSGAII, and eMOEA. Lecture Notes in Computer Science, Springer, Berlin, p 3410Google Scholar
  11. Madar J, Abonyi J, Szeifert F (2005) Interactive particle swarm optimization. International Conference on Intelligent Systems Design and Applications, IEEE, Wroclaw, Poland, pp 314–319Google Scholar
  12. Messac A, Chen X (2000) Visualizing the optimization process in real-time using physical programming. Eng Optim 32(6):721–747CrossRefGoogle Scholar
  13. Michalek J, Papalambros P (2002) Interactive design optimization of architectural layouts. Eng Optim 34(5):485–501CrossRefGoogle Scholar
  14. Miettinen K, Makela MM (2006) Synchronous approach in interactive multiobjective optimization. Eur J Oper Res 170:909–922MATHCrossRefGoogle Scholar
  15. North C (2006) Toward measuring visualization insight. IEEE Comp Graph Appl 26(3):6–9CrossRefGoogle Scholar
  16. Okabe T, Jin Y, Sendhoff B (2003) A critical survey of performance indices for multi-objective optimisation. Evol Comput 2:878–885Google Scholar
  17. Price K, Storn R, Lampinen J (2005) Differential evolution: a practical approach to global optimization. Springer, BerlinMATHGoogle Scholar
  18. Robic T, Filipic B (2005) DEMO: Differential evolution for multiobjective optimization. Third international conference on evolutionary multi-criterion optimization, Guanajuato, Mexico, Springer, pp 520–533Google Scholar
  19. Scott SD, Lesh N, Klau GW (2002) Investigating Human--Computer optimization. CHI’02, Minneapolis MN 4(1):155–162Google Scholar
  20. Shanteau J (1992) Competence in experts: the role of task characteristics. Organ Behavior and Human Decis 53(2):252–266CrossRefGoogle Scholar
  21. Simpson TW, Donndelinger JA, Yukish M, Stump G (2007) Visual steering commands for trade space exploration: user-guided sampling with example. In: Proceedings of the ASME 2007 international design engineering technical conferences & computers and information in engineering conference, Las Vegas, NV, DETC2007/DAC-34684Google Scholar
  22. Stump G, Yukish M, Simpson TW (2004a) The advanced trade space visualizer: an engineering decision-making tool. 10th AIAA/ISSMO multidisciplinary analysis and optimization conference, Albany, NY, AIAA-2004-4568Google Scholar
  23. Stump G, Yukish M, Simpson TW, Harris EN, O’Hara JJ (2004b) Trade space exploration of satellite datasets using a design by shopping paradigm. IEEE Aerospace Conference, IEEE, Big Sky, MTGoogle Scholar
  24. Tang Y, Reed PM, Wagener T (2005) How effective and efficient are mulitobjective evolutionary algorithms at hydrologic model calibration. Hydrol Earth Syst Sci Discuss 2:2465–2520CrossRefGoogle Scholar
  25. Wang GG, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. ASME J Mech Des 129(4):370–380MathSciNetCrossRefGoogle Scholar
  26. Wilson TD, Schooler JW (1991) Thinking too much: introspection can reduce the quality of preferences and decisions. J Pers Soc Psychol 60(2):181–192CrossRefGoogle Scholar
  27. Winer EH, Bloebaum CL (2001) Visual design steering for optimization solution improvement. Struct Optim 22(3):219–229CrossRefGoogle Scholar
  28. Winer EH, Bloebaum CL (2002) Development of visual design steering as an aid in large-scale multidisciplinary design optimization Part I: method development. Struct Multidiscipl Optim 23(6):412–424CrossRefGoogle Scholar
  29. Wright H, Brodlie K, David T (2000) Navigating high-dimensional spaces to support design steering. Proceedings of IEEE visualization 2000, IEEE Computer Society Press, Salt Lake City, UT, pp 291–296Google Scholar
  30. Wu J, Azarm S (2001) Metrics for quality assessment of a multiobjective design optimization solution set. ASME J Mech Des 123(1):18–25CrossRefGoogle Scholar
  31. Zitzler E (1999) Evolutionary algorithms for multiobjective: methods and applications. Swiss Federal Institute of Technology, Zurich, SwitzerlandGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Timothy W. Simpson
    • 1
  • Dan Carlsen
    • 1
  • Matthew Malone
    • 1
  • Joshua Kollat
    • 1
  1. 1.Mechanical & Nuclear Engineering and Industrial & Manufacturing Engineering, The Pennsylvania State UniversityUniversity ParkUSA

Personalised recommendations