Skip to main content
SpringerLink
Log in

An approach to the extraction of preference-related information from design team language

  • Original Paper
  • Published:
Research in Engineering Design Aims and scope Submit manuscript

Abstract

The process of selecting among design alternatives is an important activity in the early stages of design. A designer is said to express design preferences when assigning priorities to a set of possible design choices. However, the assignment of preferences becomes more challenging on both a practical and theoretical level when performed by a group. This paper presents a probabilistic approach for estimating a team’s overall preference-related information known as preferential probabilities that extracts information from the natural language used in team discussion transcripts without aggregation of individual team member opinions. Assessment of the method is conducted by surveying a design team to obtain quantitative ratings of alternatives. Two different approaches are applied to convert these ratings into values that may be compared to the results of transcript analysis: the application of a modified Logit model and simulation based on the principle of maximum entropy. The probabilistic approach proposed in the paper represents how likely a choice is to be “most preferred” by a design team over a given period of time. A preliminary design selection experiment was conducted as an illustrative case example of the method. Correlations were found between the preferential probabilities estimated from transcripts and those computed from the surveyed preferences. The proposed methods may provide a formal way to understand and represent informal, unstructured design information using a low overhead information extraction method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Arnold K (2001) Making team decision. In: Biech E (ed) The Pfeiffer book of successful team building tools. Jossey-Bass/Pfeiffer, San Francisco

  • Arrow KJ (1970) Social choice and individual values. Yale University Press, New Haven

    Google Scholar 

  • Arrow KJ, Raynaud H (1986) Social choice and multicriterion decision-making. MIT, Cambridge, MA

    MATH  Google Scholar 

  • Ben-Akiva M, Lerman SR (1985) Discrete choice analysis. MIT, Cambridge, MA

    Google Scholar 

  • Bertrand M, Mullainathan S (2001) Do people mean what they say? Implications for subjective survey data. Am Econ Rev 91(2):67–72

    Article  Google Scholar 

  • Bilmes JA (1998) A gentle tutorial of the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models. Technical report. International Computer Science Institute, Berkeley, CA, USA

  • Bockenholt U (2002) A Thurstonian analysis of preference change. J Math Psychol 46(3):300–314

    Article  MathSciNet  Google Scholar 

  • Brans JP, Vincke P (1985) A preference ranking organisation method: (The PROMETHEE method for multiple criteria decision-making). Manag Sci 31(6):647–656

    Article  MathSciNet  MATH  Google Scholar 

  • Brockman JB (1996) Evaluation of student design processes. The 26th annual frontiers in education conference, Salt Lake City, UT

  • Busemeyer JR, Diederich A (2002) Survey of decision field theory. Math Soc Sci 43(3):345–370

    Article  MathSciNet  MATH  Google Scholar 

  • Cross N, Christiaans H, Dorst K (1996) Analysing design activity. Wiley, Chichester

    Google Scholar 

  • Dempster A, Laird N, Rubin D (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B 39(1):1–38

    MathSciNet  MATH  Google Scholar 

  • Dong A (2005) The latent semantic approach to studying design team communication. Des Stud 26(5):445–461

    Google Scholar 

  • Dong A (2006a) Concept formation as knowledge accumulation: a computational linguistics study. Artif Intell Eng Des Anal Manuf 20(1):35–53

    Google Scholar 

  • Dong A (2006b) How am I doing? The language of appraisal in design. Design computing and cognition ‘06 (DCC06). J S Gero. Kluwer, Dordrecht, The Netherlands, pp 385–404

  • Dym CL, Wood WH, Scott MJ (2002) Rank ordering engineering designs: pairwise comparison charts and Borda counts. Res Eng Des 13(4):236–242

    Google Scholar 

  • Fishburn PC (1978) Choice probabilities and choice functions. J Math Psychol 18:205–219

    Article  MathSciNet  MATH  Google Scholar 

  • Fisher RA (1922) On the mathematical foundations of theoretical statistics. Philos Trans R Soc 222:309–368

    Article  Google Scholar 

  • Geslin MM (2006) An argumentation-based approach to negotiation in collaborative engineering design. Department of Aerospace And Mechanical Engineering, University of Southern California, Los Angeles

    Google Scholar 

  • Giffin M, De Weck O, Bounova G, Keller R, Eckert C, Clarkson PJ (2009) Change propagation analysis in complex technical systems. J Mech Des 131(8):081001

    Article  Google Scholar 

  • Gigone D, Hastie R (1997) The impact of information on small group choice. J Pers Soc Psychol 72(1):132–140

    Article  Google Scholar 

  • Green PE, Srinivasan V (1990) Conjoint analysis in marketing: new developments with implications for research and practice. J Market 54(4):3–19

    Article  Google Scholar 

  • Grefenstette G (1993) Automatic thesaurus generation from raw text using knowledge-poor techniques. The 9th annual conference of the UW centre for the new OED and text research, Oxford, England

  • Hanley N, Mourato S, Wright RE (2001) Choice modelling approaches: A superior alternative for environmental valuation? J Econ Surv 15(3):435–462

    Google Scholar 

  • Hauser JR, Clausing D (1988) The house of quality. Harv Bus Rev 66(3):63–73

    Google Scholar 

  • Hazelrigg GA (1998) A framework for decision-based engineering design. J Mech Des 120(4):653–658

    Article  Google Scholar 

  • Hensher DA, Johnson LW (1981) Applied discrete choice modeling. Halsted Press, New York

    Google Scholar 

  • Hey JD (1998) Do rational people make mistakes? Game theory, experience, rationality. In: Leinfellner W, Kohler E (eds) Kluwer, The Netherlands, pp 55–66

  • Honda T, Yang MC, Dong A, Ji H (2010) A comparison of formal methods for evaluating the language of preference in engineering design. ASME design engineering technical conferences. Montreal, Canada

  • Jabeur K, Martel J-M, Khelifa SB (2004) A distance-based collective preorder integrating the relative importance of the group’s members. Group Decis Negot 13(4):327–349

    Article  Google Scholar 

  • Jain VK, Sobek DK II (2006) Linking design process to customer satisfaction through virtual design of experiments. Res Eng Des 17(2):59–71

    Article  Google Scholar 

  • Jaynes ET (1957) Information theory and statistical mechanics. Phys Rev 106(4):620–630

    Article  MathSciNet  Google Scholar 

  • Jaynes ET (1968) Prior probabilities. IEEE Trans Syst Sci Cybern 4(3):227–241

    Article  MATH  Google Scholar 

  • Ji H, Yang MC, Honda T (2007) A probabilistic approach for extracting design preferences from design team discussion. In: Proceedings of ASME 2007 international design engineering technical conferences and computers and information in engineering conference

  • Keeney RL, Raiffa H (1976) Decisions with multiple objectives: preferences and value tradeoffs. Wiley, New York

    Google Scholar 

  • Kelley CT (2003) Solving nonlinear equations with Newton’s method. Society for Industrial and Applied Mathematics, Philadelphia

    Book  MATH  Google Scholar 

  • Kohrs A, Merialdo B (2000) Using category-based collaborative filtering in the ActiveWebMuseum. The 2000 IEEE international conference on multimedia and expo, vol 1, pp 351–354

  • Krantz DH, Luce RD, Suppes P, Tversky A (1971) Foundations of measurement volume 1. Academic Press, New York, NY

    Google Scholar 

  • Kulok M, Lewis K (2005) Preference consistency in multiattribute decision making. ASME conference proceedings 2005 (4742Xa), pp 291–300

  • Li W, Jin Y (2006) Fuzzy preference evaluation for hierarchical co-evolutionary design concept generation. ASME conference proceedings (4255X), pp 31–41

  • Luce RD (1959) Individual choice behavior. Wiley, New York

    MATH  Google Scholar 

  • Mabogunje A, Leifer LJ (1996) 210-NP: measuring the mechanical engineering design process. Frontiers in Education Conference. FIE ‘96. In: Proceedings of 26th annual conference, vol 3, pp 1322–1328

  • Manski CF (1977) The structure of random utility models. Theory Decis 8:229–254

    Article  MathSciNet  MATH  Google Scholar 

  • Miller GA, Beckwith R, Fellbaum C, Gross D, Miller K (1990) WordNet: an on-line lexical database. Int J Lexicogr 3(4):235–244

    Google Scholar 

  • Otto KN, Antonsson EK (1991) Trade-off strategies in engineering design. Res Eng Des 3(2):87–104

    Article  Google Scholar 

  • Otto KN, Antonsson EK (1993) The method of imprecision compared to utility theory for design selection problems. ASME 1993 design theory and methodology conference

  • Packard DJ (1979) Preference relations. J Math Psychol 19(3):295–306

    Article  MathSciNet  MATH  Google Scholar 

  • Press WH, Teukolsky SA, Vetterling WT, Flannery BP (2007) Numerical recipes: the art of scientific computing. Cambridge University Press, New York

    MATH  Google Scholar 

  • Pugh S (1991) Total design: integrated methods for successful product engineering. Addison-Wesley, Wokingham

    Google Scholar 

  • Reich Y (2010) My method is better!. Res Eng Des 21(3):137–142

    Article  Google Scholar 

  • Ross SM (2006) Simulation. Academic Press, Burlington, MA

    MATH  Google Scholar 

  • Saaty TL (2000) Fundamentals of decision making and priority theory with the analytic hierarchy process. RWS Publications, Pittsburgh

    Google Scholar 

  • Scott MJ, Antonsson EK (1998) Aggregation functions for engineering design trade-offs. Fuzzy Sets Syst 99(3):253–264

    Article  Google Scholar 

  • Scott MJ, Antonsson EK (1999) Arrow’s theorem and engineering design decision making. Res Eng Des 11(4):218–228

    Article  Google Scholar 

  • Scott MJ, Antonsson EK (2005) Compensation and weights for trade-offs in engineering design: beyond the weighted sum. J Mech Des 127(6):1045–1055

    Article  Google Scholar 

  • See T-K, Lewis K (2006) A formal approach to handling conflicts in multiattribute group decision making. J Mech Des 128(4):678–688

    Article  Google Scholar 

  • Shah JJ, Vargas-Hernandez N, Summers JD, Kulkarni S (2001) Collaborative sketching (C-Sketch)—an idea generation technique for engineering design. J Creat Behav 35(3):168–198

    Article  Google Scholar 

  • Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27(3):379–423, 623–656

  • Song S, Dong A, Agogino AM (2003) Time variation of design “story telling” in engineering design teams. In: Proceedings of the 14th international conference on engineering design (ICED 03), Stockholm, Sweden

  • Thompson LL (2003) Making the team: a guide for managers. Prentice Hall, Upper Saddle River

    Google Scholar 

  • Thurston D (1991) A formal method for subjective design evaluation with multiple attributes. Res Eng Des 3(2):105–122

    Article  Google Scholar 

  • Tribus M (1969) Rational descriptions, decisions, and designs. Pergamon Press, New York

    Google Scholar 

  • Ueda N, Nakano R (1998) Deterministic annealing EM algorithm. Neural Netw 11(2):271–282

    Article  Google Scholar 

  • Ueda N, Nakano R, Ghahramani Z, Hinton GE (2000) SMEM algorithm for mixture models. Neural Comput 12(9):2109–2128

    Article  Google Scholar 

  • von Neumann J, Morgenstern O (1947) Theory of games and economic behaviour. Princeton University Press, Princeton

    Google Scholar 

  • Wang J (1997) A fuzzy outranking method for conceptual design evaluation. Int J Prod Res 35(4):995–1010

    Article  MATH  Google Scholar 

  • Wassenaar HJ, Chen W (2003) An approach to decision based design with discrete choice analysis for demand modeling. J Mech Des 125(3):490–497

    Article  Google Scholar 

  • Wassenaar HJ, Chen W, Cheng J, Sudjianto A (2005) Enhancing discrete choice demand modeling for decision-based design. J Mech Des 127(4):514–523

    Article  Google Scholar 

  • Wood KL, Antonsson EK (1989) Computations with imprecise parameters in engineering design: background and theory. ASME J Mech Transm Autom Des 111(4):616–625

    Article  Google Scholar 

  • Yang MC (2003) Concept generation and sketching: correlations with design outcome. ASME Conf Proc 37017b:829–834

    Google Scholar 

  • Yang MC (2005) A study of prototypes, design activity, and design outcome. Des Stud 26(6):649–669

    Article  Google Scholar 

  • Yang MC, Ji H (2007) A text-based analysis approach to representing the design selection process. In: Proceedings of the 16th international conference on engineering design (ICED 07)

  • Yang MC, Wood WH, Cutkosky MR (2005) Design information retrieval: a thesauri-based approach for reuse of informal design information. Eng Comput 21(2):177–192

    Article  Google Scholar 

Download references

Acknowledgments

The work described in this paper was supported in part by the National Science Foundation under Awards CMMI-0830134 and CMMI-0900255. The opinions, findings, conclusions, and recommendations expressed are those of the authors and do not necessarily reflect the views of the sponsors.

Author information

Authors and Affiliations

  1. Yahoo! Inc., 701 First Ave, Sunnyvale, CA, 94089, USA

    Haifeng Ji

  2. Department of Mechanical Engineering and Engineering Systems Division, Massachusetts Institute of Technology, 77 Massachusetts Ave, 3-449B, Cambridge, MA, 02139, USA

    Maria C. Yang

  3. Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Ave, 3-446, Cambridge, MA, 02139, USA

    Tomonori Honda

Authors
  1. Haifeng Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maria C. Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tomonori Honda
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maria C. Yang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ji, H., Yang, M.C. & Honda, T. An approach to the extraction of preference-related information from design team language. Res Eng Design 23, 85–103 (2012). https://doi.org/10.1007/s00163-011-0116-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00163-011-0116-7

Keywords

Search

Navigation