Abstract
Design novices have limited design experience and typically lack the skills or confidence to create good design, however, they may be able to recognize good design. To assess this ability, 53 novice designers and 52 expert designers participated in an online study where they evaluated a series of websites based on aesthetic appeal using two different modes of comparison. Results show that both experts and novices are able to recognize good design and that novices are able to do so almost as well as experts (76.5% accuracy compared to 81.2%). The greatest determinant of whether a participant would correctly identify a higher-rated design was the difference in the two websites’ ground-truth aesthetic ratings. However, expertise and the mode by which the comparison was presented had a significant impact on accuracy (Keep-the-Best = 83.6% and Tournament = 74.1%).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bogumil, R. J. (1985). The reflective practitioner: How professionals think in action. Proceedings of the IEEE [Internet]. [cited 2017 Sep 15], 73(4), 845–846. Available from: http://ieeexplore.ieee.org/document/1457478/
Buxton, B. (2010). Sketching user experiences: Getting the design right and the right design [Internet]. [cited 2017 Sep 15]. Available from: https://books.google.com/books?hl=en&lr=&id=2vfPxocmLh0C&oi=fnd&pg=PP1&dq=Sketching+User+Experiences:+Getting+the+Design+Right+and+the+Right+Design&ots=06Kuhjl6VO&sig=5VvNqRGGzjzH0ufiVvL8A8UlZXo
Christiaans, H. H. C. M., & Dorst, K. H. (1992). Cognitive models in industrial design engineering: A protocol study. American Society of Mechanical Engineers Design Engineering Division.
Cross, N. (2004). Expertise in design: An overview. Design Studies. [Internet]. Elsevier; Sep 1 [cited 2017 Sep 15];25(5), 427–441. Available from: http://www.sciencedirect.com/science/article/pii/S0142694X04000316
Dow, S., & Klemmer, S. R. (2010). The efficacy of prototyping under time constraints. Design Thinking. [Internet]. [cited 2017 Sep 15], pp. 111–128. Available from: http://dl.acm.org/citation.cfm?id=1640260
Dow, S. P., Glassco, A., Kass, J., Schwarz, M., Schwartz, D. L., & Klemmer, S. R. (2012). Parallel prototyping leads to better design results, more divergence, and increased self-efficacy. Design Thinking Research Studies. Co-Creation Practice [Internet]. [cited 2017 Sep 15], pp. 127–153. Available from: http://link.springer.com/chapter/10.1007/978-3-642-21643-5_8
Faste, R. (1995). The role of aesthetics in engineering. Japan Society of Mechanical Engineering Journal. [Internet]. [cited 2017 Sep 15]. Available from: http://www.fastefoundation.org/publications/the_role_of_aesthetics.pdf
Faste, H. (2017). Intuition in design: Reflections on the iterative aesthetics of form. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems– CHI EA ’17 [Internet]. [cited 2017 Sep 15], pp. 3403–3413. Available from: http://dl.acm.org/citation.cfm?id=3025534
Hartmann, B., Klemmer, S., Bernstein, M., Abdulla, L., Burr, B., Robinson-Mosher, A., et al. (2006) Reflective physical prototyping through integrated design, test, and analysis. In Proceedings of 19th Annual ACM Symposium on User interface Software Technology [Internet]. [cited 2017 Sep 15], pp. 299–308. Available from: http://dl.acm.org/citation.cfm?id=1166300
Hassenzahl, M. (2004, December 1). The interplay of beauty, goodness, and usability in interactive products. Human-Computer Interactions. [Internet]. L. Erlbaum Associates [cited 2017 Sep 15], 19(4), 319–349. Available from: http://www.tandfonline.com/doi/abs/10.1207/s15327051hci1904_2
Kulkarni, C., Wei, K. P., Le, H., Chia, D., Papadopoulos, K., & Cheng, J., et al. (2013). Peer and self assessment in massive online classes. ACM Transactions on Computer-Human Interaction. [Internet]. [cited 2017 Sep 15], 20(6), 1–31. Available from: http://link.springer.com/chapter/10.1007/978-3-319-06823-7_9
Kurosu, M., & Kashimura, K. (1995). Creativity of. Apparent usability vs. inherent usability: Experimental analysis on the determinants of the apparent usability. In Proceedings of ACM Conference on Human Factors in Computer Systems [Internet]. New York, New York, USA: ACM Press; [cited 2017 Sep 15], 292–293. Available from: http://portal.acm.org/citation.cfm?doid=223355.223680
Lloyd, P., & Scott, P. (1994). Discovering the design problem. Design Studies. [Internet]. [cited 2017 Sep 15], 15(2), 125–140. Available from: http://www.sciencedirect.com/science/article/pii/0142694X94900205
Luther, K., Pavel, A., Wu, W., Tolentino, J., Agrawala, M., & Hartmann, B., et al. (2014) CrowdCrit: Crowdsourcing and aggregating visual design critique. In Proceedings of Companion Publ. 17th ACM Conference on Computer Supported Cooperative Work and Social Computing – CSCW Companion ’14 [Internet]. [cited 2017 Sep 15], pp. 21–24. Available from: http://dl.acm.org/citation.cfm?id=2556788
Luther, K., Tolentino, J., Wu, W., Pavel, A., Bailey, B. P., Agrawala, M., et al. (2015) Structuring, aggregating, and evaluating crowdsourced design critique. In Proceedings of 18th ACM Conference on Computer Supported Cooperative Work and Social Computing – CSCW ’15 [Internet]. [cited 2017 Sep 15], pp. 473–485. Available from: http://dl.acm.org/citation.cfm?id=2675283
Marks, J., Ruml, W., Ryall, K., Seims, J., Shieber, S., & Andalman, B., et al. (1997). Design galleries: a general approach to setting parameters for computer graphics and animation. In Proceedings of 24th Annual Conference on Computer Graphics Interactive Technology – SIGGRAPH ’97 [Internet]. [cited 2017 Sep 15], pp. 389–400. Available from: http://dl.acm.org/citation.cfm?id=258887
O’Donovan, P., Agarwala, A., & Hertzmann, A. (2015). DesignScape: Design with interactive layout suggestions. In Proceedings of 33rd Annual Conference on Human Factors in Computer Systems – CHI ’15 [Internet]. [cited 2017 Sep 15], pp. 1221–1224. Available from: http://dl.acm.org/citation.cfm?id=2702149
Petitmengin-peugeot, C. (1999). The intuitive experience. Journal of Consciousness Studies. [Internet]. [cited 2017 Sep 15], 2, 43–77. Available from: http://www.ingentaconnect.com/content/imp/jcs/1999/00000006/f0020002/928
Reinecke, K., & Gajos, K. Z. (2014). Quantifying visual preferences around the world. In Proceedings of 32rd Annual ACM Conference on Human Factors in Computer Systems– CHI ’14 [Internet]. ACM Press, New York, USA; [cited 2017 Sep 15], pp. 11–20. Available from: http://dl.acm.org/citation.cfm?doid=2556288.2557052
Salter, A., & Whyte, J. (2004). Serious play: How the world’s best companies simulate to innovate [Internet]. Technovation. [cited 2017 Sep 15], pp. 277–278. Available from: https://books.google.com/books?hl=en&lr=&id=3f6UdmTaAH0C&oi=fnd&pg=PR9&dq=Serious+Play:+How+the+World’s+Best+Companies+Simulate+to+Innovate&ots=RFdJu21jIC&sig=rHiLvl2c91_MH8Zao_nEZmabVhU
Tidwell, J. (2005). Designing interfaces: Patterns for effective interaction design [Internet]. OReilly Media Inc. [cited 2017 Sep 19]. p. 352. Available from: https://books.google.com/books?hl=en&lr=&id=5gvOU9X0fu0C&oi=fnd&pg=PR11&dq=Designing+Interfaces:+Patterns+for+Effective+Interaction+Design&ots=sSZ0N7X9VT&sig=eeDGYaJ91-vKulFUKyqqGI2fhBk
Todi, K., Weir, D., & Oulasvirta, A. (2016). Sketchplore: Sketch and explore with a layout optimiser. In Proceedings of 2016 ACM Conference [Internet]. [cited 2017 Sep 15]; Available from: http://dl.acm.org/citation.cfm?id=2901817
Tohidi, M., Buxton, W., Baecker, R., & Sellen, A. (2006). Getting the right design and the design right: Testing many is better than one. In Proceedings of CHI 2006 Conference on Human Factors on Computer Systems [Internet]. [cited 2017 Sep 15], pp. 1243–1252. Available from: http://dl.acm.org/citation.cfm?id=1124960
Van Duyne, D. K., Landay, J. A., & Hong, J. I. (2007). The design of sites: Patterns for creating winning web sites [Internet]. [cited 2017 Sep 19]. Available from: https://books.google.com/books?hl=en&lr=&id=eE2TxLtDsL8C&oi=fnd&pg=PR13&dq=The+Design+of+Sites:+Patterns+for+Creating+Winning+Web+Sites&ots=v9JzHZPmCj&sig=FwQSMbSU8bI_Nno-Gu0-1Wgh9uc
Willett, W., Heer, J., & Agrawala, M. (2012). Strategies for crowdsourcing social data analysis. In Proceedings of 2012 ACM Annual Conference on Human Factors on Computer Systems – CHI ’12 [Internet]. [cited 2017 Sep 15], p. 227. Available from: http://dl.acm.org/citation.cfm?id=2207709
Xu, A., & Bailey, B. P. (2012) What do you think? A case study of benefit, expectation, and interaction in a large online critique community. In Proceedings of 15th ACM Conference on Computer Supported Cooperative Work and Social Computing – CSCW’12 [Internet]. [cited 2017 Sep 15], 295. Available from: http://dl.acm.org/citation.cfm?id=2145252
Xu, A., Huang, S.-W., & Bailey, B. (2014). Voyant: generating structured feedback on visual designs using a crowd of non-experts. In Proceedings of 17th ACM Conference on Computer Supported Cooperative Work and Social Computing – CSCW ’14 [Internet]. [cited 2017 Sep 15], pp. 1433–1444. Available from: http://dl.acm.org/citation.cfm?id=2531604
Xu, A., Rao, H., Dow, S. P., & Bailey, B. P. (2015). A classroom study of using crowd feedback in the iterative design process. In Proceedings of 18th ACM Conference on Computer Supported Cooperative Work and Social Computing – CSCW ’15 [Internet]. [cited 2017 Sep 15], pp. 1637–1648. Available from: http://dl.acm.org/citation.cfm?id=2675140
Yuan, A., Luther, K., Krause, M., Vennix, S. I., Dow, S. P., & Hartmann, B. (2016). Almost an expert: The effects of rubrics and expertise on perceived value of crowdsourced design critiques. In Proceedings of 19th ACM Conference on Computer Supported Cooperative Work and Social Computing – CSCW ’16 [Internet]. [cited 2017 Sep 15], pp. 1003–1015. Available from: http://dl.acm.org/citation.cfm?id=2819953
Acknowledgments
We’d like to thank Jacob O. Wobbrock, Rob Semmens, Kurt Gee and Roy Roy for their assistance with statistics. Any opinions, findings, conclusions or recommendations expressed in this work are those of the authors and do not necessarily reflect those of any supporter listed above.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Tanner, K., Landay, J. (2019). “I Know It When I See It”: How Experts and Novices Recognize Good Design. In: Meinel, C., Leifer, L. (eds) Design Thinking Research. Understanding Innovation. Springer, Cham. https://doi.org/10.1007/978-3-319-97082-0_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-97082-0_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97081-3
Online ISBN: 978-3-319-97082-0
eBook Packages: Business and ManagementBusiness and Management (R0)