Abstract
Identifying icon functions differs from naming pictures in that strong semantic links between pictures and their names have been formed over a long period of time whereas the meaning of icons has often to be learned. This paper examines roles of icon characteristics such as complexity, concreteness, familiarity and aesthetic appeal in determining how easily icons can be learned and identified. The role of these characteristics is seen as dynamic, changing as the user learns the icon set. It is argued that the way in which users learn icon meanings is similar to the processes involved in language learning. Icon meanings are learned by drawing on rich multimodal representations which are the result of our world experience. This approach could lead to a better understanding of how multimodal information can be most usefully presented on interfaces.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Green, A.J.K., Barnard, P.J.: Iconic interfacing: The role of icon distinctiveness and fixed or variable screen locations. In: Diaper, D., et al. (eds.) Human-computer interaction – Interact 1990. Elsevier, North-Holland (1990)
Rogers, Y., Oborne, D.J.: Pictorial communication of abstract verbs in related to human-computer interaction. British Journal of Psychology 78, 99–112 (1987)
Stotts, D.B.: The usefulness of icons on the computer interface: Effect of graphical abstraction and functional representation on experienced and novice users. In: Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, pp. 453–457. Human Factors and Ergonomics Society, Santa Monica (1988)
Byrne, M.D.: Using icons to find documents: Simplicity is critical. In: Proceedings of INTERCHI 1993, pp. 446–453 (1993)
Scott, D.: Visual search in modern human-computer interfaces. Behaviour and Information Technology 12, 174–189 (1993)
Forsythe, A., Mulhern, G., Sawey, M.: Confounds in pictorial sets: The role of complexity in basic-level picture processing. Behavior Research Methods 40, 116–129 (2008)
McDougall, S.J.P., de Bruijn, O., Curry, M.B.: Exploring the effects of icon characteristics on user performance: The role of icon concreteness, complexity, and distinctiveness. Journal of Experimental Psychology: Applied 6, 291–306 (2000)
Goonetilleke, R.S., Shih, H.M., On, H.K., Fritsch, J.: Effects of training and representational characteristics in icon design. International Journal of Human-Computer Studies 55, 741–760 (2001)
Isherwood, S., McDougall, S., Curry, M.: Icon identification in context: The role of icon characteristics and user experience. Human Factors 49, 465–476 (2007)
Reppa, I., McDougall, S.: Visual aesthetic appeal speeds processing of complex but not simple icons. In: Proceedings of the 52nd Annual Meeting of the Human Factors and Ergonomics Society 2008, pp. 1155–1159 (2008)
McKeown, D., Isherwood, S.: Mapping candidate within-vehicle auditory displays to their referents. Human Factors 49, 417–428 (2007)
Petocz, A., Keller, P., Stevens, C.: Auditory warnings, signal-referent relations and natural indicators: re-thinking theory and application. J. Exp. Psychol: Applied 14, 165–178 (2008)
Stephan, K.L., Smith, S.E., Martin, R.L., Parker, S.P.A., McAnally, K.: Learning and retention of associations between auditory icons and denotative referents: implications for the design of auditory warnings. Human Factors 48, 288–299 (2006)
Arend, U., Muthig, K.-P., Wandmacher, J.: Evidence for global feature superiority in menu selection by icons. Behaviour and Information Technology 6, 411–426 (1987)
Stammers, R.B., George, D.A., Carey, M.S.: An evaluation of abstract and concrete icons for a CAD package. In: Megaw, E.D. (ed.) Contemporary ergonomics 1989, pp. 416–421. Taylor & Francis, London (1989)
Stammers, R.B., Hoffman, J.: Transfer between icon sets and ratings of icon concreteness and appropriateness. In: Proceedings of the Human Factors Society 35th Annual Meeting, pp. 354–358. Human Factors and Ergonomics Society, Santa Monica (1991)
Stevens, K., Petocz, A.: The user knows: Considering the cognitive contribution of the user in the design of auditory warnings. In: Proceedings of HCI International 2009 (2009) (in press)
Isherwood, S.J.: Graphics and semantics: The relationship between what is seen and what is meant in icon design. In: Proceedings of HCI International 2009 (2009) (in press)
McDougall, S.J.P., Isherwood, S.J.: What’s in a name? The role of graphics, functions, and their interrelationships in icon identification. Behaviour Research Methods (in press)
McDougall, S.J.P., Curry, M.B., de Bruijn, O.: The effects of visual information on users’ mental models: An evaluation of pathfinder analysis as a measure of icon usability. International Journal of Cognitive Ergonomics 5, 59–84 (2001)
Lambon-Ralph, M., Graham, K.S., Ellis, A.W., Hodges, J.R.: Naming in semantic dementia: What matters? Neuropsychologia 36, 775–784 (1998)
Forsythe, A.M.: Visual complexity: Is that all there is? In: Proceedings of HCI International 2009 (2009) (in press)
Garcia, M., Badre, A.N., Stasko, T.: Development and validation of icons varying in their abstractness. Interacting with Computers 6, 191–211 (1994)
Alario, F.-X., Ferrand, L., Laganaro, M., New, B., Frauenfelder, U.H., Segui, J.: Predictors of picture naming speed. Behavior Research Methods, Instruments & Computers 36, 140–155 (2004)
Humphreys, G.W., Riddoch, M.J., Quinlan, P.T.: Cascade processes in picture identification. Cognitive Neuropsychology 5, 67–103 (1988)
Ellis, A.W., Morrison, C.M.: Real age-of-acquisition effects in lexical retrieval. Journal of Experimental Psychology: Learning, Memory & Cognition 24, 515–523 (1998)
Barry, C., Morrison, C.M., Ellis, A.W.: Naming Snodgrass and Vanderwart pictures: Effects of age of acquisition, frequency, and name agreement. Quarterly Journal of Experimental Psychology 50A, 560–585 (1997)
Bonin, P., Peereman, R., Malardier, N., Méot, A., Chalard, M.: A new set of 299 pictures for psycholinguistic studies: French norms for name agreement, image agreement, conceptual familiarity, visual complexity, image variability, age of acquisition, and naming latencies. Behavior Research Methods, Instruments, & Computers 35, 158–167 (2003)
Cuetos, F., Ellis, A.W., Alvarez, B.: Naming times for the Snodgrass and Vanderwart pictures in Spanish. Behavior Research Methods, Instruments and Computers 31, 650–658 (1999)
Paivio, A., Clark, J.M., Digdon, N., Bons, T.: Referential processing: Reciprocity and correlates of naming and imaging. Memory & Cognition 17, 163–174 (1989)
Snodgrass, J.G., Yuditsky, T.: Naming times for the Snodgrass and Vanderwart pictures. Behavior Research Methods, Instruments, & Computers 28, 516–536 (1996)
McDougall, S., Reppa, I.: Why do I like it? The relationship between icon characteristics, user performance and aesthetic appeal. In: Proceedings of the 52nd Annual Meeting of the Human Factors and Ergonomics Society 2008, pp. 1257–1261 (2008)
Berlyne, D.E.: Studies in the new experimental aesthetics. Hemisphere, Washington (1974)
Jacobsen, T., Hofel, L.: Aesthetic judgments of novel graphic patterns: Analyses of individual judgments. Perceptual & Motor Skills 95, 755–766 (2002)
Bauerly, M., Liu, Y.: Computational modeling and experimental investigation of the effects of compositional elements on interface and design aesthetics. In: Proceedings of the 8th Annual International Conference on Industrial Engineering – Theory, Applications and Practice (2003)
Vartanian, O., Goel, V.: Neuroanatomical correlates of aesthetic preferences for paintings. Neuroreport 15, 893–897 (2004)
Kawabata, H., Zeki, S.: Neural correlates of beauty. Journal of Neurophysiology 91, 1699–1705 (2004)
Bornstein, R.F.: Exposure and affect: Overview and meta-analysis of research, 1968-1987. Psychological Bulletin 106, 265–289 (1989)
Reber, R., Schwarz, N., Winkielman, P.: Processing fluency and aesthetic pleasure: Is beauty in the perceiver’s processing experience? Personality and Social Psychology 8, 364–382 (2004)
McDougall, S., Reppa, I., Smith, G., Playfoot, D.: Beyond emoticons: Combining affect and cognition in icon design. In: Proceedings of HCI International 2009 (2009) (in press)
Barsalou, L.W.: Grounded cognition. Annual Review of. Psychology 59, 617–645 (2008)
Clark, A.: Being there: Putting brain, body, and world together again. MIT Press, Cambridge (1997)
Lakoff, G., Johnson, M.: Metaphors we live by. University Chicago Press, Chicago (1980)
Gibson, J.J.: The ecological approach to visual perception. Houghton Mifflin, New York (1979)
Glenberg, A.M., Schroeder, J.L., Robertson, D.A.: Averting the gaze disengages the environment and facilitates remembering. Mem. Cogn. 26, 651–658 (1998)
Gillette, J., Glietman, H., Glietman, L., Lederer, A.: Human simulations of vocabulary learning. Cognition 73, 135–176 (1999)
Yu, C., Ballard, D.H.: A multimodal learning interface for grounding spoken language in sensory perceptions. ACM Transactions on Applied Perceptions 1, 57–80 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
McDougall, S., Forsythe, A., Isherwood, S., Petocz, A., Reppa, I., Stevens, C. (2009). The Use of Multimodal Representation in Icon Interpretation. In: Harris, D. (eds) Engineering Psychology and Cognitive Ergonomics. EPCE 2009. Lecture Notes in Computer Science(), vol 5639. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02728-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-02728-4_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02727-7
Online ISBN: 978-3-642-02728-4
eBook Packages: Computer ScienceComputer Science (R0)