Abstract
Previous eye-tracking studies of whether recipients look at speakers. gestures have yielded conflicting results but have also differed in method. This study aims to isolate the effect of the medium of presentation on recipients’ fixation behaviour towards speakers’ gestures by comparing fixations of the same gestures either performed live in a face-to-face condition or presented on video ceteris paribus. The results show that although fewer gestures are fixated on video, fixation behaviour towards gestures is largely similar across conditions. In discussing the effect of the absence of a live interlocutor vs. the projection size as a source for this reduction, we touch on some underlying mechanisms governing gesture fixations. The results are pertinent to man-machine interface issues as well as to the ecological validity of video-based paradigms needed to study the relationship between visual and cognitive attention to gestures and Sign Language.
We gratefully acknowledge the financial support of Birgit and Gad Rausing’s Foundation for Research in the Humanities, and of the Max Planck Society.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Gullberg, M., Holmqvist, K.: Keeping an Eye on Gestures: Visual Perception of Gestures in Face-to-Face Communication. Pragmatics & Cognition 7 (1999) 35–63
Argyle, M.: The Psychology of Interpersonal Behaviour. Penguin, Harmondsworth (1976)
Kendon, A.: Conducting Interaction. Cambridge University Press, Cambridge (1990)
Nielsen, G.: Studies in Self Confrontation: Viewing a Sound Motion Picture of Self and Another Person in a Stressful Dyadic Interaction. Munksgaard, Copenhagen (1962)
Fehr, B., Exline, R.: Social Visual Interaction: A Conceptual and Literature Review. In: Siegman, A., Feldstein, S. (eds.): Nonverbal behavior and communication. 2nd ed. Erlbaum, Hillsdale, NJ (1987) 225–326
Nobe, S., Hayamizu, S., Hasegawa, O., Takahashi, H.: Are Listeners Paying Attention to the Hand Gestures of an Anthropomorphic Agent? An Evaluation Using a Gaze Tracking Method. In: Wachsmuth, I., Fröhlich, M. (eds.): Gesture and Sign Language in Human-Computer Interaction. Springer Verlag, Berlin (1998) 49–59
Nobe, S., Hayamizu, S., Hasegawa, O., Takahashi, H.: Hand Gestures of an Anthropomorphic Agent: Listeners’ Eye Fixation and Comprehension. Cognitive Studies. Bulletin of the Japanese Cognitive Science Society 7 (2000) 86–92
McNeill, D.: Hand and Mind. Chicago University Press, Chicago (1992)
Cassell, J., McNeill, D., McCullough, K. E.: Speech-Gesture Mismatches: Evidence for one Underlying Representation of Linguistic and Nonlinguistic Information. Pragmatics & Cognition 7 (1999) 1–33
Beattie, G., Shovelton, H.: Do Iconic Hand Gestures Really Contribute Anything to the Semantic Information Conveyed by Speech? Semiotica 123 (1999) 1–30
Beattie, G., Shovelton, H.: Mapping the Range of Information Contained in the Iconic Hand Gestures that Accompany Spontaneous Speech. J. of Lang. and Social Psy. 18 (1999) 438–462
Gullberg, M.: Gesture as a Communication Strategy in Second Language Discourse. Lund University Press, Lund (1998)
Streeck, J., Knapp, M. L.: The Interaction of Visual and Verbal Features in Human Communication. In: Poyatos, F. (ed.): Advances in Nonverbal Communication. Benjamins, Amsterdam (1992) 3–23
Kendon, A.: Some Relationships between Body Motion and Speech: An Analysis of an Example. In: Siegman, A.W., Pope, B. (eds.): Studies in Dyadic Communication. Pergamon, New York (1972) 177–210
Kendon, A.: Gesticulation and Speech: Two Aspects of the Process of Utterance. In: Key, M. (ed.): The Relationship of Verbal and Nonverbal Communication. Mouton, The Hague (1980) 207–227
Latham, K., Whitaker, D.: A Comparison of Word Recognition and Reading Performance in Foveal and Peripheral Vision. Vision Research 37 (1996) 2665–2674
Baron-Cohen, S.: The Eye Direction Detector (EDD) and the Shared Attention Mechanism (SAM): Two cases for evolutionary Psychology. In: Moore, C., Dunham, P.J. (eds.): Joint attention. Erlbaum, Hillsdale (1995) 41–59
Langton, S. R. H., Watt, R. J., Bruce, V.: Do the eyes have it? Cues to the direction of social attention. Trends in Cognitive Sciences 4 (2000) 50–59
Tomasello, M.: Joint attention as social cognition. In: Moore, C., Dunham, P. J. (eds.): Joint attention. Erlbaum, Hillsdale (1995) 103–130
Streeck, J.: Gesture as Communication I: Its Coordination with Gaze and Speech. Communication Monographs 60 (1993) 275–299
Goodwin, C.: Gestures as a resource for the organization of mutual orientation. Semiotica 62 (1986) 29–49
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gullberg, M., Holmqvist, K. (2002). Visual Attention towards Gestures in Face-to-Face Interaction vs. on Screen. In: Wachsmuth, I., Sowa, T. (eds) Gesture and Sign Language in Human-Computer Interaction. GW 2001. Lecture Notes in Computer Science(), vol 2298. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-47873-6_23
Download citation
DOI: https://doi.org/10.1007/3-540-47873-6_23
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43678-2
Online ISBN: 978-3-540-47873-7
eBook Packages: Springer Book Archive