Abstract
We describe here our efforts for modeling multimodal signals exchanged by interlocutors when interacting face-to-face. This data is then used to control embodied conversational agents able to engage into a realistic face-to-face interaction with human partners. This paper focuses on the generation and rendering of realistic gaze patterns. The problems encountered and solutions proposed claim for a stronger coupling between research fields such as audiovisual signal processing, linguistics and psychosocial sciences for the sake of efficient and realistic human-computer interaction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Argyle, M., Cook, M.: Gaze and mutual gaze. Cambridge University Press, London (1976)
Bailly, G., Elisei, F., Badin, P., Savariaux, C.: Degrees of freedom of facial movements in face-to-face conversational speech. In: International Workshop on Multimodal Corpora, Genoa - Italy, pp. 33–36 (2006)
Baron-Cohen, S., Leslie, A., Frith, U.: Does the autistic child have a “theory of mind”? Cognition 21, 37–46 (1985)
Benoît, C., Grice, M., Hazan, V.: The SUS test: A method for the assessment of text-to-speech synthesis intelligibility using Semantically Unpredictable Sentences. Speech Communication 18, 381–392 (1996)
Carpenter, M., Tomasello, M.: Joint attention, cultural learning and language acquisition: Implications for children with autism. In: Wetherby, A.M., Prizant, B.M. (eds.) Communicative and language intervention series. Autism spectrum disorders: A transactional perspective, Baltimore, vol. 9, pp. 30–54. Paul H. Brooks Publishing (2000)
Cassell, J., Sullivan, J., Prevost, S., Churchill, E.: Embodied Conversational Agents. MIT Press, Cambridge (2000)
Clodic, A., Fleury, S., Alami, R., Chatila, R., Bailly, G., Brèthes, L., Cottret, M., Danès, P., Dollat, X., Elisei, F., Ferrané, I., Herrb, M.: Rackham: an interactive robot-guide. In: IEEE International Workshop on Robots and Human Interactive Communications, Hatfield, UK (2006)
Driver, J., Davis, G., Riccardelli, P., Kidd, P., Maxwell, E., Baron-Cohen, S.: Shared attention and the social brain: gaze perception triggers automatic visuospatial orienting in adults. Visual Cognition 6(5), 509–540 (1999)
Geiger, G., Ezzat, T., Poggio, T.: Perceptual evaluation of video-realistic speech. CBCL Paper #224/AI Memo #2003-003, Cambridge, MA, Massachusetts Institute of Technology (2003)
Grossberg, S.: How does the cerebral cortex work? development, learning, attention, and 3d vision by laminar circuits of visual cortex. Behavioral and Cognitive Neuroscience Reviews 2, 47–76 (2003)
Gullberg, M., Holmqvist, K.: Visual attention towards gestures in face-to-face interaction vs. on screen. In: International Gesture Workshop, London, UK, pp. 206–214 (2001)
Itti, L., Dhavale, N., Pighin, F.: Realistic avatar eye and head animation using a neurobiological model of visual attention. In: SPIE 48th Annual International Symposium on Optical Science and Technology, San Diego, CA, pp. 64–78 (2003)
Kendon, A.: Some functions of gaze-direction in social interaction. Acta Psychologica 26, 22–63 (1967)
Langton, S., Bruce, V.: Reflexive visual orienting in response to the social attention of others. Visual Cognition 6(5), 541–567 (1999)
Langton, S., Watt, J., Bruce, V.: Do the eyes have it? Cues to the direction of social attention. Trends in Cognitive Sciences 4(2), 50–59 (2000)
Lee, S.P., Badler, J.B., Badler, N.: Eyes alive. ACM Transaction on Graphics 21(3), 637–644 (2002)
Matsusaka, Y., Tojo, T., Kobayashi, T.: Conversation Robot Participating in Group Conversation. IEICE Transaction of Information and System E86-D(1), 26–36 (2003)
Posner, M.I.: Orienting of attention. Quarterly Journal of Experimental Psychology 32, 3–25 (1980)
Pourtois, G., Sander, D., Andres, M., Grandjean, D., Reveret, L., Olivier, E., Vuilleumier, P.: Dissociable roles of the human somatosensory and superior temporal cortices for processing social face signals. European Journal of Neuroscience 20, 3507–3515 (2004)
Premack, D., Woodruff, G.: Does the chimpanzee have a theory of mind? Behavioral and brain sciences 1, 515–526 (1978)
Raidt, S., Bailly, G., Elisei, F.: Does a virtual talking face generate proper multimodal cues to draw user’s attention towards interest points? In: Language Ressources and Evaluation Conference (LREC), Genova, Italy, pp. 2544–2549 (2006)
Riva, G., Davide, F., IJsselsteijn, W.A.: Being there: concepts, effects and measurements of user presence in synthetic environments. IOS Press, Amsterdam (2003)
Scassellati, B.: Foundations for a theory of mind for a humanoid robot. In: Department of Computer Science and Electrical Engineering, Boston - MA, p. 174. MIT, Cambridge (2001)
Simons, D.J., Chabris, C.F.: Gorillas in our midst: sustained inattentional blindness for dynamic events. Perception 28, 1059–1074 (1999)
Stork, D.G., Hennecke, M.E.: Speechreading by Humans and Machines. Springer, Berlin, Germany (1996)
Sun, Y.: Hierarchical object-based visual attention for machine vision. PhD Thesis. Institute of Perception, Action and Behaviour. School of Informatics. Edinburgh, University of Edinburgh, p. 169 (2003)
Thórisson, K.: Natural turn-taking needs no manual: computational theory and model from perception to action. In: Granström, B., House, D., Karlsson, I. (eds.) Multimodality in language and speech systems, pp. 173–207. Kluwer Academic, Dordrecht, The Netherlands (2002)
Vatikiotis-Bateson, E., Eigsti, I.-M., Yano, S., Munhall, K.G.: Eye movement of perceivers during audiovisual speech perception. Perception & Psychophysics 60, 926–940 (1998)
Xu, Y., Chun, M.M.: Dissociable neural mechanisms supporting visual short-term memory for objects. Nature 440, 91–95 (2006)
Yarbus, A.L.: Eye movements during perception of complex objects. In: Riggs, L.A. (ed.) Eye Movements and Vision, vol. VII, pp. 171–196. Plenum Press, New York (1967)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bailly, G., Elisei, F., Raidt, S., Casari, A., Picot, A. (2006). Embodied Conversational Agents: Computing and Rendering Realistic Gaze Patterns. In: Zhuang, Y., Yang, SQ., Rui, Y., He, Q. (eds) Advances in Multimedia Information Processing - PCM 2006. PCM 2006. Lecture Notes in Computer Science, vol 4261. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11922162_2
Download citation
DOI: https://doi.org/10.1007/11922162_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-48766-1
Online ISBN: 978-3-540-48769-2
eBook Packages: Computer ScienceComputer Science (R0)