Journal on Multimodal User Interfaces

, Volume 13, Issue 1, pp 19–30 | Cite as

The sense of social agency in gaze leading

  • Samuel Recht
  • Ouriel GrynszpanEmail author
Original Paper


Social interactions entail reciprocal reactions where one’s communicative acts triggers responses in others. Fluent interpersonal exchange relies on the ability to discriminate behaviors produced by others that are responses to one’s actions, thus involving a social sense of agency. Given the pivotal role of gaze in human communication, we propose to use gaze following as a model for studying the sense of agency in social actions. The experiment investigates the influence of sensory expertise and timing of the action’s effects by comparing feedback provided by a human avatar versus a nonfigurative animated object (an arrow) and by varying the control exerted by participants’ gaze on the feedback (avatar vs arrow). Results revealed a linear relationship between the judgement of agency and feedback latencies and higher agency discriminating performances with the avatar. These outcomes suggest that classical cognitive accounts of the sense of agency can be expanded to the realm of social actions and provide important information for designing virtual agents to train social gaze interactions.


Agency Social cognition Forward model Eye-tracking Avatar 



This work was performed within the Labex SMART (ANR-11-LABX-65) supported by French state funds managed by the ANR within the Investissements d’Avenir programme under Reference ANR-11-IDEX-0004-02. This work was also supported by a grant from La Fondation Orange (Project #71/2012, coordinator: O. Grynszpan). The data was collected in collaboration with the Centre Multidisciplinaire de Sciences Comportementales Sorbonne Universités-INSEAD. The funding sources were not involved in the study design, data processing, interpretation or writing of the article.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.


  1. 1.
    Bandura A (2006) Toward a psychology of human agency. Perspect Psychol Sci 1:164–180. CrossRefGoogle Scholar
  2. 2.
    Nadel J, Guérini C, Pezé A, Rivet C (1999) The evolving nature of imitation as a format for communication. In: Nadel J, Butterworth G (eds) Imitation in infancy. Cambridge University Press, New York, pp 209–234Google Scholar
  3. 3.
    Nadel J (2002) Imitation and imitation recognition: functional use in preverbal infants and nonverbal children with autism. In: Meltzoff AN, Prinz W (eds) The imitative mind: development, evolution, and brain bases. Cambridge University Press, Cambridge, pp 42–62CrossRefGoogle Scholar
  4. 4.
    Grynszpan O, Nadel J, Martin J-C et al (2012) Self-monitoring of gaze in high functioning autism. J Autism Dev Disord 42:1642–1650. CrossRefGoogle Scholar
  5. 5.
    Loveland KA (2005) Social-emotional impairment and self-regulation in autism spectrum disorders. In: Nadel J, Muir D (eds) Emotional development: recent research advances. Oxford University Press, Oxford, pp 365–382Google Scholar
  6. 6.
    Haggard P, Chambon V (2012) Sense of agency. Curr Biol 22:R390–R392. CrossRefGoogle Scholar
  7. 7.
    Gallagher S (2000) Philosophical conceptions of the self: implications for cognitive science. Trends Cogn Sci 4:14–21. CrossRefGoogle Scholar
  8. 8.
    Jeannerod M (2003) The mechanism of self-recognition in humans. Behav Brain Res 142:1–15CrossRefGoogle Scholar
  9. 9.
    van der Wel RPRD, Sebanz N, Knoblich G (2012) The sense of agency during skill learning in individuals and dyads. Conscious Cogn 21:1267–1279. CrossRefGoogle Scholar
  10. 10.
    Obhi SS, Hall P (2011) Sense of agency and intentional binding in joint action. Exp Brain Res 211:655–662. CrossRefGoogle Scholar
  11. 11.
    Pfister R, Obhi SS, Rieger M, Wenke D (2014) Action and perception in social contexts: intentional binding for social action effects. Front Hum Neurosci. Google Scholar
  12. 12.
    Herwig A, Horstmann G (2011) Action–effect associations revealed by eye movements. Psychon Bull Rev 18:531–537. CrossRefGoogle Scholar
  13. 13.
    Verschoor SA, Spapé M, Biro S, Hommel B (2013) From outcome prediction to action selection: developmental change in the role of action–effect bindings. Dev Sci 16:801–814. Google Scholar
  14. 14.
    Blakemore SJ, Wolpert DM, Frith CD (2002) Abnormalities in the awareness of action. Trends Cogn Sci 6:237–242CrossRefGoogle Scholar
  15. 15.
    Wolpert DM, Doya K, Kawato M (2003) A unifying computational framework for motor control and social interaction. Philos Trans R Soc Lond B Biol Sci 358:593–602. CrossRefGoogle Scholar
  16. 16.
    Sperry RW (1950) Neural basis of the spontaneous optokinetic response produced by visual inversion. J Comp Physiol Psychol 43:482–489CrossRefGoogle Scholar
  17. 17.
    von Holst E, Mittelstaedt H (1950) Das Reafferenzprinzip. Naturwissenschaften 37:464–476. CrossRefGoogle Scholar
  18. 18.
    Wurtz RH (2008) Neuronal mechanisms of visual stability. Vis Res 48:2070–2089CrossRefGoogle Scholar
  19. 19.
    Wolpert DM, Ghahramani Z, Jordan MI (1995) An internal model for sensorimotor integration. Science 269:1880–1882CrossRefGoogle Scholar
  20. 20.
    Wolpert DM, Miall RC (1996) Forward models for physiological motor control. Neural Netw Off J Int Neural Netw Soc 9:1265–1279CrossRefzbMATHGoogle Scholar
  21. 21.
    Haggard P (2005) Conscious intention and motor cognition. Trends Cogn Sci 9:290–295. CrossRefGoogle Scholar
  22. 22.
    Wegner DM (2003) The mind’s best trick: how we experience conscious will. Trends Cogn Sci 7:65–69. CrossRefGoogle Scholar
  23. 23.
    Farrer C, Valentin G, Hupé JM (2013) The time windows of the sense of agency. Conscious Cogn 22:1431–1441. CrossRefGoogle Scholar
  24. 24.
    Kawabe T, Roseboom W, Nishida S (2013) The sense of agency is action–effect causality perception based on cross-modal grouping. Proc R Soc Lond B Biol Sci 280:20130991. CrossRefGoogle Scholar
  25. 25.
    Sato A, Yasuda A (2005) Illusion of sense of self-agency: discrepancy between the predicted and actual sensory consequences of actions modulates the sense of self-agency, but not the sense of self-ownership. Cognition 94:241–255. CrossRefGoogle Scholar
  26. 26.
    Shanks DR, Dickinson A (1991) Instrumental judgment and performance under variations in action-outcome contingency and contiguity. Mem Cognit 19:353–360. CrossRefGoogle Scholar
  27. 27.
    Shanks DR, Pearson SM, Dickinson A (1989) Temporal contiguity and the judgement of causality by human subjects. Q J Exp Psychol Sect B 41:139–159. Google Scholar
  28. 28.
    Moore JW, Fletcher PC (2012) Sense of agency in health and disease: a review of cue integration approaches. Conscious Cogn 21:59–68. CrossRefGoogle Scholar
  29. 29.
    Synofzik M, Vosgerau G, Newen A (2008) Beyond the comparator model: a multifactorial two-step account of agency. Conscious Cogn 17:219–239. CrossRefGoogle Scholar
  30. 30.
    David N, Newen A, Vogeley K (2008) The “sense of agency” and its underlying cognitive and neural mechanisms. Conscious Cogn 17:523–534. CrossRefGoogle Scholar
  31. 31.
    Grynszpan O, Simonin J, Martin J-C, Nadel J (2012) Investigating social gaze as an action-perception online performance. Front Hum Neurosci 6:94. CrossRefGoogle Scholar
  32. 32.
    Pfeiffer UJ, Schilbach L, Jording M et al (2012) Eyes on the mind: investigating the influence of gaze dynamics on the perception of others in real-time social interaction. Front Psychol 3:537. CrossRefGoogle Scholar
  33. 33.
    Kobayashi H, Kohshima S (2001) Unique morphology of the human eye and its adaptive meaning: comparative studies on external morphology of the primate eye. J Hum Evol 40:419–435. CrossRefGoogle Scholar
  34. 34.
    Posner MI (1980) Orienting of attention. Q J Exp Psychol 32:3–25. CrossRefGoogle Scholar
  35. 35.
    Frischen A, Bayliss AP, Tipper SP (2007) Gaze cueing of attention: visual attention, social cognition, and individual differences. Psychol Bull 133:694–724. CrossRefGoogle Scholar
  36. 36.
    Lachat F, Conty L, Hugueville L, George N (2012) Gaze cueing effect in a face-to-face situation. J Nonverbal Behav 36:177–190. CrossRefGoogle Scholar
  37. 37.
    Driver J (1999) Gaze perception triggers reflexive visuospatial orienting. Vis Cogn 6:509–540. CrossRefGoogle Scholar
  38. 38.
    Friesen CK, Kingstone A (1998) The eyes have it! Reflexive orienting is triggered by nonpredictive gaze. Psychon Bull Rev 5:490–495. CrossRefGoogle Scholar
  39. 39.
    Emery NJ (2000) The eyes have it: the neuroethology, function and evolution of social gaze. Neurosci Biobehav Rev 24:581–604. CrossRefGoogle Scholar
  40. 40.
    Charman T (2003) Why is joint attention a pivotal skill in autism? Philos Trans R Soc Lond B Biol Sci 358:315–324. CrossRefGoogle Scholar
  41. 41.
    Scaife M, Bruner JS (1975) The capacity for joint visual attention in the infant. Nature 253:265–266. CrossRefGoogle Scholar
  42. 42.
    Mundy P, Newell L (2007) Attention, joint attention, and social cognition. Curr Dir Psychol Sci 16:269–274. CrossRefGoogle Scholar
  43. 43.
    Edwards SG, Stephenson LJ, Dalmaso M, Bayliss AP (2015) Social orienting in gaze leading: a mechanism for shared attention. Proc R Soc B 282:20151141. CrossRefGoogle Scholar
  44. 44.
    Brignani D, Guzzon D, Marzi CA, Miniussi C (2009) Attentional orienting induced by arrows and eye-gaze compared with an endogenous cue. Neuropsychologia 47:370–381. CrossRefGoogle Scholar
  45. 45.
    Friesen CK, Ristic J, Kingstone A (2004) Attentional effects of counterpredictive gaze and arrow cues. J Exp Psychol Hum Percept Perform 30:319–329. CrossRefGoogle Scholar
  46. 46.
    Guzzon D, Brignani D, Miniussi C, Marzi CA (2010) Orienting of attention with eye and arrow cues and the effect of overtraining. Acta Psychol (Amst) 134:353–362. CrossRefGoogle Scholar
  47. 47.
    Hietanen JK, Leppänen JM, Nummenmaa L, Astikainen P (2008) Visuospatial attention shifts by gaze and arrow cues: an ERP study. Brain Res 1215:123–136. CrossRefGoogle Scholar
  48. 48.
    Hietanen JK, Nummenmaa L, Nyman MJ et al (2006) Automatic attention orienting by social and symbolic cues activates different neural networks: an fMRI study. NeuroImage 33:406–413. CrossRefGoogle Scholar
  49. 49.
    Marotta A, Lupiáñez J, Martella D, Casagrande M (2012) Eye gaze versus arrows as spatial cues: two qualitatively different modes of attentional selection. J Exp Psychol Hum Percept Perform 38:326–335. CrossRefGoogle Scholar
  50. 50.
    Stevens SA, West GL, Al-Aidroos N et al (2008) Testing whether gaze cues and arrow cues produce reflexive or volitional shifts of attention. Psychon Bull Rev 15:1148–1153. CrossRefGoogle Scholar
  51. 51.
    Tipples J (2008) Orienting to counterpredictive gaze and arrow cues. Percept Psychophys 70:77–87. CrossRefGoogle Scholar
  52. 52.
    Tipples J (2002) Eye gaze is not unique: automatic orienting in response to uninformative arrows. Psychon Bull Rev 9:314–318. CrossRefGoogle Scholar
  53. 53.
    Hooker CI, Paller KA, Gitelman DR et al (2003) Brain networks for analyzing eye gaze. Cogn Brain Res 17:406–418. CrossRefGoogle Scholar
  54. 54.
    Bayliss AP, Murphy E, Naughtin CK et al (2013) “Gaze leading”: initiating simulated joint attention influences eye movements and choice behavior. J Exp Psychol Gen 142:76–92. CrossRefGoogle Scholar
  55. 55.
    Dalmaso M, Edwards GS, Bayliss AP (2016) Re-encountering individuals who previously engaged in joint gaze modulates subsequent gaze cueing. J Exp Psychol Learn Mem Cogn 42:271–284. CrossRefGoogle Scholar
  56. 56.
    Dratsch T, Schwartz C, Yanev K et al (2013) Getting a grip on social gaze: control over others’ gaze helps gaze detection in high-functioning autism. J Autism Dev Disord 43:286–300. CrossRefGoogle Scholar
  57. 57.
    Grynszpan O, Nadel J, Martin J-C, Fossati P (2017) The awareness of joint attention: a study using gaze contingent avatars. Interact Stud 18(2):234–253. CrossRefGoogle Scholar
  58. 58.
    Kim K, Mundy P (2012) Joint attention, social-cognition, and recognition memory in adults. Front Hum Neurosci 6:1. Google Scholar
  59. 59.
    Oberwelland E, Schilbach L, Barisic I et al (2016) Look into my eyes: investigating joint attention using interactive eye-tracking and fMRI in a developmental sample. NeuroImage 130:248–260. CrossRefGoogle Scholar
  60. 60.
    Schilbach L, Wilms M, Eickhoff SB et al (2010) Minds made for sharing: initiating joint attention recruits reward-related neurocircuitry. J Cogn Neurosci 22:2702–2715. CrossRefGoogle Scholar
  61. 61.
    Wilms M, Schilbach L, Pfeiffer U et al (2010) It’s in your eyes—using gaze-contingent stimuli to create truly interactive paradigms for social cognitive and affective neuroscience. Soc Cogn Affect Neurosci 5:98. CrossRefGoogle Scholar
  62. 62.
    Daprati E, Franck N, Georgieff N et al (1997) Looking for the agent: an investigation into consciousness of action and self-consciousness in schizophrenic patients. Cognition 65:71–86CrossRefGoogle Scholar
  63. 63.
    Farrer C, Franck N, Georgieff N et al (2003) Modulating the experience of agency: a positron emission tomography study. NeuroImage 18:324–333. CrossRefGoogle Scholar
  64. 64.
    Sperduti M, Delaveau P, Fossati P, Nadel J (2011) Different brain structures related to self- and external-agency attribution: a brief review and meta-analysis. Brain Struct Funct 216:151–157. CrossRefGoogle Scholar
  65. 65.
    Faul F, Erdfelder E, Lang A-G, Buchner A (2007) G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 39:175–191. CrossRefGoogle Scholar
  66. 66.
    Courgeon M, Clavel C (2013) MARC: a framework that features emotion models for facial animation during human–computer interaction. J Multimodal User Interfaces 7:311–319. CrossRefGoogle Scholar
  67. 67.
    Courgeon M, Rautureau G, Martin J-C, Grynszpan O (2014) Joint attention simulation using eye-tracking and virtual humans. IEEE Trans Affect Comput 5:238–250. CrossRefGoogle Scholar
  68. 68.
    Wilson HR, Wilkinson F, Lin L-M, Castillo M (2000) Perception of head orientation. Vis Res 40:459–472. CrossRefGoogle Scholar
  69. 69.
    Horowitz TS, Wolfe JM, Alvarez GA et al (2009) The speed of free will. Q J Exp Psychol 62:2262–2288. CrossRefGoogle Scholar
  70. 70.
    Sakia RM (1992) The Box–Cox transformation technique: a review. Statistician 41:169–178CrossRefGoogle Scholar
  71. 71.
    Al Moubayed S, Edlund J, Beskow J (2012) Taming Mona Lisa: communicating gaze faithfully in 2D and 3D facial projections. ACM Trans Interact Intell Syst 1:25CrossRefGoogle Scholar
  72. 72.
    Senju A, Southgate V, White S, Frith U (2009) Mindblind eyes: an absence of spontaneous theory of mind in Asperger syndrome. Science 325:883–885. CrossRefGoogle Scholar
  73. 73.
    Senju A, Tojo Y, Dairoku H, Hasegawa T (2004) Reflexive orienting in response to eye gaze and an arrow in children with and without autism. J Child Psychol Psychiatry 45:445–458CrossRefGoogle Scholar
  74. 74.
    Kajopoulos J, Wong AHY, Yuen AWC et al (2015) Robot-assisted training of joint attention skills in children diagnosed with autism. In: Tapus A, André E, Martin J-C et al (eds) Social robotics. Springer International Publishing, Cham, pp 296–305CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.CNRS, UMR 7222, Institut des Systèmes Intelligents et de Robotique (ISIR)Université Pierre et Marie CurieParisFrance

Personalised recommendations