I See What You See: Eye Movements in Real-World Scenes Are Affected by Perceived Direction of Gaze

  • Monica S. Castelhano
  • Mareike Wieth
  • John M. Henderson
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4840)


In this chapter, we report an investigation the influence of the saliency of another person’s direction of gaze on an observer’s eye movements through real-world scenes. Participants’ eye movements were recorded while they viewed a sequence of scene photographs that told a story. A subset of the scenes contained an actor. The actor’s face was highly likely to be fixated, and when it was, the observer’s next saccade was more likely to be toward the object that was the focus of the actor’s gaze than in any other direction. Furthermore, when eye movement patterns did not show an immediate saccade to the focused object, observers were nonetheless more likely to fixate the focused object than a control object within close temporal proximity of fixation on the face. We conclude that during real-world scene perception, observers are sensitive to another’s direction of gaze and use it to help guide their own eye movements.


Joint Attention Salient Object Focus Object Close Temporal Proximity Scene Viewing 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ando, S.: Luminance-induced shift in the apparent direction of gaze. Perception. 31, 657–674 (2002)CrossRefGoogle Scholar
  2. 2.
    Ando, S.: Luminance-induced shift in the apparent direction of gaze. Perception. 33, 1173–1184 (2004)CrossRefGoogle Scholar
  3. 3.
    Argyle, M., Cook, M.: Gaze and mutual gaze. Cambridge University Press, Cambridge, England (1976)Google Scholar
  4. 4.
    Bayliss, A.P., di Pellegrino, G., Tipper, S.P.: Orienting of attention via observed eye gaze is head-centred. Cognition 94, B1–B10 (2004)Google Scholar
  5. 5.
    Baron-Cohen, S.: Mindblindness: An essay on autism and theory of mind. MIT Press, Cambridge, MA (1995)Google Scholar
  6. 6.
    Barresi, J., Moore, C.: Intentional relations and social understanding. Behavioral and Brain Sciences 19, 107–154 (1996)CrossRefGoogle Scholar
  7. 7.
    Brooks, R., Meltzoff, A.N.: The importance of eyes: How infants interpret adult looking behavior. Developmental Psychology 38, 958–966 (2002)CrossRefGoogle Scholar
  8. 8.
    Buswell, G.T.: How people look at pictures. The University of Chicago press, Chicago, IL (1935)Google Scholar
  9. 9.
    Butler, S.C., Caron, A.J., Brooks, R.: Infant understanding of the referential nature of looking. Journal of Cognition and Development 1, 359–377 (2000)CrossRefGoogle Scholar
  10. 10.
    Cline, M.G.: The perception of where a person is looking. American Journal of Psychology 80, 41–50 (1967)CrossRefGoogle Scholar
  11. 11.
    Corkum, V., Moore, C.: The origins of joint visual attention in infants. In: Moore, C., Dunham, P.J. (eds.) Joint attention: Its origins and role in development, pp. 61–83. Lawrence Erlbaum, Hillsdale, NJ (1998)Google Scholar
  12. 12.
    D’Entremont, B., Hains, S., Muir, D.: A demonstration of gaze following in 3- to 6-month olds. Infant Behavior and Development 20, 569–572 (1997)CrossRefGoogle Scholar
  13. 13.
    Driver, J., Davis, G., Ricciardelli, P., Kidd, P., Maxwell, E., Baron-Cohen, S.: Gaze perception triggers visuospatial orienting by adults in a reflexive manner. Visual Cognition 6, 509–540 (1999)CrossRefGoogle Scholar
  14. 14.
    Emery, N.J.: The eyes have it: the neuroethology, function and evolution of social gaze. Neuroscience and Biobehavioral Reviews 24, 581–604 (2000)CrossRefGoogle Scholar
  15. 15.
    Friedman, A.: Framing pictures: the role of knowledge in automatized encoding and memory for gist. Journal of Experimental Psychology: General 108, 316–355 (1979)CrossRefGoogle Scholar
  16. 16.
    Friesen, C.K., Kingstone, A.: The eyes have it!: Reflexive orienting is triggered by nonpredictive gaze. Psychonomic Bulletin and Review 5, 490–495 (1998)CrossRefGoogle Scholar
  17. 17.
    Friesen, C.K., Kingstone, A.: Abrupt onsets and gaze direction cues trigger independent reflexive attentional effects. Cognition 87, B1–B10 (2003)Google Scholar
  18. 18.
    Friesen, C.K., Ristic, J., Kingstone, A.: Attentional effects of counterpredictive gaze and arrow cues. Journal of Experimental Psychology-Human Perception and Performance 30(2), 319–329 (2004)CrossRefGoogle Scholar
  19. 19.
    Gibson, J.J., Pick, A.D.: Perception of another person’s looking behavior. American Journal of Psychology 76, 386–394 (1963)CrossRefGoogle Scholar
  20. 20.
    Henderson, J.M.: Human gaze control in real-world scene perception. Trends in Cognitive Sciences 7, 498–504 (2003)CrossRefGoogle Scholar
  21. 21.
    Henderson, J.M., Brockmole, J.R., Castelhano, M.S., Mack, M.: Image salience versus cognitive control of eye movements in real-world scenes: Evidence from visual search. In: van Gompel, R., Fischer, M., Murray, W., Hill, R. (eds.) Eye movement research: Insights into mind and brain, pp. 537–562. Elsevier, Oxford (2007)CrossRefGoogle Scholar
  22. 22.
    Henderson, J.M., Ferreira, F.: Scene perception for psycholinguists. In: Henderson, J.M., Ferreira, F. (eds.) The interface of language, vision, and action: Eye movements and the visual world, pp. 1–58. Psychology Press, New York (2004)Google Scholar
  23. 23.
    Henderson, J.M., Weeks Jr., P.A., Hollingworth, A.: Effects of semantic consistency on eye movements during scene viewing. Journal of Experimental Psychology: Human Perception and Performance 25, 210–228 (1999)Google Scholar
  24. 24.
    Henderson, J.M., Williams, C.C., Castelhano, M.S., Falk, R.J.: Eye movements and picture processing during recognition. Perception & Psychophysics 65, 725–734 (2003)CrossRefGoogle Scholar
  25. 25.
    Hietanen, J.K., Nummenmaa, L., Nyman, M.J., Parkkola, R., Hamalainen, H.: Automatic attention orienting by social and symbolic cues activates different neural networks: An fMR1 study. Neuroimage 33(1), 406–413 (2006)CrossRefGoogle Scholar
  26. 26.
    Hollingworth, A., Henderson, J.M.: Accurate visual memory for previously attended objects in natural scenes. Journal of Experimental Psychology: Human Perception and Performance 28, 113–136 (2002)Google Scholar
  27. 27.
    Itti, L., Koch, C.: Computational modeling of visual attention. Nature Reviews: Neuroscience 2, 194–203 (2001)CrossRefGoogle Scholar
  28. 28.
    Kingstone, A., Smilek, D., Ristic, J., Friesen, C.K., Eastwood, J.D.: Attention, Researchers! It is time to take a look at the real world. Current Direction in Psychological Science 12(5), 176–184 (2003)CrossRefGoogle Scholar
  29. 29.
    Kellerman, J., Lewis, J., Laird, J.D.: Looking and loving: The effects of mutual gaze on feelings of romantic love. Journal of Research in Personality 23, 145–161 (1989)CrossRefGoogle Scholar
  30. 30.
    Kleinke, C.L.: Gaze and eye contact: A research review. Psychological Bulletin 100, 78–100 (1986)CrossRefGoogle Scholar
  31. 31.
    Langdon, R., Smith, P.: Spatial cueing by social versus nonsocial directional signals. Visual Cognition 12(8), 1497–1527 (2005)CrossRefGoogle Scholar
  32. 32.
    Langton, S.R.H.: The mutual influence of gaze and head orientation of the analysis of social attention direction. The Quarterly Journal of Experimental Psychology A 53, 825–845 (2000)CrossRefGoogle Scholar
  33. 33.
    Langton, S.R.H., Bruce, V.: Reflexive social orienting. Visual Cognition 6, 541–567 (1999)CrossRefGoogle Scholar
  34. 34.
    Langton, S.R.H., O’Donnell, C., Riby, D.M, Ballantyne, C.J.: Gaze cues influence the allocation of attention in natural scene viewing. Quarterly Journal of Experimental Psychology 59 (12), 2056–2064 (2006)CrossRefGoogle Scholar
  35. 35.
    Loftus, G., Mackworth, N.: Cognitive determinants of fixation location during picture viewing. Journal of Experimental Psychology: Human Perception and Performance 4(4), 565–572 (1978)Google Scholar
  36. 36.
    Lungarella, M., Metta, G., Pfeifer, R., Sandini, G.: Developmental robotics: a survey. Connection Science 15, 151–190 (2003)CrossRefGoogle Scholar
  37. 37.
    Mackworth, N.H., Morandi, A.J.: The gaze selects informative details within pictures. Perception & Psychophysics 2, 547–552 (1967)CrossRefGoogle Scholar
  38. 38.
    Mansfield, E.M., Farroni, T., Johnson, M.H.: Does gaze perception facilitate overt orienting? Visual Cognition 10, 7–14 (2000)CrossRefGoogle Scholar
  39. 39.
    Maurer, D.: Infants’ perception of facedness. In: Field, T., Fox, M. (eds.) Social Perception in Infants, Ablex, Norwood, NJ (1995)Google Scholar
  40. 40.
    Morita, A., Yoshikawa, Y., Hosoda, K., Asada, M.: Joint attention with strangers based on generalization through joint attention with caregivers. In: IROS. Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 4, pp. 3744–3749 (2004)Google Scholar
  41. 41.
    Morton, J., Johnson, M.: CONSPEC and CONLEARN: A two-process theory of infant face recognition. Psychological Review 98, 164–181 (1991)CrossRefGoogle Scholar
  42. 42.
    Nagai, Y., Asada, M., Hosoda, K.: A developmental approach accelerates learning of joint attention. In: ICDL 2002. Proceedings of the 2nd International Conference on Development and Learning, pp. 277–282 (2002)Google Scholar
  43. 43.
    Nagai, Y., Hosoda, K., Asada, M.: Joint attention emerges through bootstrap learning. In: Proceedings of the IEEE/RSJ International Conference on Interlligent Robots and Systems, pp. 168–173 (2003)Google Scholar
  44. 44.
    Nelson, W.W., Loftus, G.R.: The functional visual field during picture viewing. Journal of Experimental Psychology: Human Learning and Memory 6(4), 391–399 (1980)Google Scholar
  45. 45.
    Parkhurst, D., Law, K., Niebur, E.: Modeling the role of salience in the allocation of overt visual attention. Vision Research 42, 107–123 (2002)CrossRefGoogle Scholar
  46. 46.
    Perrett, D., Emery, N.J.: Understanding the intentions of others from visual signals: Neuropsychological evidence. Cahiers de Psychologie Cognitive 13, 683–694 (1994)Google Scholar
  47. 47.
    Ricciardelli, P., Briocolo, E., Aglioti, S.M., Chelazzi, L.: My eyes want to look where your eyes are looking: Exploring the tendency to imitate another individual’s gaze. Neuroreport 13, 2259–2264 (2002)CrossRefGoogle Scholar
  48. 48.
    Scaife, M., Bruner, J.S.: The capacity for joint visual attention in the infant. Nature 253, 265–266 (1975)CrossRefGoogle Scholar
  49. 49.
    Scassellati, B.: Theory of mind for a humanoid robot. Autonomous Robots 8(1), 13–24 (2002)CrossRefzbMATHGoogle Scholar
  50. 50.
    Smilek, D., Birmingham, E., Cameron, D., Bischof, W., Kingstone, A.: Cognitive ethology and exploring attention in real-world scenes. Brain Research 1080, 101–119 (2006)CrossRefGoogle Scholar
  51. 51.
    Smilek, D., Eastwood, J.D., Reynolds, M.G., Kingstone, A.: Metacognitive errors in change detection: Missing the gap between lab and life. Consciousness and Cognition 16(1), 52–57 (2007)CrossRefGoogle Scholar
  52. 52.
    Torralba, A., Oliva, A., Castelhano, M.S., Henderson, J.M.: Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. Psychological Review 113, 766–786 (2006)CrossRefGoogle Scholar
  53. 53.
    von Grünau, M., Anston, C.: The detection of gaze direction: A stare-in-the-crowd effect. Perception 24, 1297–1313 (1995)CrossRefGoogle Scholar
  54. 54.
    Yarbus, I.A.: Eye movements and vision. Plenum Press, New York (1967)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Monica S. Castelhano
    • 1
  • Mareike Wieth
    • 2
  • John M. Henderson
    • 3
  1. 1.Queen’s University, Kingston ON K7L 3N6Canada
  2. 2.Albion College, Albion MI 49224USA
  3. 3.University of Edinburgh, Edinburgh EH8 9YLUK

Personalised recommendations