Experimental Brain Research

, Volume 217, Issue 1, pp 125–136 | Cite as

Predictive eye movements in natural vision

  • Mary M. HayhoeEmail author
  • Travis McKinney
  • Kelly Chajka
  • Jeff B. Pelz
Research Article


In the natural world, the brain must handle inherent delays in visual processing. This is a problem particularly during dynamic tasks. A possible solution to visuo-motor delays is prediction of a future state of the environment based on the current state and properties of the environment learned from experience. Prediction is well known to occur in both saccades and pursuit movements and is likely to depend on some kind of internal visual model as the basis for this prediction. However, most evidence comes from controlled laboratory studies using simple paradigms. In this study, we examine eye movements made in the context of demanding natural behavior, while playing squash. We show that prediction is a pervasive component of gaze behavior in this context. We show in addition that these predictive movements are extraordinarily precise and operate continuously in time across multiple trajectories and multiple movements. This suggests that prediction is based on complex dynamic visual models of the way that balls move, accumulated over extensive experience. Since eye, head, arm, and body movements all co-occur, it seems likely that a common internal model of predicted visual state is shared by different effectors to allow flexible coordination patterns. It is generally agreed that internal models are responsible for predicting future sensory state for control of body movements. The present work suggests that model-based prediction is likely to be a pervasive component in natural gaze control as well.


Saccadic eye movements Prediction Internal models Squash Gaze pursuit 



This work was supported by NIH Grant EY05729. The authors wish to acknowledge the assistance in data collection of Jason Droll and also players Eric Hernady and Mithun Mukherjee, as well as Martin Heath and members of the University of Rochester Squash Team.

Supplementary material

Supplementary material 1 (AVI 263 kb)

Supplementary material 2 (AVI 137 kb)

Supplementary material 3 (AVI 219 kb)


  1. Abernethy B (1990) Expertise, visual search, and information pick-up in squash. Perception 19:63–77PubMedCrossRefGoogle Scholar
  2. Aivar P, Hayhoe M, Mruczek R (2005) Role of spatial memory in saccadic targeting in natural tasks. J Vis 5:177–193PubMedCrossRefGoogle Scholar
  3. Assad JA, Maunsell JH (1995) Neuronal correlates of inferred motion in primate posterior parietal cortex. Nature 373(6514):518–521PubMedCrossRefGoogle Scholar
  4. Babcock JS, Pelz JB (2004) Building a lightweight eyetracking headgear. In: ACM SIGCHI eye tracking research & applications symposium, pp 109–114Google Scholar
  5. Bahill AT, LaRitz T (1984) Why can’t batters keep their eye on the ball? Am Sci 72:249–253Google Scholar
  6. Bahill AT, Baldwin DG, Venkateswaran J (2005) Predicting a baseball’s path. Am Sci 93:218–225Google Scholar
  7. Barborica A, Ferrera VP (2003) Estimating invisible target speed from neuronal activity in monkey frontal eye field. Nat Neurosci 6:66–74PubMedCrossRefGoogle Scholar
  8. Barnes G, Collins C (2008) Evidence for a link between the extra-retinal component of random-onset pursuit and the anticipatory pursuit of predictable object motion. J Neurophysiol 100:1135–1146PubMedCrossRefGoogle Scholar
  9. Battaglia PW, Schrater P (2007) Humans trade off viewing time and movement duration to improve visuomotor accuracy in a fast reaching task. J Neurosci 27(26):6984–6994PubMedCrossRefGoogle Scholar
  10. Battaglia PW, Schrater P, Kersten D (2005) Auxiliary object knowledge influences visually-guided interception behavior. In: Proceedings of 2nd symposium APGV, pp 145–152Google Scholar
  11. Becker W, Fuchs A (1985) Prediction in the oculomotor system: smooth pursuit during the transient disappearance of a visual target. Exp Brain Res 57:562–575PubMedCrossRefGoogle Scholar
  12. Blakemore SJ, Frith CD, Wolpert DW (2001) The cerebellum is involved in predicting the sensory consequences of action. Neuroreport 12(9):1879–1884PubMedCrossRefGoogle Scholar
  13. Brouwer A, Knill DC (2007) The role of memory in visually guided reaching. J Vis 7(5):6 (1–12)Google Scholar
  14. Chen-Harris H, Joiner WM, Ethier V, Zee DS, Shadmehr R (2008) Adaptive control of saccades via internal feedback. J Neurosci 28(11):2804–2813Google Scholar
  15. Churchland M, Chou I, Lisberger S (2003) Evidence for object permanence in the smooth pursuit movements of monkeys. J Neurophysiol 90:2205–2218PubMedCrossRefGoogle Scholar
  16. Collins C, Barnes GR (1999) Independent control of head and gaze movements during head-free pursuit in humans. J Physiol 515.1:299–314Google Scholar
  17. Ferrera V, Barborica A (2010) Internally generated error signals in monkey frontal eye field during an inferred motion task. J Neurosci 30(35):11612–11623PubMedCrossRefGoogle Scholar
  18. Findlay JM (1981) Spatial and temporal factors in the predictive generation of saccadic eye movements. Vision Res 21(3):347–354Google Scholar
  19. Flanagan J, Wing A (1997) The role of internal models in motion planning and control: evidence from grip force adjustments during movements of hand-held loads. J Neurosci 17:1519–1528PubMedGoogle Scholar
  20. Fukushima K, Akao T, Shichinohe N, Nitta T, Kurkin S, Fukushima J (2008) Predictive signals in the pursuit area of the monkey frontal eye fields. Prog Brain Res 171:433–440PubMedCrossRefGoogle Scholar
  21. Fukushima K, Kasahara S, Akao T, Kurkin S, Fukushima J, Peterson B (2009) Eye-pursuit and reafferent head movement signals carried by pursuit neurons in the caudal part of the frontal eye fields during head-free pursuit. Cereb Cortex 19:263–275PubMedCrossRefGoogle Scholar
  22. Hayhoe M (2009) Visual memory in motor planning and action. In: Brockmole J (ed) Memory for the visual world. Psychology Press, UK, pp 117–139Google Scholar
  23. Hayhoe M, Ballard D (2005) Eye movements in natural behavior. Trends Cogn Sci 9(4):188–193PubMedCrossRefGoogle Scholar
  24. Hayhoe M, Mennie N, Sullivan B, Gorgos K (2005) The role of internal models and prediction in catching balls. In: Proceedings of AAAI fall symposium seriesGoogle Scholar
  25. Isotalo E, Lasker AG, Zee DS (2005) Cognitive influences on predictive saccadic tracking. Exp Brain Res 165:461–469PubMedCrossRefGoogle Scholar
  26. Itti L, Koch C (2001) Computational modeling of visual attention. Nat Rev Neurosci 2:194–203PubMedCrossRefGoogle Scholar
  27. Johansson RS (1996) Sensory control of dextrous manipulation in humans. In: Wing A, Haggard P, Flanagan J (eds) Hand and brain: the neurophysiology and psychology of hand movements. Academic Press, San Diego, pp 381–414Google Scholar
  28. Joiner W, Shelhamer M (2006) Pursuit and saccadic tracking exhibit a similar dependence on movement preparation time. Exp Brain Res 173:572–586PubMedCrossRefGoogle Scholar
  29. Jovancevic-Misic J, Hayhoe M (2009) Adaptive gaze control in natural environments. J Neurosci 29(19):6234–6238PubMedCrossRefGoogle Scholar
  30. Kawato M (1999) Internal models for motor control and trajectory planning. Curr Opin Neurobiol 9(6):718–727Google Scholar
  31. Koerding K, Wolpert D (2004) Bayesian integration in sensorimotor learning. Nature 427:244–247CrossRefGoogle Scholar
  32. Koerding K, Wolpert D (2006) Bayesian decision theory in sensorimotor control. Trends Cogn Sci 10:319–326CrossRefGoogle Scholar
  33. Kowler E, Martins A, Pavel M (1984) The effect of expectations on slow oculomotor control- IV Anticipatory smooth eye movements depend on prior target motions. Vis Res 24:197–210PubMedCrossRefGoogle Scholar
  34. Krauzlis R (2005) The control of voluntary eye movements: new perspectives. Neuroscientist 11(2):124–137PubMedCrossRefGoogle Scholar
  35. Land M (2004) Eye movements in daily life. In: Chalupa L, Werner J (eds) The visual neurosciences, vol 2. MIT Press, Cambridge, pp 1357–1368Google Scholar
  36. Land MF, Furneaux S (1997) The knowledge base of the oculomotor system. Philos Trans R Soc Lond B Biol Sci 352:1231–1239PubMedCrossRefGoogle Scholar
  37. Land MF, McLeod P (2000) From eye movements to actions: how batsmen hit the ball. Nat Neurosci 3:1340–1345PubMedCrossRefGoogle Scholar
  38. Lopez-Moliner J, Field DT, Wann JP (2007) Interceptive timing: prior knowledge matters. J Vis 7:1–8Google Scholar
  39. Madelain L, Krauzlis RJ (2003) Effects of learning on smooth pursuit during transient disappearance of a visual target. J Neurophysiol 90:972–982PubMedCrossRefGoogle Scholar
  40. Mulliken G, Andersen R (2009) Forward modles and state estimation in parietal cortex. In: Gazzaniga MS (ed) The cognitive neurosciences IV. MIT Press, Cambridge, pp 599–611Google Scholar
  41. Nyffeler T, Rivaud-Pechoux S, Wattiez N, Gaymard N (2008) Involvement of the supplementary eye field in oculomotor predictive behavior. J Cog Neurosci 20(9):1583–1594CrossRefGoogle Scholar
  42. Pierrot-Deseilligny Ch, Müri A, Nyffeler B, Milea D (2005) The role of the human dorsolateral prefrontal cortex in ocular motor behavior. Ann NY Acad Sci 1039:239–251PubMedCrossRefGoogle Scholar
  43. Saunders JA, Knill DC (2003) Humans use continuous visual feedback from the hand to control fast reaching movements. Exp Brain Res 152:341–352PubMedCrossRefGoogle Scholar
  44. Schlicht E, Schrater P (2007) Reach-to-grasp trajectories adjust for uncertainty in the location of visual targets. Exp Brain Res 182:47–57PubMedCrossRefGoogle Scholar
  45. Shadmehr R, Mussa-Ivaldi FA (1994) Adaptive representation of dynamics during learning of a motor task. J Neurosci 14:3208–3224PubMedGoogle Scholar
  46. Shelhamer M, Joiner WM (2003) Saccades exhibit abrupt transition between reactive and predictive, predictive saccade sequence have long-term correlations. J Neurophysiol 90:2763–2769PubMedCrossRefGoogle Scholar
  47. Shichinohe N, Akao T, Kurkin S, Fukushima J, Kaneko C, Fukushima K (2009) Memory and decision making in the frontal cortex during visual motion processing for smooth pursuit eye movements. Neuron 62:717–732PubMedCrossRefGoogle Scholar
  48. Simo LS, Krisky CM, Sweeney JA (2005) Functional neuroanatomy of anticipatory behavior: dissociation between sensory-driven and memory-driven systems. Cereb Cortex 15:1982–1991PubMedCrossRefGoogle Scholar
  49. Sprague N, Ballard D, Robinson A (2007) Modeling embodied visual behaviors. ACM Trans Appl Percept 4(2):11Google Scholar
  50. Tabata H, Muira K, Kawano K (2008) Trial-by-trial updating of the gain in preparation for smooth pursuit eye movement based on past experience in humans. J Neurophysiol 99:747–758PubMedCrossRefGoogle Scholar
  51. Tassinari H, Hudson TE, Landy MS (2006) Combining priors and noisy visual cues in a rapid pointing task. J Neurosci 26:10154–10163PubMedCrossRefGoogle Scholar
  52. Tresilian JR, Houseman JH (2005) Systematic variation in performance of an interceptive action with changes in the temporal constraints. Q J Exp Psychol 58A:447–466Google Scholar
  53. Wolpert D (2007) Probabilistic models in human sensorimotor control. Hum Mov Sci 26:511–524PubMedCrossRefGoogle Scholar
  54. Wolpert D, Miall C, Kawato M (1998) Internal models in the cerebellum. Trends Cogn Sci 2:338–347PubMedCrossRefGoogle Scholar
  55. Xaio Q, Barborica A, Ferrera V (2007) Modulation of visual responses in macaque frontal eye field during covert tracking of invisible targets. Cereb Cortex 17(4):918–928CrossRefGoogle Scholar
  56. Zago M, Bosco G, Maffei V, Iosa M, Ivanenko Y, Lacquaniti F (2004) Internal models of target motion: expected dynamics overrides measured kinematics in timing manual interceptions. J Neurophysiol 91:1620–1634PubMedCrossRefGoogle Scholar
  57. Zago M, McIntyre J, Patrice Senot P, Lacquaniti F (2008) Internal models and prediction of visual gravitational motion. Vis Res 48:1532–1538PubMedCrossRefGoogle Scholar
  58. Zago M, McIntyre J, Patrice Senot P, Lacquaniti F (2009) Visuo-motor coordination and internal models for object interception. Exp Brain Res 192(4):571–604PubMedCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  • Mary M. Hayhoe
    • 1
    Email author
  • Travis McKinney
    • 1
  • Kelly Chajka
    • 2
  • Jeff B. Pelz
    • 3
  1. 1.Center for Perceptual SystemsUniversity of Texas at AustinAustinUSA
  2. 2.School of OptometrySUNYNew YorkUSA
  3. 3.Center for Imaging ScienceRochester Institute of TechnologyRochesterUSA

Personalised recommendations