pp 1-15 | Cite as

A Practical Guide to Functional Magnetic Resonance Imaging with Simultaneous Eye Tracking for Cognitive Neuroimaging Research

  • Michael Hanke
  • Sebastiaan Mathôt
  • Eduard Ort
  • Norman Peitek
  • Jörg Stadler
  • Adina Wagner
Part of the Neuromethods book series


The simultaneous acquisition of functional magnetic resonance imaging (fMRI) with in-scanner eye tracking promises to combine the advantages of full-brain coverage of brain activity measurements with a fast and unobtrusive capture of eye movement behavior and attentional deployment. Despite its applicability to a wide variety of research questions, ranging from investigations of gaze control and attention guidance to the use of eye movement events as a response modality for gaze-contingent fMRI experiments, only few studies employ this kind of data acquisition. In this chapter we identify technical challenges, describe all necessary components and procedures for conducting such a study, and give practical advice on how these can be integrated in a common MRI laboratory setup. The chapter concludes with notes on the analysis of such datasets and summarizes key data properties and their implications of a joint analysis of fMRI and eye tracking data.


Functional magnetic resonance imaging Eye tracking Hardware Gaze-contingent stimulation Eye movement event detection Pupillometry 


  1. 1.
    Ort E et al (2019) The role of proactive and reactive cognitive control for target selection in multiple-target search. BioRxiv:559500.
  2. 2.
    Geringswald F, Baumgartner F, Pollmann S (2012) Simulated loss of foveal vision eliminates visual search advantage in repeated displays. Front Hum Neurosci 6:134. Scholar
  3. 3.
    Hansen DW, Ji Q (2010) In the eye of the beholder: a survey of models for eyes and gaze. IEEE Trans Pattern Anal Mach Intell 32:478–500Google Scholar
  4. 4.
    Van der Stigchel S, Meeter M, Theeuwes J (2006) Eye movement trajectories and what they tell us. Neurosci Biobehav Rev 30(5):666–679Google Scholar
  5. 5.
    Theeuwes J, Belopolsky A, Olivers CN (2009) Interactions between working memory, attention and eye movements. Acta Psychol 132(2):106–114Google Scholar
  6. 6.
    Van Gerven M et al (2004) Memory load and the cognitive pupillary response in aging. Psychophysiology 41:167–174Google Scholar
  7. 7.
    Mathôt S et al (2015) Large pupils predict goal-driven eye movements. J Exp Psychol Gen 144(3):513–521. Scholar
  8. 8.
    Lean Y, Shan F (2011) Brief review on physiological and biochemical evaluations of human mental workload. Hum Factors Man 22(3):177–187Google Scholar
  9. 9.
    Gordon PC et al (2006) Similarity-based interference during language comprehension: evidence from eye tracking during reading. Exp Psychol Learn Mem Cogn 32(6):1304–1321Google Scholar
  10. 10.
    Rayner K (2009) Eye movements and attention in reading, scene perception, and visual search. Q J Exp Psychol (Hove) 62(8):1457–1506Google Scholar
  11. 11.
    Peitek N et al (2018) Simultaneous measurement of program comprehension with fMRI and eye tracking: a case study. In: Proceedings of the international symposium of empirical software engineering and measurement (ESEM). ACM, New York, pp 24:1–24:10Google Scholar
  12. 12.
    Henderson JM (1997) Transsaccadic memory and integration during real-world object perception. Psychol Sci 8(1):51–55Google Scholar
  13. 13.
    Mathôt S, Theeuwes J (2011) Visual attention and stability. Philos Trans R Soc Lond Ser B Biol Sci 366(1564):516–527. Scholar
  14. 14.
    Hanke M et al (2016) A studyforrest extension, simultaneous fMRI and eye gaze recordings during prolonged natural stimulation. Sci Data 3:160092Google Scholar
  15. 15.
    Holmqvist K et al (2011) Eye tracking: a comprehensive guide to methods and measures. OUP, OxfordGoogle Scholar
  16. 16.
    Richlan F et al (2013) A new high-speed visual stimulation method for gaze-contingent eye movement and brain activity studies. Front Syst Neurosci 7:24. Scholar
  17. 17.
    Andersson R et al (2017) One algorithm to rule them all? An evaluation and discussion of ten eye movement event-detection algorithms. Behav Res Methods 49(2):616–637. Scholar
  18. 18.
    Dalmaijer ES, Mathôt S, Van der Stigchel S (2013) PyGaze: an open-source, crossplatform toolbox for minimal-effort programming of eyetracking experiments. Behav Res Methods 46:913–921. Scholar
  19. 19.
    Mathôt S, Schreij D, Theeuwes J (2012) OpenSesame: an open-source, graphical experiment builder for the social sciences. Behav Res Methods 44:314–324. Scholar
  20. 20.
    Dar AH, Wagner AS, Hanke M (2019) REMoDNaV: robust eye movement detection for natural viewing. bioRxiv:619254.
  21. 21.
    Bahill AT, Clark MR, Stark L (1975) The main sequence, a tool for studying human eye movements. Math Biosci 24(3–4):191–204. Scholar
  22. 22.
    Mathôt S (2018) Pupillometry: psychology, physiology, and function. J Cogn 1(1):1–16. Scholar
  23. 23.
    Goldwater BC (1972) Psychological significance of pupillary movements. Psychol Bull 77(5):340–355. Scholar
  24. 24.
    Kret ME, Sjak-Shie EE (2018) Preprocessing pupil size data: guidelines and code. Behav Res Methods:1–7.
  25. 25.
    Mathôt S et al (2018) Safe and sensible preprocessing and baseline correction of pupil-size data. Behav Res Methods:1–13.
  26. 26.
    Hershman R, Henik A, Cohen N (2018) A novel blink detection method based on pupillometry noise. Behav Res Methods 50(1):107–114. Scholar
  27. 27.
    Mathôt S (2013) A simple way to reconstruct pupil size during eye blinks.
  28. 28.
    Mathôt S, Melmi JB, Castet E (2015) Intrasaccadic perception triggers pupillary constriction. PeerJ 3(e1150):1–16. Scholar
  29. 29.
    Brisson J et al (2013) Pupil diameter measurement errors as a function of gaze direction in corneal reflection eyetrackers. Behav Res Methods 45(4):1322–1331. Scholar
  30. 30.
    Gagl B, Hawelka S, Hutzler F (2011) Systematic influence of gaze position on pupil size measurement: analysis and correction. Behav Res Methods 43(4):1171–1181. Scholar
  31. 31.
    Hayes TR, Petrov AA (2016) Mapping and correcting the influence of gaze position on pupil size measurements. Behav Res Methods 48(2):510–527. Scholar
  32. 32.
    Wierda SM, van Rijn H, Taatgen NA, Martens S (2012) Pupil dilation deconvolution reveals the dynamics of attention at high temporal resolution. Proc Natl Acad Sci 109(22):8456–8460. Scholar
  33. 33.
    Hoeks B, Levelt WJ (1993) Pupillary dilation as a measure of attention: a quantitative system analysis. Behav Res Methods Instrum Comput 25(1):16–26. Scholar
  34. 34.
    Joshi S et al (2016) Relationships between pupil diameter and neuronal activity in the locus Coeruleus, Colliculi, and cingulate cortex. Neuron 89(1):221–234. Scholar

Copyright information

© Springer Science+Business Media, LLC 2019

Authors and Affiliations

  • Michael Hanke
    • 1
    • 2
  • Sebastiaan Mathôt
    • 3
  • Eduard Ort
    • 4
  • Norman Peitek
    • 5
  • Jörg Stadler
    • 5
  • Adina Wagner
    • 6
  1. 1.Institute of Neuroscience and Medicine, Brain and Behaviour (INM-7)Research Centre JülichJülichGermany
  2. 2.Medical Faculty, Institute of Systems NeuroscienceHeinrich Heine University DüsseldorfDüsseldorfGermany
  3. 3.Department of PsychologyUniversity of GroningenGroningenThe Netherlands
  4. 4.Department of Experimental and Applied Psychology, Institute for Brain and BehaviourVrije Universiteit AmsterdamAmsterdamThe Netherlands
  5. 5.Leibniz Institute for NeurobiologyMagdeburgGermany
  6. 6.Institute of Neuroscience and Medicine, Brain and Behaviour (INM-7)Research Centre JülichJülichGermany

Personalised recommendations