Abstract
The human eye plays an essential role in information acquisition from external world, and much of our contemporary information technology relies on visual processing. The eye-mind hypothesis suggests that human attention is connected to where our eyes are looking (Just and Carpenter 1980). Taken together with the continual movement of our eyes and the limited area of high-acuity human vision, eye-tracking methods are considered to offer theoretically reliable measures of visual attention and search task activities. We first briefly review cognitive factors of interest to information search and the “traditional” methods of their measurement. We then present examples of eye tracking tools and how they capture data before examining how eye-tracking data has been used to assess select cognitive factors in information search.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Oxford dictionary, https://www.lexico.com/en/definition/relevance.
References
Ajanki A, Hardoon DR, Kaski S, Puolamäki K, Shawe-Taylor J (2009) Can eyes reveal interest? implicit queries from gaze patterns. User Model User-Adap Inter 19(4):307–339. https://doi.org/10.1007/s11257-009-9066-4
Anderson LW, Krathwohl DR, Bloom BS. (2001) A taxonomy for learning, teaching, and assessing: a revision of Bloom’s taxonomy of educational objectives. Longman
Arapakis I, Athanasakos K, Jose JM (2010) A comparison of general vs personalised affective models for the prediction of topical relevance. In: Proceedings of the 33rd international ACM SIGIR conference on research and development in information retrieval. ACM, New York, NY, USA, pp 371–378. https://doi.org/10.1145/1835449.1835512
Arguello J, Choi B (2019) The effects of working memory, perceptual speed, and inhibition in aggregated search. ACM Trans Inf Syst 37(3):36:1–36:34. https://doi.org/10.1145/3322128
Belkin NJ (2016) People, interacting with information. SIGIR Forum 49(2):13–27. https://doi.org/10.1145/2888422.2888424
Bhattacharya N, Gwizdka J (2019) Measuring learning during search: differences in interactions, eye-gaze, and semantic similarity to expert knowledge. In: Proceedings of the 2019 conference on human information interaction and retrieval. ACM, New York, NY, USA, pp 63–71. https://doi.org/10.1145/3295750.3298926
Biedert R, Hees J, Dengel A, Buscher G (2012) A robust realtime reading-skimming classifier. In: Proceedings of the symposium on eye tracking research and applications. ACM, New York, NY, USA, pp 123–130. https://doi.org/10.1145/2168556.2168575
Borlund P (2003) The concept of relevance in IR. J Am Soc Inform Sci Technol 54(10):913–925. https://doi.org/10.1002/asi.10286
Brennan K, Kelly D, Arguello J (2014) The effect of cognitive abilities on information search for tasks of varying levels of complexity. In: Proceedings of the 5th information interaction in context symposium. ACM, New York, NY, USA, pp 165–174. https://doi.org/10.1145/2637002.2637022
Buettner R, Sauer S, Maier C, Eckhardt A (2015) Towards ex ante prediction of user performance: a novel NeuroIS methodology based on real-time measurement of mental effort. In: 2015 48th Hawaii international conference on system sciences. Presented at the 2015 48th Hawaii international conference on system sciences, pp 533–542. https://doi.org/10.1109/HICSS.2015.70
Buscher G, Dengel A, Biedert R, Elst LV (2012) Attentive documents: eye tracking as implicit feedback for information retrieval and beyond. ACM Trans Interact Intell Syst 1(2):9:1–9:30. https://doi.org/10.1145/2070719.2070722
Campbell CS, Maglio PP (2001) A robust algorithm for reading detection. In: Proceedings of the 2001 workshop on perceptive user interfaces. ACM, New York, NY, USA, pp 1–7. https://doi.org/10.1145/971478.971503
Carroll JB (1993) Human cognitive abilities: a survey of factor analytic studies. Cambridge University Press, Cambridge
Chen S, Epps J, Ruiz N, Chen F (2011) Eye activity as a measure of human mental effort in HCI. In: Proceedings of the 15th international conference on intelligent user interfaces. ACM, New York, NY, USA, pp 315–318. https://doi.org/10.1145/1943403.1943454
Chen SY, Magoulas GD, Dimakopoulos D (2005) A flexible interface design for web directories to accommodate different cognitive styles. J Am Soc Inf Sci Technol 56(1):70–83. https://doi.org/10.1002/asi.20103
Chin J, Anderson E, Chin C-L, Fu W-T (2015) Age differences in information search: an exploration-exploitation tradeoff model. Proc Hum Factors Ergon Soc Annu Meet 59(1):85–89. https://doi.org/10.1177/1541931215591018
Cole MJ, Gwizdka J, Liu C, Belkin NJ, Zhang X (2013) Inferring user knowledge level from eye movement patterns. Inf Process Manag 49(5):1075–1091. https://doi.org/10.1016/j.ipm.2012.08.004
Cole MJ, Gwizdka J, Liu C, Bierig R, Belkin NJ, Zhang X (2011) Task and user effects on reading patterns in information search. Interact Comput 23(4):346–362. https://doi.org/10.1016/j.intcom.2011.04.007
Cole M, Liu J, Belkin N, Bierig R, Gwizdka J, Liu C et al (2009) Usefulness as the criterion for evaluation of interactive information retrieval. In: Third workshop on human computer interaction and information retrieval (HCIR ’09). Presented at the HCIR’09, pp 1–4
Cosijn E, Ingwersen P (2000) Dimensions of relevance. Inf Process Manag 36(4):533–550. https://doi.org/10.1016/S0306-4573(99)00072-2
Delabarre EB (1898) A method of recording eye-movements. Am J Psychol 9(4):572–574. https://doi.org/10.2307/1412191
Dillon A (2004) Designing usable electronic text. 2nd edition, Boca Raton: CRC Press
Dillon A (2016) Theory for design: the case of reading. In: Sonnenwald DH (ed) Theory development in the information sciences. University of Texas Press, Austin, TX, USA, pp 222–238
Dinet J, Bastien JMC, Kitajima M (2010) What, where and how are young people looking for in a search engine results page?: impact of typographical cues and prior domain knowledge. In: Proceedings of the 22nd conference on L’Interaction Homme-Machine. ACM, New York, NY, USA, pp 105–112. https://doi.org/10.1145/1941007.1941022
Eickhoff C, Gwizdka J, Hauff C, He J (2017) Introduction to the special issue on search as learning. Inf Retr J 20(5):399–402. https://doi.org/10.1007/s10791-017-9315-9
Findlay JM (2003) Visual selection, covert attention and eye movements. In: Active vision: the psychology of looking and seeing. Oxford University Press. Accessed 6 Feb 2019
Findlay JJM, Gilchrist ID (2003) Active vision: the psychology of looking and seeing. Oxford University Press, Incorporated
Ford N, Miller D, Moss N (2005) Web search strategies and human individual differences: cognitive and demographic factors, Internet attitudes, and approaches. J Am Soc Inf Sci Technol 56(7):741–756. https://doi.org/10.1002/asi.20168
Freund L, Gwizdka J, Hansen P, Kando N, Rieh SY (2013) From searching to learning. In: Agosti M, Fuhr N, Toms E, Vakkari P (eds) Evaluation methodologies in information retrieval, vol 13441, pp 102–105. http://drops.dagstuhl.de/opus/volltexte/2014/4433
Freund L, He J, Gwizdka J, Kando N, Hansen P, Rieh SY (2014) Searching As Learning (SAL) workshop 2014. In: Proceedings of the 5th information interaction in context symposium. ACM, New York, NY, USA, pp 7–7. https://doi.org/10.1145/2637002.2643203
Frias-Martinez E, Chen SY, Liu X (2009) Evaluation of a personalized digital library based on cognitive styles: adaptivity vs. adaptability. Int J Inf Manage 29(1):48–56. https://doi.org/10.1016/j.ijinfomgt.2008.01.012
Goodale P, David Clough P, Fernando S, Ford N, Stevenson M (2014) Cognitive styles within an exploratory search system for digital libraries. J Doc 70(6):970–996. https://doi.org/10.1108/JD-03-2014-0045
de Greef T, Lafeber H, van Oostendorp H, Lindenberg J (2009) Eye movement as indicators of mental workload to trigger adaptive automation. In: Schmorrow DD, Estabrooke IV Grootjen M (eds) Foundations of augmented cognition. Neuroergonomics and operational neuroscience. Springer, Berlin, Heidelberg, pp 219–228. http://link.springer.com.ezproxy.lib.utexas.edu/chapter/10.1007/978-3-642-02812-0_26. Accessed 16 Dec 2013
Guo X, Li R, Alm C, Yu Q, Pelz J, Shi P, Haake A (2014) Infusing perceptual expertise and domain knowledge into a human-centered image retrieval system: a prototype application. In: Proceedings of the symposium on eye tracking research and applications. ACM, New York, NY, USA, pp 275–278. https://doi.org/10.1145/2578153.2578196
Gwizdka J (2009) What a difference a tag cloud makes: effects of tasks and cognitive abilities on search results interface use. Inf Res 14(4). http://informationr.net/ir/14-4/paper414.html. Accessed 15 Sept 2013
Gwizdka J (2010) Distribution of cognitive load in web search. J Am Soc Inf Sci Technol 61(11):2167–2187. https://doi.org/10.1002/asi.21385
Gwizdka J (2013a) Effects of working memory capacity on users’ search effort. In: Proceedings of the international conference on multimedia, interaction, design and innovation. ACM, New York, NY, USA, pp 11:1–11:8. https://doi.org/10.1145/2500342.2500358
Gwizdka J (2013b) Searchers switch tactics under increased mental load. In: Proceedings of the 76th ASIS&T annual meeting: beyond the cloud: rethinking information boundaries, vol 50. American Society for Information Science, Silver Springs, MD, USA, pp 146:1–146:3. http://dl.acm.org/citation.cfm?id=2655780.2655926. Accessed 15 Dec 2017
Gwizdka J (2014) Characterizing relevance with eye-tracking measures. In: Proceedings of the 5th information interaction in context symposium. ACM, New York, NY, USA, pp 58–67. https://doi.org/10.1145/2637002.2637011
Gwizdka J (2017) I can and so I search more: effects of memory span on search behavior. In: Proceedings of the 2017 conference on conference human information interaction and retrieval. ACM, New York, NY, USA, pp 341–344. https://doi.org/10.1145/3020165.3022148
Gwizdka J (2018) Inferring web page relevance using pupillometry and single channel EEG. In: Davis FD, Riedl R, vom Brocke J, Léger P-M, Randolph AB (eds) Information systems and neuroscience. Presented at the NeuroIS 2017. Springer, Cham, Switzerland, pp 175–183. https://doi.org/10.1007/978-3-319-67431-5_20
Gwizdka J, Hansen P, Hauff C, He J, Kando N (2016) Search As Learning (SAL) workshop 2016. In: Proceedings of the 39th international ACM SIGIR conference on research and development in information retrieval. ACM, New York, NY, USA, pp 1249–1250. https://doi.org/10.1145/2911451.2917766
Gwizdka J, Hosseini R, Cole M, Wang S (2017) Temporal dynamics of eye-tracking and EEG during reading and relevance decisions. J Assoc Inf Sci Technol 68(10):2299–2312. https://doi.org/10.1002/asi.23904
Gwizdka J, Zhang Y (2015) Differences in eye-tracking measures between visits and revisits to relevant and irrelevant web pages. In: Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval. ACM, New York, NY, USA, pp 811–814. https://doi.org/10.1145/2766462.2767795
Hansen P, Rieh SY (2016) Editorial recent advances on searching as learning: an introduction to the special issue. J Inf Sci 42(1):3–6. https://doi.org/10.1177/0165551515614473
Heinstrőm J (2003) Five personality dimensions and their influence on information behaviour. Inf Res 9(1):05.2.2008
Holmqvist K, Nyström M, Andersson R, Dewhurst R, Jarodzka H, van de Weijer J (2011) Eye tracking: a comprehensive guide to methods and measures. Oxford University Press, Oxford, UK
Huey EB (1898) Preliminary experiments in the physiology and psychology of reading. Am J Psychol 9(4):575–586. https://doi.org/10.2307/1412192
Jacucci G, Barral O, Daee P, Wenzel M, Serim B, Ruotsalo T et al (2019) Integrating neurophysiologic relevance feedback in intent modeling for information retrieval. J Assoc Inf Sci Technol 70(9):917–930. https://doi.org/10.1002/asi.24161
Just MA, Carpenter PA (1980) A theory of reading: from eye fixations to comprehension. Psychol Rev 87(4):329–354. https://doi.org/10.1037/0033-295X.87.4.329
Kahneman D, Beatty J (1966) Pupil diameter and load on memory. Science 154(3756):1583–1585. https://doi.org/10.1126/science.154.3756.1583
Karanam S, Jorge-Botana G, Olmos R, van Oostendorp H (2017a) The role of domain knowledge in cognitive modeling of information search. Inf Retr J:1–24. https://doi.org/10.1007/s10791-017-9308-8
Karanam S, van Oostendorp H, Sanchiz M, Chevalier A, Chin J, Fu W-T (2017b) Cognitive modeling of age-related differences in information search behavior. J Assoc Inf Sci Technol 68(10):2328–2337. https://doi.org/10.1002/asi.23893
Karanam S, van Oostendorp H (2016a) Modeling individual differences in information search. In: Proceedings of the 8th Indian conference on human computer interaction. ACM, New York, NY, USA, pp 12–23. https://doi.org/10.1145/3014362.3014363
Karanam S, van Oostendorp H (2016b) Age-related differences in the content of search queries when reformulating. In: Proceedings of the 2016 CHI conference on human factors in computing systems. ACM, New York, NY, USA, pp 5720–5730. https://doi.org/10.1145/2858036.2858444
Karanam S, van Oostendorp H (2017) Age-related effects of task difficulty on the semantic relevance of query reformulations. In: Bernhaupt R, Dalvi G, Joshi A, Balkrishan DK, O’Neill J, Winckler M (eds) Human-computer interaction—INTERACT 2017. Springer, pp 77–96
Komogortsev OV, Khan JI (2008) Eye movement prediction by Kalman filter with integrated linear horizontal oculomotor plant mechanical model. In: Proceedings of the 2008 symposium on eye tracking research & applications. ACM, New York, NY, USA, pp 229–236. https://doi.org/10.1145/1344471.1344525
Krugman HE (1964) Some applications of pupil measurement. JMR, J Mark Res (pre-1986) 1(000004):15
Liu C, Liu J, Cole M, Belkin NJ, Zhang X (2012) Task difficulty and domain knowledge effects on information search behaviors. Proc Am Soc Inf Sci Technol 49:1–10. https://doi.org/10.1002/meet.14504901142
Lopatovska I (2011) Emotional correlates of information retrieval behaviors. In: 2011 IEEE Workshop on Affective Computational Intelligence (WACI). Presented at the 2011 IEEE Workshop on Affective Computational Intelligence (WACI), pp 1–7. https://doi.org/10.1109/WACI.2011.5953145
Lopatovska I (2014) Toward a model of emotions and mood in the online information search process. J Assoc Inf Sci Technol 65(9):1775–1793. https://doi.org/10.1002/asi.23078
Lu Q, Zhang J, Chen J, Li J (2018) Predicting readers’ domain knowledge based on eye-tracking measures. Electron Libr 36(6):1027–1042. https://doi.org/10.1108/EL-05-2017-0108
Marshall SP (2002) The Index of Cognitive Activity: measuring cognitive workload. In: Proceedings of the 2002 IEEE 7th conference on human factors and power plants, 2002. Presented at the Proceedings of the 2002 IEEE 7th conference on human factors and power plants, 2002, IEEE, pp 7-5-7–9. https://doi.org/10.1109/HFPP.2002.1042860
Moray N (1979) Models and measures of mental workload. In Moray N (ed) Mental workload: its theory and measurement. Springer US, Boston, MA, pp 13–21. https://doi.org/10.1007/978-1-4757-0884-4_2
Oliveira FTP, Aula A, Russell DM (2009) Discriminating the relevance of web search results with measures of pupil size. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, NY, USA, pp 2209–2212. https://doi.org/10.1145/1518701.1519038
Onorati F, Barbieri R, Mauri M, Russo V, Mainardi L (2013) Characterization of affective states by pupillary dynamics and autonomic correlates. Front Neuroeng 6:9. https://doi.org/10.3389/fneng.2013.00009
Palmquist RA, Kim K-S (2000) Cognitive style and on-line database search experience as predictors of web search performance. J Am Soc Inf Sci 51(6):558–566. https://doi.org/10.1002/(SICI)1097-4571(2000)51:6%3c558:AID-ASI7%3e3.0.CO;2-9
Pirolli P (2009) Powers of 10: modeling complex information-seeking systems at multiple scales. Computer 42(3):33–40. https://doi.org/10.1109/MC.2009.94
Pirolli P, Card S (1999) Information foraging. Psychol Rev 106(4):643–675
Preuschoff K, Hart BM, Einhäuser W (2011) Pupil dilation signals surprise: evidence for noradrenaline’s role in decision making. Front Decis Neurosci 5:115. https://doi.org/10.3389/fnins.2011.00115
Rayner K (1975) Parafoveal identification during a fixation in reading. Acta Psychol 39(4):271–281. https://doi.org/10.1016/0001-6918(75)90011-6
Rayner K (1998) Eye movements in reading and information processing: 20 years of research. Psychol Bull 124(3):372–422. https://doi.org/10.1037/0033-2909.124.3.372
Rayner K (2009) Eye movements and attention in reading, scene perception, and visual search. Q J Exp Psychol 62(8):1457–1506. https://doi.org/10.1080/17470210902816461
Rayner K, Pollatsek A, Ashby J, Clifton C Jr (2011) Psychology of reading, 2nd edn. Psychology Press
Reichle ED, Pollatsek A, Rayner K (2006) EZ Reader: a cognitive-control, serial-attention model of eye-movement behavior during reading. Model Eye-Mov Control Read 7(1):4–22
Reichle ED, Rayner K, Pollatsek A (2003) The E-Z Reader model of eye-movement control in reading: comparisons to other models. Behav Brain Sci 26(04):445–476. https://doi.org/10.1017/S0140525X03000104
Rieh SY, Collins-Thompson K, Hansen P, Lee H-J (2016) Towards searching as a learning process: a review of current perspectives and future directions. J Inf Sci 42(1):19–34. https://doi.org/10.1177/0165551515615841
Salvucci DD, Goldberg JH (2000) Identifying fixations and saccades in eye-tracking protocols. In: Proceedings of the 2000 symposium on eye tracking research & applications. ACM, New York, NY, USA, pp 71–78. https://doi.org/10.1145/355017.355028
Saracevic T (1996) Relevance reconsidered. Information science: integration in perspectives. In: Proceedings of the second conference on conceptions of library and information science, Copenhagen, Denmark, pp 201–218
Saracevic T (2007) Relevance: a review of the literature and a framework for thinking on the notion in information science. Part II: nature and manifestations of relevance. J Am Soc Inf Sci Technol 58(13):1915–1933. https://doi.org/10.1002/asi.20682
Saracevic T (2016) The notion of relevance in information science: everybody knows what relevance is. But, what is it really? Morgan & Claypool Publishers
Schotter ER, Angele B, Rayner K (2011) Parafoveal processing in reading. Atten Percept Psychophys 74(1):5–35. https://doi.org/10.3758/s13414-011-0219-2
Simon HA (1956) Rational choice and the structure of the environment. Psychol Rev 63(2):129–138. https://doi.org/10.1037/h0042769
Sonnenwald DH (2016) Theory development in the information sciences. University of Texas Press, Austin, TX, USA
Sweller J, Ayres P, Kalyuga S (2011) Measuring cognitive load. In: Cognitive load theory. Springer New York, New York, NY, pp 71–85. http://www.springerlink.com/content/h7221p68q5658303/. Accessed 23 Jan 2012
Tang R, Solomon P (1998) Toward an understanding of the dynamics of relevance judgment: an analysis of one person’s search behavior. Inf Process Manag 34(2):237–256. https://doi.org/10.1016/S0306-4573(97)00081-2
Tatler BW (2007) The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. J Vis 7(14):4–4. https://doi.org/10.1167/7.14.4
Veltman JA, Gaillard AWK (1998) Physiological workload reactions to increasing levels of task difficulty. Ergonomics 41(5):656–669. https://doi.org/10.1080/001401398186829
van der Wel P, van Steenbergen H (2018) Pupil dilation as an index of effort in cognitive control tasks: a review. Psychon Bull Rev 25(6):2005–2015. https://doi.org/10.3758/s13423-018-1432-y
Wildemuth BM (2004) The effects of domain knowledge on search tactic formulation. J Am Soc Inf Sci Technol 55(3):246–258
Acknowledgements
The authors wish to thank the research support from the 2011 Google Faculty Research Award to Jacek Gwizdka, Institute for Museum and Library Studies (IMLS) Career Development Grant to Jacek Gwizdka #RE-04-11-0062-11A, and the Portuguese Foundation for Science and Technology and the Digital Media Program at University of Texas at Austin. Andrew Dillon wishes to thank the V.M. Daniel Professorship for research support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Gwizdka, J., Dillon, A. (2020). Eye-Tracking as a Method for Enhancing Research on Information Search. In: Fu, W., van Oostendorp, H. (eds) Understanding and Improving Information Search. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-030-38825-6_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-38825-6_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-38824-9
Online ISBN: 978-3-030-38825-6
eBook Packages: Computer ScienceComputer Science (R0)