Abstract
We study a new research problem, where an implicit information retrieval query is inferred from eye movements measured when the user is reading, and used to retrieve new documents. In the training phase, the user’s interest is known, and we learn a mapping from how the user looks at a term to the role of the term in the implicit query. Assuming the mapping is universal, that is, the same for all queries in a given domain, we can use it to construct queries even for new topics for which no learning data is available. We constructed a controlled experimental setting to show that when the system has no prior information as to what the user is searching, the eye movements help significantly in the search. This is the case in a proactive search, for instance, where the system monitors the reading behaviour of the user in a new topic. In contrast, during a search or reading session where the set of inspected documents is biased towards being relevant, a stronger strategy is to search for content-wise similar documents than to use the eye movements.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Budzik, J. Hammond, K.J.: User interactions with everyday applications as context for just-in-time information access. In: 5th International Conference on Intelligent User Interfaces, pp. 44–51. ACM, New Orleans (2000)
Claypool, M., Le, P., Wased, M., Brown, D.: Implicit interest indicators. In: 6th International Conference on Intelligent User Interfaces, pp. 33–40. Santa Fe (2001)
Conati C., Mertena C.: Eye-tracking for user modeling in exploratory learning environments: an empirical evaluation. Knowl. Based Syst. 20, 557–574 (2007)
Czerwinski, M., Dumais, S., Robertson, G., Dziadosz, S., Tiernan, S., van Dantzich, M.: Visualizing implicit queries for information management and retrieval. In: SIGCHI Conference on Human Factors in Computing Systems, pp. 560–567. Pittsburgh (1999)
Dhanjal, C., Gunn, S.R., Shawe-Taylor, J.: Sparse feature extraction using generalised partial least squares. In: IEEE International Workshop on Machine Learning for Signal Processing, pp. 27–32. Maynooth (2006)
DiCiccio T.J., Efron B.: Bootstrap confidence intervals. Stat. Sci. 11, 189–228 (1996)
Dumais, S., Cutrell, E., Sarin, R., Horvitz, E.: Implicit queries (IQ) for contextualized search. In: 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, p. 594. ACM, Sheffield (2004)
Farquhar J.D.R., Hardoon D.R., Meng H., Shawe-Taylor J., Szedmak S.: Two view learning: SVM-2K, theory and practice. In: Weiss, Y., Schölkopf, B., Platt, J. (eds) Advances in Neural Information Processing Systems, pp. 355–362. MIT Press, Cambridge (2006)
Fono, D., Vertegaal, R.: EyeWindows: evaluation of eye-controlled zooming windows for focus selection. In: SIGCHI Conference on Human Factors in Computing Systems, pp. 151–160. ACM Press, Portland (2005)
Fox S., Karnawat K., Mydland M., Dumais S., White T.: Evaluating implicit measures to improve web search. ACM Trans. Inf. Syst. 23, 147–168 (2005)
Granaas M.M., McKay T.D., Laham R.D., Hurt L.D., Juola J.F.: Reading moving text on a CRT screen. Hum. Factors 26, 97–104 (1984)
Hardoon D.R., Szedmak S.R., Shawe-Taylor J.R.: Canonical correlation analysis: an overview with application to learning methods. Neural Comput. 16, 2639–2664 (2004)
Hardoon, D.R., Ajanki, A., Puolamaki, K., Shawe-Taylor, J., Kaski, S.: Information retrieval by inferring implicit queries from eye retrieval by inferring implicit queries from eye movements. In: 11th Intelligence and Statistics. San Juan. Electronic proceedings at http://www.stat.umn.edu/~aistat/proceedings/start.htm (2007)
Howard D.L., Crosby M.E.: Snapshots from the eye: towards strategies for viewing biblographic citations. In: Savendy, G., Smith, M.J. (eds) Human-Computer Interaction: Software and Hardware Interfaces, pp. 488–493. Elsevier, Amsterdam (1993)
Joachims, T., Granka, L., Pan, B., Hembrooke, H., Gay, G.: Accurately interpreting clickthrough data as implicit feedback. In: 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 154–161. ACM, Salvador (2005)
Kelly D., Teevan J.: Implicit feedback for inferring user preference: a bibliography. ACM SIGIR Forum 37, 18–28 (2003)
King, L.: The relationship between scene and eye movements. In: 35th Annual Hawaii International Conference on System Sciences, p. 136b. Big Island (2002)
Levy-Schoen A., O’Regan K.: The control of eye movements in reading. In: Kolers, P.A., Wrolstad, M.E., Bouma, H. (eds) Processing of visible language, pp. 7–36. Plenum Press, New York (1979)
Maglio P.P., Campbell C.S.: Attentive agents. Commun. ACM 46, 47–51 (2003)
Maglio, P.P., Barrett, R., Campbell, C.S., Selker, T.: SUITOR: an attentive information system. In: International Conference on Intelligent User Interfaces, pp. 169–176. ACM, New Orleans (2000)
Meng, H., Hardoon, D.R., Shawe-Taylor, J., Szedmak, S.: Generic object recognition by distinct features combination in machine learning. In: 17th Annual Symposium on Electronic Imaging, pp. 90–98. San Jose (2005)
Puolamäki, K., Kaski, S. (eds.): Proceedings of the NIPS 2005 Workshop on Machine Learning for Implicit Feedback and User Modeling. Helsinki University of Technology, Otaniemi. http://www.cis.hut.fi/inips2005/ (2006)
Puolamäki, K., Salojärvi, J., Savia, E., Simola, J. Kaski, S.: Combining eye movements and collaborative filtering for proactive information retrieval. In: 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 146–153. Salvador, (2005)
Rafter, R., Smyth, B.: Passive profiling from server logs in an online recruitment environment. In: Workshop on Intelligent Techniques for Web Personalization, pp. 35–41. Seattle (2001)
Rayner K.: Eye movements in reading and information processing: 20 years of research. Psychol. Bull. 124, 372–422 (1998)
Rosipal R., Trejo L.J.: Kernel partial least squares regression in reproducing kernel Hilbert space. J. Mach. Learn. Res. 2, 97–123 (2001)
Salojärvi, J., Kojo, I., Simola, J., Kaski, S.: Can relevance be inferred from eye movements in information retrieval?. In: Workshop on Self-Organizing Maps. Kyushu Institute of Technology, Hibikino, pp. 261–266. Kitakyushu (2003)
Salojärvi J., Puolamäki K., Kaski S.: Implicit relevance feedback from eye movements. In: Duch, W., Kacprzyk, J., Oja, E., Zadrozny, S. (eds) Artificial Neural Networks: Biological Inspirations—ICANN 2005, Lecture Notes in Computer Science 3696, pp. 513–518. Springer, Berlin (2005)
Salojärvi, J., Puolamäki, K., Simola, J., Kovanen, L., Kojo, I. Kaski, S.: Inferring relevance from eye movements: feature extraction. Technical Report A82, Helsinki University of Technology, Publications in Computer and Information Science. http://www.cis.hut.fi/eyechallenge2005/ (2005b)
Salton G., McGill M.J.: Introduction to Modern Information Retrieval. McGraw-Hill, New York (1983)
Turpin, A., Scholer, F.: User performance versus precision measures for simple search tasks. In: 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 11–18. Seattle (2006)
Ward D.J., MacKay D.J.C.: Fast hands-free writing by gaze direction. Nature 418, 838 (2002)
Author information
Authors and Affiliations
Corresponding author
Additional information
The authors appear in alphabetical order.
Rights and permissions
About this article
Cite this article
Ajanki, A., Hardoon, D.R., Kaski, S. et al. Can eyes reveal interest? Implicit queries from gaze patterns. User Model User-Adap Inter 19, 307–339 (2009). https://doi.org/10.1007/s11257-009-9066-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11257-009-9066-4