Recognizing Cognitive Activities Through Eye Tracking

  • Sara Moraleda
  • Javier de Lope AsiainEmail author
  • Manuel Graña
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11486)


Eye detection and tracking is usually performed by using specific devices that allow to determine the pupil position in many different situations. We propose to use these techniques for recognizing cognitive activities that a potential user is carrying out in front of a computer. We use the images captured by a conventional web camera located over the computer display. Those image are processed and, after the face and facial landmarks are found, the user gaze is analyzed and the ethogram and several statistics associated to the eyes and gaze destination are computed. They are used for determining what is doing the user from a set of predefined activities.


Neuroethology Activities recognition Eye tracking Screen-based eye tracker Non-invasive techniques 



This work has been partially supported by FEDER funds through MINECO project TIN2017-85827-P.


  1. 1.
    Hof, R.: How do you Google? New eye tracking study reveals huge changes. Accessed 11 Feb 2019
  2. 2.
    Tsai, M.-J., Hou, H.-T., Lai, M.-L., Liu, W.-Y., Yang, F.-Y.: Visual attention for solving multiple-choice science problem: an eye-tracking analysis. Comput. Educ. 58, 375–385 (2012)CrossRefGoogle Scholar
  3. 3.
    Weigel, P., Raab, M., Wollny, R.: Tactical decision making in team sports: a model of cognitive processes. Int. J. Sports Sci. 5(49), 128–138 (2015)Google Scholar
  4. 4.
    Gidlöf, K., Wallin, A., Dewhurst, R., Holmqvist, K.: Using eye tracking to trace a cognitive process: Gaze behaviour during decision making in a natural environment. J. Eye Mov. Res. 6(1), 1–14 (2013)Google Scholar
  5. 5.
    Al-Rahayfeh, A., Faezipour, M.: Eye tracking and head movement detection: a state-of-art survey. IEEE J. Transl. Eng. Health Med. 1, 1–12 (2013)CrossRefGoogle Scholar
  6. 6.
    Krafka, K., et al.: Eye tracking for everyone. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2176–2184 (2016)Google Scholar
  7. 7.
    Ferhat, O., Vilariño, F.: Low cost eye tracking: the current panorama. Comput. Intell. Neurosci. 2016, 1–14 (2016)CrossRefGoogle Scholar
  8. 8.
    Obaidellah, U., Al Haek, M., Cheng, P.C.-H.: A survey on the usage of eye-tracking in computer programming. ACM Comput. Surv. 51(1), 1–58 (2018)CrossRefGoogle Scholar
  9. 9.
    Baluja, S., Pomerleau, D.: Non-intrusive gaze tracking using artificial neural networks. Technical report CMU-CS-94-102. Carnegie Mellon University. Pittsburgh, Pennsylvania (1994)Google Scholar
  10. 10.
    Sewell, W., Komogortsev, O.: Real-time eye gaze tracking with anunmodified commodity webcam employing a neural network. In: ACM CHI Conference Human Factors in Computing Systems, pp. 3739–3744. Atlanta, Georgia (2010)Google Scholar
  11. 11.
    Vora, S., Rangesh, A., Trivedi, M.M.: On generalizing driver gaze zone estimation using convolutional neural networks. In: IEEE Intelligent Vehicles Symposium, pp. 849–854. Redondo Beach, California (2017)Google Scholar
  12. 12.
    Naqvi, R.A., Arsalan, M., Batchuluun, G., Yoon, H.S., Park, K.R.: Deep learning-based gaze detection systems for automobile drivers using a NIR camera sensor. Sensors 18, 1–34 (2018)CrossRefGoogle Scholar
  13. 13.
    Blakley, B.W., Chan, L.: Methods considerations for nystagmography. J. Otolaryngol. Head Neck Surg. 44, 25 (2015)CrossRefGoogle Scholar
  14. 14.
    Cassin, B., Solomon, S.: Dictionary of Eye Terminology. Triad Publishing Company, Gainesville (1990)Google Scholar
  15. 15.
    Dalal, N., Triggs, B.: Histogram of Oriented Gradients for Human Detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 886–893 (2005)Google Scholar
  16. 16.
    Kazemi, V., Sullivan, J.: One millisecond face alignment with anensemble of regression trees. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1867–1874. Ohio, USA (2014)Google Scholar
  17. 17.
    Soukupová, T., Čech, J.: Real-time eye blink detection using facial landmarks. In: 21st Computer Vision Winter Workshop (2016)Google Scholar
  18. 18.
    Anderson, D.J., Perona, P.: Toward a science of computational ethology. Neuron 84, 18–31 (2014)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Sara Moraleda
    • 1
  • Javier de Lope Asiain
    • 1
    Email author
  • Manuel Graña
    • 2
  1. 1.Computational Cognitive Robotics Group, Department of Artificial IntelligenceUniversidad Politécnica de Madrid (UPM)MadridSpain
  2. 2.Computational Intelligence GroupUniversity of the Basque Country (UPV/EHU)San SebastiánSpain

Personalised recommendations