Assessing Students’ Clinical Reasoning Using Gaze and EEG Features

  • Imène JraidiEmail author
  • Asma Ben Khedher
  • Maher Chaouachi
  • Claude Frasson
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11528)


The purpose of this work is to analyze the learners’ visual and brain behaviors during clinical reasoning. An experimental study was conducted to record gaze and EEG data of 15 novice medical students as they interacted with a computer-based learning environment in order to treat medical cases. We describe our approach to track the learners’ reasoning process using the visual scanpath followed during the clinical diagnosis and present our methodology to assess the learners’ brain activity using the engagement and the workload cerebral indexes. We determine which visual and EEG features are related to the students’ performance and analyze the relationship between the students’ visual behavior and brain activity.


Eye tracking Scanpath EEG Engagement Workload Clinical reasoning Learning performance 



This work was supported by NSERC (National Science and Engineering Research Council) and SSHRC (Social and Human Research Council) through the LEADS project. We also thank Issam Tanoubi from the University of Montreal for his collaboration in the design of the Amnesia environment.


  1. 1.
    Jraidi, I., Frasson, C.: Student’s uncertainty modeling through a multimodal sensor-based approach. J. Educ. Technol. Soc. 16, 219–230 (2013)Google Scholar
  2. 2.
    Hou, H.-T.: Integrating cluster and sequential analysis to explore learners’ flow and behavioral patterns in a simulation game with situated-learning context for science courses: a video-based process exploration. Comput. Hum. Behav. 48, 424–435 (2015)CrossRefGoogle Scholar
  3. 3.
    Ben Khedher, A., Jraidi, I., Frasson, C.: Static and dynamic eye movement metrics for students’ performance assessment. Smart Learning Environments 5(1), (2018)
  4. 4.
    D’Mello, S.K., et al.: AutoTutor detects and responds to learners affective and cognitive states. In: Presented at the Workshop on Emotional and Cognitive Issues at the International Conference on Intelligent Tutoring Systems (2008)Google Scholar
  5. 5.
    Pardo, A., Han, F., Ellis, R.A.: Combining university student self-regulated learning indicators and engagement with online learning events to predict academic performance. IEEE Transact. Learn. Technol. 10, 82–92 (2017)CrossRefGoogle Scholar
  6. 6.
    Berka, C., et al.: EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviat. Space Environ. Med. 78, B231–B244 (2007)Google Scholar
  7. 7.
    Ben Khedher, A., Jraidi, I., Frasson, C.: Tracking students’ mental engagement using EEG signals during an interaction with a virtual learning environment. J. Intell. Learn. Syst. Appl. 11, 1–14 (2019)Google Scholar
  8. 8.
    Maynard, O.M., Munafò, M.R., Leonards, U.: Visual attention to health warnings on plain tobacco packaging in adolescent smokers and non-smokers. Addiction 108, 413–419 (2013)CrossRefGoogle Scholar
  9. 9.
    Ben Khedher, A., Jraidi, I., Frasson, C.: What can eye movement patterns reveal about learners’ performance? In: 14th International Conference on Intelligent Tutoring Systems (ITS 2018). LNCS, vol. 10858, pp. 415–417. Springer (2018)Google Scholar
  10. 10.
    Poitras, E.G., Doleck, T., Lajoie, S.P.: Towards detection of learner misconceptions in a medical learning environment: a subgroup discovery approach. Educ. Tech. Res. Dev. 66, 129–145 (2018)CrossRefGoogle Scholar
  11. 11.
    Lajoie, S.P., Naismith, L., Poitras, E., Hong, Y.-J., Cruz-Panesso, I., Ranellucci, J., Mamane, S., Wiseman, J.: Technology-rich tools to support self-regulated learning and performance in medicine. In: Azevedo, R., Aleven, V. (eds.) International Handbook of Metacognition and Learning Technologies. SIHE, vol. 28, pp. 229–242. Springer, New York (2013). Scholar
  12. 12.
    D’Mello, S., Olney, A., Williams, C., Hays, P.: Gaze tutor: a gaze-reactive intelligent tutoring system. Int. J. Hum Comput Stud. 70, 377–398 (2012)CrossRefGoogle Scholar
  13. 13.
    Lallé, S., Conati, C., Carenini, G.: Predicting confusion in information visualization from eye tracking and interaction data. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pp. 2529–2535. AAAI Press (2016)Google Scholar
  14. 14.
    Ben Khedher, A., Jraidi, I., Frasson, C.: Local sequence alignment for scan path similarity assessment. Int. J. Inf. Educ. Technol. 8(7), 482–490 (2018). Scholar
  15. 15.
    Slanzi, G., Balazs, J., Velasquez, J.: Combining eye tracking, pupil dilation and EEG analysis for predicting web users click intention. Inf. Fusion 35, 51–57 (2017). Scholar
  16. 16.
    Brouwer, A.-M., Hogervorst, M.A., Oudejans, B., Ries, A.J., Touryan, J.: EEG and eye tracking signatures of target encoding during structured visual search. Front. Hum. Neurosci. 11, 264 (2017). Scholar
  17. 17.
    El-Abbasy, K., Angelopoulou, A., Towell, T.: Measuring the Engagement of the Learner in a Controlled Environment using Three Different Biosensors. Presented at the 10th International Conference on Computer Supported Education February 8 (2019)Google Scholar
  18. 18.
    Alhasan, K., Chen, L., Chen, F.: An experimental study of learning behaviour in an elearning environment. In: The IEEE 20th International Conference on High Performance Computing and Communications, pp. 1398–1403 (2018)Google Scholar
  19. 19.
    Muldner, K., Burleson, W.: Utilizing sensor data to model students’ creativity in a digital environment. Comput. Hum. Behav. 42, 127–137 (2015)CrossRefGoogle Scholar
  20. 20.
    Makransky, G., Terkildsen, T.S., Mayer, R.E.: Role of subjective and objective measures of cognitive processing during learning in explaining the spatial contiguity effect. Learn. Instr. 61, 23–34 (2019)CrossRefGoogle Scholar
  21. 21.
    Ben Khedher, A., Jraidi, I., Frasson, C.: Tracking students’ analytical reasoning using visual scan paths. In: 17th IEEE International Conference on Advanced Learning Technologies (ICALT), pp. 53–54. IEEE (2017)Google Scholar
  22. 22.
    Ben Khedher, A., Jraidi, I., Frasson, C.: Exploring students’ eye movements to assess learning performance in a serious game. In: EdMedia + Innovate Learning: Association for the Advancement of Computing in Education, pp. 394–401. AACE (2018)Google Scholar
  23. 23.
    Swanson, H.L., O’Connor, J.E., Cooney, J.B.: An information processing analysis of expert and novice teachers’ problem solving. Am. Educ. Res. J. 27, 533–556 (1990)CrossRefGoogle Scholar
  24. 24.
    Chaouachi, M.: Modélisation de l’engagement et de la charge mentale de travail dans les Systèmes Tutoriels Intelligents. Ph.D. thesis, Université de Montréal (2015).
  25. 25.
    Pope, A.T., Bogart, E.H., Bartolome, D.S.: Biocybernetic system evaluates indices of operator engagement in automated task. Biol. Psychol. 40, 187–195 (1995)CrossRefGoogle Scholar
  26. 26.
    Chaouachi, M., Jraidi, I., Frasson, C.: Modeling mental workload using EEG features for intelligent systems. In: User Modeling, Adaption and Personalization, pp. 50–61 (2011)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Imène Jraidi
    • 1
    Email author
  • Asma Ben Khedher
    • 1
  • Maher Chaouachi
    • 2
  • Claude Frasson
    • 1
  1. 1.Department of Computer Science and Operations ResearchUniversity of MontrealMontrealCanada
  2. 2.Department of Educational and Counselling PsychologyMcGill UniversityMontrealCanada

Personalised recommendations