Skip to main content
SpringerLink
Log in

Automatic gaze-based user-independent detection of mind wandering during computerized reading

  • Published:
User Modeling and User-Adapted Interaction Aims and scope Submit manuscript

Abstract

Mind wandering is a ubiquitous phenomenon where attention involuntarily shifts from task-related thoughts to internal task-unrelated thoughts. Mind wandering can have negative effects on performance; hence, intelligent interfaces that detect mind wandering can improve performance by intervening and restoring attention to the current task. We investigated the use of eye gaze and contextual cues to automatically detect mind wandering during reading with a computer interface. Participants were pseudorandomly probed to report mind wandering while an eye tracker recorded their gaze during the reading task. Supervised machine learning techniques detected positive responses to mind wandering probes from eye gaze and context features in a user-independent fashion. Mind wandering was detected with an accuracy of 72 % (expected accuracy by chance was 60 %) when probed at the end of a page and an accuracy of 67 % (chance was 59 %) when probed in the midst of reading a page. Global gaze features (gaze patterns independent of content, such as fixation durations) were more effective than content-specific local gaze features. An analysis of the features revealed diagnostic patterns of eye gaze behavior during mind wandering: (1) certain types of fixations were longer; (2) reading times were longer than expected; (3) more words were skipped; and (4) there was a larger variability in pupil diameter. Finally, the automatically detected mind wandering rate correlated negatively with measures of learning and transfer even after controlling for prior knowledge, thereby providing evidence of predictive validity. Possible improvements to the detector and applications that utilize the detector are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Baayen, R., Harald, P., Richard, G.: The CELEX Lexical Database (Release 2) (1995)

  • Baer, R.A.: Mindfulness training as a clinical intervention: a conceptual and empirical review. Clin. Psychol. 10(2), 125–143 (2003)

    Google Scholar 

  • Barbuceanu, F., Antonya, C., Duguleana, M., et al.: Attentive User Interface for Interaction Within Virtual Reality Environments Based on Gaze Analysis. Human-Computer Interaction, pp. 204–213. Springer, Interaction Techniques and Environments, New York (2011)

  • Ba, Sileye O., Odobez, J-M.: A Study on Visual Focus of Attention Recognition from Head Pose in a Meeting Room, pp. 75–87. Springer, In Machine Learning for Multimodal Interaction, New York (2006)

  • Bergasa, L.M., Nuevo, J., Sotelo, M.A., et al.: Real-time system for monitoring driver vigilance. IEEE Trans. Intell. Transp. Sys. 7(1), 63–77 (2006)

    Article  Google Scholar 

  • Bixler, R., D’Mello, S.: Toward fully automated person-independent detection of mind wandering. In: User Modeling, Adaptation, and Personalization, pp. 37–48. Springer, New York (2014)

  • Bixler, R., Kopp, K., D’Mello, S.: Evaluation of a Personalized Method for Proactive Mind Wandering Reduction. In: Proceedings of the 4th Workshop on Personalization Approaches for Learning Environments (PALE 2014), pp. 33, (2014)

  • Blanchard, N., Bixler, R., Joyce, T., et al.: Automated physiological-based detection of mind wandering during learning. In: Intelligent Tutoring Systems, pp. 55–60. Springer, New York (2014)

  • Börner, D., Kalz, M., Specht, M.: Lead me gently: facilitating knowledge gain through attention-aware ambient learning displays. Comput. & Educ. 78, 10–19 (2014)

    Article  Google Scholar 

  • Chawla, N.V., Bowyer, K. W., Hall, L.O., et al.: SMOTE: Synthetic Minority Over-Sampling Technique.’ J. Artif. Intell. Res. 16(1), 321–357 (2011)

  • Christoff, K., Gordon, A.M., Smallwood, J., et al.: experience Sampling during fMRI reveals default network and executive system contributions to mind wandering. Proc. Natl. Acad. Sci. 106(21), 8719–8724 (2009)

    Article  Google Scholar 

  • Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20(1), 37–46 (1960)

    Article  Google Scholar 

  • Davidson, R.J.: Empirical explorations of mindfulness: conceptual and methodological conundrums. Emotion 10(1), 8–11 (2010)

    Article  Google Scholar 

  • D’Mello, S., Cobian, J., Hunter, M.: Automatic gaze-based detection of mind wandering during reading. Proceedings of the 6th International Conference on Educational Data Mining, International Educational Data Mining Society, pp. 364–365 (2013)

  • Dong, L., Di, H., Tao, L., et al.: Visual Focus of Attention Recognition in the Ambient Kitchen. In: Computer Vision–ACCV 2009, pp. 548–559. Springer, New York (2010)

  • Dong, Y., Hu, Z., Uchimura, K., et al.: Driver inattention monitoring system for intelligent vehicles: a review. IEEE Trans. Intell. Transp. Syst. 12(2), 596–614 (2011)

    Article  Google Scholar 

  • D’Orazio, T., Leo, M., Guaragnella, C., et al.: A visual approach for driver inattention detection. Pattern Recognit. 40(8), 2341–2355 (2007)

    Article  MATH  Google Scholar 

  • Drummond, J., Litman, D.: In the Zone: towards Detecting Student Zoning Out Using Supervised Machine Learning, pp. 306–308. Springer, New York, In Intelligent Tutoring Systems (2010)

  • Feng, S., D’Mello, S., Graesser, A.C.: Mind wandering while reading easy and difficult texts. Psychon. Bull. & Rev. 20(3), 586–592 (2013)

    Article  Google Scholar 

  • Fletcher, L., Zelinsky, A.: Driver inattention detection based on eye gaze-road event correlation. Int. J. Robot. Res. 28(6), 774–801 (2009)

    Article  Google Scholar 

  • Forster, S., Lavie, N.: Distracted by your mind? Individual differences in distractibility predict mind wandering. J. Exp. Psychol. 40(1), 251–260 (2014)

    Google Scholar 

  • Foulsham, T., Farley, J., Kingstone, A.: Mind wandering in sentence reading: decoupling the link between mind and eye. Can. J. Exp. Psychol./Rev. Can de Psychol Exp 67(1), 51–59 (2013)

    Google Scholar 

  • Frank, D.J., Nara, B., Zavagnin, M., et al.: Validating older adults’ reports of less mind-wandering: an examination of eye movements and dispositional influences. Psychol. Aging 30(2), 266–278 (2015)

    Article  Google Scholar 

  • Franklin, M.S., Broadway, J.M., Mrazek, M.D., et al.: Window to the wandering mind: pupillometry of spontaneous thought while reading. Q. J. Exp. Psychol. 66(12), 2289–2294 (2013)

    Article  Google Scholar 

  • Franklin, M.S., Smallwood, J., Schooler, J.W.: Catching the mind in flight: using behavioral indices to detect mindless reading in real time. Psychon. Bull. & Rev. 18(5), 992–997 (2011)

    Article  Google Scholar 

  • Furugori, S., Yoshizawa, N., Iname, C., et al.: Estimation of driver fatigue by pressure distribution on seat in long term driving. Rev. Automot. Eng. 26(1), 053–058 (2005)

    Google Scholar 

  • Grandchamp, R., Braboszcz, C., Delorme, A.: Oculometric variations during mind wandering. Front. Psychol. 5, 1000–1078 (2014)

    Article  Google Scholar 

  • Gruberger, M., Ben-Simon, E., Levkovitz, Y., et al.: Towards a neuroscience of mind-wandering. Front. Hum. Neurosci. 5, 56–60 (2011)

    Article  Google Scholar 

  • Hall, M.A.: Correlation-based feature selection for machine learning. PhD Thesis, department of Computer Science, University of Waikato, New Zealand. (1999)

  • Hall, M., Frank, E., Holmes, G., et al.: The WEKA data mining software: an update. ACM SIGKDD Explor. Newsl. 11(1), 10–18 (2009)

    Article  Google Scholar 

  • Halpern, D.F., Millis, K., Graesser, A.C., et al.: Operation ARA: a computerized learning game that teaches critical thinking and scientific reasoning. Think. Skills Creat. 7(2), 93–100 (2012)

    Article  Google Scholar 

  • Holmqvist, K., Nyström, M., Andersson, R., et al.: Eye Tracking: a Comprehensive Guide to Methods and Measures. Oxford University Press, Oxford (2011)

    Google Scholar 

  • Horvitz, E., Jacobs, A., Hovel D.: Attention-sensitive alerting’. In: Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, pp. 305–313. Morgan Kaufmann Publishers Inc. (1999)

  • Horvitz, E., Kadie, C., Paek, T., et al.: Models of attention in computing and communication: from principles to applications. Commun. ACM 46(3), 52–59 (2003)

    Article  Google Scholar 

  • Jha, A.P., Krompinger, J., Baime, M.J.: Mindfulness training modifies subsystems of attention. Cogn. Affect. & Behav. Neurosci. 7(2), 109–119 (2007)

    Article  Google Scholar 

  • Just, M.A., Carpenter, P.A.: A theory of reading: from eye fixations to comprehension. Psychol. Rev. 87(4), 329 (1980)

    Article  Google Scholar 

  • Kane, M.J., Brown, L.H., McVay, J.C., et al.: For whom the mind wanders, and when an experience-sampling study of working memory and executive control in daily life. Psychol. Sci. 18(7), 614–621 (2007)

    Article  Google Scholar 

  • Killingsworth, M.A., Gilbert, D.T.: A wandering mind is an unhappy mind. Science 330(6006), 932–932 (2010)

    Article  Google Scholar 

  • Knipling, R.F., Wierwille, W.W.: America, IVHS: Vehicle-Based Drowsy Driver Detection: Current Status and Future Prospects. National Highway Traffic Safety Administration,Office of Crash Avoidance Research (1994)

  • Kopp, K., Bixler, R., D’Mello, S.: Identifying Learning Conditions that Minimize Mind Wandering by Modeling Individual Attributes, pp. 94–103. Springer, New York, In Intelligent Tutoring Systems (2014)

  • Lutz, A., Slagter, H.A., Rawlings, N.B., et al.: Mental training enhances attentional stability: neural and behavioral evidence. J. Neurosci. 29(42), 13418–13427 (2009)

    Article  Google Scholar 

  • Mason, M.F., Norton, M.I., Van Horn, J.D., et al.: Wandering minds: the default network and stimulus-independent thought. Science 315(5810), 393–395 (2007)

    Article  Google Scholar 

  • Matsumoto, Y., Ogasawara, T., Zelinsky, A.: Behavior recognition based on head pose and gaze direction measurement. In: Intelligent Robots and Systems, 2000. (IROS 2000). Proceedings. 2000 IEEE/RSJ International Conference on, IEEE pp. 2127–2132 (2000)

  • Mills, C., D’Mello, S.: In Press. Toward a real-time (Day) dreamcatcher: detecting mind wandering episodes during online reading. In: Proceedings of the 8th International Conference on Educational Data Mining, International Educational Data Mining Society pp. XX–XX

  • Mooneyham, B.W., Schooler, J.W.: The costs and benefits of mind-wandering: a review. Can. J. Exp. Psychol./Rev. Can. de Psychol. Exp. 67(1), 11–18 (2013)

    Google Scholar 

  • Mrazek, M.D., Franklin, M.S., Phillips, D.T., et al.: Mindfulness training improves working memory capacity and gre performance while reducing mind wandering. Psychol. Sci. 24(5), 776–781 (2013)

    Article  Google Scholar 

  • Mrazek, M.D., Smallwood, J., Schooler, J.W.: Mindfulness and mind-wandering: finding convergence through opposing constructs. Emotion 12(3), 442–448 (2012)

    Article  Google Scholar 

  • Muir, M., Conati, C.: An analysis of attention to student—adaptive hints in an educational game. In: Cerri, S.A., Clancey, W.J., Papadourakis, G., et al. (eds.) Intelligent Tutoring Systems, Lecture Notes in Computer Science, pp. 112–122. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  • Navalpakkam, V., Kumar, R., Li, L., et al.: Attention and selection in online choice tasks. In: Masthoff, J., Mobasher, B., Desmarais, M.C., et al. (eds.) User Modeling, Adaptation, and Personalization, Lecture Notes in Computer Science, pp. 200–211. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  • Pham, P., Wang, J.: Attentive learner: improving mobile MOOC learning via implicit heart rate tracking. In: Conati, C., Heffernan, N., Mitrovic, A., et al. (eds.) Artificial Intelligence in Education, pp. 367–376. Springer International Publishing, New York (2015)

    Chapter  Google Scholar 

  • Randall, J.G., Oswald, F.L., Beier, M.E.: Mind-wandering, cognition, and performance: a theory-driven meta-analysis of attention regulation. Psychol. Bull. 140(6), 1411–1431 (2014)

    Article  Google Scholar 

  • Rayner, K.: Eye movements in reading and information processing: 20 years of research. Psychol. Bull. 124(3), 372 (1998)

    Article  Google Scholar 

  • Reichle, E.D., Pollatsek, A., Fisher, D.L., et al.: Toward a model of eye movement control in reading. Psychol. Rev. 105(1), 125 (1998)

    Article  Google Scholar 

  • Reichle, E.D., Reineberg, A.E., Schooler, J.W.: Eye movements during mindless reading. Psychol. Sci. 21(9), 1300–1310 (2010)

    Article  Google Scholar 

  • Robertson, L.H., Manly, T., Andrade, J., et al.: Oops!: performance correlates of everyday attentional failures in traumatic brain injured and normal subjects. Neuropsychologia 35(6), 747–758 (1997)

    Article  Google Scholar 

  • Roda, C., Nabeth, T.: Supporting attention in learning environments: attention support services, and information management. In: Creating New Learning Experiences on a Global Scale, pp. 277–291. Springer, New York (2007)

  • Roda, C., Thomas, J.: Attention aware systems. Encycl. Hum. Comput. Interact. 58, 38 (2006)

    Article  Google Scholar 

  • Schad, D.J., Nuthmann, A., Engbert, R.: Your mind wanders weakly, your mind wanders deeply: objective measures reveal mindless reading at different levels. Cognition 125(2), 179–194 (2012)

    Article  Google Scholar 

  • Schooler, J.W., Reichle, E.D., Halpern D.V.: Zoning out while reading: evidence for dissociations between experience and metaconsciousness. In: Daniel T.L. (ed.) Thinking and Seeing: Visual Metacognition in Adults and Children, pp. 203–226. Cambridge, Mass.: MIT Press (2004)

  • Schooler, J.W., Smallwood, J., Christoff, K., et al.: Meta-awareness, perceptual decoupling and the wandering mind. Trends Cognit. Sci. 15(7), 319–326 (2011)

    Google Scholar 

  • Seibert, P.S., Ellis, H.C.: Irrelevant thoughts, emotional mood states, and cognitive task performance. Mem. & Cognit. 19(5), 507–513 (1991)

    Article  Google Scholar 

  • Selker, T.: Visual attentive interfaces. BT Technol. J. 22(4), 146–150 (2004)

    Article  Google Scholar 

  • Sewell, W., Komogortsev, O.: Real-time eye gaze tracking with an unmodified commodity webcam employing a neural network. In: CHI’10 Extended Abstracts on Human Factors in Computing Systems, ACM pp. 3739–3744, (2010)

  • Smallwood, J.: Mind-wandering while reading: attentional decoupling, mindless reading and the cascade model of inattention. Lang. Linguist. Compass 5(2), 63–77 (2011)

    Article  Google Scholar 

  • Smallwood, J., Beach, E., Schooler, J.W., et al.: Going AWOL in the brain: mind wandering reduces cortical analysis of external events. J. Cognit. Neurosci. 20(3), 458–469 (2008)

    Article  Google Scholar 

  • Smallwood, J., Brown, K.S., Tipper, C., et al.: Pupillometric evidence for the decoupling of attention from perceptual input during offline thought ’ed. Sam Gilbert. PLoS ONE 6(3), e18298 (2011)

    Article  Google Scholar 

  • Smallwood, J., Davies, J.B., Heim, D., et al.: Subjective experience and the attentional lapse: task engagement and disengagement during sustained attention. Consci. Cognit. 13(4), 657–690 (2004)

    Article  Google Scholar 

  • Smallwood, J., Fishman, D.J., Schooler, J.W.: Counting the cost of an absent mind: mind wandering as an underrecognized influence on educational performance. Psychon. Bull. & Rev. 14(2), 230–236 (2007)

    Article  Google Scholar 

  • Smallwood, J., McSpadden, M., Schooler, J.W.: When attention matters: the curious incident of the wandering mind. Mem. & Cognit. 36(6), 1144–1150 (2008)

    Article  Google Scholar 

  • Smallwood, J., Schooler, J.W.: The restless mind. Psychol. Bull. 132(6), 946–958 (2006)

    Article  Google Scholar 

  • Smilek, D., Carriere, J.S.A., Cheyne, J.A.: Out of mind, out of sight: eye blinking as indicator and embodiment of mind wandering. Psychol. Sci. 21(6), 786–789 (2010)

    Article  Google Scholar 

  • Stiefelhagen, R.: Tracking focus of attention in meetings. In: Proceedings of the 4th IEEE International Conference on Multimodal Interfaces, IEEE Computer Society pp. 273, (2002)

  • Stiefelhagen, R., Yang, J., Waibel, A.: Estimating focus of attention based on gaze and sound. In: Proceedings of the 2001 workshop on Perceptive user interfaces, ACM pp. 1–9 (2001)

  • Stiefelhagen, R., Zhu, J.: Head orientation and gaze direction in meetings. In: CHI’02 Extended Abstracts on Human Factors in Computing Systems, ACM pp. 858–859 (2002)

  • Su, Mu-Chun, Hsiung, Chao-Yueh, Huang De-Yuan (2006) ’A Simple Approach to Implementing a System for Monitoring Driver Inattention’. In Systems, Man and Cybernetics, 2006. SMC’06. IEEE International Conference on, IEEE pp. 429–433

  • Sussman, E., Bishop, H., Madnick, B., et al.: Driver inattention and highway safety. Transp. Res. Rec. 1047, 40–48 (1985)

    Google Scholar 

  • Tawari, A., Sivaraman, S., Trivedi, M.M. et al.: Looking-in and looking-out vision for urban intelligent assistance: estimation of driver attentive state and dynamic surround for safe merging and braking. In: Intelligent Vehicles Symposium Proceedings, 2014 IEEE, IEEE pp. 115–120 (2014)

  • Toet, A.: Gaze directed displays as an enabling technology for attention aware systems. Comput. Hum. Behav. 22(4), 615–647 (2006)

    Article  Google Scholar 

  • Torkkola, K., Massey, N., Wood. C.: Driver inattention detection through intelligent analysis of readily available sensors. In: Intelligent Transportation Systems, 2004. Proceedings. The 7th International IEEE Conference on, IEEE pp. 326–331 (2004)

  • Ugurlu, Y.: User attention analysis for e-learning systems using gaze and speech information. In: Information Science, Electronics and Electrical Engineering (ISEEE), 2014 International Conference on, IEEE pp. 1–5 (2014)

  • Uzzaman, S., Steve, J.: The eyes know what you are thinking: eye movements as an objective measure of mind wandering. Consci. Cognit. 20(4), 1882–1886 (2011)

    Article  Google Scholar 

  • Vertegaal, R., Shell, J.S., Chen, D., et al.: Designing for augmented attention: towards a framework for attentive user interfaces. Comput. Hum. Behav. 22(4), 771–789 (2006)

    Article  Google Scholar 

  • Voit, M., Stiefelhagen, R.: Deducing the visual focus of attention from head pose estimation in dynamic multi-view meeting scenarios. In: Proceedings of the 10th International Conference on Multimodal Interfaces, ACM pp. 173–180 (2008)

  • Voßkühler, A., Nordmeier, V., Kuchinke, L., et al.: open gaze and mouse analyzer): open-source software designed to analyze eye and mouse movements in slideshow study design. Behav. Res. Methods 40(4), 1150–1162 (2008)

    Article  Google Scholar 

  • Warren, T., White, S.J., Reichle, E.D.: Investigating the causes of wrap-up effects: evidence from eye movements and e-z reader. Cognition 111(1), 132–137 (2009)

    Article  Google Scholar 

  • Yeo, M.V.M., Li, X., Shen, K., et al.: ’Can SVM be used for automatic EEG detection of drowsiness during car driving? Saf. Sci. 47(1), 115–124 (2009)

    Article  Google Scholar 

  • Yonetani, R., Kawashima, H., Matsuyama, T.: Multi-mode saliency dynamics model for analyzing gaze and attention. Proceedings of the Symposium on Eye Tracking Research and Applications, ETRA ’12, New York, NY, pp. 115–122. ACM, New York (2012)

    Chapter  Google Scholar 

  • Zeidan, F., Johnson, S.K., Diamond, B.J., et al.: Mindfulness meditation improves cognition: evidence of brief mental training. Consci. Cognit. 19(2), 597–605 (2010)

    Article  Google Scholar 

Download references

Acknowledgments

We would first like to thank our collaborators at the University of Memphis for assistance with data collection. We also thank Kris Kopp, Caitlin Mills, Nigel Bosch, Jennifer Neale, Jacqueline Kory, Jonathan Cobian, and Matthew Hunter for help with data collection and analysis. The authors would also like to thank the individuals who reviewed the initial draft of this paper prior to publication. This research was supported by the National Science Foundation (NSF) (ITR 0325428, HCC 0834847, DRL 1235958).

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Notre Dame, Notre Dame, IN, 46556, USA

    Robert Bixler & Sidney D’Mello

Authors
  1. Robert Bixler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sidney D’Mello
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Robert Bixler.

Appendix 1

Appendix 1

1.1 Sample items from learning assessments

1.1.1 Sample item from posttest

Which scenario would be the best to use a double blind study?

(a):

Researchers are testing the relationship between test anxiety and GPA (thematic miss)

(b):

Researchers are testing the effects of lotion on making people look younger (correct answer)

(c):

Researchers are testing the effect of a new soap on bacteria cells (near miss)

(d):

None of the above (distractor)

Table 10 Performance on learning assessments
Full size table

1.1.2 Sample item from transfer test

A sports psychologist tested whether visualizations alone have an impact on muscle tone. Visualization involves picturing an activity (e.g., shooting a basketball) in your mind, but not physically doing the activity. Earlier studies compared physical exercise only to a group that did physical and visualization exercise, but the sports psychologist added two groups to her study: (1) visualization exercise only and (2) a control group that did neither. All participants were first given a standardized test of muscle tone (biceps) by a trained professional. Participants were randomly assigned to one of the four conditions. For four weeks, participants (except the control group) spent 20 minutes every morning performing exercises (physical, visualized, or physical and visualized) designed to strengthen biceps. After four weeks, bicep muscle tone was measured by the same expert, unaware of participants’ condition. Both physical exercise and visualization exercise had significant effects on muscle tone, confirming earlier results that visualization exercise effectively increases muscle tone. The researcher repeated the study and found the same results.

The conclusions being drawn by the researchers are...

(a):

Causal because the researcher measured muscle tone before and after the treatment conditions.

(b):

Correlational because the researcher measured muscle tone before and after the treatment conditions.

(c):

Causal because the participants were randomly assigned to conditions (correct answer)

(d):

Correlational because the participants were randomly assigned to conditions. Performance on the learning assessments is shown in Table 10.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bixler, R., D’Mello, S. Automatic gaze-based user-independent detection of mind wandering during computerized reading. User Model User-Adap Inter 26, 33–68 (2016). https://doi.org/10.1007/s11257-015-9167-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11257-015-9167-1

Keywords

Search

Navigation