Abstract
This research predicted behavioral disengagement using quitting behaviors while learning from instructional texts. Supervised machine learning algorithms were used to predict if students would quit an upcoming text by analyzing reading behaviors observed in previous texts. Behavioral disengagement (quitting) at any point during the text was predicted with an accuracy of 76.5% (48% above chance), before students even began engaging with the text. We also predicted if a student would quit reading on the first page of a text or continue reading past the first page with an accuracy of 88.5% (29% above chance), as well as if students would quit sometime after the first page with an accuracy of 81.4% (51% greater than chance). Both actual quits and predicted quits were significantly related to learning, which provides some evidence for the predictive validity of our model. Implications and future work related to ITSs are also discussed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kelly, K.M., Heffernan, N., D’Mello, S., Namais, J., Strain, A.: Added Teacher-Created Motiational Video to an ITS. In: The Twenty-Sixth International FLAIRS Conference, pp. 503–508. AAAI Press, Menlo Park (2013)
Calvo, R.A., D’Mello, S.: Affect detection: An interdisciplinary review of models, methods, and their applications. IEEE Trans. on Affect. Comput. 1, 18–37 (2010)
Pekrun, R., Linnenbrink-Garcia, L.: Academic emotions and student engagement. In: Handbook of Research on Student Engagement, pp. 259–282. Springer (2012)
Baker, R.S., Corbett, A.T., Koedinger, K.R.: Detecting student misuse of intelligent tutoring systems. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 531–540. Springer, Heidelberg (2004)
Beck, J.E.: Using response times to model student disengagement. In: Proceedings of the ITS 2004 Workshop on Social and Emotional Intelligence in Learning Environments, pp. 13–20 (2004)
D’Mello, S., Cobian, J., Hunter, M.: Automatic Gaze-Based Detection of Mind Wandering during Reading. In: Proceedings of the 6th International Conference on Educational Data Mining, pp. 364–365. International Educational Data Mining Society (2013)
Forbes-Riley, K., Litman, D.: When does disengagement correlate with learning in spoken dialog computer tutoring? In: Biswas, G., Bull, S., Kay, J., Mitrovic, A. (eds.) AIED 2011. LNCS, vol. 6738, pp. 81–89. Springer, Heidelberg (2011)
Rowe, J.P., McQuiggan, S.W., Robison, J.L., Lester, J.C.: Off-Task Behavior in Narrative-Centered Learning Environments. In: AIED, pp. 99–106 (2009)
Jang, H.: Supporting students’ motivation, engagement, and learning during an uninteresting activity. UMAP 2012 100, 798 (2008)
Baker, R.S.J.: Modeling and understanding students’ off-task behavior in intelligent tutoring systems. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1059–1068 (2007)
Baker, R.S.J.d., et al.: Adapting to when students game an intelligent tutoring system. In: Ikeda, M., Ashley, K.D., Chan, T.-W. (eds.) ITS 2006. LNCS, vol. 4053, pp. 392–401. Springer, Heidelberg (2006)
Cocea, M., Weibelzahl, S.: Eliciting motivation knowledge from log files towards motivation diagnosis for Adaptive Systems. In: Conati, C., McCoy, K., Paliouras, G. (eds.) UM 2007. LNCS (LNAI), vol. 4511, pp. 197–206. Springer, Heidelberg (2007)
Cocea, M., Weibelzahl, S.: Disengagement Detection in Online Learning: Validation Studies and Perspectives. IEEE Trans. Learn. Technol. 4, 114–124 (2011)
Baker, R.S.J., Rossi, L.M.: Assessing the Disengaged Behaviors of Learners. Des. Recomm. Intell. Tutoring Syst. 155 (2013)
Fredricks, J.A., Blumenfeld, P.C., Paris, A.H.: School engagement: Potential of the concept, state of the evidence. Rev. Educ. Res. 74, 59–109 (2004)
D’Mello, S., Graesser, A.C.: The half-life of cognitive-affective states during complex learning. Cogn. Emot. 25, 1299–1308 (2011)
Brusilovsky, P., Schwarz, E., Weber, G.: ELM-ART: An intelligent tutoring system on World Wide Web. In: Lesgold, A.M., Frasson, C., Gauthier, G. (eds.) ITS 1996. LNCS, vol. 1086, pp. 261–269. Springer, Heidelberg (1996)
McNamara, D.S., Levinstein, I.B., Boonthum, C.: iSTART: Interactive strategy training for active reading and thinking. Behav. Res. Methods Instrum. Comput. 36, 222–233 (2004)
Millis, K., Forsyth, C., Butler, H., Wallace, P., Graesser, A.C., Halpern, D.: Operation ARIES!: A serious game for teaching scientific inquiry. Serious Games Edutainment Appl., 169–195 (2011)
Rosenthal, R., Rosnow, R.L.: Essentials of behavioral analysis: Methods and data analysis. McGraw-Hill, New York (1984)
Graesser, A.C., Person, N.K.: Question asking during tutoring. Am. Educ. Res. J. 31, 104–137 (1994)
Davis, J., Goadrich, M.: The relationship between Precision-Recall and ROC curves. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 233–240 (2006)
Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20, 37–46 (1960)
Baker, R.S.J., De Carvalho, A.: Labeling student behavior faster and more precisely with text replays. In: Proceedings of the 1st International Conference on Educational Data Mining, pp. 38–47 (2008)
Jang, H.: Supporting students’ motivation, engagement, and learning during an uninteresting activity. J. Educ. Psychol. 100, 798 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Mills, C., Bosch, N., Graesser, A., D’Mello, S. (2014). To Quit or Not to Quit: Predicting Future Behavioral Disengagement from Reading Patterns. In: Trausan-Matu, S., Boyer, K.E., Crosby, M., Panourgia, K. (eds) Intelligent Tutoring Systems. ITS 2014. Lecture Notes in Computer Science, vol 8474. Springer, Cham. https://doi.org/10.1007/978-3-319-07221-0_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-07221-0_3
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07220-3
Online ISBN: 978-3-319-07221-0
eBook Packages: Computer ScienceComputer Science (R0)