Abstract
Once a tutoring system is able to detect students’ emotions, it is not obvious how to change the tutor’s behavior to leverage this emotion detection for the benefit of the student. For instance, if students state that they are excited, then providing harder problems may be appropriate in one case, while providing actions to calm them down so that they can better focus may be the best response in other cases. Both the cognitive and emotional states are important when choosing the tutor’s actions. The purpose of this chapter is to describe the elements necessary for a tutoring system that makes appropriate actions based on a detected affective state. This is broken down into three parts. First we describe several methods for emotion detection. Then we present a study using Wayang Outpost, our math tutor, using sensors to detect the student’s emotion and taking actions based on that emotion. Then we discuss potential actions for the detected emotions. We conclude with future steps needed to improve the actions of tutoring systems in general.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Afzal, S., & Robinson, P. (2011). Natural affect data: collection and annotation. In R. Calvo & S. D’Mello (Eds.), Affective prospecting (Explorations in the learning sciences, instructional systems and performance). New York: Springer.
Amershi, S., Conati, C., & McLaren, H. (2006). Using feature selection and unsupervised clustering to identify affective expressions in educational games. Workshop on Motivational and Affective Issues in ITS, 8th International Conference on Intelligent Tutoring Systems (pp. 21–28).
Ammar, M. B., Neji, M., & Alimi, A. M. (2005). The integration of an emotional system in the intelligent system. The 3rd ACS/IEEE International Conference on Computer Systems and Applications, 2005 (pp. 145–148). Cairo, Egypt.
Arroyo, I., Cooper, D. G., Burleson, W., Woolf, B. P., Muldner, K., & Christopherson, R. (2009). Emotion sensors go to school. Proceeding of the 2009 conference on Artificial Intelligence in Education, (pp. 17–24). Brighton, UK.
Arroyo, I., Woolf, B. P., Royer, J. M., & Tai, M. (2009). Affective gendered learning companions. Proceeding of the 2009 conference on Artificial Intelligence in Education, (pp. 41–48). Brighton, UK.
Conati, C., Chabbal, R., & Maclaren, H. (2003). A study on using biometric sensors for monitoring user emotions in educational games. Proceedings User Modeling Workshop on “Assessing and Adapting to User Attitudes and Effect: Why, When, and How?”, in conjunction with UM’03, 9th International Conference on User Modeling, Pittsburgh, USA.
Cooper, D. G., Arroyo, I., Woolf, B. P., Muldner, K., Burleson, W., & Christopherson, R. (2009). Sensors model student self concept in the classroom. UMAP ‘09: Proceedings of the 17th International Conference on User Modeling, Adaptation, and Personalization, (pp. 30–41). Trento, Italy.
Cooper, D. G., Muldner, K., Arroyo, I., Woolf, B. P., & Burleson, W. (2010). Ranking feature sets for emotion models used in classroom based intelligent tutoring systems. UMAP (pp. 135–146).
Cowie, R., Douglas-Cowie, E., Apolloni, B., Taylor, J., Romano, A., & Fellenz, W. (1999). What a neural net needs to know about emotion words. Computational Intelligence and Applications.
Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W. (2001). Emotion recognition in human-computer interaction. Signal Processing Magazine, IEEE, 18, 32–80.
D’Mello, S., Craig, S., Witherspoon, A., McDaniel, B., & Graesser, A. (2008). Automatic detection of learner’s affect from conversational cues. User Modeling and User-Adapted Interaction, 18, 45–80.
D’Mello, S., Dowell, N., & Graesser, A. (2009) Cohesion relationships in tutorial dialogue as predictors of affective states. Proceeding of the 2009 conference on Artificial Intelligence in Education, (pp. 9–16). Amsterdam: IOS Press.
D’Mello, S. & Graesser, A. (2007). Mind and body: Dialogue and posture for affect detection in learning environments. Proceeding of the 2007 conference on Artificial Intelligence in Education (pp. 161–168). Amsterdam: IOS Press.
Derbali, L. & Frasson, C. (2010). Players’ motivation and EEG waves patterns in a serious game environment. International Conference on Intelligent Tutoring Systems (pp. 297–299).
Dweck, C. S. (2000). Self-theories: Their role in motivation, personality, and development. London, UK: Psychology Press.
Ekman, P., Levenson, R., & Friesen, W. (1983). Autonomic nervous system activity distinguishes among emotions. Science, 221, 1208–1210.
el Kaliouby, R., & Robinson, P. (2004). Real-time inference of complex mental states from facial expressions and head gestures. Proc. Int’l Conf. Computer Vision & Pattern Recognition, 3, 154–173.
Graesser, A., McDaniel, B., Chipman, P., Witherspoon, A., D’Mello, S., & Gholson, B. (2006). Detection of emotions during learning with AutoTutor. Proceedings of the 28th Annual Meetings of the Cognitive Science Society, (pp. 285–290). Mahwah, NJ: Erlbaum.
Heraz, A. & Frasson, C. (2009). Predicting learner answers correctness through brainwaves assesment and emotional dimensions. Proceeding of the 2009 conference on Artificial Intelligence in Education, (pp. 49–56). Amsterdam: IOS Press.
Heraz, A. & Frasson, C. (2010). Theoretical model for interplay between some learning situations and brainwaves. International Conference on Intelligent Tutoring Systems (pp. 337–339).
Mandryk, R. L., Atkins, M. S., & Inkpen, K. M. (2006). A continuous and objective evaluation of emotional experience with interactive play environments. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (p. 1036). New York: ACM Press.
McQuiggan, S., Lee, S., & Lester, J. (2007). Early prediction of student frustration (pp. 698–709). Interaction: Affective Computing and Intelligent.
Mota, S., & Picard, R. W. (2003). Automated posture analysis for detecting learner’s interest level. Computer Vision and Pattern Recognition Workshop, 5, 49.
Muldner, K., Burleson, W., & VanLehn, K. (2010). “Yes”: using tutor and sensor data to predict moments of delight during instructional activities. User Modeling, Adaptation, and Personalization, 6075, 159–170.
Murray, T. & Arroyo, I. (2002). Toward measuring and maintaining the zone of proximal development in adaptive instructional systems. International Conference on Intelligent Tutoring Systems (pp. 133–145).
Nkambou, R. (2006). A framework for affective intelligent tutoring systems. Information Technology Based Higher Education and Training, 2006. ITHET ‘06. 7th International Conference on (pp. nil2–nil8).
Qi, Y. & Picard, R. W. (2002). Context-sensitive Bayesian classifiers and application to mouse pressure pattern classification. Proceedings of 16th International Conference on Pattern Recognition, 2002, vol 3 (pp. 448–451).
Ruvolo, P., Fasel, I. R., & Movellan, J. R. (2008). Auditory mood detection for social and educational robots. ICRA (pp. 3551–3556).
Sarrafzadeh, A., Alexander, S., Dadgostar, F., Fan, C., & Bigdeli, A. (2006). See me, teach me: facial expression and gesture recognition for intelligent tutoring systems. Innovations in Information Technology, 2006, 1–5.
Strauss, M., Reynolds, C., Hughes, S., Park, K., McDarby, G., & Picard, R. (2005). The handwave bluetooth skin conductance sensor. Affective Computing and Intelligent Interaction, (pp. 699–706).
Truong, K. P., & van Leeuwen, D. A. (2007). Automatic discrimination between laughter and speech. Speech Communication, 49, 144–158.
Varlander, S. (2008). The role of students emotions in formal feedback situations. Teaching in Higher Education, 13, 145–156.
Woolf, B., Arroyo, I., Muldner, K., Burleson, W., Cooper, D., Dolan, R., & Christopherson, R. (2010). The effect of motivational learning companions on low achieving students and students with disabilities. International Conference on Intelligent Tutoring Systems (pp. 327–337).
Xiangjie, Q., Zhiliang, W., Jun, Y., & Xiuyan, M. (2006). An affective intelligent tutoring system based on artificial psychology. ICICIC ‘06. First International Conference on Innovative Computing, Information and Control, 2006, 3, 402–405.
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009). A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31, 39–58.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Cooper, D.G., Arroyo, I., Woolf, B.P. (2011). Actionable Affective Processing for Automatic Tutor Interventions. In: Calvo, R., D'Mello, S. (eds) New Perspectives on Affect and Learning Technologies. Explorations in the Learning Sciences, Instructional Systems and Performance Technologies, vol 3. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-9625-1_10
Download citation
DOI: https://doi.org/10.1007/978-1-4419-9625-1_10
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4419-9624-4
Online ISBN: 978-1-4419-9625-1
eBook Packages: Humanities, Social Sciences and LawEducation (R0)