Skip to main content
SpringerLink
Log in

Affect-Targeted Interviews for Understanding Student Frustration

  • Conference paper
  • First Online:
Artificial Intelligence in Education (AIED 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12748))

Included in the following conference series:

Abstract

Frustration is a natural part of learning in AIED systems but remains relatively poorly understood. In particular, it remains unclear how students’ perceptions about the learning activity drive their experience of frustration and their subsequent choices during learning. In this paper, we adopt a mixed-methods approach, using automated detectors of affect to signal classroom researchers to interview a specific student at a specific time. We hand-code the interviews using grounded theory, then distill particularly common associations between interview codes and affective patterns. We find common patterns involving student perceptions of difficulty, system helpfulness, and strategic behavior, and study them in greater depth. We find, for instance, that the experience of difficulty produces shifts from engaged concentration to frustration that lead students to adopt a variety of problem-solving strategies. We conclude with thoughts on both how this can influence the future design of AIED systems, and the broader potential uses of data mining-driven interviews in AIED research and development.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Grawemeyer, B., Wollenschlaeger, A., Santos, S.G., Holmes, W., Mavrikis, M., Poulovassilis, A.: Using Graph-based Modelling to explore changes in students’ affective states during exploratory learning tasks. In: Proceedings of the International Conference on Educational Data Mining, pp. 382–383 (2017)

    Google Scholar 

  2. DeFalco, J.A., et al.: Detecting and addressing frustration in a serious game for military training. Int. J. Artif. Intell. Educ. 28(2), 152–193 (2018)

    Google Scholar 

  3. Sottilare, R., Goldberg, B.: Designing adaptive computer-based tutoring systems to accelerate learning and facilitate retention. Cogn. Technol. 17(1), 19–33 (2012)

    Google Scholar 

  4. Forbes-Riley, K., Rotaru, M., Litman, D.: The relative impact of student affect on performance models in a spoken dialogue tutoring system. User Model. User-Adap. Inter. 18, 11–43 (2007)

    Article  Google Scholar 

  5. D’Mello, S.K., Lehman, B., Person, N.: Monitoring affect states during effortful problem solving activities. Int. J. Artif. Intell. Educ. 20(4), 361–389 (2010)

    Google Scholar 

  6. Pardos, Z.A., Baker, R.S., San Pedro, M.O.C.Z., Gowda, S.M., Gowda, S.M.: Affective states and state tests: investigating how affect and engagement during the school year predict end of year learning outcomes. J. Learn. Anal. 1(1), 107–128 (2014)

    Google Scholar 

  7. Liu, Z., Pataranutaporn, V., Ocumpaugh, J., Baker, R.: Sequences of frustration and confusion, and learning. In: Proceedings of the International Conference on Educational Data Mining (2013)

    Google Scholar 

  8. Gee, J.P.: Good video games+ good learning: Collected essays on video games, learning, and literacy. Peter Lang Pub Incorporated, Bern, Switzerland (2007)

    Google Scholar 

  9. Richey, J.E., et al.: More confusion and frustration, better learning: the impact of erroneous examples. Comput. Educ. 139, 173–190 (2019)

    Google Scholar 

  10. Sabourin, J., Rowe, J.P., Mott, B.W., Lester, J.C.: When off-task is on-task: the affective role of off-task behavior in narrative-centered learning environments. In: Biswas, G., Bull, S., Kay, J., Mitrovic, A. (eds.) AIED 2011. LNCS (LNAI), vol. 6738, pp. 534–536. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21869-9_93

    Chapter  Google Scholar 

  11. Baker, R.S., Moore, G.R., Wagner, A.Z., Kalka, J., Salvi, A., Karabinos, M., Yaron, D.: The dynamics between student affect and behavior occurring outside of educational software. In: Proceedings of the International Conference on Affective Computing and Intelligent Interaction, pp. 14–24 (2011)

    Google Scholar 

  12. Valitutti, A.: Action decomposition and frustration regulation in the assisted execution of difficult tasks. In: Proceedings of the AIED 2009 Workshops, Brighton, UK (2009)

    Google Scholar 

  13. Miller, M.K., Mandryk, R.L.: Differentiating in-game frustration from at-game frustration using touch pressure. In: Proceedings of the 2016 ACM International Conference on Interactive Surfaces and Spaces, pp. 225–234 (2016)

    Google Scholar 

  14. McCuaig, J., Pearlstein, M., Judd, A.: Detecting learner frustration: towards mainstream use cases. In: Aleven, V., Kay, J., Mostow, J. (eds.) ITS 2010. LNCS, vol. 6095, pp. 21–30. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13437-1_3

    Chapter  Google Scholar 

  15. Buono, S., Zdravkovic, A., Lazic, M., Woodruff, E.: The effect of emotions on self-regulated-learning (SRL) and story comprehension in emerging readers. Front. Educ. 5, 218 (2020)

    Google Scholar 

  16. Huber, G.P., Power, D.J.: Retrospective reports of strategic-level managers: guidelines for increasing their accuracy. Strateg. Manag. J. 6(2), 171–180 (1985)

    Article  Google Scholar 

  17. D’Mello, S.: A selective meta-analysis on the relative incidence of discrete affective states during learning with technology. J. Educ. Psychol. 105(4), 1082 (2013)

    Article  Google Scholar 

  18. D’Mello, S., Graesser, A.: The half-life of cognitive-affective states during complex learning. Cogn. Emot. 25(7), 1299–1308 (2011)

    Article  Google Scholar 

  19. Botelho, A.F., Baker, R., Ocumpaugh, J., Heffernan, N.: Studying affect dynamics and chronometry using sensor-free detectors. In: Proceedings of the 11th International Conference on Educational Data Mining, pp. 157–166 (2018)

    Google Scholar 

  20. Lazar, J., Bessiere, K., Ceaparu, I., Robinson, J., Shneiderman, B.: Help! I’m lost: user frustration in web navigation. IT Soc. 1(3), 18–26 (2003)

    Google Scholar 

  21. Taylor, B., Dey, A., Siewiorek, D., Smailagic, A.: Using physiological sensors to detect levels of user frustration induced by system delays. In: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 517–528 (2015)

    Google Scholar 

  22. Canossa, A., Drachen, A., Sørensen, J.R.M.: Arrrgghh!!! blending quantitative and qualitative methods to detect player frustration. In: Proceedings of the 6th International Conference on Foundations of Digital Games, pp. 61–68 (2011)

    Google Scholar 

  23. Leelawong, K., Biswas, G.: Designing learning by teaching agents: the Betty’s Brain system. Int. J. Artif. Intell. Educ. 18(3), 181–208 (2008)

    Google Scholar 

  24. Munshi, A., Rajendran, R., Ocumpaugh, J., Biswas, G., Baker, R., Paquette, L.: Modeling learners’ cognitive and affective states to scaffold SRL in open-ended learning environments. In: Proceedings of the 25th Conference on User Modeling, Adaptation, and Personalization, pp. 131–138 (2018)

    Google Scholar 

  25. Jiang, Y., et al.: Expert feature-engineering vs. deep neural networks: which is better for sensor-free affect detection? In: Proceedings of the International Conference on Artificial Intelligence in Education, pp. 198–211. Springer, Cham (2018)

    Google Scholar 

  26. Weston, C., Gandell, T., Beauchamp, J., McAlpine, L., Wiseman, C., Beauchamp, C.: Analyzing interview data: the development and evolution of a coding system. Qual. Sociol. 24(3), 381–400 (2001)

    Article  Google Scholar 

  27. Charmaz, K.: The grounded theory method: an explication and interpretation. Contemp. Field Res. 109–126 (1983)

    Google Scholar 

  28. D’Mello, S., Graesser, A.: Dynamics of affective states during complex learning. Learn. Instr. 22(2), 145–157 (2012)

    Article  Google Scholar 

  29. Winne, P.H., Hadwin, A.F.: Studying as self-regulated engagement in learning. In: Hacker, D., Dunlosky, J.,Hillsdale, G.A. (eds.) Metacognition in Educational Theory and Practice, Erlbaum, pp. 277–3048 (1998)

    Google Scholar 

  30. Andres, J.M.A.L., et al.: Affect sequences and learning in Betty's Brain. In: Proceedings of the 9th International Learning Analytics and Knowledge Conference, pp. 383–390 (2019)

    Google Scholar 

  31. Azevedo, R., Gašević, D.: Analyzing multimodal multichannel data about self-regulated learning with advanced learning technologies: issues and challenges. Comput. Hum. Behav. 96, 207–210 (2019)

    Google Scholar 

  32. Chi, M., VanLehn, K., Litman, D., Jordan, P.: Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies. User Model. User-Adap. Inter. 21(1), 137–180 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Graduate School of Education, University of Pennsylvania, Philadelphia, USA

    Ryan S. Baker, Nidhi Nasiar, Jaclyn L. Ocumpaugh, Stephen Hutt, Juliana M. A. L. Andres, Stefan Slater & Matthew Schofield

  2. Vanderbilt University, Nashville, USA

    Allison Moore, Anabil Munshi & Gautam Biswas

  3. University of Illinois at Urbana-Champaign, Champaign, USA

    Luc Paquette

Authors
  1. Ryan S. Baker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nidhi Nasiar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jaclyn L. Ocumpaugh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stephen Hutt
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Juliana M. A. L. Andres
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Stefan Slater
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Matthew Schofield
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Allison Moore
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Luc Paquette
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Anabil Munshi
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Gautam Biswas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ryan S. Baker .

Editor information

Editors and Affiliations

  1. Technion – Israel Institute of Technology, Haifa, Israel

    Ido Roll

  2. Arizona State University, Tempe, AZ, USA

    Danielle McNamara

  3. Utrecht University, Utrecht, The Netherlands

    Sergey Sosnovsky

  4. London Knowledge Lab, London, UK

    Rose Luckin

  5. University of Leeds, Leeds, UK

    Vania Dimitrova

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Baker, R.S. et al. (2021). Affect-Targeted Interviews for Understanding Student Frustration. In: Roll, I., McNamara, D., Sosnovsky, S., Luckin, R., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2021. Lecture Notes in Computer Science(), vol 12748. Springer, Cham. https://doi.org/10.1007/978-3-030-78292-4_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-78292-4_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-78291-7

  • Online ISBN: 978-3-030-78292-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation