Skip to main content
SpringerLink
Log in

Affect Recognition and Expression in Narrative-Centered Learning Environments

  • Chapter
  • First Online:
New Perspectives on Affect and Learning Technologies

Abstract

While there are many open questions about the role of affect in learning, a key issue is determining how emotion impacts learning in immersive, technology-rich learning environments. We examine these issues within narrative-centered learning environments, which leverage narrative’s motivating features such as compelling plots, engaging characters, and fantastical settings. These environments offer a novel and rich setting for investigating affective reasoning in intelligent support systems that aim to improve student learning, motivation and engagement.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • AndrĂ©, E., & Mueller, M. (2003). Learning affective behavior. In J. Jacko & C. Stephanidis (Eds.), Proceedings of the 10th International Conference on Human-Computer Interaction (pp. 512–516). Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Arroyo, I., Woolf, B., Royer, J., & Tai, M. (2009). Affective gendered learning companions. In Pro-ceedings of the 14th International Conference on Artificial Intelligence in Education (pp. 41–48).

    Google Scholar 

  • Baker, R., Rodrigo, M., & Xolocotzin, U. (2007). The dynamics of affective transitions in simulation problem-solving environments. In Proceedings of the 2nd International Conference on Affective Computing and Intelligent Interactions (pp. 666–677). Lisbon, Portugal.

    Google Scholar 

  • Beal, C., & Lee, H. (2005). Creating a pedagogical model that uses student self reports of motivation and mood to adapt ITS instruction. Workshop on Motivation and Affect in Educational Software, in Conjunction with the 125th International Conference on Artificial Intelligence in Education. Amsterdam, Netherlands.

    Google Scholar 

  • Burleson, W. (2006). Affective learning companions: Strategies for empathetic agents with real-time multimodal affective sensing to foster meta-cognitive and meta-affective approaches to learning, motivation, and perseverance. PhD thesis, Massachusetts Institute of Technology, Cambridge, MA.

    Google Scholar 

  • Chaffar, S., & Frasson, C. (2004). Using an emotional intelligent agent to improve the learner’s performance. Proceedings of the Workshop on Social and Emotional Intelligence in Learning Environments in conjunction with the International Conference on Intelligent Tutoring Systems. Maceio, Brazil.

    Google Scholar 

  • Conati, C. (2002). Probabilistic assessment of user’s emotions in educational games. Applied Artificial Intelligence, 16, 555–575.

    Article  Google Scholar 

  • Conati, C., & Mclaren, H. (2005). Data-driven refinement of a probabilistic model of user affect. In L. Andrissono, P. Brna & A. Mitrovic (Eds.), Proceedings of the 10th International Conference on User Modeling (pp. 40–49). New York: Springer.

    Google Scholar 

  • Craig, S. D., Graesser, A. C., Sullins, J., & Gholson, B. (2004). Affect and learning: An exploratory look into the role of affect in learning with AutoTutor. Journal of Educational Media, 29(3), 241–250.

    Google Scholar 

  • D’Mello, S., Jackson, T., Craig, S., Morgan, B., Chipman, P., White, H., et al. (2008). AutoTutor detects and responds to learners affective and cognitive states. In Proceedings of the Workshop on Emotional and Cognitive issues in ITS in conjunction with the 9th International Conference on Intelligent Tutoring Systems (pp. 31–43).

    Google Scholar 

  • D’Mello, S., Taylor, R. S., & Graesser, A. (2007). Monitoring affective trajectories during complex learning. In Proceedings of the 29th Annual Meeting of the Cognitive Science Society (pp. 203–208). Austin, TX.

    Google Scholar 

  • Davis, M. (1994). Empathy: A social psychological approach. Madison, WI: Brown and Benchmark Publishers.

    Google Scholar 

  • de Vicente, A., & Pain, H. (2002). Informing the detection of the students’ motivational state: An empirical study. In S. Cerri, G. Gouardères & F. Paraguaçu (Eds.), Proceedings of the 6th International Conference on Intelligent Tutoring Systems (pp. 933–943). New York: Springer.

    Google Scholar 

  • Ekman, P., & Friesen, W. (1978). The facial action coding system: A technique for the measurement of facial movement. Palo Alto, CA: Consulting Psychologists Press.

    Google Scholar 

  • Forbes-Riley, K., Rotaru, M., & Litman, D. (2008). The relative impact of student affect on performance models in a spoken dialogue tutoring system. User Modeling and User-Adapted Interaction, 18(1–2), 11–43.

    Article  Google Scholar 

  • Frijda, N. H. (1986). The emotions. New York: Cambridge University Press.

    Google Scholar 

  • Gilleade, K., & Allanson, J. (2003). A toolkit for exploring affective interface adaptation in videogames. In Proceedings of Human–Computer Interaction International (pp. 370–374). Crete, Greece.

    Google Scholar 

  • Graesser, A., Jeon, M., & Dufty, D. (2008). Agent technologies designed to facilitate interactive knowledge construction. Discourse Processes, 45(4–5), 298–322.

    Article  Google Scholar 

  • Graesser, A. C., Person, N., & Magliano, J. (1995). Collaborative dialog patterns in naturalistic ­one-on-one tutoring. Applied Cognitive Psychology, 9, 359–387.

    Article  Google Scholar 

  • Gratch, J., & Marsella, S. (2004). A domain-independent framework for modeling emotion. Journal of Cognitive Systems Research, 5(4), 269–306.

    Article  Google Scholar 

  • Healey, J. (2000). Wearable and automotive systems for affect recognition from physiology. PhD thesis, Massachusetts Institute of Technology, Cambridge, MA.

    Google Scholar 

  • Johnson, L., & Rizzo, P. (2004). Politeness in tutoring dialogs: “Run the factory, that’s what I’d do.” In J. Lester, R. M. Vicari & F. Paraguaçu (Eds.), Proceedings of the 7th International Conference on Intelligent Tutoring Systems (pp. 67–76). New York: Springer.

    Google Scholar 

  • Kort, B., Reilly, R., & Picard, R. (2001). An affective model of interplay between emotions and learning: Reengineering educational pedagogy – building a learning companion. In T. Okamoto, R. Hartley & J. P. Kinsuk (Eds.), Proceedings of IEEE International Conference on Advanced Learning Technology: Issues, Achievements and Challenges (pp. 43–48). Madison, WI: IEEE Computer Society.

    Google Scholar 

  • Lester, J., Towns, S., & FitzGerald, P. (1999). Achieving affective impact: Visual emotive communication in lifelike pedagogical agents. The International Journal of Artificial Intelligence in Education, 10(3–4), 278–291.

    Google Scholar 

  • Linnenbrink, E., & Pintrich, P. (2001). Multiple goals, multiple contexts: The dynamic interplay between personal goals and contextual goal stresses. In S. Volet & S. Jarvela (Eds.), Motivation in learning contexts: Theoretical advances and methodological implications (pp. 251–269). New York: Elsevier.

    Google Scholar 

  • Malone, T., & Lepper, M. (1987). Making learning fun: A taxonomy of intrinsic motivations for learning. In R. Snow & M. Farr (Eds.), Aptitude, learning, and instruction: Cognitive and affective process analyses (Vol. 3, pp. 223–253). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Marsella, S., & Gratch, J. (2009). EMA: A model of emotional dynamics. Journal of Cognitive Systems Research, 10(1), 70–90.

    Article  Google Scholar 

  • McQuiggan, S., Lee, S., & Lester, J. (2007). Early prediction of student frustration. In A. Paiva, R. Prada & R. W. Picard (Eds.), Proceedings of the 2nd International Conference on Affective Computing and Intelligent Interaction (pp. 698–709). Lisbon, Portugal: Springer.

    Google Scholar 

  • McQuiggan, S., & Lester, J. (2007). Modeling and evaluating empathy in embodied companion agents. International Journal of Human Computer Studies, 65(4), 348–360.

    Article  Google Scholar 

  • McQuiggan, S., Mott, B., & Lester, J. (2008). Modeling self-efficacy in intelligent tutoring systems: An inductive approach. User Modeling and User-Adapted Interaction, 18(1–2), 81–123.

    Article  Google Scholar 

  • McQuiggan, S., Robison, J., & Lester, J. (2010). Affective transitions in narrative-centered learning environments. Educational Technology & Society, 13(1), 40–53.

    Google Scholar 

  • McQuiggan, S., Robison, J., Phillips, R., & Lester, J. (2008). Modeling parallel and reactive empathy in virtual agents: An inductive approach. In Proceedings of the 7th International Joint Conference on Autonomous Agents and Multi-Agent Systems (pp. 167–174). Estoril, Portugal: International Foundation for Autonomous Agents and Multiagent Systems.

    Google Scholar 

  • McQuiggan, S., Robison, J., Phillips, R., & Lester, J. (2008). Modeling parallel and reactive empathy in virtual agents: An inductive approach. In L. Padgham, D. Parkes, J. MĂĽller & S. Parsons (Eds.), Proceedings of the 7th International Joint Conference on Autonomous Agents and Multi-Agent Systems (pp. 167–174). Estoril, Portugal: International Foundation for Autonomous Agents and Multiagent Systems.

    Google Scholar 

  • Mekeig, S., & Inlow, M. (1993). Lapses in alertness: Coherence of fluctuations in performance and EEG spectrum. Electroencephalography and Clinical Neurophysiology, 86, 23–25.

    Article  Google Scholar 

  • Meyer, D., & Turner, J. (2007). Scaffolding emotions in classrooms. In P. Schutz & R. Pekrun (Eds.), Emotion in education (pp. 243–258). New York: Elsevier.

    Google Scholar 

  • Paiva, A., Dias, J., Sobral, D., Aylett, R., Woods, S., Hall, L., et al. (2005). Learning by feeling: Evoking empathy with synthetic characters. Applied Artificial Intelligence, 19, 235–266.

    Article  Google Scholar 

  • Pekrun, R. (1992). The impact of emotions on learning and achievement: Toward a theory of cognitive/motivational mediators. Applied Psychology: An International Review, 41(4), 359–376.

    Article  Google Scholar 

  • Perry, N. (1998). Young children’s self-regulated learning and the contexts that support it. Journal of Educational Psychology, 90, 715–729.

    Article  Google Scholar 

  • Picard, R. (1997). Affective computing. Boston: MIT.

    Google Scholar 

  • Picard, R., Papert, S., Bender, W., Blumberg, B., Breazeal, C., Cavallo, D., et al. (2004). Affective learning – a manifesto. BT Technology Journal, 22(4), 153–189.

    Article  Google Scholar 

  • Pope, A., Bogart, E., & Bartolome, D. (1995). Biocybernetic system evaluates indices of operator engagement in automated yask. Biological Psychology, 40, 187–195.

    Article  Google Scholar 

  • Porayska-Pomsta, K., & Pain, H. (2004). Providing cognitive and affective scaffolding through teaching strategies. In Proceedings of the 7th International Conference on Intelligent Tutoring Systems (pp. 77–86). New York: Springer.

    Google Scholar 

  • Prendinger, H., & Ishizuka, M. (2005). The empathic companion: A character-based interface that addresses users’ affective states. Applied Artificial Intelligence, 19, 267–285.

    Article  Google Scholar 

  • Prendinger, H., Mayer, S., Mori, J., & Ishizuka, M. (2003). Persona effect revisited: Using biosignals to measure and reflect the impact of character-based interfaces. In T. Rist, R. Aylett, D. Ballin & J. Rickel (Eds.), Proceedings of the 4th International Working Conference on Intelligent Virtual Agents (Kloster Irsee, Germany, September 15–17) (pp. 283–291). New York: Springer.

    Google Scholar 

  • Robison, J., McQuiggan, S., & Lester, J. (2009). Evaluating the consequences of affective feedback in intelligent tutoring systems. In Proceedings of the International Conference on Affective Computing & Intelligent Interaction (pp. 37–42). Amsterdam, The Netherlands.

    Google Scholar 

  • Rodrigo, M. T., & Baker, R. (2011). Comparing the incidence and persistence of learners’ affect during interactions with different educational software packages. In R. Calvo & S. D’Mello (Eds.), Explorations in the learning sciences, instructional systems and performance technologies. New York: Springer.

    Google Scholar 

  • Rusting, C. (1998). Personality, mood, and cognitive processing of emotional information: Three conceptual frameworks. Psychological Bulletin, 124(2), 165–196.

    Article  Google Scholar 

  • Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.

    Article  Google Scholar 

  • Smith, C., & Lazarus, R. (1990). Emotion and adaptation. In L. A. Pervin (Ed.), Handbook of personality: Theory and research (pp. 609–637). New York: Guildford.

    Google Scholar 

  • Wang, N., Johnson, W. L., Mayer, R., Rizzo, P., Shaw, E., & Collins, H. (2008). The politeness effect: Pedagogical agents and learning outcomes. International Journal of Human Computer Studies, 66, 98–112.

    Article  Google Scholar 

Download references

Acknowledgments

The authors wish to thank the members of the IntelliMedia research lab for their assistance in implementing Crystal Island, Omer Sturlovich and Pavel Turzo for use of their 3D model libraries, Valve Software for access to the Source™ engine and SDK. This research was supported by the National Science Foundation under REC-0632450, DRL-0822200, ­CNS-0540523, IIS-0812291, DRL-1007962. This material is based upon work supported under a National Science Foundation Graduate Research Fellowship. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

Author information

Authors and Affiliations

  1. Department of Computer Science, North Carolina State University, Engineering Building II, 890 Oval Drive, Raleigh, NC, 27695-8206, USA

    James C. Lester

Authors
  1. James C. Lester
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Scott W. McQuiggan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jennifer L. Sabourin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to James C. Lester .

Editor information

Editors and Affiliations

  1. , School of Electrical and Information Eng, The University of Sydney, Sydney, 2006, Australia

    Rafael A. Calvo

  2. , Department of Psychology, The University of Memphis, Memphis, 38152, USA

    Sidney K. D'Mello

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Lester, J.C., McQuiggan, S.W., Sabourin, J.L. (2011). Affect Recognition and Expression in Narrative-Centered Learning Environments. In: Calvo, R., D'Mello, S. (eds) New Perspectives on Affect and Learning Technologies. Explorations in the Learning Sciences, Instructional Systems and Performance Technologies, vol 3. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-9625-1_7

Download citation

Publish with us

Policies and ethics

Search

Navigation