Comparing Player Responses to Choice-Based Interactive Narratives Using Facial Expression Analysis

  • John T. MurrayEmail author
  • Raquel Robinson
  • Michael Mateas
  • Noah Wardrip-Fruin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11318)


Interactive storytelling balances the desire to create dynamic, engaging experiences around characters and situations with the practical considerations of the cost of producing content. We describe a method for assessing player experience by analyzing player facial expressions following key content events in The Wolf Among Us by Telltale Games. Two metrics, engagement and valence, are extracted for six participants who play the first episode of the game. An analysis of the variance and distribution of responses relative to emotionally charged content events and choices suggests that content is designed around events that serve to anchor player emotions while providing the freedom to respond through emotionally-motivated choice selections and content elicitors.


Analyses and evaluation of systems Media annotation Facial expression analysis Interactive storytelling Emotion 


  1. 1.
    Bruni, L.E., Baceviciute, S., Arief, M.: Narrative cognition in interactive systems: suspense-surprise and the P300 ERP component. In: Mitchell, A., Fernández-Vara, C., Thue, D. (eds.) ICIDS 2014. LNCS, vol. 8832, pp. 164–175. Springer, Cham (2014). Scholar
  2. 2.
    Drenikow, B., Mirza-Babaei, P.: Vixen: interactive visualization of gameplay experiences. In: Proceedings of the International Conference on the Foundations of Digital Games, p. 3. ACM (2017).
  3. 3.
    Ekman, P.: An argument for basic emotions. Cogn. Emot. 6(3–4), 169–200 (1992)CrossRefGoogle Scholar
  4. 4.
    Friesen, W.V., Ekman, P.: Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Pyschologists Press, Palo Alto (1978)Google Scholar
  5. 5.
    Hjortsjö, C.H.: Man’s Face and Mimic Language. Studen litteratur (1969)Google Scholar
  6. 6.
    Jalbert, J., Rank, S.: Exit 53: physiological data for improving non-player character interaction. In: Nack, F., Gordon, A.S. (eds.) ICIDS 2016. LNCS, vol. 10045, pp. 25–36. Springer, Cham (2016). Scholar
  7. 7.
    Kaltman, E., Osborn, J., Wardrip-Fruin, N., Mateas, M.: Getting the GISST: a toolkit for the creation, analysis and reference of game studies resources. In: Proceedings of the International Conference on the Foundations of Digital Games, FDG 2017, pp. 16:1–16:10. ACM, New York (2017).
  8. 8.
    Kim, J.H., Gunn, D.V., Schuh, E., Phillips, B., Pagulayan, R.J., Wixon, D.: Tracking real-time user experience (TRUE): a comprehensive instrumentation solution for complex systems. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2008, pp. 443–452. ACM, New York (2008)Google Scholar
  9. 9.
    Lombardo, V., Damiano, R.: Narrative annotation and editing of video. In: Aylett, R., Lim, M.Y., Louchart, S., Petta, P., Riedl, M. (eds.) ICIDS 2010. LNCS, vol. 6432, pp. 62–73. Springer, Heidelberg (2010). Scholar
  10. 10.
    Martinez, B., Valstar, M.F., Jiang, B., Pantic, M.: Automatic analysis of facial actions: a survey. IEEE Trans. Affect. Comput. (2017)Google Scholar
  11. 11.
    McDuff, D., Mahmoud, A., Mavadati, M., Amr, M., Turcot, J., Kaliouby, R.E.: AFFDEX SDK: a cross-platform real-time multi-face expression recognition toolkit. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA 2016, pp. 3723–3726. ACM, New York (2016)Google Scholar
  12. 12.
    Medler, B., John, M., Lane, J.: Data cracker: developing a visual game analytic tool for analyzing online gameplay. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2365–2374. (2011)Google Scholar
  13. 13.
    Mirza-Babaei, P.: Biometric storyboards: a games user research approach for improving qualitative evaluations of player experience. Ph.D. thesis, University of Sussex (2014)Google Scholar
  14. 14.
    Nacke, L.: Affective ludology: scientific measurement of user experience in interactive entertainment. Ph.D. thesis, Blekinge Institute of Technology, Karlskrona (2009)Google Scholar
  15. 15.
    Nacke, L., Lindley, C.A.: Affective ludology, flow and immersion in a First-Person shooter: measurement of player experience. The Journal of the Canadian Game Studies Association (2009)Google Scholar
  16. 16.
    Nacke, L.E.: An introduction to physiological player metrics for evaluating games. In: Seif El-Nasr, M., Drachen, A., Canossa, A. (eds.) Game Analytics. Springer, London (2013). Scholar
  17. 17.
    Paradeda, R., Ferreira, M.J., Martinho, C., Paiva, A.: Using interactive storytelling to identify personality traits. In: Nunes, N., Oakley, I., Nisi, V. (eds.) ICIDS 2017. LNCS, vol. 10690, pp. 181–192. Springer, Cham (2017). Scholar
  18. 18.
    Poria, S., Cambria, E., Bajpai, R., Hussain, A.: A review of affective computing: from unimodal analysis to multimodal fusion. Int. J. Inf. Fusion 37, 98–125 (2017). Scholar
  19. 19.
    R Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2017).
  20. 20.
    Ravaja, N., et al.: The psychophysiology of video gaming: Phasic emotional responses to game events. In: Proceedings of DiGRA 2005 Conference: Changing Views - Worlds in Play, pp. 1–13 (2005)Google Scholar
  21. 21.
    Ravaja, N., Turpeinen, M., Saari, T., Puttonen, S., Keltikangas-Järvinen, L.: The psychophysiology of james bond: phasic emotional responses to violent video game events. Emotion 8(1), 114–120 (2008). Scholar
  22. 22.
    Robinson, R., Isbister, K., Rubin, Z.: All the feels: introducing biometric data to online gameplay streams. In: Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts, CHI PLAY Companion 2016, pp. 261–267. ACM, New York (2016).
  23. 23.
    Robinson, R., Murray, J., Isbister, K.: You’re giving me mixed signals! A comparative analysis of methods that capture players’ emotional response to games. In: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, CHI EA 2018, pp. LBW567:1-LBW567:6. ACM, New York (2018).
  24. 24.
    Scherer, K.R., Schorr, A., Johnstone, T.: Appraisal Processes in Emotion: Theory, Methods, Research. Oxford University Press, Oxford (2001)Google Scholar
  25. 25.
    Schoenau-Fog, H.: Hooked! Evaluating engagement as continuation desire in interactive narratives. In: Si, M., Thue, D., André, E., Lester, J.C., Tanenbaum, J., Zammitto, V. (eds.) ICIDS 2011. LNCS, vol. 7069, pp. 219–230. Springer, Heidelberg (2011). Scholar
  26. 26.
    Szilas, N., Ilea, I.: Objective metrics for interactive narrative. In: Mitchell, A., Fernández-Vara, C., Thue, D. (eds.) ICIDS 2014. LNCS, vol. 8832, pp. 91–102. Springer, Cham (2014). Scholar
  27. 27.
    Telltale Games: The Wolf Among Us (2013)Google Scholar
  28. 28.
    Vermeulen, I.E., Roth, C., Vorderer, P., Klimmt, C.: Measuring user responses to interactive stories: towards a standardized assessment tool. In: Aylett, R., Lim, M.Y., Louchart, S., Petta, P., Riedl, M. (eds.) ICIDS 2010. LNCS, vol. 6432, pp. 38–43. Springer, Heidelberg (2010). Scholar
  29. 29.
    Yannakakis, G.N., Spronck, P., Loiacono, D., André, E.: Player modeling. In: Dagstuhl Follow-Ups, vol. 6 (2013).

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.University of Central FloridaOrlandoUSA
  2. 2.University of SaskatchewanSaskatoonUSA
  3. 3.Expressive Intelligence StudioUniversity of CaliforniaSanta CruzUSA

Personalised recommendations