Skip to main content
SpringerLink
Log in

Towards a Socially Adaptive Virtual Agent

  • Conference paper
  • First Online:
Intelligent Virtual Agents (IVA 2015)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9238))

Included in the following conference series:

Abstract

This paper presents a socially adaptive virtual agent that can adapt its behaviour according to social constructs (e.g. attitude, relationship) that are updated depending on the behaviour of its interlocutor. We consider the context of job interviews with the virtual agent playing the role of the recruiter. The evaluation of our approach is based on a comparison of the socially adaptive agent to a simple scripted agent and to an emotionally-reactive one. Videos of these three different agents in situation have been created and evaluated by 83 participants. This subjective evaluation shows that the simulation and expression of social attitude is perceived by the users and impacts on the evaluation of the agent’s credibility. We also found that while the emotion expression of the virtual agent has an immediate impact on the user’s experience, the impact of the virtual agent’s attitude expression’s impact is stronger after a few speaking turns.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://semaine-project.eu/.

References

  1. Acosta, J.C., Ward, N.G.: Achieving rapport with turn-by-turn, user-responsive emotional coloring. Speech Commun. 53(9–10), 1137–1148 (2011)

    Article  Google Scholar 

  2. Anderson, K., André, E., Baur, T., Bernardini, S., Chollet, M., Chryssafidou, E., Damian, I., Ennis, C., Egges, A., Gebhard, P., Jones, H., Ochs, M., Pelachaud, C., Porayska-Pomsta, K., Rizzo, P., Sabouret, N.: The TARDIS framework: intelligent virtual agents for social coaching in job interviews. In: Reidsma, D., Katayose, H., Nijholt, A. (eds.) ACE 2013. LNCS, vol. 8253, pp. 476–491. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  3. Argyle, M.: Bodily Communication. University paperbacks, Methuen (1988)

    Google Scholar 

  4. Ballin, D., Gillies, M., Crabtree, B.: A framework for interpersonal attitude and non-verbal communication in improvisational visual media production. In: First European Conference on Visual Media Production, pp. 203–210 (2004)

    Google Scholar 

  5. Bickmore, T.W., Picard, R.W.: Establishing and maintaining long-term human-computer relationships. ACM Trans. Comput. Hum. Interact. (TOCHI) 12(2), 293–327 (2005)

    Article  Google Scholar 

  6. Burgoon, J.K., Buller, D.B., Hale, J.L., Turck, M.A.: Relational messages associated with nonverbal behaviors. Hum. Commun. Res. 10(3), 351–378 (1984)

    Article  Google Scholar 

  7. Buschmeier, H., Baumann, T., Dosch, B., Kopp, S., Schlangen, D.: Combining incremental language generation and incremental speech synthesis for adaptive information presentation. In: Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 295–303 (2012)

    Google Scholar 

  8. Cafaro, A., Vilhjálmsson, H.H., Bickmore, T., Heylen, D., Jóhannsdóttir, K.R., Valgarsson, G.S.: First impressions: users’ judgments of virtual agents’ personality and interpersonal attitude in first encounters. In: Nakano, Y., Neff, M., Paiva, A., Walker, M. (eds.) IVA 2012. LNCS, vol. 7502, pp. 67–80. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  9. Carney, D.R., Hall, J.A., LeBeau, L.S.: Beliefs about the nonverbal expression of social power. J. Nonverbal Behav. 29(2), 105–123 (2005)

    Article  Google Scholar 

  10. Chollet, M., Ochs, M., Pelachaud, C.: From non-verbal signals sequence mining to bayesian networks for interpersonal attitudes expression. In: Bickmore, T., Marsella, S., Sidner, C. (eds.) IVA 2014. LNCS, vol. 8637, pp. 120–133. Springer, Heidelberg (2014)

    Google Scholar 

  11. Chollet, M., Ochs, M., Pelachaud, C.: Mining a multimodal corpus for non-verbal signals sequences conveying attitudes. In: International Conference on Language Resources and Evaluation, Reykjavik, Iceland, May 2014

    Google Scholar 

  12. Chollet, M., Sratou, G., Shapiro, A., Morency, L.-P., Scherer, S.: An interactive virtual audience platform for public speaking training. In: Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems, AAMAS 2014, pp. 1657–1658. Richland, SC (2014)

    Google Scholar 

  13. DeVault, D., Artstein, R., Benn, G., Dey, T., Fast, E., Gainer, A., Georgila, K., Gratch, J., Hartholt, A., Lhommet, M., et al.: Simsensei kiosk: a virtual human interviewer for healthcare decision support. In: Proceedings of the 2014 International Conference on Autonomous agents and Multi-Agent Systems, pp. 1061–1068 (2014)

    Google Scholar 

  14. Gebhard, P.: ALMA - A layered model of affect. Artificial Intelligence, pp. 0–7 (2005)

    Google Scholar 

  15. Hoque, M.E., Courgeon, M., Martin, J.-C., Mutlu, B., Picard, R.W.: Mach: my automated conversation coach. In: Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 697–706. ACM (2013)

    Google Scholar 

  16. Isbister, K.: Better Game Characters by Design: A Psychological Approach (The Morgan Kaufmann Series in Interactive 3D Technology). Morgan Kaufmann Publishers Inc., San Francisco (2006)

    Google Scholar 

  17. Jaillet, S., Laurent, A., Teisseire, M.: Sequential patterns for text categorization. Intell. Data Anal. 10(3), 199–214 (2006)

    Google Scholar 

  18. Johnson, W.L., Rickel, J.: Steve: an animated pedagogical agent for procedural training in virtual environments. ACM SIGART Bull. 8(1–4), 16–21 (1997)

    Article  Google Scholar 

  19. Jones, H., Sabouret, N.: TARDIS - A simulation platform with an affective virtual recruiter for job interviews. In: IDGEI (2013)

    Google Scholar 

  20. Keltner, D.: Signs of appeasement: evidence for the distinct displays of embarrassment, amusement, and shame. J. Pers. Soc. Psychol. 68, 441–454 (1995)

    Article  Google Scholar 

  21. Kendon, A.: Movement coordination in social interaction: some examples described. Acta Psychol. 32, 101–125 (1970)

    Article  Google Scholar 

  22. Mateas, M., Stern, A.: Structuring content in the façade interactive drama architecture. In: Proceedings of Artificial Intelligence and Interactive Digital Entertainment (AIIDE), pp. 93–98 (2005)

    Google Scholar 

  23. Mehrabian, A.: Pleasure-arousal-dominance: a general framework for describing and measuring individual differences in temperament. Curr. Psychol. 14(4), 261 (1996)

    Article  MathSciNet  Google Scholar 

  24. Ortony, A., Clore, G.L., Collins, A.: The Cognitive Structure of Emotions. Cambridge University Press, Cambridge (1988)

    Book  Google Scholar 

  25. Poggi, I., Pelachaud, C., de Rosis, F., Carofiglio, V., De Carolis, B.: Greta. a believable embodied conversational agent. In: Stock, O., Zancanaro, M. (eds.) Multimodal Intelligent Information Presentation, pp. 3–25. Springer, The Netherlands (2005)

    Chapter  Google Scholar 

  26. Prendinger, H., Ishizuka, M.: The empathic companion: a character-based interface that addresses users’ affective states. Appl. Artif. Intell. 19(3–4), 267–285 (2005)

    Article  Google Scholar 

  27. Ravenet, B., Ochs, M., Pelachaud, C.: From a user-created corpus of virtual agent’s non-verbal behavior to a computational model of interpersonal attitudes. In: Aylett, R., Krenn, B., Pelachaud, C., Shimodaira, H. (eds.) IVA 2013. LNCS, vol. 8108, pp. 263–274. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  28. Schroder, M., Bevacqua, E., Cowie, R., Eyben, F., Gunes, H., Heylen, D., Ter Maat, M., McKeown, G., Pammi, S., Pantic, M., et al.: Building autonomous sensitive artificial listeners. IEEE Trans. Affect. Comput. 3(2), 165–183 (2012)

    Article  Google Scholar 

  29. Snyder, M.: The influence of individuals on situations: implications for understanding the links between personality and social behavior. J. Pers. 51(3), 497–516 (1983)

    Article  Google Scholar 

  30. Srikant, R., Agrawal, R.: Mining sequential patterns: generalizations and performance improvements. Adv. Database Technol. 1–17, 1996 (1057)

    Google Scholar 

  31. Tan, P.-N., Steinbach, M., Kumar, V.: Introduction to Data Mining, 1st edn. Addison-Wesley Longman Publishing Co. Inc., Boston (2005)

    Google Scholar 

  32. Vardoulakis, L.P., Ring, L., Barry, B., Sidner, C.L., Bickmore, T.: Designing relational agents as long term social companions for older adults. In: Nakano, Y., Neff, M., Paiva, A., Walker, M. (eds.) IVA 2012. LNCS, vol. 7502, pp. 289–302. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  33. Wagner, J., Lingenfelser, F., Baur, T., Damian, I., Kistler, F., André, E.: The social signal interpretation (ssi) framework-multimodal signal processing and recognition in real-time. In: Proceedings of the 21st ACM International Conference on Multimedia, Barcelona, Spain (2013)

    Google Scholar 

  34. Wegener, D.T., Petty, R.E., Klein, D.J.: Effects of mood on high elaboration attitude change: the mediating role of likelihood judgments. Eur. J. Soc. Psychol. 24(1), 25–43 (1994)

    Article  Google Scholar 

  35. With, S., Kaiser, W.S.: Sequential patterning of facial actions in the production and perception of emotional expressions. Swiss J. Psychol. 70(4), 241–252 (2011)

    Article  Google Scholar 

  36. Yang, Z., Metallinou, A., Narayanan, S.: Analysis and predictive modeling of body language behavior in dyadic interactions from multimodal interlocutor cues. IEEE Trans. Multimedia 16(6), 1766–1778 (2014)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. LIMSI-CNRS UPR 3251, University Paris-Sud, Orsay, France

    Atef Ben Youssef & Nicolas Sabouret

  2. CNRS LTCI, Telecom-ParisTech, Paris, France

    Mathieu Chollet & Catherine Pelachaud

  3. SupAgro, UMR ITAP, Montpellier, France

    Hazaël Jones

  4. University Aix-Marseille, LSIS, DiMag, Marseille, France

    Magalie Ochs

Authors
  1. Atef Ben Youssef
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mathieu Chollet
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hazaël Jones
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nicolas Sabouret
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Catherine Pelachaud
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Magalie Ochs
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Atef Ben Youssef .

Editor information

Editors and Affiliations

  1. Delft University of Technology, Delft, The Netherlands

    Willem-Paul Brinkman

  2. Delft University of Technology, Delft, The Netherlands

    Joost Broekens

  3. University of Twente, Enschede, The Netherlands

    Dirk Heylen

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Ben Youssef, A., Chollet, M., Jones, H., Sabouret, N., Pelachaud, C., Ochs, M. (2015). Towards a Socially Adaptive Virtual Agent. In: Brinkman, WP., Broekens, J., Heylen, D. (eds) Intelligent Virtual Agents. IVA 2015. Lecture Notes in Computer Science(), vol 9238. Springer, Cham. https://doi.org/10.1007/978-3-319-21996-7_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-21996-7_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-21995-0

  • Online ISBN: 978-3-319-21996-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation