Skip to main content
SpringerLink
Log in

From Non-verbal Signals Sequence Mining to Bayesian Networks for Interpersonal Attitudes Expression

  • Conference paper
Intelligent Virtual Agents (IVA 2014)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 8637))

Included in the following conference series:

Abstract

In this paper, we present a model and its evaluation for expressing attitudes through sequences of non-verbal signals for Embodied Conversational Agents. To build our model, a corpus of job interviews has been annotated at two levels: the non-verbal behavior of the recruiters as well as their expressed attitudes was annotated. Using a sequence mining method, sequences of non-verbal signals characterizing different interpersonal attitudes were automatically extracted from the corpus. From this data, a probabilistic graphical model was built. The probabilistic model is used to select the most appropriate sequences of non-verbal signals that an ECA should display to convey a particular attitude. The results of a perceptive evaluation of sequences generated by the model show that such a model can be used to express interpersonal attitudes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Agrawal, R., Srikant, R.: Fast algorithms for mining association rules in large databases. In: Very Large Data Bases, pp. 487–499. Morgan Kaufmann Publishers Inc., San Francisco (1994)

    Google Scholar 

  2. Allwood, J., Kopp, S., Grammer, K., Ahlsen, E., Oberzaucher, E., Koppensteiner, M.: The analysis of embodied communicative feedback in multimodal corpora: a prerequisite for behavior simulation. Language Resources and Evaluation 41, 255–272 (2007)

    Article  Google Scholar 

  3. Argyle, M.: Bodily Communication. University paperbacks. Methuen (1988)

    Google Scholar 

  4. Aylett, M.P., Pidcock, C.J.: The CereVoice Characterful Speech Synthesiser SDK. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722, pp. 413–414. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  5. Aylett, R., Paiva, A., Dias, J., Hall, L., Woods, S.: Affective agents for education against bullying. In: Affective Information Processing, pp. 75–90. Springer (2009)

    Google Scholar 

  6. Ballin, D., Gillies, M., Crabtree, B.: A framework for interpersonal attitude and non-verbal communication in improvisational visual media production. In: First European Conference on Visual Media Production, pp. 203–210 (2004)

    Google Scholar 

  7. Bickmore, T.W., Picard, R.W.: Establishing and maintaining long-term human-computer relationships. ACM Transactions in Computer-Human Interaction 12(2), 293–327 (2005)

    Article  Google Scholar 

  8. Boersma, P., Weenink, D.: Praat, a system for doing phonetics by computer. Glot International 5(9/10), 341–345 (2001)

    Google Scholar 

  9. Burgoon, J.K., Buller, D.B., Hale, J.L., de Turck, M.A.: Relational Messages Associated with Nonverbal Behaviors. Human Communication Research 10(3), 351–378 (1984)

    Article  Google Scholar 

  10. Cafaro, A., Vilhjálmsson, H.H., Bickmore, T., Heylen, D., Jóhannsdóttir, K.R., Valgarðsson, G.S.: First impressions: Users’ judgments of virtual agents’ personality and interpersonal attitude in first encounters. In: Nakano, Y., Neff, M., Paiva, A., Walker, M. (eds.) IVA 2012. LNCS, vol. 7502, pp. 67–80. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  11. Carney, D.R., Hall, J.A., LeBeau, L.S.: Beliefs about the nonverbal expression of social power. Journal of Nonverbal Behavior 29(2), 105–123 (2005)

    Article  Google Scholar 

  12. Chollet, M., Ochs, M., Clavel, C., Pelachaud, C.: A multimodal corpus approach to the design of virtual recruiters. In: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, ACII 2013, pp. 19–24 (2013)

    Google Scholar 

  13. Chollet, M., Ochs, M., Pelachaud, C.: A multimodal corpus approach to the design of virtual recruiters. In: Workshop Multimodal Corpora, Intelligent Virtual Agents, IVA 2013, pp. 36–41 (2013)

    Google Scholar 

  14. Cowie, R., Cox, C., Martin, J.C., Batliner, A., Heylen, D., Karpouzis, K.: Issues in data labelling. In: Cowie, R., Pelachaud, C., Petta, P. (eds.) Emotion-Oriented Systems. Cognitive Technologies, pp. 213–241. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  15. Ferreira, P.G., Azevedo, P.J.: Protein sequence classification through relevant sequence mining and bayes classifiers. In: Bento, C., Cardoso, A., Dias, G. (eds.) EPIA 2005. LNCS (LNAI), vol. 3808, pp. 236–247. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  16. Fricker, D., Zhang, H., Yu, C.: Sequential pattern mining of multimodal data streams in dyadic interactions. In: 2011 IEEE International Conference on Development and Learning (ICDL), vol. 2, pp. 1–6 (2011)

    Google Scholar 

  17. Graesser, A., Chipman, P., King, B., McDaniel, B., D’Mello, S.: Emotions and learning with autotutor. In: Proceedings of the 2007 conference on Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work, pp. 569–571. IOS Press, Amsterdam (2007)

    Google Scholar 

  18. Guillame-Bert, M., Crowley, J.L.: Learning temporal association rules on symbolic time sequences. In: Proceedings of the 4th Asian Conference on Machine Learning, pp. 159–174 (2012)

    Google Scholar 

  19. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: An update. Association for Computing Machinery’s Special Interest Group on Knowledge Discovery and Data Mining Explorations Newsletter 11(1), 10–18 (2009)

    Google Scholar 

  20. Jaillet, S., Laurent, A., Teisseire, M.: Sequential patterns for text categorization. Intelligent Data Analysis 10(3), 199–214 (2006)

    Google Scholar 

  21. Keltner, D.: Signs of appeasement: Evidence for the distinct displays of embarrassment, amusement, and shame. Journal of Personality and Social Psychology 68, 441–454 (1995)

    Article  Google Scholar 

  22. LaFrance, M.: Posture mirroring and rapport. In: Davis, M. (ed.) Interaction Rhythms: Periodicity in Communicative Behavior, pp. 279–299. Human Sciences Press, New York (1982)

    Google Scholar 

  23. Lee, J., Marsella, S.: Modeling side participants and bystanders: The importance of being a laugh track. In: Vilhjálmsson, H.H., Kopp, S., Marsella, S., Thórisson, K.R. (eds.) IVA 2011. LNCS, vol. 6895, pp. 240–247. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  24. Lee, J., Marsella, S.C.: Predicting speaker head nods and the effects of affective information. IEEE Transactions on Multimedia, 552–562 (2010)

    Google Scholar 

  25. Mancini, M., Pelachaud, C.: Dynamic behavior qualifiers for conversational agents. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722, pp. 112–124. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  26. Mancini, M., Pelachaud, C.: The FML - APML language. In: The First FML Workshop, AAMAS 2008, Estoril, Portugal (May 2008)

    Google Scholar 

  27. Martínez, H.P., Yannakakis, G.N.: Mining multimodal sequential patterns: a case study on affect detection. In: Proceedings of the 13th International Conference on Multimodal Interfaces, pp. 3–10. ACM, New York (2011)

    Chapter  Google Scholar 

  28. McQuiggan, S.W., Robison, J.L., Phillips, R., Lester, J.C.: Modeling parallel and reactive empathy in virtual agents: An inductive approach. In: Proceedings of 7th International Conference on Autonomous Agents and Multiagent Systems, pp. 167–174 (2008)

    Google Scholar 

  29. Moridis, C.N., Economides, A.A.: Affective learning: Empathetic agents with emotional facial and tone of voice expressions. IEEE Transactions on Affective Computing 3(3), 260–272 (2012)

    Article  Google Scholar 

  30. Niewiadomski, R., Hyniewska, S.J., Pelachaud, C.: Constraint-based model for synthesis of multimodal sequential expressions of emotions. IEEE Transaction on Affective Computing 2(3), 134–146 (2011)

    Article  Google Scholar 

  31. Pan, X., Gillies, M., Sezgin, T.M., Loscos, C.: Expressing complex mental states through facial expressions. In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 745–746. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  32. Prepin, K., Ochs, M., Pelachaud, C.: Beyond backchannels: co-construction of dyadic stancce by reciprocal reinforcement of smiles between virtual agents. In: Proceedings of the 35th Annual Meeting of the Cognitive Science Society (2013)

    Google Scholar 

  33. Ravenet, B., Ochs, M., Pelachaud, C.: From a user-created corpus of virtual agent’s non-verbal behavior to a computational model of interpersonal attitudes. In: Aylett, R., Krenn, B., Pelachaud, C., Shimodaira, H. (eds.) IVA 2013. LNCS (LNAI), vol. 8108, pp. 263–274. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  34. Scherer, K.R.: What are emotions? and how can they be measured? Social Science Information 44, 695–729 (2005)

    Article  Google Scholar 

  35. Srikant, R., Agrawal, R.: Mining sequential patterns: Generalizations and performance improvements. In: Apers, P.M.G., Bouzeghoub, M., Gardarin, G. (eds.) EDBT 1996. LNCS, vol. 1057, pp. 1–17. Springer, Heidelberg (1996)

    Chapter  Google Scholar 

  36. Vilhjálmsson, H.H., et al.: The behavior markup language: Recent developments and challenges. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722, pp. 99–111. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  37. With, S.: Structural analysis of temporal patterns of facial actions: Measurement and implications for the study of emotion perception through facial expressions. Ph.D. thesis, University of Geneva (2010)

    Google Scholar 

  38. Wittenburg, P., Brugman, H., Russel, A., Klassmann, A., Sloetjes, H.: Elan: a professional framework for multimodality research. In: Language Resources and Evaluation (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut Mines-Telecom, Telecom Paristech, CNRS-LTCI, 46 rue Barrault, 75013, Paris, France

    Mathieu Chollet

  2. CNRS-LTCI, Telecom Paristech, 46 rue Barrault, 75013, Paris, France

    Magalie Ochs & Catherine Pelachaud

Authors
  1. Mathieu Chollet
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Magalie Ochs
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Catherine Pelachaud
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. College of Computer and Information Science, Northeastern University, 202 West Village H, 360 Huntington Avenue, 02115, Boston, MA, USA

    Timothy Bickmore  & Stacy Marsella  & 

  2. Department of Computer Science, Worcester Polytechnic Institute, 01609, Worcester, MA, USA

    Candace Sidner

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Chollet, M., Ochs, M., Pelachaud, C. (2014). From Non-verbal Signals Sequence Mining to Bayesian Networks for Interpersonal Attitudes Expression. In: Bickmore, T., Marsella, S., Sidner, C. (eds) Intelligent Virtual Agents. IVA 2014. Lecture Notes in Computer Science(), vol 8637. Springer, Cham. https://doi.org/10.1007/978-3-319-09767-1_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-09767-1_15

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-09766-4

  • Online ISBN: 978-3-319-09767-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation