Selecting and Expressing Communicative Functions in a SAIBA-Compliant Agent Framework

  • Angelo CafaroEmail author
  • Merijn BruijnesEmail author
  • Jelte van Waterschoot
  • Catherine Pelachaud
  • Mariët Theune
  • Dirk Heylen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10498)


In SAIBA-compliant agent systems, the Function Markup Language (FML) is used to describe the agent’s communicative functions that are transformed into utterances accompanied with appropriate non-verbal behaviours. In the context of the ARIA Framework, we propose a template-based approach, grounded in the DIT++ taxonomy, as an interface between the dialogue manager (DM) and the non-verbal behaviour generation (NVBG) components of this framework. Our approach enhances our current FML-APML implementation of FML with the capability of receiving on-the-fly generated natural language and socio-emotional parameters (e.g. emotional stance) for transforming the agent’s intents in believable verbal and non-verbal behaviours in an adaptive manner.


Dialogue management Communicative function FML Multimodal behaviour SAIBA 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aylett, M.P., Pidcock, C.J.: The cerevoice characterful speech synthesiser SDK. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722, pp. 413–414. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-74997-4_65CrossRefGoogle Scholar
  2. 2.
    Bruijnes, M.: Believable suspect agents: response and interpersonal style selection for an artificial suspect. Ph.D. thesis, University of Twente (2016). sIKS dissertation no. 2016–39Google Scholar
  3. 3.
    Bunt, H., Alexandersson, J., Choe, J.W., Fang, A.C., Hasida, K., Petukhova, V., Popescu-Belis, A., Traum, D.R.: Iso 24617–2: A semantically-based standard for dialogue annotation. In: LREC, pp. 430–437 (2012)Google Scholar
  4. 4.
    Cafaro, A., Vilhjálmsson, H.H., Bickmore, T., Heylen, D., Pelachaud, C.: Representing communicative functions in saiba with a unified function markup language. In: Bickmore, T., Marsella, S., Sidner, C. (eds.) IVA 2014. LNCS (LNAI), vol. 8637, pp. 81–94. Springer, Cham (2014). doi: 10.1007/978-3-319-09767-1_11CrossRefGoogle Scholar
  5. 5.
    Goldberg, L.R.: An alternative “description of personality”: the big-five factor structure. Journal of Personality and Social Psychology 59(6), 1216–1229 (1990)CrossRefGoogle Scholar
  6. 6.
    Keizer, S., Bunt, H., Petukhova, V.: Multidimensional dialogue management. In: van den Bosch, A., Bouma, G. (eds.) Interactive Multi-modal Question-Answering, pp. 57–86. Springer (2011)Google Scholar
  7. 7.
    Kopp, S., Krenn, B., Marsella, S., Marshall, A.N., Pelachaud, C., Pirker, H., Thórisson, K.R., Vilhjálmsson, H.: Towards a common framework for multimodal generation: the behavior markup language. In: Gratch, J., Young, M., Aylett, R., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 205–217. Springer, Heidelberg (2006). doi: 10.1007/11821830_17CrossRefGoogle Scholar
  8. 8.
    Larsson, S., Traum, D.R.: Information state and dialogue management in the TRINDI Dialogue Move Engine Toolkit. Natural Language Engineering 6(3&4), 323–340 (2000)CrossRefGoogle Scholar
  9. 9.
    Leuski, A., Traum, D.: NPCEditor: Creating Virtual Human Dialogue Using Information Retrieval Techniques. AI Magazine 32(2), 42–56 (2011)CrossRefGoogle Scholar
  10. 10.
    Lison, P.: Structured probabilistic modelling for dialogue management. Ph.D. thesis, University of Oslo (2013)Google Scholar
  11. 11.
    ter Maat, M., Heylen, D.: Flipper: an information state component for spoken dialogue systems. In: Vilhjálmsson, H.H., Kopp, S., Marsella, S., Thórisson, K.R. (eds.) IVA 2011. LNCS (LNAI), vol. 6895, pp. 470–472. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-23974-8_67CrossRefGoogle Scholar
  12. 12.
    Mairesse, F., Walker, M.: PERSONAGE: personality generation for dialogue. In: Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, vol. 45, pp. 496–503. Association for Computational Linguistics (2007)Google Scholar
  13. 13.
    Mancini, M., Pelachaud, C.: The FML-APML language. In: Why Conversational Agents do what they do. Workshop on Functional Representations for Generating Conversational Agents Behavior at AAMAS (2008)Google Scholar
  14. 14.
    Morbini, F., DeVault, D., Sagae, K., Gerten, J., Nazarian, A., Traum, D.: FLoReS: a forward looking, reward seeking, dialogue manager. In: Natural Interaction with Robots, Knowbots and Smartphones, pp. 313–325. Springer (2014)Google Scholar
  15. 15.
    Ochs, M., Sabouret, N., Corruble, V.: Simulation of the dynamics of nonplayer characters’ emotions and social relations in games. IEEE Transactions on Computational Intelligence and AI in Games 1(4), 281–297 (2009)CrossRefGoogle Scholar
  16. 16.
    Poggi, I.: Mind, hands, face and body: A goal and belief view of multimodal communication. Weidler Buchverlag Berlin (2007)Google Scholar
  17. 17.
    Rich, C., Sidner, C.L.: Using Collaborative Discourse Theory to Partially Automate Dialogue Tree Authoring. In: Nakano, Y., Neff, M., Paiva, A., Walker, M. (eds.) IVA 2012. LNCS (LNAI), vol. 7502, pp. 327–340. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33197-8_34CrossRefGoogle Scholar
  18. 18.
    Wagner, J., Lingenfelser, F., Baur, T., Damian, I., Kistler, F., André, E.: The social signal interpretation (SSI) framework: multimodal signal processing and recognition in real-time. In: Proceedings of the 21st ACM International Conference on Multimedia, pp. 831–834 (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Angelo Cafaro
    • 2
    Email author
  • Merijn Bruijnes
    • 1
    Email author
  • Jelte van Waterschoot
    • 1
  • Catherine Pelachaud
    • 2
  • Mariët Theune
    • 1
  • Dirk Heylen
    • 1
  1. 1.Human Media InteractionUniversity of TwenteEnschedeThe Netherlands
  2. 2.CNRS-ISIR, Pierre and Marie Curie UniversityParisFrance

Personalised recommendations