Skip to main content
SpringerLink
Log in
Book cover

International Workshop on Multimodal Analyses Enabling Artificial Agents in Human-Machine Interaction

MA3HMI 2014: Multimodal Analyses enabling Artificial Agents in Human-Machine Interaction pp 45–56Cite as

Modelling User Experience in Human-Robot Interactions

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 8757))

Abstract

In human-human interaction, the participants’ multimodal behaviour has impact on the interaction as a whole, and similarly in spoken human-robot interactions, the interlocutors’ multimodal signals seem to correlate with the user’s experience and impressions of the interaction. We explored in more detail how some aspects of multimodal behaviour (gazing, facial expressions, body posture) can predict the user’s evaluation of the robot’s behaviour (Responsiveness, Expressiveness, Interface, Usability, Overall impression). The results indicate that the user’s assessment concerning the evaluation categories Interface and Usability, and to some extent the categories Expressiveness and Overall correlate with their behaviour in a statistically significant manner. The work contributes to our understanding of how the interlocutors’ engagement and active participation relate to their assessment of the success of communication, and points towards automating evaluation of human-robot interactions.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   34.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   44.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Anastasiou, D., Jokinen, K., Wilcock, G.: Evaluation of WikiTalk – user studies of human-robot interaction. In: Kurosu, M. (ed.) HCII/HCI 2013, Part IV. LNCS, vol. 8007, pp. 32–42. Springer, Heidelberg (2013)

    Google Scholar 

  2. Bavelas, J.B., Black, A., Lemery, C.R., Mullett, J.: I show how you feel: motor mimicry as a communicative act. J. Pers. Soc. Psychol. 50(2), 322–329 (1986)

    Article  Google Scholar 

  3. Beringer, N., Kartal, U., Louka, K., Schiel, F., Türk, U.: PROMISE: a procedure for multimodal interactive system evaluation. In: Proceedings of the Workshop Multimodal Resources and Multimodal Systems Evaluation, Las Palmas (2002)

    Google Scholar 

  4. Breazeal, C., Scassellati, B.: A context-dependent attention system for a social robot. In: Proceedings of the 16th International Joint Conference in Artificial Intelligence (IJCAI 1999), Stockholm, pp. 1146–1151 (1999)

    Google Scholar 

  5. Campbell, N., Scherer, S.: Comparing measures of synchrony and alignment in dialogue speech timing with respect to turn-taking activity. In: Proceedings of Interspeech 2010, Makuhari (2010)

    Google Scholar 

  6. Csapo, A., Gilmartin, E., Grizou, J., Han, J., Meena, R., Anastasiou, D., Jokinen, K., Wilcock, G.: Multimodal conversational interaction with a humanoid robot. In: Proceedings of 3rd IEEE International Conference on Cognitive Infocommunications (CogInfoCom 2012), Kosice, pp. 667–672 (2012)

    Google Scholar 

  7. Edlund, J., Heldner, M., Hirschberg, J.: Pause and gap length in face-to-face interaction. In: Proceedings of Interspeech 2009, Brighton (2009)

    Google Scholar 

  8. Endsley, M.R.: Theoretical underpinnings of situation awareness: a critical review. In: Endsley, M.R., Garland, D.J. (eds.) Situation Awareness Analysis and Measurement. LEA, Mahwah (2000)

    Google Scholar 

  9. Goodwin, C.: Action and embodiment within situated human interaction. J. Pragmat. 32, 1489–1522 (2000)

    Article  Google Scholar 

  10. Han, J., Campbell, N., Jokinen, K., Wilcock, G.: Investigating the use of non-verbal cues in human-robot interaction with a Nao robot. In: Proceedings of 3rd IEEE International Conference on Cognitive Infocommunications (CogInfoCom 2012), Kosice, pp. 679–683 (2012)

    Google Scholar 

  11. Ishii, R., Nakano, Y., Nishida, T.: Gaze awareness in conversational agents: Estimating a user’s conversational engagement from eye gaze. ACM Trans. Interact. Intell. Syst. 3(2), 11:1–11:25 (2013)

    Article  Google Scholar 

  12. Jokinen, K., McTear, M.: Spoken Dialogue Systems. Morgan and Claypool, San Rafael (2009)

    Google Scholar 

  13. Jokinen, K., Wilcock, G.: User experience in human-robot interactions. In: Schatz, R., Hossfeld, T. (eds.) Proceedings of the Fourth Workshop on Perceptual Quality of Systems (PQS 2013), FTW-TECHREPORT-128, FTW, Vienna, Austria (2013)

    Google Scholar 

  14. Levitan, R., Gravano, A., Hirschberg, J.: Entrainment in speech preceding backchannels. In: Proceedings of ACL 2011, pp. 113–117 (2011)

    Google Scholar 

  15. Levitski, A., Radun, J., Jokinen, K.: Visual interaction and conversational activity. In: Proceedings of the 4th Workshop on Eye Gaze in Intelligent Human Machine Interaction: Eye Gaze and Multimodality, at the 14th ACM International Conference on Multimodal Interaction (ICMI 2012), Santa Monica, California, USA (2012)

    Google Scholar 

  16. Nakano, Y., Reinstein, G., Stocky, T., Cassell, J.: Towards a model of face-to-face grounding. In: Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, vol. 1, pp. 553–561 (2003)

    Google Scholar 

  17. Steinfeld, A., Fong, T., Kaber, D., Scholtz, J., Schultz, A., Goodrich, M.: Common metrics for human-robot interaction. In: 2006 Human-Robot Interaction Conference (2006)

    Google Scholar 

  18. Traum, D., Robinson, S., Stephan, J.: Evaluation of multi-party virtual reality dialogue interaction. In: Proceedings of Fourth International Conference on Language Resources and Evaluation (LREC 2004), Lisbon, pp. 1699–1702 (2004)

    Google Scholar 

  19. Walker, M., Litman, D., Kamm, C., Abella, A.: PARADISE: a framework for evaluating spoken dialogue agents. In: Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics, Madrid (1997)

    Google Scholar 

  20. Wilcock, G.: WikiTalk: a spoken Wikipedia-based open-domain knowledge access system. In: Proceedings of the COLING 2012 Workshop on Question Answering in Complex Domains, Mumbai, India, pp. 57–69 (2012)

    Google Scholar 

  21. Witten, I., Frank, E., Hall, M.: Data Mining: Practical Machine Learning Tools and Techniques, 3rd edn. Morgan Kaufmann, San Francisco (2011)

    Google Scholar 

  22. Wittenburg, P., Brugman, H., Russel, A., Klassmann, A., Sloetjes, H.: ELAN: a professional framework for multimodality research. In: Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC 2006), pp. 1556–1559 (2006)

    Google Scholar 

Download references

Acknowledgements

We would like to thank all the participants as well as the annotators of the video files.

Author information

Authors and Affiliations

  1. University of Tartu, Tartu, Estonia

    Kristiina Jokinen

  2. University of Helsinki, Helsinki, Finland

    Kristiina Jokinen & Graham Wilcock

Authors
  1. Kristiina Jokinen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Graham Wilcock
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kristiina Jokinen .

Editor information

Editors and Affiliations

  1. Otto von Guericke University, Magdeburg, Germany

    Ronald Böck

  2. Trinity College, Dublin, Ireland

    Francesca Bonin

  3. Trinity College, Dublin, Ireland

    Nick Campbell

  4. Utrecht University, Utrecht, The Netherlands

    Ronald Poppe

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Jokinen, K., Wilcock, G. (2015). Modelling User Experience in Human-Robot Interactions. In: Böck, R., Bonin, F., Campbell, N., Poppe, R. (eds) Multimodal Analyses enabling Artificial Agents in Human-Machine Interaction. MA3HMI 2014. Lecture Notes in Computer Science(), vol 8757. Springer, Cham. https://doi.org/10.1007/978-3-319-15557-9_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-15557-9_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-15556-2

  • Online ISBN: 978-3-319-15557-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation