Skip to main content
SpringerLink
Log in

Towards Affect Sensitive and Socially Perceptive Companions

  • Chapter
Book cover Your Virtual Butler

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 7407))

Abstract

This chapter investigates affect sensitivity as an important requirement for socially perceptive companions. Challenges and issues arising in the design of an affect recognition framework for artificial companions are identified. A multi-level approach to the analysis of non-verbal affective expressions in human-companion interaction is also presented. The chapter ends with a discussion on the importance of affect recognition for the generation of empathic reactions and the establishment of long-term human-companion relationships.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Picard, R.W.: Affective Computing. MIT Press (1997)

    Google Scholar 

  2. Robins, B., Dickerson, P., Stribling, P., Dautenhahn, K.: Robot-mediated joint attention in children with autism: A case study in robot-human interaction. Interaction Studies 5(2), 161–198 (2004)

    Article  Google Scholar 

  3. Castellano, G., Leite, I., Pereira, A., Martinho, C., Paiva, A., McOwan, P.W.: Affect Recognition for Interactive Companions: Challenges and Design in Real World Scenarios. Journal on Multimodal User Interfaces 3(1), 89–98 (2010)

    Article  Google Scholar 

  4. Dautenhahn, K.: Socially Intelligent Robots: Dimensions of Human-Robot Interaction. Philosophical Transactions of the Royal Society B: Biological Sciences 362(1480), 679–704 (2007)

    Article  Google Scholar 

  5. Breazeal, C.: Role of expressive behaviour for robots that learn from people. Philosophical Transactions of the Royal Society B 364, 3527–3538 (2009)

    Article  Google Scholar 

  6. Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(1) (2009)

    Google Scholar 

  7. You, Z.-J., Shen, C.-Y., Chang, C.-W., Liu, B.-J., Chen, G.-D.: A Robot as a Teaching Assistant in an English Class. In: Proc. of the Sixth Int’l Conf. on Advanced Learning Technologies (2006)

    Google Scholar 

  8. Tanaka, F., Cicourel, A., Movellan, J.R.: Socialization between toddlers and robots at an early childhood education center. PNAS 104(46), 17954–17958 (2007)

    Article  Google Scholar 

  9. www.lirec.eu

  10. Kapoor, A., Picard, R.W.: Multimodal affect recognition in learning environments. In: Proceedings of the ACM International Conference on Multimedia, pp. 677–682 (2005)

    Google Scholar 

  11. Kapoor, A., Burleson, W., Picard, R.W.: Automatic Predictionof Frustration. International Journal of Human-Computer Studies 65(8), 724–736 (2007)

    Article  Google Scholar 

  12. Peters, C., Asteriadis, S., Karpouzis, K.: Investigating shared attention with a virtual agent using a gaze-based interface. Journal on Multimodal User Interfaces 3(1-2), 119–130 (2010)

    Article  Google Scholar 

  13. Nakano, Y.I., Ishii, R.: Estimating user’s engagement from eye-gaze behaviors in human-agent conversations. In: Proceeding of the 14th International Conference on Intelligent User Interfaces, IUI 2010, pp. 139–148. ACM Press, New York (2010)

    Google Scholar 

  14. Kulic, D., Croft, E.A.: Affective state estimation for human-robot interaction. IEEE Transactions on Robotics 23(25), 991–1000 (2007)

    Article  Google Scholar 

  15. Rich, C., Ponsler, B., Holroyd, A., Sidner, C.L.: Recognizing engagement in human-robot interaction. In: Proceeding of the 5th ACM/IEEE International Conference on Human-Robot Interaction, HRI 2010, pp. 375–382. ACM Press, New York (2010)

    Chapter  Google Scholar 

  16. Castellano, G., Pereira, A., Leite, I., Paiva, A., McOwan, P.W.: Detecting user engagement with a robot companion using task and social interaction-based features. In: Proceedings of the International Conference on Multimodal Interfaces and Workshop on Machine Learning for Multimodal Interaction (ICMI-MLMI 2009), pp. 119–126. ACM Press, Cambridge (2009)

    Chapter  Google Scholar 

  17. Sanghvi, J., Castellano, G., Leite, I., Pereira, A., McOwan, P.W., Paiva, A.: Automatic analysis of affective postures and body motion to detect engagement with a game companion. In: Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction, Lausanne, Switzerland (2011)

    Google Scholar 

  18. Leite, I., Martinho, C., Pereira, A., Paiva, A.: iCat: An affective game buddy based on anticipatory mechanisms. In: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems, vol. 3, pp. 1229–1232 (2008)

    Google Scholar 

  19. Scherer, K.R.: Subsystem synchronization and multimodal behavioural organization. In: Synchronization Workshop, HUMAINE/SCAS Summer School 2006, Genova (2006)

    Google Scholar 

  20. Scherer, K.R.: On the nature and function of emotion: A component process approach. In: Scherer, K.R., Ekman, P. (eds.) Approaches to Emotion, pp. 293–317. Erlbaum, Hillsdale (1984)

    Google Scholar 

  21. Scherer, K.R.: Which emotions can be induced by music? What are the underlying mechanisms? And how can we measure them? Journal of New Music Research 33(3), 239–251 (2004)

    Article  MathSciNet  Google Scholar 

  22. Castellano, G., Caridakis, G., Camurri, A., Karpouzis, K., Volpe, G., Kollias, S.: Body Gesture and Facial Expression Analysis for Automatic Affect Recognition. In: Scherer, K.R., Baenziger, T., Roesch, E.B. (eds.) Blueprint for Affective Computing: A Sourcebook. Oxford University Press, Oxford (2010)

    Google Scholar 

  23. Castellano, G., Peters, C.: Socially Perceptive Robots: Challenges and Concerns. Interaction Studies 11(2) (2010)

    Google Scholar 

  24. Scherer, K., Baenziger, T.: On the use of actor portrayals in research on emotional expression. In: Scherer, K.R., Baenziger, T., Roesch, E.B. (eds.) Blueprint for Affective Computing: A Sourcebook. Oxford University Press, Oxford (2010)

    Google Scholar 

  25. Afzal, S., Robinson, P.: Natural affect data – collection and annotation in learning context. In: Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction (ACII 2009), pp. 22–28. IEEE (2009)

    Google Scholar 

  26. Castellano, G., Leite, I., Pereira, A., Martinho, C., Paiva, A., McOwan, P.W.: Inter-ACT: An affective and contextually rich multimodal video corpus for studying interaction with robots. In: Proceedings of the ACM International Conference on Multimedia, pp. 1031–1034. ACM (2010)

    Google Scholar 

  27. Vinciarelli, A., Pantic, M., Bourlard, H., Pentland, A.: Social signal processing: State-of-the-art and future perspectives of an emerging domain. In: Proceedings of the ACM International Conference on Multimedia (MM 2008), Vancouver, Canada, pp. 1061–1070 (2008)

    Google Scholar 

  28. Breazeal, C., Edsinger, A., Fitzpatrick, P., Scassellati, B.: Active vision for sociable robots. IEEE Transactions on Systems, Man and Cybernetics-Part A 31(5), 443–453 (2001)

    Article  Google Scholar 

  29. Deshmukh, A., Castellano, G., Lim, M.Y., Aylett, R., McOwan, P.W.: Ubiquitous Social Perception Abilities for Interaction Initiation in Human-Robot Interaction. In: Proceedings of the 3rd International Workshop on Affect Interaction in Natural Environments, ACM Multimedia 2010, Florence, Italy (2010)

    Google Scholar 

  30. Camurri, A., Lagerlof, I., Volpe, G.: Recognizing emotion from dance movement: Comparison of spectator recognition and automated techniques. International Journal of Human-Computer Studies 59(1-2), 213–225 (2003)

    Article  Google Scholar 

  31. Castellano, G., Mortillaro, M., Camurri, A., Volpe, G., Scherer, K.: Automated Analysis of Body Movement in Emotionally Expressive Piano Performances. Music Perception 26(2), 103–119 (2008)

    Article  Google Scholar 

  32. Peters, C.: A perceptually-based theory of mind model for agent interaction initiation. International Journal of Humanoid Robotics (IJHR), Special issue on Achieving Human-Like Qualities in Interactive Virtual and Physical Humanoids 3(3), 321–340 (2006)

    Google Scholar 

  33. Castellano, G., Kessous, L., Caridakis, G.: Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech. In: Peter, C., Beale, R. (eds.) Affect and Emotion in HCI. LNCS, vol. 4868, pp. 92–103. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  34. Bartel, C.A., Saavedra, R.: The collective construction of work group moods. Administrative Science Quarterly 45, 197–231 (2000)

    Article  Google Scholar 

  35. Castellano, G., Mancini, M.: Analysis of Emotional Gestures for the Generation of Expressive Copying Behaviour in an Embodied Agent. In: Sales Dias, M., Gibet, S., Wanderley, M.M., Bastos, R. (eds.) GW 2007. LNCS (LNAI), vol. 5085, pp. 193–198. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  36. Van Baaren, R., Holland, R., Kawakami, K., van Knippenberg, A.: Mimicry and prosocial behavior. Psychological Science 15(1), 71–74 (2004)

    Article  Google Scholar 

  37. Gratch, J., Okhmatovskaia, A., Lamothe, F., Marsella, S.C., Morales, M., van der Werf, R.J., Morency, L.-P.: Virtual Rapport. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 14–27. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  38. Leite, I., Pereira, A., Mascarenhas, S., Castellano, G., Martinho, C., Prada, R., Paiva, A.: Closing the Loop: from Affect Recognition to Empathic Interaction. In: Proceedings of the 3rd International Workshop on Affect Interaction in Natural Environments, ACM Multimedia 2010, Florence, Italy (2010)

    Google Scholar 

  39. Castellano, G.: Movement Expressivity Analysis in Affective Computers: From Recognition to Expression of Emotion, Ph.D.Thesis, Department of Communication, Computer and System Sciences, University of Genova, Italy (2008)

    Google Scholar 

  40. Conati, C., Marsella, S., Paiva, A.: Affective interactions: the computer in the affective loop. In: Proceedings of the 10th International Conference on Intelligent User Interfaces, IUI 2005, p. 7. ACM, New York (2005)

    Google Scholar 

  41. Maatman, R.M., Gratch, J., Marsella, S.C.: Natural Behavior of a Listening Agent. In: Panayiotopoulos, T., Gratch, J., Aylett, R.S., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS (LNAI), vol. 3661, pp. 25–36. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  42. Kopp, S., Sowa, T., Wachsmuth, I.: Imitation Games with an Artificial Agent: From Mimicking to Understanding Shape-Related Iconic Gestures. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 436–447. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  43. Reidsma, D., Nijholt, A., Poppe, R., Rienks, R., Hondorp, H.: Virtualrap dancer: Invitation to dance. In: CHI 2006 Extended Abstracts on Human Factors in Computing Systems, pp. 263–266. ACM (2006)

    Google Scholar 

  44. Riek, L., Paul, P., Robinson, P.: When my robot smiles at me: Enabling human-robot rapport via real-time head gesture mimicry. Journal on Multimodal User Interfaces 3(1), 99–108 (2010)

    Article  Google Scholar 

  45. Lester, J.C., Towns, S.G., FitzGerald, P.J.: Achieving affective impact: visual emotive communication in lifelike pedagogical agents. The International Journal of Artificial Intelligence in Education 10, 278–291 (1999)

    Google Scholar 

  46. Bickmore, T., Picard, R.W.: Establishing and maintaining long-term human-computer relationships. ACM Transactions on Computer-Human Interaction 12(2), 293–327 (2004)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronic, Electrical and Computer Engineering, University of Birmingham, UK

    Ginevra Castellano

  2. School of Electronic Engineering and Computer Science, Queen Mary University of London, UK

    Peter W. McOwan

Authors
  1. Ginevra Castellano
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peter W. McOwan
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Austrian Research Institute for Artificial Intelligence (OFAI), Austrian Society for Cypernetic Studies (OSGK), Freyung 6/6, 1010, Vienna, Austria

    Robert Trappl

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Castellano, G., McOwan, P.W. (2013). Towards Affect Sensitive and Socially Perceptive Companions. In: Trappl, R. (eds) Your Virtual Butler. Lecture Notes in Computer Science(), vol 7407. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37346-6_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-37346-6_5

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-37345-9

  • Online ISBN: 978-3-642-37346-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation