Skip to main content
SpringerLink
Log in

Simulating empathic behavior in a social assistive robot

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

When used as an interface in the context of Ambient Assisted Living (AAL), a social robot should not just provide a task-oriented support. It should also try to establish a social empathic relation with the user. To this aim, it is crucial to endow the robot with the capability of recognizing the user’s affective state and reason on it for triggering the most appropriate communicative behavior. In this paper we describe how such an affective reasoning has been implemented in the NAO robot for simulating empathic behaviors in the context of AAL. In particular, the robot is able to recognize the emotion of the user by analyzing communicative signals extracted from speech and facial expressions. The recognized emotion allows triggering the robot’s affective state and, consequently, the most appropriate empathic behavior. The robot’s empathic behaviors have been evaluated both by experts in communication and through a user study aimed at assessing the perception and interpretation of empathy by elderly users. Results are quite satisfactory and encourage us to further extend the social and affective capabilities of the robot.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. The name of the project, NICA, is also used to indicate the robot.

  2. In this work, since the robot had to deal with European elderly people, we used color combinations that are suitable for the European culture. Of course, different cultures may require different colors for conveying the same feelings, but this does not change the basics of our approach (a different color can be easily used for each culture).

References

  1. Anderson C, Keltner D (2002) The role of empathy in the formation and maintenance of social bonds. Behav Brain Sci 25:21–22

    Google Scholar 

  2. Baron-Cohen S (2011) The science of evil: on empathy and the origins of cruelty. Basic Books, New York, NY

    Google Scholar 

  3. Beadle JN, Sheehan AH, Dahlben B, Gutchess AH (2013) Aging, empathy, and prosociality. J Gerontol Ser B Psychol Sci Soc Sci:1–10

  4. Boersma P, Weenink D (2007) Praat: doing phonetics by computer (version 4.5.15) [computer program]. http://www.praat.org/. Retrieved 24.02.2007

  5. Broekens J, Heerink M, Rosendal H (2009) Assistive social robots in elderly care: a review. Gerontechnology 8(2):94–103. doi:10.4017/gt.2009.08.02.002.00

    Article  Google Scholar 

  6. Cesta A, Cortellessa G, Pecora F, Rasconi R (2007) Supporting interaction in the robocare intelligent assistive environment, AAAI Spring Symposium: Interaction Challenges for Intelligent Assistants, pp. 18–25

  7. Coradeschi S, Cesta A, Cortellessa G, Coraci L, Galindo C, Gonzalez J, Karlsson L, Forsberg A, Frennert S, Furfari F, Loutfi A, Orlandini A, Palumbo F, Pecora F, von Rump S, Stimec A, Ullberg J, Ötslund B (2014) GiraffPlus: a system for monitoring activities and physiological parameters and promoting social interaction for elderly in human-computer systems interaction: backgrounds and applications 3. Adv Intell Syst Comput 300:261–271. doi:10.1007/978-3-319-08491-6_22

    Article  Google Scholar 

  8. Cramer H, Goddijn J, Wielinga B, Evers V (2010) Effects of (In) accurate empathy and situational valence on attitudes towards robots. ACM/IEEE International Conference on Human Robot Interaction (HRI). 141-2. New York. March 2005

  9. Davis M (1980) A multidimensional approach to individual differences in empathy. Cat Sel Doc Psychol 10:85

    Google Scholar 

  10. De Carolis B, Ferilli S, Greco D (2013) Towards a caring home for assisted living. In: Proceedings of the workshop “the challenge of ageing society: technological roles and opportunities for artificial intelligence” co-located with the 13th Conference of the Italian Association for Artificial Intelligence (AI*IA 2013), Turin, 6 Dec 2013. http://ceur-ws.org/Vol-1122/

  11. De Carolis B, Mazzotta I, Novielli N, Pizzutilo S (2013) User modeling in social interaction with a Caring Agent. In: UMADR - User modeling and adaptation for daily routines, Springer HCI Series. Springer

  12. de Rosis F, De Carolis B, Carofiglio V, Pizzutilo S (2003) Shallow and inner forms of emotional intelligence in advisory dialog simulation. In: Prendinger H, Ishizuka M (eds) “Life-like characters. Tools, affective functions and applications. pp 271-294. Springer

  13. Eisenberg N, Miller PA (1987) The relation of empathy to prosocial and related behaviors. Psychol Bull 101:91–119

    Article  Google Scholar 

  14. Ekman P, Oster H (1979) Facial expressions of emotion. Annu Rev Psychol 30:527–554

    Article  Google Scholar 

  15. Elliott C, Siegle G (1993) Variables influencing the intensity of simulated affective states. In Proceedings of the AAAI Spring Symposium on Mental States.’93:58-67

  16. Graf B, Hans M, Shaft RD (2004) Care-O-bot II – development of a next generation robotics home assistant. Auton Robots 16(2):193–205

  17. Hoffman ML (1981) The development of empathy. In: Rushton J, Sorrentino R (eds) Altruism and helping behavior: Social personality and developmental perspectives. Erlbaum, Hillsdale, pp. 41–63

    Google Scholar 

  18. Jensen FV (2001) Bayesian networks and decision graphs, statistics for engineering and information science. Springer New York, New York, doi:10.1007/978-1-4757-3502-4

  19. Jiang H, Vidal JM, Huhns MN (2007) EBDI: an architecture for emotional agents, Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems, May 14-18, Honolulu, Hawaii, ACM, p. 11

  20. Johnson DO, Cuijpers RH, van der Pol D (2013) Imitating human emotions with artificial facial expressions. Int J Soc Robot 5(4):503–513

    Article  Google Scholar 

  21. Klein J, Moon Y, Picard R (1999) This computer responds to user frustration. In Proceedings of the Conference on Human Factors in Computing Systems. Pittsburgh, USA, ACM Press, pp 242–243

  22. Leite I, Martinho C, Paiva A (2013) Social robots for long-term interaction: a survey. Int J Soc Robot. doi:10.1007/s12369-013-0178-y

    Google Scholar 

  23. Mataric M, Tapus A, Feil-Seifer D (2007) Personalized socially assistive robotics, workshop on intelligent systems for assisted cognition, rochester, New York, USA

  24. Mehrabian A (1968) Communication without words. Psychol Today 2:53–56

    Google Scholar 

  25. Milborrow S, Nicolls F (2008) Locating facial features with an extended active shape model. In Computer Vision–ECCV 2008 (pp. 504-513). Springer Berlin Heidelberg

  26. Milborrow S, Nicolls F (2014) Active shape models with SIFT descriptors and MARS, In Proceedings of the 9th International Conference on Computer Vision Theory and Applications, Lisbon, pp. 5–8

  27. Nehmer J, Becker M, Karshmer A, Lamm R (2006) Living assistance systems: an ambient intelligence approach, in International Conference on Software Engineering, ACM, pp. 43–50

  28. Niewiadomski R, Ochs M, Pelachaud C (2008) Expressions of empathy in ECAs, in Proceedings of the 8t Int.Conf. on IVA, LNAI, vol. 5208. Springer-Verlag, Berlin, Heidelberg, pp 37-44

  29. Nijholt A, de Ruyter B, Heylen D, Privender S (2006) Social interfaces for ambient intelligence environments. Chapter 14. In: Aarts E, Encarnaçao J (eds) True visions: the emergence of ambient intelligence. Springer, New York, pp. 275–289

    Chapter  Google Scholar 

  30. Oatley K, Johnson-Laird PN (1987) Towards a cognitive theory of emotions. Cognit Emot 29–50

  31. Ortony A, Clore GL, Collins A (1988) The cognitive structure of emotions. Cambridge University Press

  32. Paiva A, Dias J, Sobral D, Aylett R, Sobreperez P, Woods S, Hall L (2004) Caring for agents and agents that care: building empathic relations with synthetic agents. In Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems-Volume 1 (pp. 194-201). IEEE Computer Society., ACM Press

  33. Paiva A et al. (2011) Empathy in social agents. Int J sVirtual Real 10(1):65–68

    Google Scholar 

  34. Palestra G, Pettinicchio A, Del Coco M, Carcagnì P, Leo M, Distante C (2015) Improved performance in facial expression recognition using 32 geometric features. ICIAP, lecture notes in computer science, Springer, pp. 518–528

  35. Picard R (1997) Affective computing. MIT Press, Cambridge, MA

    Book  Google Scholar 

  36. Pineau J, Montemerlo M, Pollack M, Roy N, Thrun S (2003) Towards robotic assistants in nursing homes: challenges and results, robotics and autonomous systems 42(3–4):271–281

  37. Reeves B, Nass C (1996) The media equation: how people treat computers, television, and new media like real people and places. Cambridge University Press, New York, NY

    Google Scholar 

  38. Reiter E, Dale R (2000) Building natural language generation systems. Studies in natural language processing. Cambridge University Press, Cambridge, United Kingdom. ISBN 0-521-62036-8

  39. Russell J (1980) A circumplex model of affect. J Pers Soc Psychol 39:1161–1178

    Article  Google Scholar 

  40. Speigel C. Greczek J, Matarić M (2016) Creating a baseline for empathy in socially assistive robotics. http://robotics.usc.edu/~cspiegel/naoempathy4.pdf

  41. van Ruiten AM, Haitas D, Bingley P, Hoonhout HCM, Meerbeek BW, Terken JMB (2007) Attitude of elderly towards a robotic game-and-train- buddy: evaluation of empathy and objective control. In: Cowie R, de Rosis F (eds) Proceedings of the Doctoral consortium, in the scope of ACII2007 Conference

  42. Viola PA, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  43. Vogt T, Andrié E, Bee N (2008) EmoVoice - a framework for online recognition of emotions from voice. In Proceedings of the 4th IEEE tutorial and research workshop on Perception and Interactive Technologies for Speech-Based Systems: Perception in Multimodal Dialogue Systems (PIT '08), Springer-Verlag, Berlin, 188-199

Download references

Acknowledgments

This work fulfills the research objectives of the PON02_00563_3489339 project "PUGLIA@SERVICE - funded by the Italian Ministry of University and Research (MIUR). Our thanks go to Isabella Poggi and Francesca D’Errico for their useful suggestions on the evaluation of the robot’s behaviors.

Author information

Authors and Affiliations

  1. Dipartimento di Informatica, Univerisita’ di Bari, Aldo Moro, Via Orabona 4, 70126, Bari, Italy

    Berardina De Carolis, Stefano Ferilli & Giuseppe Palestra

Authors
  1. Berardina De Carolis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefano Ferilli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Giuseppe Palestra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Berardina De Carolis.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

De Carolis, B., Ferilli, S. & Palestra, G. Simulating empathic behavior in a social assistive robot. Multimed Tools Appl 76, 5073–5094 (2017). https://doi.org/10.1007/s11042-016-3797-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3797-0

Keywords

Search

Navigation