Abstract
Independent and autonomous at-home care can solve many current societal issues. However, with increased life expectancy and the rise of aging populations, further improvements to interactive robots are needed to increase Quality-of-Life. The standalone Toyota Human Support Robot has the ability to conduct independent at-home care. However, it lacks the establishment of autonomous social behaviors. We show that, by synthesizing a 3D multi-modal social interactive agent, it is capable of performing active listening in conjunction with the physical HSR. We perform a user analysis between social behaviors of standalone HSR and proposed by us – SIA-HSR. Experimental results have shown the effectiveness of our proposed approach, enhanced user experience, and improved rapport-building with HRI.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Admoni, H., Hayes, B., Feil-Seifer, D., Ullman, D., Scassellati, B.: Are you looking at me? Perception of robot attention is mediated by gaze type and group size. In: 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 389–395. IEEE (2013)
Admoni, H., Scassellati, B.: Social eye gaze in human-robot interaction: a review. J. Hum.-Rob. Interact. 6(1), 25–63 (2017)
Breazeal, C.: Toward sociable robots. Robot. Auton. Syst. 42(3), 167–175 (2003). Socially Interactive Robots
Brown, T.B., et al.: Language models are few-shot learners (2020)
Bruce, A., Nourbakhsh, I., Simmons, R.: The role of expressiveness and attention in human-robot interaction. In: Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No. 02CH37292), vol. 4, pp. 4138–4142. IEEE (2002)
Charrier, L., Rieger, A., Galdeano, A., Cordier, A., Lefort, M., Hassas, S.: The rope scale: a measure of how empathic a robot is perceived. In: 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 656–657. IEEE (2019)
Dieter, J., Wang, T., Chaganty, A.T., Angeli, G., Chang, A.: Mimic and rephrase: reflective listening in open-ended dialogue. In: Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pp. 393–403 (2019)
Ekman, P.: An argument for basic emotions. Cogn. Emot. 6(3/4), 169–200 (1992)
Fitrianie, S., Bruijnes, M., Li, F., Abdulrahman, A., Brinkman, W.P.: The artificial-social-agent questionnaire: establishing the long and short questionnaire versions. In: Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents, pp. 1–8 (2022)
Gonsior, B., et al.: Improving aspects of empathy and subjective performance for HRI through mirroring facial expressions. In: 2011 RO-MAN, pp. 350–356. IEEE (2011)
Grigore, E.C., Pereira, A., Zhou, I., Wang, D., Scassellati, B.: Talk to me: verbal communication improves perceptions of friendship and social presence in human-robot interaction. In: Traum, D., Swartout, W., Khooshabeh, P., Kopp, S., Scherer, S., Leuski, A. (eds.) IVA 2016. LNCS (LNAI), vol. 10011, pp. 51–63. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-47665-0_5
Hart, J., et al.: RoboCup@Home 2022: rules and regulations (2022). www.athome.robocup.org/rules/2022 rulebook.pdf
Johal, W., Calvary, G., Pesty, S.: Non-verbal signals in HRI: interference in human perception. In: ICSR 2015. LNCS (LNAI), vol. 9388, pp. 275–284. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-25554-5_28
Kim, J.: Bimodal emotion recognition using speech and physiological changes. Robust Speech Recogn. Underst. 265, 280 (2007)
Kirby, R., Forlizzi, J., Simmons, R.: Affective social robots. Robot. Auton. Syst. 58(3), 322–332 (2010)
Kobayashi, Y., Yamamoto, D., Koga, T., Yokoyama, S., Doi, M.: Design targeting voice interface robot capable of active listening. In: 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 161–162. IEEE (2010)
Lisetti, C., Amini, R., Yasavur, U.: Now all together: overview of virtual health assistants emulating face-to-face health interview experience. KI-Kunstliche Intell. 29(2), 161–172 (2015)
Lugrin, B., Pelachaud, C., Traum, D. (eds.): The Handbook on Socially Interactive Agents: 20 Years of Research on Embodied Conversational Agents, Intelligent Virtual Agents, and Social Robotics Volume 1: Methods, Behavior, Cognition, vol. 37, 1 edn. Association for Computing Machinery, New York (2021)
Manyika, J.: Google Bard. https://ai.google/static/documents/google-aboutbard.pdf
Miller, W.R., Rollnick, S.: Motivational Interviewing: Helping People Change, 3rd edn. Guilford Press (2013)
Resnicow, K., McMaster, F.: Motivational interviewing: moving from why to how with autonomy support. Int. J. Behav. Nutr. Phys. Act. 9, 19 (2012)
Riek, L.D., Paul, P.C., Robinson, P.: When my robot smiles at me: enabling human-robot rapport via real-time head gesture mimicry. J. Multimodal User Interfaces 3(1), 99–108 (2010)
Ritschel, H., Aslan, I., Mertes, S., Seiderer, A., Andre, E.: Personalized synthesis of intentional and emotional non-verbal sounds for social robots. In: 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 1–7. IEEE (2019)
Rogers, C.R., Farson, R.E.: Active listening. Organ. Psychol. (1984)
Thoppilan, R., et al.: LaMDA: language models for dialog applications (2022)
Touvron, H., et al.: LLaMA: open and efficient foundation language models (2023)
Ulhoi, J.P., Norskov, S.: The emergence of social robots: adding physicality and agency to technology. J. Eng. Tech. Manage. 65, 101703 (2022)
Vaswani, A., et al.: Attention is all you need (2017)
Wuth, J., Correa, P., Nunez, T., Saavedra, M., Yoma, N.B.: The role of speech technology in user perception and context acquisition in HRI. Int. J. Soc. Robot. 13, 949–968 (2021)
Yamamoto, T., Terada, K., Ochiai, A., Saito, F., Asahara, Y., Murase, K.: Development of human support robot as the research platform of a domestic mobile manipulator. ROBOMECH J. 6(1), 4 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pasternak, K., Duarte, C., Ojalvo, J., Lisetti, C., Visser, U. (2024). 3D Multimodal Socially Interactive Robot with ChatGPT Active Listening. In: Buche, C., Rossi, A., Simões, M., Visser, U. (eds) RoboCup 2023: Robot World Cup XXVI. RoboCup 2023. Lecture Notes in Computer Science(), vol 14140. Springer, Cham. https://doi.org/10.1007/978-3-031-55015-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-55015-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-55014-0
Online ISBN: 978-3-031-55015-7
eBook Packages: Computer ScienceComputer Science (R0)