Effects of adapting to user pitch on rapport perception, behavior, and state with a social robotic learning companion

Abstract

Social robots such as learning companions, therapeutic assistants, and tour guides are dependent on the challenging task of establishing a rapport with their users. People rarely communicate with just words alone; facial expressions, gaze, gesture, and prosodic cues like tone of voice and speaking rate combine to help individuals express their words and convey emotion. One way that individuals communicate a sense of connection with one another is entrainment, where interaction partners adapt their way of speaking, facial expressions, or gestures to each other; entrainment has been linked to trust, liking, and task success and is thought to be a vital phenomenon in how people build rapport. In this work, we introduce a social robot that combines multiple channels of rapport-building behavior, including forms of social dialog and prosodic entrainment. We explore how social dialog and entrainment contribute to both self-reported and behavioral rapport responses. We find prosodic adaptation enhances perceptions of social dialog, and that social dialog and entrainment combined build rapport. Individual differences indicated by gender mediate these social responses; an individual’s underlying rapport state, as indicated by their verbal rapport behavior, is exhibited and triggered differently depending on gender. These results have important repercussions for assessing and modeling a user’s social responses and designing adaptive social agents.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. ALICE: A.L.I.C.E AI Foundation (2002). http://www.alicebot.org/. Accessed 21 Apr 2015

  2. Aneja, D., Hoegen, R., McDuff, D., Czerwinski, M.: Designing style matching conversational agents. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM (2019)

  3. Arroyo, I., Burleson, W., Tai, M., Muldner, K., Woolf, B.P.: Gender differences in the use and benefit of advanced learning technologies for mathematics. J. Educ. Psychol. 105(4), 957–969 (2013). https://doi.org/10.1037/a0032748

    Google Scholar 

  4. Babel, M.: Dialect divergence and convergence in New Zealand English. Lang. Soc. 39, 437–456 (2010)

    Google Scholar 

  5. Babel, M.: Evidence for phonetic and social selectivity in spontaneous phonetic imitation. J. Phon. 40, 177–189 (2011)

    Google Scholar 

  6. Babel, M., Bulatov, D.: The role of fundamental frequency in phonetic accommodation. Lang. Speech 55(2), 231–248 (2012)

    Google Scholar 

  7. Bales, R.F.: A set of categories for the analysis of small group interaction. Am. Sociol. Rev. 15(2), 257–263 (1950)

    Google Scholar 

  8. Beal, C., Mitra, S., Cohen, P.: Modeling learning patterns of students with a tutoring system using Hidden Markov Model. In: Luckin, R., et al. (eds.) Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED), pp. 238–245. Marina del Rey (2007)

  9. Bechade, L., Dubuisson Duplessis, G., Sehili, M., Devillers, L.: Behavioral and emotional spoken cues related to mental states in human–robot social interaction. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, November 2015, pp. 347–350. ACM (2015)

  10. Bell, L., Gustafson, J., Heldner, M.: Prosodic adaptation in human–computer interaction. In: Proceedings of ICPHS, vol. 3 (2003)

  11. Bell, D., Arnold, H., Haddock, R.: Linguistic politeness and peer tutoring. Learn. Assist. Rev. 14(1), 37–54 (2009)

    Google Scholar 

  12. Bengio, Y., Frasconi, P.: An input output HMM architecture. In: Advances in Neural Information Processing Systems, pp. 427–434 (1995)

  13. Beňuš, Š., Levitan, R., Hirschberg, J., Gravano, A., Darjaa, S.: Entrainment in Slovak collaborative dialogues. In: 2014 5th IEEE Conference on Cognitive Infocommunications (CogInfoCom), November 2014, pp. 309–313. IEEE (2014)

  14. Beňuš, Š., Levitan, R., Hirschberg, J.: Entrainment in spontaneous speech: the case of filled pauses in Supreme Court hearings. In: 3rd IEEE Conference on Cognitive Infocommunications, Kosice, Slovakia (2012)

  15. Beňuš, Š.: Are we in sync’: turn-taking in collaborative dialogues. In: Tenth Annual Conference of the International Speech Communication Association (2009)

  16. Bergner, Y., Walker, E., Ogan, A.: Dynamic Bayesian network models for peer tutoring interactions. In: Innovative Assessment of Collaboration, pp. 249–268. Springer, Cham (2017)

  17. Bickmore, T.W.: Relational Agents: Effecting Change Through Human-Computer Relationships (Doctoral dissertation, Massachusetts Institute of Technology) (2003)

  18. Bickmore, T., Cassell, J.: Relational agents: a model and implementation of building user trust. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 396–403 (2001)

  19. Bickmore, T.W., Picard, R.W.: Establishing and maintaining long-term human-computer relationships. ACM Trans. Comput.-Human. Interact. (TOCHI) 12(2), 293–327 (2005)

    Google Scholar 

  20. Bickmore, T.W., Vardoulakis, L.M.P., Schulman, D.: Tinker: a relational agent museum guide. Auton. Agent. Multi-Agent Syst. 27(2), 254–276 (2013)

    Google Scholar 

  21. Biocca, F., Harms, C.: Defining and measuring social presence: contribution to the networked minds theory and measure. In: Proceedings of PRESENCE, 2002, pp. 1–36 (2002)

  22. Boersma, P.: Praat: doing phonetics by computer. http://www.praat.org/ (2006). Accessed 31 Mar 2014

  23. Bone, D., Lee, C.C., Chaspari, T., Black, M.P., Williams, M.E., Lee, S., Levitt, P., Narayanan, S.: Acoustic-prosodic, turn-taking, and language cues in child–psychologist interactions for varying social demand. In: INTERSPEECH-2013, pp. 2400–2404 (2013)

  24. Bonin, F., De Looze, C., Ghosh, S., Gilmartin, E., Vogel, C., Polychroniou, A., Salamin, H., Vinciarelli, A., Campbell, N.: Investigating fine temporal dynamics of prosodic and lexical accommodation. In: INTERSPEECH-2013, pp. 539–543 (2013)

  25. Borrie, S.A., Lubold, N., Pon-Barry, H.: Disordered speech disrupts conversational entrainment: a study of acoustic-prosodic entrainment and communicative success in populations with communication challenges. Front. Psychol. 6, 1187 (2015)

    Google Scholar 

  26. Boyer, K.E., Phillips, R., Ingram, A., Ha, E.Y., Wallis, M., Vouk, M., Lester, J.: Characterizing the effectiveness of tutorial dialogue with hidden markov models. In: International Conference on Intelligent Tutoring Systems, June 2010, pp. 55–64. Springer, Berlin, Heidelberg (2010)

  27. Breazeal, C.: Emotion and sociable humanoid robots. Int. J. Human-Comput. Stud. 59(1–2), 119–155 (2003)

    Google Scholar 

  28. Brown, P.: How and why are women more polite: some evidence from a Mayan community. In: McConnell-Ginet, S., Borker, R., Furman, N. (eds.) Women and Language in Literature and Society, pp. 111–136. Praeger, New York (1980)

    Google Scholar 

  29. Brown, P., Levinson, S.: Politeness. Some Universals in Language Usage. CUP, Cambridge (1987). Originally published as Universals in language usage: politeness phenomenon. In: Goody, E. (ed.) Questions and Politeness: Strategies in Social Interaction. CUP, Cambridge (1978)

    Google Scholar 

  30. Burleson, W., Picard, R.W.: Gender-specific approaches to developing emotionally intelligent learning companions. IEEE Intell. Syst. 22(4), 62–69 (2007). https://doi.org/10.1109/MIS.2007.69

    Google Scholar 

  31. Cassell, J., Bickmore, T.: Negotiated collusion: modeling social language and its relationship effects in intelligent agents. User Model. User-Adap. Inter. 13(1–2), 89–132 (2003)

    Google Scholar 

  32. Chalupnik, M., Christie, C., Mullany, L.: (Im)politeness and gender. In: Culpeper, J., Haugh, M., Kádár, D. (eds.) The Palgrave Handbook of Linguistic (Im)politeness, pp. 517–537. Palgrave Macmillan, London (2017)

    Google Scholar 

  33. Chidambaram, V., Chiang, Y.-H., Mutlu, B.: Designing persuasive robots: how robots might persuade people using vocal and nonverbal cues. In: 7th Annual ACM/IEEE International Conference on Human–Robot Interaction (HRI’12), pp. 293–300 (2012). http://doi.org/10.1145/2157689.2157798

  34. Csapo, A., Gilmartin, E., Grizou, J., Han, J., Meena, R., Anastasiou, D., Jokinen, K., Wilcock, G.: Multimodal conversational interaction with a humanoid robot. In: 2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom), pp. 667–672. IEEE (2012)

  35. Coates, J.: Women, Men and Language: A Sociolinguistic Account of Gender Differences in Language. Routledge, Abingdon (2015)

    Google Scholar 

  36. Darves, C., Oviatt, S.: Adaptation of users’ spoken dialogue patterns in a conversational interface. In: Seventh International Conference on Spoken Language Processing (2002)

  37. De Carolis, B., Ferilli, S., Palestra, G., Carofiglio, V.: Modeling and simulating empathic behavior in social assistive robots. In: Proceedings of the 11th Biannual Conference on Italian SIGCHI Chapter, September 2015, pp. 110–117. ACM (2015)

  38. Drolet, A.L., Morris, M.W.: Rapport in conflict resolution: accounting for how face-to-face contact fosters mutual cooperation in mixed-motive conflicts. J. Exp. Soc. Psychol. 36(1), 26–50 (2000)

    Google Scholar 

  39. Foster, M.E., Gaschler, A., Giuliani, M.: How can I help you’: comparing engagement classification strategies for a robot bartender. In: Proceedings of the 15th ACM on International Conference on Multimodal Interaction, December 2013, pp. 255–262. ACM (2013)

  40. Giles, H., Smith, P.: Accommodation theory: optimal levels of convergence. In: Howard, G., Robert, N. (eds.) Language and Social Psychology, pp. 45–65. St Clair (1979)

  41. Gordon, G., Spaulding, S., Westlund, J.K., Lee, J.J., Plummer, L., Martinez, M., Breazeal, C., et al.: Affective personalization of a social robot tutor for children’s second language skills. In: Thirtieth AAAI Conference on Artificial Intelligence, March 2016 (2016)

  42. Gratch, J., Wang, N., Gerten, J., Fast, E., Duffy, R.: Creating rapport with virtual agents. In: International Workshop on Intelligent Virtual Agents, September 2007, pp. 125–138. Springer, Berlin, Heidelberg (2007)

  43. Gravano, A., Benus, S., Levitan, R., Hirschberg, J.: Three ToBI-based measures of prosodic entrainment and their correlations with speaker engagement. In: 2014 IEEE Spoken Language Technology Workshop (SLT), December 2014, pp. 578–583. IEEE (2014)

  44. Gregory, S.W., Dagan, K., Webster, S.: Evaluating the relation of vocal accommodation in conversation partners’ fundamental frequencies to perceptions of communication quality. J. Nonverbal Behav. 21(1), 23–43 (1997)

    Google Scholar 

  45. Gulz, A., Haake, M., Silvervarg, A.: Extending a teachable agent with a social conversation module—effects on student experiences and learning. In: International Conference on Artificial Intelligence in Education, June 2011, pp. 106–114. Springer, Berlin, Heidelberg (2011)

  46. Gweon, G., Jain, M., McDonough, J., Raj, B., Rosé, C.P.: Measuring prevalence of other-oriented transactive contributions using an automated measure of speech style accommodation. Int. J. Comput. Support. Collab. Learn. 8(2), 245–265 (2013)

    Google Scholar 

  47. Hess, U., Blairy, S.: Facial mimicry and emotional contagion to dynamic emotional facial expressions and their influence on decoding accuracy. Int. J. Psychophysiol. 40(2), 129–141 (2001)

    Google Scholar 

  48. Holmes, J.: Women, Men and Politeness. Longman, London (1995)

    Google Scholar 

  49. Hong, J.-C., Hwang, M.-Y.: Gender differences in help-seeking and supportive dialogue during on-line game. Procedia Soc. Behav. Sci. 64(2007), 342–351 (2012). https://doi.org/10.1016/j.sbspro.2012.11.041

    Google Scholar 

  50. Huang, L., Morency, L.P., Gratch, J.: Virtual rapport 2.0. In: International Workshop on Intelligent Virtual Agents, September 2011, pp. 68–79. Springer, Berlin, Heidelberg (2011)

  51. Jacq, A., Lemaignan, S., Garcia, F., Dillenbourg, P., Paiva, A.: Building successful long child–robot interactions in a learning context. In: 2016 11th ACM/IEEE International Conference on Human–Robot Interaction (HRI), March 2016, pp. 239–246. IEEE (2016)

  52. Jokinen, K., Hurtig, T.: User expectations and real experience on a multimodal interactive system. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, January 2006, vol. 2, pp. 1049–1052 (2006)

  53. Kanda, T., Hirano, T., Eaton, D., Ishiguro, H.: Interactive robots as social partners and peer tutors for children: a field trial. Hum. Comput. Interact. 19(1), 61–84 (2004)

    Google Scholar 

  54. Kasap, Z., Magnenat-Thalmann, N.: Building long-term relationships with virtual and robotic characters: the role of remembering. Vis. Comput. 28(1), 87–97 (2012)

    Google Scholar 

  55. Kasap, Z., Magnenat-Thalmann, N.: Towards episodic memory-based long-term affective interaction with a human-like robot. In: 19th International Symposium in Robot and Human Interactive Communication, September 2010, pp. 452–457. IEEE (2010)

  56. Kory-Westlund, J.M., Breazeal, C.: Exploring the effects of a social robot’s speech entrainment and backstory on young children’s emotion, rapport, relationship, and learning. Front. Robot. AI 6, 54 (2019)

    Google Scholar 

  57. Kose-Bagci, H., Ferrari, E., Dautenhahn, K., Syrdal, D.S., Nehaniv, C.L.: Effects of embodiment and gestures on social interaction in drumming games with a humanoid robot. Adv. Robot. 23(14), 1951 (2009). https://doi.org/10.1163/016918609X12518783330360

    Google Scholar 

  58. Kousidis, S., Dorran, D.: Monitoring convergence of temporal features in spontaneous dialogue speech. In: First Young Researchers Workshop on Speech Technology, Dublin, Ireland, January 2009 (2009)

  59. Krämer, N.C., Karacora, B., Lucas, G., Dehghani, M., Rüther, G., Gratch, J.: Closing the gender gap in STEM with friendly male instructors? On the effects of rapport behavior and gender of a virtual agent in an instructional interaction. Comput. Educ. 99, 1–13 (2016)

    Google Scholar 

  60. Kumar, R., Ai, H., Beuth, J.L., Rosé, C.P.: Socially capable conversational tutors can be effective in collaborative learning situations. In: Aleven, V., Kay, J., Mostow, J. (eds.) Intelligent Tutoring Systems, pp. 156–164. Springer, Berlin (2010)

    Google Scholar 

  61. Lakin, J.L., Chartrand, T.L.: Using nonconscious behavioral mimicry to create affiliation and rapport. Psychol. Sci. 14(4), 334–339 (2003)

    Google Scholar 

  62. Lakens, D., Stel, M.: If they move in sync, they must feel in sync: Movement synchrony leads to attributions of rapport and entitativity. Soc. Cognit. 29(1), 1–14 (2011)

    Google Scholar 

  63. Lee, C.C., Katsamanis, A., Black, M.P., Baucom, B.R., Georgiou, P.G., Narayanan, S.S.: An analysis of PCA-based vocal entrainment measures in married couples’ affective spoken interactions. In: Proceedings of Interspeech, Florence, Italy (2011)

  64. Lee, M.K., Forlizzi, J., Kiesler, S., Rybski, P., Antanitis, J., Savetsila, S.: Personalization in HRI: a longitudinal field experiment. In: 2012 7th ACM/IEEE International Conference on Human–Robot Interaction (HRI), March 2012, pp. 319–326. IEEE (2012)

  65. Lee, N., Shin, H., Sundar, S.S.: Utilitarian vs. hedonic robots: role of parasocial tendency and anthropomorphism in shaping user attitudes. In: Proceedings of the 6th International Conference on Human–Robot Interaction, March 2011, pp. 183–184 (2011)

  66. Leelawong, K., Biswas, G.: Designing learning by teaching agents: the Betty’s Brain system. Int. J. Artif. Intell. Educ. 18(3), 181–208 (2008)

    Google Scholar 

  67. Levitan, R., Hirschberg, J.: Measuring acoustic-prosodic entrainment with respect to multiple levels and dimensions. In: Interspeech 2011 (2011)

  68. Levitan, R., Benus, S., Galvez, R.H., Gravano, A., Savoretti, F., Trnka, M., Hirschberg, J.: Implementing acoustic-prosodic entrainment in a conversational avatar. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, 08–12 September 2016, pp. 1166–1170 (2016). http://doi.org/10.21437/Interspeech.2016-985

  69. Levitan, R., Beňuš, Š., Gravano, A., Hirschberg, J.: Acoustic-prosodic entrainment in Slovak, Spanish, English and Chinese: a cross-linguistic comparison. In: Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 325–334 (2015)

  70. Levitan, R., Gravano, A., Willson, L., Benus, S., Hirschberg, J., Nenkova, A.: Acoustic-prosodic entrainment and social behavior. In: Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2012, pp. 11–19. Association for Computational Linguistics (2012)

  71. Longster, J.: Concatenative speech synthesis: a framework for reducing perceived distortion when using the TD-PSOLA algorithm. Dissertation, Bournemouth University (2003)

  72. Looije, R., van der Zalm, A., Neerincx, M.A., Beun, R.J.: Help, I need somebody: the effect of embodiment on playful learning, pp. 718–724. IEEE (2012). http://dx.doi.org/10.1109/ROMAN.2012.6343836

  73. Lubold, N., Pon-Barry, H., Walker, E.: Naturalness and rapport in a pitch adaptive learning companion. In: 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), December 2015, pp. 103–110. IEEE (2015)

  74. Lubold, N., Pon-Barry, H.: Acoustic-prosodic entrainment and rapport in collaborative learning dialogues. In: Proceedings of the 2014 ACM workshop on Multimodal Learning Analytics Workshop and Grand Challenge, November 2014, pp. 5–12 (2014)

  75. Lubold, N., Walker, E., Pon-Barry, H., Ogan, A.: Automated pitch convergence improves learning in a social, teachable robot for middle school mathematics. In: International Conference on Artificial Intelligence in Education, pp. 282–296. Springer, Cham (2018)

  76. Lubold, N.: Producing acoustic-prosodic entrainment in a robotic learning companion to build learner rapport. Doctoral dissertation, Arizona State University (2018)

  77. Lutfi, S., Fernández-Martínez, F., Lorenzo-Trueba, J., Barra-Chicote, R., Montero, J.: I feel you: the design and evaluation of a domotic affect-sensitive spoken conversational agent. Sensors 13(8), 10519–10538 (2013)

    Google Scholar 

  78. Meena, R., Jokinen, K., Wilcock, G.: Integration of gestures and speech in human–robot interaction. In: 2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom), December 2012, pp. 673–678. IEEE (2012)

  79. Mehrabian, A.: Nonverbal Communication. Transaction Publishers, Piscataway (1972)

    Google Scholar 

  80. Murphy, K.: The Bayes net toolbox for Matlab. Comput. Sci. Stat. 33(2), 1024–1034 (2001)

    Google Scholar 

  81. Mushin, I., Stirling, L., Fletcher, J., Wales, R.: Discourse structure, grounding, and prosody in task-oriented dialogue. Discourse Process. 35(1), 1–31 (2003)

    Google Scholar 

  82. Natale, M.: Convergence of mean vocal intensity in dyadic communication as a function of social desirability. J. Pers. Soc. Psychol. 32(5), 790 (1975)

    Google Scholar 

  83. Nenkova, A., Gravano, A., Hirschberg, J.: High frequency word entrainment in spoken dialogue. In: Proceedings of ACL-08: HLT, Short Papers, pp. 169–172 (2008)

  84. Novick, D., Gris, I.: Building rapport between human and ECA: a pilot study. In: International Conference on Human–Computer Interaction, June 2014, pp. 472–480. Springer, Cham (2014)

  85. Nwe, T.L., Foo, S.W., De Silva, L.C.: Speech emotion recognition using hidden Markov models. Speech Commun. 41(4), 603–623 (2003)

    Google Scholar 

  86. Ogan, A., Finkelstein, S., Walker, E., Carlson, R., Cassell, J.: Rudeness and rapport: insults and learning gains in peer tutoring. In: Cerri, S.A., Clancey, W.J., Papadourakis, G., Panourgia, K. (eds.) Intelligent Tutoring Systems, pp. 11–21. Springer, Berlin (2012)

    Google Scholar 

  87. Pantic, M., Pentland, A., Nijholt, A., Huang, T.S.: Human computing and machine understanding of human behavior: a survey. In: Artificial Intelligence for Human Computing, pp. 47–71. Springer, Berlin, Heidelberg (2007)

  88. Powers, A., Kiesler, S., Fussell, S., Torrey, C.: Comparing a computer agent with a humanoid robot. In: Proceedings of the ACM/IEEE International Conference on Human–Robot Interaction, March 2007, pp. 145–152 (2007)

  89. Sadoughi, N., Pereira, A., Jain, R., Leite, I., Lehman, J.F.: Creating prosodic synchrony for a robot co-player in a speech-controlled game for children, pp. 91–99 (2017). https://doi.org/10.1145/2909824.3020244

  90. Saerbeck, M., Schut, T., Bartneck, C., Janse, M.D.: Expressive robots in education: varying the degree of social supportive behavior of a robotic tutor. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1613–1622. ACM (2010)

  91. Saint-Aimé, S., Le-Pevedic, B., Duhaut, D., Shibata, T.: EmotiRob: companion robot project. In: RO-MAN 2007-The 16th IEEE International Symposium on Robot and Human Interactive Communication, pp. 919–924. IEEE (2007)

  92. Schermerhorn, P., Scheutz, M., Crowell, C.R.: Robot social presence and gender: do females view robots differently than males?. In: Proceedings of the 3rd ACM/IEEE International Conference on Human–Robot Interaction, March 2008, pp. 263–270 (2008)

  93. Schweitzer, A., Lewandowski, N.: Convergence of articulation rate in spontaneous speech. In: INTERSPEECH, August 2013, pp. 525–529 (2013)

  94. Scissors, L.E., Gill, A.J., Geraghty, K., Gergle, D.: In CMC we trust: the role of similarity. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, April 2009, pp. 527–536. ACM (2009)

  95. Seo, S.H., Geiskkovitch, D., Nakane, M., King, C., Young, J.E.: Poor thing! Would you feel sorry for a simulated robot?. In: Proceedings of International Conference on Human–Robot Interaction—HRI’15, pp. 125–132. ACM (2015). https://doi.org/10.1145/2696454.2696471

  96. Shawar, B.A., Atwell, E.: A comparison between Alice and Elizabeth chatbot systems. University of Leeds, School of Computing research report 2002.19 (2002)

  97. Shawar, B.A., Atwell, E.: Chatbots: are they really useful?. In: Ldv Forum, January 2007, vol. 22, no. 1, pp. 29–49 (2007)

  98. Sidaras, S.K.: Hearing what you expect to hear: the interaction of social and cognitive mechanisms underlying vocal accommodation. Doctoral dissertation, Emory University (2011)

  99. Sinha, T., Cassell, J.: We click, we align, we learn: impact of influence and convergence processes on student learning and rapport building. In: Proceedings of the 1st Workshop on Modeling INTERPERsonal SynchrONy And infLuence, pp. 13–20 (2015). http://doi.org/10.1145/2823513.2823516

  100. Spencer-Oatey, H.: (Im)Politeness, face and perceptions of rapport: unpackaging their bases and interrelationships. Polit. Res. 1(1), 95–119 (2005). https://doi.org/10.1515/jplr.2005.1.1.95

    Google Scholar 

  101. Stewart, M., Brown, J.B., Boon, H., Galajda, J., Meredith, L., Sangster, M.: Evidence on patient–doctor communication. Cancer 25(1999), 30 (1999)

    Google Scholar 

  102. Strait, M., Briggs, P., Scheutz, M.: Gender, more so than age, modulates positive perceptions of language-based human–robot interactions. In: 4th International Symposium on New Frontiers in Human Robot Interaction, April 2015, pp. 21–22 (2015)

  103. Tanaka, F., Matsuzoe, S.: Children teach a care-receiving robot to promote their learning: field experiments in a classroom for vocabulary learning. J. Hum. Robot Interact. 1(1), 78–95 (2012). https://doi.org/10.5898/JHRI.1.1.Tanaka

    Google Scholar 

  104. Tannen, D.: Gender and Discourse. Oxford University Press, Oxford (1994)

    Google Scholar 

  105. Thomason, J., Nguyen, H.V., Litman, D.: Prosodic entrainment and tutoring dialogue success. In: International Conference on Artificial Intelligence in Education, July 2013, pp. 750–753. Springer, Berlin, Heidelberg (2013)

  106. Tickle-Degnen, L., Rosenthal, R.: The nature of rapport and its nonverbal correlates. Psychol. Inq. 1(4), 285–293 (1990)

    Google Scholar 

  107. Vail, A.K., Boyer, K.E., Wiebe, E.N., Lester, J.C.: The mars and venus effect: the influence of user gender on the effectiveness of adaptive task support. In: International Conference on User Modeling, Adaptation, and Personalization, June 2015, pp. 265–276. Springer, Cham (2015)

  108. Vaughan, B.: Prosodic synchrony in co-operative task-based dialogues: a measure of agreement and disagreement. In: Twelfth Annual Conference of the International Speech Communication Association (2011)

  109. Walker, E., Girotto, V., Kim, Y., Muldner, K.: The effects of physical form and embodied action in a teachable robot for geometry learning. In: 2016 IEEE 16th International Conference on Advanced Learning Technologies (ICALT), July 2016, pp. 381–385. IEEE (2016)

  110. Wallace, R.: The elements of AIML style. Alice AI Foundation (2003)

  111. Wang, N., Gratch, J.: Rapport and facial expression. In: 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, September 2009, pp. 1–6. IEEE (2009)

  112. Web Speech API: https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API. Accessed 26 Oct 2018

  113. Weizenbaum, J.: ELIZA—a computer program for the study of natural language communication between man and machine. Commun. ACM 9(1), 36–45 (1966)

    Google Scholar 

  114. Westlund, J.K., Lee, J.J., Plummer, L., Faridi, F., Gray, J., Berlin, M., Dos Santos, K., et al.: Tega: a social robot. In: The Eleventh ACM/IEEE International Conference on Human Robot Interaction, March 2016, pp. 561–561. IEEE Press (2016)

  115. Wheldall, K., Mettem, P.: Behavioural peer tutoring: training 16-year-old tutors to employ the ‘pause, prompt and praise’method with 12-year-old remedial readers. Educ. Psychol. 5(1), 27–44 (1985)

    Google Scholar 

Download references

Funding

This work was supported by the National Robotics Initiative and the National Science Foundation, Grant # CISE-IIS-1637809.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Nichola Lubold.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This paper is void of plagiarism or self-plagiarism as defined by the Committee on Publication Ethics and Springer Guidelines.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Rapport Measures: Participants responded on a Likert scale from 1 to 5

I felt I had a connection with Quinn

I felt I was able to engage Quinn

I think that Quinn and I understood each other

I felt that Quinn was interested in what I had to say

I felt that Quinn was warm and caring

I felt that Quinn was intensely involved in the interaction

I felt that Quinn seemed to find the interaction stimulating

I felt that Quinn was respectful to me

I felt that Quinn showed enthusiasm while talking to me

Social Presence Measures: Participants responded on a Likert scale from 1 to 7

Quinn was easily distracted

I was easily distracted

Quinn tended to ignore me

I tended to ignore Quinn

I sometimes pretend to pay attention to Quinn

Quinn sometimes pretended to pay attention to me

Quinn paid close attention to me

Coding Scheme:

Politeness: “P” is polite to Quinn, follows conversational niceties (like saying hello)

  • Ex 1: Thank you, Quinn

  • Ex 2: ah step four please

Complimenting or praising: “P” praises Quinn

  • Ex 1: good job Quinn

  • Ex 2: great! Now I factor out the two

  • Ex 3: nice!

Name usage: “P” uses Quinn’s name

  • Ex 1: Nice job Quinn (this would contain checks in both the praise column and the name column)

Inclusive: “P” includes Quinn, for example by using ‘inclusive’ language such as “us,” “we,”, “together”, “let’s”

  • Ex 1: Let’s do problem one!

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lubold, N., Walker, E. & Pon-Barry, H. Effects of adapting to user pitch on rapport perception, behavior, and state with a social robotic learning companion. User Model User-Adap Inter (2020). https://doi.org/10.1007/s11257-020-09267-3

Download citation

Keywords

  • Rapport
  • Social robot
  • Pitch
  • Gender
  • Adaptation
  • Social dialog