A Video Communication System with a Virtual Pupil CG Superimposed on the Partner’s Pupil

  • Yoshihiro SejimaEmail author
  • Ryosuke Maeda
  • Daichi Hasegawa
  • Yoichiro Sato
  • Tomio Watanabe
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10904)


Pupil response plays an important role in expression of talker’s affect. Focusing on the pupil response in human voice communication, we analyzed the pupil response in embodied interaction, and demonstrated that the speaker’s pupil was clearly dilated during the burst-pause of utterance. In addition, it was confirmed that the pupil response is effective for enhancing affective conveyance by using the developed system in which an interactive CG character generates the pupil response based on the burst-pause of utterance. In this study, we develop a video communication system with a virtual pupil CG superimposed on the partner’s pupil for enhancing affective conveyance. This system generates a virtual pupil response in synchronization of the talker’s utterance. The effectiveness of the system is demonstrated by means of sensory evaluations of 12 pairs of subjects in video communication.


Human interaction Nonverbal communication Affective conveyance Line-of-sight Pupil response 



This work was supported by JSPS KAKENHI Grant Numbers JP16K01560, JP26280077.


  1. 1.
    Schoeffmann, K., Hudelist, M.A., Huber, J.: Video interaction tools: a survey of recent work. ACM Comput. Surv. 48(1), 14:1–14:34, (2015)CrossRefGoogle Scholar
  2. 2.
    Watanabe, T.: Human-Entrained Embodied Interaction and Communication Technology, Emotional Engineering, pp. 161–177. Springer, London (2011). Scholar
  3. 3.
    Condon, W.S., Sander, L.W.: Neonate movement is synchronized with adult speech. Science 183, 99–101 (1974)CrossRefGoogle Scholar
  4. 4.
    Sejima, Y., Watanabe, T., Jindai, M.: Speech-driven embodied entrainment character system with pupillary response. Bull. JSME Mech. Eng. J. 3(4), 832–837 (2016). Paper No. 15-00314Google Scholar
  5. 5.
    Argyle, M., Dean, J.: Eye contact, distance and affiliation. Sociometry 41(3), 289–304 (1965)CrossRefGoogle Scholar
  6. 6.
    Hess, E.H.: Attitude and pupil size. Sci. Am. 212(4), 46–54 (1965)CrossRefGoogle Scholar
  7. 7.
    Vertgaal, R., Veer, V.G., Vons, H.: Effects of gaze on multiparty mediated communication. In: Proceedings of Graphic Interface 2000, pp. 95–102 (2000)Google Scholar
  8. 8.
    Maeda, R., Egawa, S., Sejima, Y., Sato, Y., Watanabe, T.: Development of a video chat system superimposing a pupil CG synchronized with utterance on the partner’s pupil. In: Proceedings of 23rd International Symposium on Artificial Life and Robotics (AROB2018), pp. 236–239 (2018)Google Scholar
  9. 9.
    Johnson & Johnson Co, Ltd.: The “golden ratio of pupil” of white eyes and black eyes is “1: 2: 1” (2014). Accessed 8 Feb 2018. (in Japanese)
  10. 10.
    Oyama, S., Imai, S., Wake, T.: Handbook of Sense and Perception Psychology, p. 895. Sei Shin Shobo, Tokyo (1994). (in Japanese)Google Scholar
  11. 11.
    Kawatani, H., Kashiwazaki, H., Takai Y., Takai, N.: Feature evaluation by moe-factor of ANIME character image and its application. In: IEICE Technical Report of Image Engineering, vol. 34, no. 6, pp. 113–118 (2010). (in Japanese)Google Scholar
  12. 12.
    Luce, R.D.: Individual Choice Behavior: A Theoretical Analysis. Wiley, New York (1959)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Yoshihiro Sejima
    • 1
    Email author
  • Ryosuke Maeda
    • 2
  • Daichi Hasegawa
    • 2
  • Yoichiro Sato
    • 1
  • Tomio Watanabe
    • 1
  1. 1.Faculty of Computer Science and Systems EngineeringOkayama Prefectural UniversitySoja-shiJapan
  2. 2.Graduate School of Computer Science and Systems EngineeringOkayama Prefectural UniversitySoja-shiJapan

Personalised recommendations