Proposal of Remote Face-to-Face Communication System with Line of Sight Matching Based on Pupil Detection

  • Kiyotaka FukumotoEmail author
  • Yoshiyuki Yamamoto
  • Yoshinobu Ebisawa
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 850)


In general, remote face-to-face communication systems via the Internet, it is impossible to match the eyes of a user and a communication partner with each other because their face positions displayed in PC screens, at which they look, are different from the positions of the cameras for shooting their faces. To solve this problem, we proposed a remote head-free system which was based on the combination of a wide-angle color camera located behind a 45-deg inclined magic mirror, our pupil detection technique, image deformation techniques, and so on. The same two systems were assigned to the user and the partner. The narrow region image including a whole of the user’s face image was cut out from the image obtained from the wide-angle color camera in the user’s system, was displayed in the partner’s system so that the midpoint of the detected right and left pupils of the user’s face image virtually located at the position of the color camera in the partner’s system, and vice versa. In addition, to deal with large back and forth or lateral head movements, the displayed face image of the user was adjusted in its size and distorted by using the projection transformation so that it always looked like the same size as the user’s face and looked like the front face as seen from the partner, and vice versa. The questionnaire results obtained from experiment showed the tendency that the subjects’ lines of sight matched each other, and the other questionnaire items about the system showed high evaluation.


Remote communication Pupil detection Projection transformation 


  1. 1.
    Yaskawa information systems: NetSHAKER TalkEye.
  2. 2.
    Ebisawa, Y., Fukumoto, K.: Head-free remote eye-gaze detection system based on pupil-corneal reflection method with easy calibration using two stereo-calibrated video cameras. IEEE Trans. Biomed. Eng. 60(10), 2952–2960 (2013)CrossRefGoogle Scholar
  3. 3.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Kiyotaka Fukumoto
    • 1
    Email author
  • Yoshiyuki Yamamoto
    • 1
  • Yoshinobu Ebisawa
    • 1
  1. 1.Graduate School of Integrated Science and TechnologyShizuoka UniversityHamamatsuJapan

Personalised recommendations