Proposal of Remote Face-to-Face Communication System with Line of Sight Matching Based on Pupil Detection
In general, remote face-to-face communication systems via the Internet, it is impossible to match the eyes of a user and a communication partner with each other because their face positions displayed in PC screens, at which they look, are different from the positions of the cameras for shooting their faces. To solve this problem, we proposed a remote head-free system which was based on the combination of a wide-angle color camera located behind a 45-deg inclined magic mirror, our pupil detection technique, image deformation techniques, and so on. The same two systems were assigned to the user and the partner. The narrow region image including a whole of the user’s face image was cut out from the image obtained from the wide-angle color camera in the user’s system, was displayed in the partner’s system so that the midpoint of the detected right and left pupils of the user’s face image virtually located at the position of the color camera in the partner’s system, and vice versa. In addition, to deal with large back and forth or lateral head movements, the displayed face image of the user was adjusted in its size and distorted by using the projection transformation so that it always looked like the same size as the user’s face and looked like the front face as seen from the partner, and vice versa. The questionnaire results obtained from experiment showed the tendency that the subjects’ lines of sight matched each other, and the other questionnaire items about the system showed high evaluation.
KeywordsRemote communication Pupil detection Projection transformation
- 1.Yaskawa information systems: NetSHAKER TalkEye. https://www.ysknet.co.jp/