1 Introduction

The first author and the second author are teaching staff of a special university for students who have a visual impairment or a hearing impairment. The university has a department of synthetic design which is only for hearing impaired students including deaf students. When a teacher of this department conducts a lecture to these students, the teacher uses a sign language, subtitled educational materials, and communication tools such as a whiteboard or a chatting software in usual case. If the teacher is part-time staff or does not have enough skill of sign language, a subtitle translation service will be provided. Such educational techniques are useful and effective for our students with hearing impairment to study.

However, in practical lectures such as teaching how to use a computer software or how to draw a picture, sign language and subtitle translation service are not so effective compare with using them in other normal classes. For example, when a teacher explain a difficult part of how to use a software with showing a model operation, the teacher encounters a problem. At first, he/she cannot produce his/her own sign language and the computer operation at a same time. Second, students cannot see the model operation and the explaining sign language or subtitle sentences simultaneously if they will have translation services. They should see them alternately. Third, these translation has time delay even in the highest quality service and it confuses the explanation with showing the model operation because the sign or subtitle describes slightly before operation.

To solve the problem mentioned above, we developed a teaching support software called “SynchroniZed Key points Indication Tool: SZKIT” [1, 2]. The software is composed of icons and a subtitle text window, and basically it always exists around a mouse cursor to avoid eye motion between a position of the subtitle and the model operation. These icons represent a mouse and modifier keys. They will appear when a teacher clicks a mouse button or presses modifier keys so that students can understand when the teacher clicks a mouse button or presses a key. For example, an icon of mouse with colored left button and an icon of shift-key and alt-key appear around the mouse cursor when the teacher clicks left mouse button with pressing a shift-key and an alt-kay. If the teacher makes a dragging action after that, these icons are kept to be displayed and students can understand the teacher keeps to pressing the mouse button and the shift-key. Figure 1 shows appearance of SZKIT in example described above. Addition to it, the subtitle text window shows explanation of the operation by sentences. The explanation texts are prepared in advance and are changed by the teacher arbitrarily. Hence it is not a translation software and there is no time delay.

Fig. 1.
figure 1

Appearance of Synchronized Key points Indication Tool: SZKIT

Full size image

Moreover, using SZKIT in practical lectures has another important advantage. That is every teacher can make a subtitled educational video material without any difficulty, by just recording a desktop of a computer during the lecture. The first author uses SZKIT in his class in which students learn how to use an “Adobe Illustrator,” and record his operation during the lecture to make video materials. The produced video are used for revising in self-practice by students who could not understand the contents of the lecture. Nevertheless such a revising, it was clear that some students had difficulty to understand new technique of Illustrator. Therefore, we decided to investigate where the students see when they learn new technique by the revising video by recording eye movement.

2 Experimental Procedure

Experimental circumstance was computer with dual display. One of them is for operation by subjects and the other display is for video material that explain how to use an Illustrator. This circumstance is a usual setting of the practical lecture in our university and that is the reason why we prepared them. The number of participated students who has hearing impairment is ten. The experimental procedure with which these subjects tried to do is as follows.

At first, these subjects watched a video material continuously without returning to the same segment. The material was made with SZKIT which adds subtitles around a mouse cursor. The purpose of this procedure is that we would like to observe how students see the model operation and how they make their own operation in passive studying. After that, they took an examination to clear where the point they cannot understand well was. Next to the examination, they watched the same video material again and in this time they were allowed to return to any segment they wanted, to observe where they see in active studying. During both experimental procedures, the eye movement data in the display for video material were recorded by the eye tracking system in front of them and the operating sequences were recorded by a desktop capture software and the position of mouse pointer is considered as eye focusing point if it moves (Fig. 2).

Fig. 2.
figure 2

Experimental circumstance for watching eye movement of hearing impaired students when they tried to study how to use an Illustrator.

Full size image

3 Results and Consideration

The result data from eye tracking system showed that some students who had a difficulty to learn tend to take more time to read explanation subtitles than to see model operation and it causes to miss out to see important operation in the video material.

Fig. 3.
figure 3

Durations of seeing model operation and durations of reading subtitles comparing between all subjects and subject G.

Full size image
Fig. 4.
figure 4

Duration of seeing model operation and duration of reading subtitles comparing between faster group and slower group.

Full size image

Figure 3 shows average durations of all subjects and one subject named G, who needed more time to acquire the new skill and who marked lower score in the examination compare with other subjects. In this graph, upper graphs represent durations of seeing the model operation and lower graphs represent durations of reading subtitles in the text window of SZKIT. The average time shows that all of them needed more time to read subtitles though, the subject G marked especially long time to read subtitles and the duration to see the model operation is short. The same tendency was able to be seen in other subject who needed more time to learn.

Figure 4 also shows average durations of seeing model operation and reading subtitles by two groups. Members of group A are students who took less than 10 min to acquire the skill and members of group B are students who took more than 10 min. The number of the group A is three and the number of the group B is seven. By this graph, it is clear that slower members took more time to read subtitles.