1 Introduction

Our university, Tsukuba University of Technology is a unique one, which is only for visually handicapped or hard of hearing students including blind and deaf students. We have two campuses for each type of impaired students respectively. The campus for the hard of hearing students has well equipped lecture room for them. For example, several beamers which are hanging from the ceiling in front of a whiteboard in the lecture room can be slid in lateral direction and it allows teachers to overlay their handwritings on the whiteboard with outputs by the beamer. Another example is that some tables has whiteboard surface and students and teacher can discuss with writing and drawing on the table directly. Addition to such equipment, most of teachers in the campus can talk with sign language in their classes. We also prepares captioning services for several classes which is conducted by visiting lecturers who does not know sign language.

However, in the class of learning how to use a drawing software like an Adobe Illustrator, there still remains some problems. Teachers frequently shows a model operation to their students using a beamer in a practical lecture class though, they cannot explain their operation with sign language by themselves at the same time. Even though a sign language translator or a captioning service which is effective in usual conference [1], the translation would be delay and students cannot see both of the translation information and the model operation, because these positions are far from each other. These delay and position difference are critical problems especially in the class of teaching how to use Illustrator, in which combination keys and its timing is quite important for the correct operation.

Therefore, we developed and estimated a supporting software which shows teacher’s operation using icons around a mouse cursor in past research project [2]. The appearance of the software is shown in Fig. 1. It shows which mouse button was clicked and what kind of keys was pressed. The mouse icon appears when the teacher clicks the mouse button and it keeps to display until he/she releases the button, hence students can understand that the operation was click or double click or drag. Addition to them, instruction text is displayed under the mouse cursor as well. This software is apparently effective to teach how to use Illustrator and students can acquire techniques of it in shorter period than the way before. On the other hand, few students still has difficulty to understand the timing of clicking and pressing a combination keys, because some deaf or hard of hearing students tend to take much time to read text information and it might cause to miss these icons. We need additional system to teach to such kind of students.

Fig. 1.
figure 1

Appearance of the supporting software for deaf or hard of hearing students. Indication icons and explanation texts appears besides mouse cursor.

Full size image

2 System Design and Components

To solve the problem mentioned above, a pair of vibration ring device for an index finger and a middle finger is developed. The overview of the device is shown in Fig. 2. Deaf or hard of hearing student wares this device on his/her dominant hand, and it acquaints the student which mouse button is pressed by the teacher. The ring continues to vibrate until the teacher releases the mouse button, hence the students can understand how long the teacher presses the mouse button. It means they can distinguish the action is a click or a drag by the vibration pattern. Of course when the teacher makes double click action, the ring vibrates twice and the student can be aware of it. We named the device as Synchronized Click Action Transmitter, SZCAT.

Fig. 2.
figure 2

An overview of vibration ring devices and its controller.

Full size image

These rings consist of plastic cases and Velcro straps. The size of these cases is 29.5 [mm] (length) by 19.5 [mm] (width) by 17.5 [mm] (height). Inside them, there are small solenoid coils (ROB-11015, SparkFun electronics) respectively. The end of the axis of the coil protrudes 5 [mm] from the case. These solenoids are assigned in parallel to each fingers and the axis knocks the inside wall of the case when an electric power comes. The distance between the end of the axis and the wall of the case effects the feeling of vibration. By the difference of this distance, the vibration on the index finger is smooth and light. On the contrary, the vibration on the middle finger is rough and strong. These difference adds clear information of which button was pressed.

Here, the reason of selecting solenoid coils instead of vibrating motors can be said that a feelings of the solenoid coil is stronger than that of the motor. Moreover, the fact that the solenoid vibration starts and stops immediately and clearly is one of the reason.

A controller unit for these rings is composed of an Arduino Nano and a small circuit with switching transistors. To avoid back electromotive forces by coils, diodes are assigned on the circuit. A software which uses a mouse hook function detects the action that the mouse button is pressed, then, the software sends a message to itself. When the software receives this message, a signal to control Arduino is send via USB port. Finally the switching transistor drives the solenoid coil and it knocks the case, then the student feels the ring is vibrating. The device produces approximately 100 [Hz] vibration by the software on Arduino. As a result, the vibration makes an attention and the students correctly comprehends when the mouse action starts and ends, which side of buttons are pressed.

3 System Estimation by Hard of Hearing Students

3.1 Subject Profile and Procedure

Subjects for the estimation were deaf or hard of hearing students in a department of synthetic design, in our university. They have official certification for disability person in Japan. The total number of estimating students was thirteen. At first, they were required to relearn basic skills of Adobe Illustrator with wearing SZCAT on their index finger and middle finger. All of them already know these skills. After the relearning session with SZCAT, they were requested to answer several questionnaire describes below.

3.2 Estimation Questionnaires

Prepared questionnaires were as follows;

  1. 1.

    If you had stress feeling from SZCAT, Please describe it. Especially about its weight and fitting feelings.

  2. 2.

    How correct SZCAT could transmit teacher’s mouse operation timing?

  3. 3.

    What do you think about the difference of vibration between index finger and middle finger?

  4. 4.

    Is SZCAT helpful to transmit teacher’s mouse operation to students?

  5. 5.

    Is SZCAT effective in practical lecture class?

Questionnaire from number one to three is required to answer in free-form, number four and five is required to answer in range of scores on a 5-point Likert scale.

3.3 Estimation Results

In the results of questionnaire number one, seven students answered there were no stress from waring SZCAT. Remained six students described some comment about the stress feelings. Four students of the six mentioned that the cables between solenoid cases and the controller unit is cumbersome for operation, and two students of the four mentioned that the weight was little bit heavy. Two students of the first six mentioned that the Velcro tape is obstructive. One student commented that the sound from solenoids on middle finger would be noisy in case he would uses it for a long time. The problem of the cable should be solved in the future though, the fact that half of the estimating students do not care about the ring device surprised to us. Actually some students started to talk in sign language with their hand without putting off SZCAT after the relearning session, then the experimenter scrambled to stop it to avoid to break the device.

About questionnaire number two, all of students answered optimistic. It seemed that they can feel the teacher’s mouse action correctly. One student proposed a function of adjusting the frequency of vibrate, another student commented the solenoid motion of the middle finger was little bit strong in negative meanings.

The third questionnaire reveals the difference of vibration is practical and effective. Twelve students answered that they preferred this difference and ten of these twelve commented the condition (vibration on the index finger is smooth and it on the middle finger is rough) is fit for learning. Because the click by the middle finger is not so frequently operations, hence stronger vibration is impressive to learn the timing.

The results of questionnaire number four and five are shown in Fig. 3. About questionnaire number four, six students answered “5-strongly agreed” and another six students answered “4-agreed,” last one student answered “3-intermediate.” The trend of this result is similar to the answers of questionnaire number five. Five students answered “5-strongly agreed” and another seven students answered “4-agreed,” last one student answered “3-intermediate.” Totally twelve of thirteen (92%) thought that SZCAT is effective in practical lecture class.

Fig. 3.
figure 3

Results of questionnaire number four and five.

Full size image

4 Summary

Overall, we can say that the vibration ring device which transmits teacher’s mouse button operation during making a model drawing is helpful and effective in the lecture class, since twelve students in the thirteen estimating students made positive reaction to the device. We think that the character of solenoid coil vibration is a kind of reason they preferred to it. The components of “solenoid case” will be available not only for this device but for other notification device which deaf or hard of hearing person use in daily life.