1 Introduction

The visually impaired find it very difficult to interact with computers, particularly to perceive figures. No good off-the-shelf tactile display presently allows the visually impaired to perceive and draw figures. Therefore, we developed an affordable dynamic device presenting spatial information in a tactile manner. The device features dynamic changes in tactile stimulation (electrostatic force based stimulation to a finger). The force magnitude changes by position; spatial information is thus imparted. Here, we describe our prototype and preliminary evaluation thereof. We explored whether figures were recognized when fingers were stimulated, and we discuss planned future improvements.

2 A Tactile Graphics Display

The visually impaired must recognize shapes in a tactile manner. However, conventional graphics are static. Several researchers have explored tactile displays of spatial information; for example, a pin array has been used to stimulate the fingers [1, 2]. Ohka et al. developed a “tactile mouse”; this combined a computer mouse with a pin array to present graphical information to the fingers via simple rendering. For example, when the mouse moves inside a figure, the pins representing the figure protrude to stimulate the finger. Thus, the user captures the figure edges and can trace the outline. The pin array is a form of mechanical stimulation and is thus relatively easy to use. Another tactile feedback method features electro stimulation [3]. Uematsu et al. sought to display characters using this method. However, the recognition rate was only 76.2% for large characters, thus lower than that afforded by pin arrays. Also, in the cited system, the user must wear a finger pad to allow stimulation. We also developed an electrostatic-force-based, tactile stimulation apparatus featuring a high-voltage control circuit, an electrode, and an insulator, all made of easily available materials. Recently, lateral-force-based, tactile feedback devices employing static electric fields have been developed (Senseg Inc., Bau, et al. [4]), and many researchers have explored their parameters (input frequencies, waveforms, and amplitude modulations [4,5,6,7,8]).

Bateman et al. explored whether dots could be located using a tactile electrostatic device [9]; all subjects found the dots quickly. Xu et al. investigated whether simple shapes (a circle, square, and triangle) could be recognized using an electrostatic tactile display [10]. The average recognition rate was 56%. As only simple shapes were tested, it is unclear whether more complex shapes could be recognized. We added complex figures to basic shapes when deriving recognition rates and exploring the figures captured by users.

3 A Graphics System Featuring an Electrostatic Tactile Display

The device features a high-voltage generator, an electrode, and an insulator. The generator was developed by Kajimoto et al. The device includes an mbed LPC1768 microcontroller maintaining the output voltage at a maximum of 600 \(\mathrm {V}\) by modifying the firmware. Various waveforms can be output to the electrode and used to impart different forms of electrostatic tactile feedback. The electrode is covered with an insulating plastic film 15 \(\upmu \)m in thickness that the user touches.

Fig. 1.
figure 1

A schematic of the tactile graphics system (left) and a photograph of the device (right).

Full size image

An electrostatic force is generated only when the user slides his/her finger on the display. When a high voltage is applied to the electrode, dielectric polarization is generated in the finger. In this state, the electrode applies a static attractive force to the finger, but the force is too weak to feel. However, when s/he slides his/her finger on the display, s/he feels a tactile sensation (as if the surface texture has changed). Because our electrode is a single large sheet, the system delivers only one type of force at any time. Thus, when changing the force by finger position, the system should sense the finger position, and then changes the force accordingly. We incorporated an infrared-based touch sensor (zForce AirTM 295) to that end.

The tactile display is enclosed in an acrylic frame; the user can easily grasp the display area. The width is 295 \(\mathrm {mm}\) and the height 180 \(\mathrm {mm}\). A figure is represented as an area within the tactile region; the user can feel a tactile sensation within the figure but not outside. The shape presented can be switched using a PC (Fig. 1).

4 Experiment 1: Identification of Simple Figures

We explored whether it was possible to discern figures; we measured the identification rates of simple figures.

4.1 Outline of Experiment 1

We recruited three participants (all males; no handicapped subject). We prepared four figures: a circle, a square, a triangle, and a star (Fig. 2), all 8 \(\mathrm {cm}\) high. The task was to choose the correct figure from among the four. The recognition rate was the proportion of correct answers. The input waveform for tactile feedback was a 100 \(\mathrm {Hz}\) rectangular wave. The participants were asked to touch the tactile display using the right index finger, irrespective of the dominant hand. The tracing speed was limited to 30 \(\mathrm {cm/s}\) to ensure that the sensor captured finger motion. We set no restriction on either finger pressure or the direction of movement.

Fig. 2.
figure 2

The four figures.

Full size image

Initially, the experimenter gave an overview of the task and asked all participants to complete a consent form. The device was covered during this introduction. At this point, participants were not given any information on the shapes to be displayed. Then, all participants were blindfolded while touching the display, to minimize visual information and reproduce the challenge faced by the visually impaired. Participants were then asked to review the location and the size of the display using the hands in a blinded manner, and then to practice. First, they were asked if they could feel tactile stimulation; all answered “yes”. Then, they were allowed to slide their fingers freely on the display for 1 min. During the practice session, we used only the circle, and we did not tell the participants that this was displayed. During the experiment, we displayed the four shapes five times (20 trials in total). The shapes were described to the participants before the experiment commenced. When the participant gave the name of the shape s/he felt, the next figure was immediately presented; that participant was not told whether s/he was correct. Participants were allowed 5-min breaks after every five trials. Finally, they were asked to make short comments. The experiment took about 1 h.

4.2 Results and Discussion

The average correct identification rate was 68.3%. Table 1 shows the confusion matrix for each shape. The triangle and star were relatively easily identified; the circle and square identification rates were lower than those for the triangle and star, which were rarely mistaken for other shapes. However, the circle and square were often mutually mistaken.

Table 1. The correct answer rates for each figure.
Full size table

The high recognition rates for the triangle and star may reflect their unique characteristics. Both have acute angles at the vertices. When tracing the top or bottom of the triangle or star, participants can feel these angles and identify the shapes correctly. However, tracing an edge with a finger is relatively difficult, which may explain the low recognition rates of the square and circle. When participants sought to identify shapes by tracing, they tried to move their fingers horizontally or vertically. However, because they were blindfolded, they often moved the fingers somewhat obliquely and misidentified the shape. For example, when a square is displayed, the finger expects a long edge, but if the participant traces the upper edge of the square at (even a slight) angle from the horizontal, stimulation is brief. Therefore, the participant thought that the line was curved and misidentified the square as the circle.

5 Experiment 2: Free Drawing

We next explored how accurately graphic information could be grasped in the absence of prior information, and also how shapes were captured in more realistic settings. Here, we asked participants to draw shapes felt on the device on paper.

5.1 Outline of Experiment 2

Five participants (no handicapped person; two females) were recruited. The participants were shown a shape in a tactile manner, and then asked to draw the exact shape sensed on the paper. Figure 3 shows the shapes presented by the display. By comparing the drawn to the original shapes, we explored how accurately a shape was recognized using our display.

Fig. 3.
figure 3

The figures for experiment 2.

Full size image

As for experiment 1, we first gave an overview of the test, blindfolded all participants, and had them operate the device. The input waveform was a rectangular wave of 100 \(\mathrm {Hz}\). After practice, each participant was presented with one of the shapes and asked to trace it. When each participant concluded that s/he “knew” the shape, the blindfold was removed and the shape drawn on paper. All participants were asked to preserve shape orientation when drawing. Furthermore, if they were not confident that they were correct, they were permitted to touch the tactile device again (as many times as they wished) wearing a blindfold. After drawing the figures, participants were asked to describe the figures orally, to explore how they recognized the shapes. After participants finished drawing, they took a 1-min break and then moved to the next figure. The figures were presented in random order. Finally, all participants were asked to give short comments. The experiment required about 1 h.

Fig. 4.
figure 4

The results of experiment 2.

Full size image

5.2 Results and Discussion

Figure 4 shows the figures drawn by the participants. In terms of simple shapes, the characteristics of the triangle and square were well captured. However, most participants were unable to capture the circle. As shown in Fig. 4, two participants thought that the circle was a square, as was also the case in Experiment 1. For the square, all participants captured the quadrilateral features well. However, several participants drew vertically elongated rectangles, although we displayed a square. In terms of the complex shapes (the star, arrow, and human), no participant fully captured the figures. However, parts thereof were captured by several. For the star, some participants captured protrusions of the upper or lower part. Similarly, for the arrow, the shape was well-grasped by several subjects. For the human, a foot or a bar-like structure was captured by several participants. However, none grasped the shape of the arm, and only some of the body was drawn.

Fig. 5.
figure 5

Example of how to trace a finger.

Full size image

Participants found it easy to capture figures (or parts thereof) featuring characteristic vertices and edges, thus triangles, squares, and arrows, but difficult to capture figures with curved edges such as circle and a human shape. Most of the participants sought to recognize figures by tracing horizontal or vertical lines (as in Fig. 5(a)). However, when the tracing interval is long (as in Fig. 5(b)) or the required tracing direction is not a horizontal/vertical line (as in Fig. 5(c)), mistakes were sometimes made. Particularly, for the equilateral triangle, it was simple to discern that the top was narrow and the bottom wide. In addition, as the width increases linearly from the top to the bottom, the triangle was relatively easy to identify.

It is possible that the rectangular shape of the display affects the finger sliding speed; participants thus tend to see the square as a rectangle. The width of the device is longer than the height. Therefore, when the participant is tracing, horizontal finger movement is faster than vertical movement; less time is required when tracing the square horizontally. Thus, the square tended to be recognized as a rectangle. Our method facilitates recognition of simple graphics such as triangles and squares, with acute or right angles, but it is difficult to capture the features of complex figures such as stars.

6 Future Work: Improvements

In this study, participants tried to grasp figures by searching for edges while moving the fingers (mostly) horizontally or vertically. The recognition rate can be improved, however, by delivering the shape differently. For example, we may be able to navigate the finger in the horizontal or vertical direction, or emphasize the edges of shapes more effectively. Our current device delivers a uniform stimulus when the finger is inside a shape (Fig. 6(a)). Edges and vertices can be highlighted by increasing the tactile stimulation they afford (Fig. 6(b)), thus distinguishing them from the interior. For example, a 20 \(\mathrm {Hz}\) rectangular wave can be applied to edges and a 100 \(\mathrm {Hz}\) wave to the interiors. Also, a 200 \(\mathrm {Hz}\) wave can be applied to vertices. Different tactile sensations can be delivered by changing the frequency of the rectangular wave [11].

Fig. 6.
figure 6

The current display. The display area is wholly internal (left). We will aim to display three tactile sensations (right).

Full size image

The shearing force applied to the finger also changes when the waveform is modulated. Saga et al. proposed that both large bumps and small textural changes could be simultaneously felt by changing the shearing force [12]. We will use this method to apply bumps to edges and vertices to emphasize the unique characteristics of different graphics.

7 Conclusion

We explored whether shapes were correctly identified using our electrostatic tactile display. We presented four shapes and asked volunteers to identify them. The shape recognition rate was 68%. Furthermore, we investigated to what extent a shape could be captured without prior information. We presented six types of shapes; participants were asked to draw them on paper. They found it easy to recognize graphics with characteristic vertices or edges, such as triangles and squares, but difficult to recognize curves. In the future, we will improve our method and enroll visually impaired subjects.