Keywords

1 Introduction

Blind people can have access to digital documents using specific software called “screen readers”. Screen readers can present in a linear way, either through speech synthesis or braille, the content of a document or elements of a graphical interface. However, access to graphics and other two-dimensional information is still severely limited for the blind. It is not easy for them to explore 2D structures such as mathematical formulas, maps, electronic circuit diagrams…) using a screen reader. The user is then faced with many problems such as disorientation and difficulty to memorize and to build a correct mental model.

The work presented in this paper is a first step of a larger project that aims at defining new ways for the blind to have access to electronic documents while preserving spatial layout of the document. The main idea of the project is to use a dot-matrix display to present the general spatial layout of the document. Each element of the document structure (title, paragraph, image, etc.) will be represented by a geometrical form that will reflect the size and the position of the element in the document. When the user explores this spatial layout, he/she will be able to access to the detailed content of the element that is currently under his/her fingers, through another modality such as speech synthesis or braille.

As a preliminary step, two questions should be addressed. First, which geometrical form should be used? Obviously, using rectangles is the first idea that comes in mind but is it possible to use other forms depending for instance on the information type? Second, which rendering method allows the best and faster recognition process?

2 Related Work

Different methods exist to translate graphical information into a tactile form to make it accessible to a blind person [2, 3]. 3D printing, collage, thermoforming and embossed paper [8] are great for educational purposes but they all have the same drawback: they produce static documents which prevents useful interactive operations such as zooming and scrolling. This leads to a drastic reduction of information density due to the limited resolution of the skin. Furthermore, their quality decreases with use and they require huge space to be stored.

Other devices that allow refreshable tactile display, exist. They can be classified into two main categories. The first category concerns the devices that allow a tactile exploration of a virtual large surface using a small tactile device. A typical example of such devices is the VTPlayer mouse [9, 10] that can be used as a classical mouse to explore a virtual surface while receiving tactile stimuli through the index finger thanks to its 4 * 4 Braille dots. The main advantage of this device is its low cost and portability. However, exploration is generally done using only one finger which leads to important time exploration before achieving recognition even of very simple shapes.

Another similar device is the Tactograph [11, 12]. The Tactograph includes a STReSS2 tactile display (see Fig. 1) [5] which allows the production of a variety of tactile stimuli providing richer rendering of textures using thin strips for stretching the skin of the finger. However, it still allows only a single finger exploration.

Fig. 1.
figure 1

(a) Active area of the STReSS2 tactile display, (b) STReSS2 mounted on a planar carrier, and (c) usage of the device. Extracted from Levesque’s website (http://vlevesque.com/papers/Levesque-HAPTICS08/)

Full size image

The second category concerns the devices that allow the tactile exploration of a large physical surface using several fingers of both hands [6, 7]. The surface is generally composed by a matrix of a high number of Braille dots which play the same role as pixels in screens. An example of such device is the dot-matrix display designed by Shimada et al. [4] which offers 32 × 48 Braille dots. The main drawback of this kind of devices is their cost.

In this paper, we present a study conducted using a device of this second category to identify the rendering features that allow the fastest and most reliable recognition of geometrical shapes. The protocol of this study was inspired by a study conducted by Levesque and Hayward [1] on a device of the first category (the STReSS2 device).

3 User Study

For this study, we have used a 3600-dot-matrix display (60 × 60 dots) from metec AG. The display surface is 15 × 15 cm2. The dots can be only in 2 states: up or down. The device is presented in Fig. 2. It has also a set of buttons (some of them can be used as a braille keyboard) and a scrollbar.

Fig. 2.
figure 2

The dot-matrix display used in the study

Full size image

3.1 Experimental Conditions

Shapes.

Six different shapes were used in the experiment. We choose the same shapes as the ones used in [1]. As shown in Fig. 3 these shapes are: square, circle, simple triangle, right triangle, diamond and cross.

Fig. 3.
figure 3

The six shapes used in the study

Full size image

Size of shapes.

In [1] the shapes were selected to fill a 2 or 3 cm square, leading to two different sizes: small and large. In our experiment, we used three different sizes: small, medium, and large. Our small and medium sizes correspond respectively to small and large sizes of Levesque’s study (2 and 3 cm). Our large size corresponds to a 4-cm bounding square. We added this larger size because the dot-matrix display has less resolution than the STReSS2 tactile display [5]. In the STReSS2 device, the center-to-center distance between adjacent actuators is 1.2 × 1.4 mm and the actuators can deflect toward the left or right by 0.1 mm. In our dot-matrix display, the horizontal and vertical distances between the dots centers are the same and are equal to ~2.5 mm. The diameter of each dot is ~1 mm. So, we kept the same sizes as in [1] but added a supplementary (larger) one in case recognition performances would be affected by poorer resolution of the dot-matrix display.

Rendering of shapes.

Six different rendering methods were used during the experiment depending on the way the outlineFootnote 1 and the inside of the shapes are displayed. Each of these two elements can be rendered in 3 different ways: static, vibrating, empty. The vibration effect is obtained by putting the dots up and down alternatively with a 10 Hz frequency to not damage the device. Theoretically this should lead to 9 different Rendering Methods (RM) as shown in Table 1.

Table 1. Features of different renderings of shape.
Full size table

However, if we look deeper at these 9 rendering methods, we can see that 3 of them (RM7, RM8, RM9) are not pertinent. RM9 displays nothing since both the outline and the inside of the shape are empty. RM7 represents the same rendering method as RM1 because only the size of the shape is a little smaller if we remove the outline. Similarly, RM8 and RM5 represent the same rendering method for the same reason. Since the size factor is evaluated separately, we have decided to not consider RM7 and RM8. Figure 4 illustrates examples of the 6 RMs that were kept. Note that in [1], only RM1, RM3 and RM6 were used.

Fig. 4.
figure 4

The six rendering methods used in the study.

Full size image

3.2 Participants

Data were collected from 40 sighted subjects (31 men and 9 women), aged from 18 to 40 (M age = 23.7; SD age = 5.2). Many participants were people with a computer science background. All participants filled out a background questionnaire, which was used to gather information on personal statistics such as age and education level. Our sample was composed by 34 right-handers and 6 left-handers. All participants were naive with respect to the experimental setup and purpose of the experiment.

3.3 Protocol

First, each participant is invited to sign an informed consent and then an overview of the experiment is provided. The experiment was conducted in two main phases:

  1. 1.

    A training phase (~5 min) allowing each participant to become familiar with the geometrical shapes and the rendering methods used during the experiment. The six geometrical shapes are presented to the subject (who cannot see them thanks to a raised cover that hides the dot-matrix display) and then we ask him/her to name them. This step is complete when the subject is able to recognize the six shapes.

  2. 2.

    A phase of test where subjects were asked to recognize and to name shapes as fast as possible. This phase was decomposed in three continuous sessions (with a short break between them). The duration of the whole process (3 sessions) was under 1 h of time.

During the test, shapes varied according to the geometrical form, the size, and the rendering method. The order of the forms, sizes and rendering methods was randomly generated across participants. In all, each participant had to recognize 324 shapes (6 forms × 3 sizes × 6 rendering methods × 3 sessions).

3.4 Measures

For each shape, we recorded time to recognize it (in milliseconds) and participant’s answer. The participants used a button to display/hide the figure (which starts/stops the chronometer). The answers were given verbally. We developed a program to extract dependent variables from the log files that were generated during the test.

4 Results

The results presented in this section are considered statistically significant when p < 0.05. Results are explicitly referred as a “trend” if p is between 0.05 and 0.1. We applied the Shapiro-Wilk test to verify that the variables succeed to satisfy normality assumptions. This is only verified for the recognition time variable. Recognition time was analyzed by means of ANOVAsFootnote 2 with shape, shape size, and combination of rendering methods of shape’s outline and shape’s inside. ANOVAs were calculated using Statistica 9. Post hoc comparisons used the Student’s t-test. A Chi 2 test was performed for the recognition rate.

4.1 Recognition Rate

We first analyzed the results by considering all answers given by the subjects and we conducted an analysis of Chi 2. Results show that recognition rate do not vary according to the geometrical form, the size, and the rendering method. The global mean of the recognition rate is 95%. Table 2 provides the detailed percentages of recognition in each category. Chi 2 analysis reveals that shapes are well recognized whatever the geometrical form, the size, and the rendering.

Table 2. Recognition rate according the shape, the size, and the rendering.
Full size table

4.2 Recognition Time

Shape effect.

We observed a main effect of the geometrical shape on the recognition time (F(5, 195) = 39,295, p < 0,001 see Fig. 5). Post hoc comparisons suggested that participants tended to recognize more quickly crosses, squares, and right triangles than circles, diamonds, and simple triangles. There was no significant difference between crosses, squares, and right triangles. In addition, there was no significant difference between circles, diamonds, and simple triangles.

Fig. 5.
figure 5

Recognition time according the geometrical shapes.

Full size image

Size effect.

We observed a main effect of the size on the recognition time (F(2, 78) = 86,157; p < 0,001). Post hoc comparisons suggested that participants recognized more slowly small shapes (Mean = 6219,87; SD = 1996,642) than medium (Mean = 5300,23; SD = 63838,58) or large shapes (Mean = 5242,80; SD = 1674,63). There was no significant difference between the medium and large shapes.

Rendering method effect.

We observed a main effect of the rendering method on the recognition time (F(5, 195) = 73,237, p < 0,001 see Fig. 6). Post hoc comparisons suggested that the best configuration is when the rendering method combines static outline with empty inside. Participants recognize faster the shapes with this configuration compared to other configurations. In addition, post hoc comparisons suggested that the worst configuration is when the rendering method combines vibrating outline with vibrating inside. Participants recognize more slowly the shapes with this configuration compared to other configurations. Finally post comparisons suggested that the recognition time varies according to the combination of rendering methods.

Fig. 6.
figure 6

Recognition time according the combination of inside and outline rendering.

Full size image

5 Discussion

The previous section revealed three important results.

  • First, the forms are well recognized regardless of the geometrical shape, the size, or the rendering method.

  • Second the recognition times appears to be significantly better with crosses, squares, and right triangles than with circles, diamonds, and simple triangles. This result provides an interesting cue about the exploration strategy that participants followed. Some of them said that they start by looking for right angles in the shape which help them to rapidly identify the form (only 1 right angle for right triangles, 4 for squares and a lot (12) for crosses).

  • Third, the rendering methods that include vibrations seem to disrupt the participants even if there is no impact on the recognition rate. Participants spend more time to recognize the shapes rendered with vibrations than those rendered with static outline and empty or static insides. This result differs from Levesque and Hayward study [1] which obtained better identification for the shapes rendered with vibrations or dots than the ones rendered with grating. We think that this is due to the better resolution of the STReSS2 which allows a less “aggressive” perception of the vibrations.

However, and even though the following results should be taken with care due to different experimental conditions, a comparison with Levesque and Hayward results shows that recognition of geometrical shapes is better with a dot-matrix display than with a STReSS2 device in all cases. Figures 7, 8 and 9 show the comparison between our average recognition rates (Dot-Matrix) and theirs (STReSS2) depending respectively, on shapes, sizes, and rendering methods.

Fig. 7.
figure 7

Recognition rate comparison between the STReSS2 and the dot-matrix display according to shapes.

Full size image
Fig. 8.
figure 8

Recognition rate comparison between the STReSS2 and the dot-matrix display according to size.

Full size image
Fig. 9.
figure 9

Recognition rate comparison between the STReSS2 and the dot-matrix display according to rendering method.

Full size image

Concerning the recognition time, Levesque and Hayward found that recognition was performed in 14,2 s on average, while in our study, recognition is performed in 5,6 s on average (2,5 × faster).

6 Conclusion

This article explored several haptic rendering methods to present geometrical shapes through the touch using several fingers on a large physical surface: the dot-matrix display. The presented study allowed us to collect 12960 recognition times and 12960 recognition scores (324 shapes × 40 participants). Results show that the best rendering method is the one that combines static outline with empty inside and that squares, right triangles, and crosses are more quickly recognized than circles, diamonds, and simple triangles. These results are interesting for our project concerning spatial access to documents by the blind.

The protocol of the presented study was inspired by a similar study conducted by Levesque and Hayward on a smaller device that allows exploring a virtual surface using only one finger: the STReSS2 device. The comparison of results shows that the recognition rates and times on a dot-matrix display are better in all cases. However, further investigations are needed to determine if this is due to mono-finger vs multi-finger exploration or for other reasons.

Next step of this work will be to reproduce the same experiment with visually impaired people. It would be also interesting to study the effects of different vibration frequencies and different outline widths as well to compare the performances of the dot-matrix display with those of a vibrotactile device such as in [13].