1 Introduction

After the several decades since the emergence of the virtual reality (VR) concept, the cost of the hardware for an immersive VR experience has drastically decreased [17]. Today, VR is accessible to everyone thanks to inexpensive high-end consumer-friendly head-mounted displays and input devices. However, a limited number of haptic interface displays, such as tactile or force feedback displays, have been adopted in VR systems compared with the audiovisual ones. This is mainly due to the technical difficulty in reproducing a haptic or somatosensory experience with inexpensive haptic displays.

On the other hand, innovative information displays can be designed that consider the characteristics of human perception and optimize information technologies for it. The perceptual limits on the human sensory system have often been considered to determine guidelines for designing audiovisual displays. Examples of these guidelines are video frame rates that produce smooth motion and perceptual coding in audio data compression for natural sound. In addition, human sensory illusions are often utilized to invent innovative audiovisual displays. Thus, some haptic interface displays can be built on the basis of the sensory-illusion-based approach without incurring much cost. This paper introduces a novel approach that exploits the nonlinearity of human perception and sensory integration for developing somatosensory and kinesthetic displays.

2 Haptic Displays Using Perceptual Illusions

2.1 Directed Force Perception by an Asymmetric Oscillation

Over the past two decades, a great number of force displays have been developed and studied. Most of them are grounded force displays, such as PHANToM and SPIDAR, which use mechanical linkages to establish a fulcrum relative to the ground. The fulcrum (grounding support) is required for grounded displays because of the action-reaction principle. However, since mobile devices lack a fulcrum, most conventional force display systems for mobile devices can produce neither a constant nor translational force; that is, they can generate only short-term rotational force (e.g., using the gyro effect [24] or angular momentum change [23]). Thus, in mobile devices, the haptic cues are usually limited to simple vibrotactile ones.

The author and his colleagues have succeeded in creating a force sensation of being pulled or pushed with various kinds of mobile apparatuses. The display, called Buru-Navi® [1, 2, 9], creates both a constant- and translational-force sensation by utilizing the nonlinear characteristics of human perception. The trick is to use different acceleration patterns for two directions to create a perceived force imbalance. A brief and strong force is generated in a desired direction, while a weaker one is generated over a longer period of time in the direction opposite to the desired one. Although the temporal average of the net force is physically zero (e.g., the average of the forces in each direction are the same), people who hold a device vibrating by the acceleration patterns feel as if they are being pulled to one direction because the amplitude of the weaker force is adjusted to be below a sensory threshold.

Over the past ten years, the author has been refining a method to create a sensory illusion of being pulled with a slider-crank mechanism [1,2,3,4,5] or spring-cam mechanism [6], and, during that time, he has developed various prototypes of ungrounded force displays as shown in Fig. 1. The author has succeeded in reducing the size and weight of the force display remarkably by using a linear electromagnetic actuator [7, 9]. The prototype was designed to be pinched by the fingers because the author focused on the finger pad, which is one of the most sensitive body surfaces. In the human finger pad, there are four major tactile mechanoreceptors, which are Pacinian corpuscles, Merkel disks, Meissner corpuscles, and Ruffini endings. The Pacinian corpuscles are sensitive to vibrations with high temporal frequencies from 100 to 300 Hz in the normal direction [13], but they seem to have nearly the same sensitivity to sliding directions tangential to the skin [22]. In contrast, SA-I and RA-I fibers, whose signals are thought to come mainly from the Merkel disks and Meissner corpuscles, respectively, are sensitive to vibrations with lower frequencies (less than 100 Hz), and some of them can clearly code the sliding or tangential force direction [18]. The SA-II fiber innervating Ruffini endings is also sensitive to skin stretch, particularly to tangential forces generated in the skin. Therefore, to create an illusory force sensation of being pulled, the author designed an asymmetrical oscillation pattern that contains the frequency components stimulating these receptors (i.e., less than 100 Hz) and contains the asymmetric magnitude which exceeds the thresholds of shearing displacement in one direction but not in the other.

Fig. 1.
figure 1

Prototypes of haptic displays generating an asymmetric oscillation to induce a sensation of being pulled or pushed. The second rightmost one is 18 × 18 × 37 mm (1 DoF) and the rightmost one is \( {\upvarphi} \)40 × 17 mm (2 DoF).

Full size image

Our previous studies on force direction discrimination of the asymmetric oscillation confirmed that almost all the participants felt a clear illusory force of being pulled or pushed persistently and that they were able to distinguish the force direction correctly.

2.2 Wayfinding by an Illusory Force

An illusory force of being pulled as introduced above has the potential to be applied for pedestrian navigation without visual or auditory information, as if someone is being led by the hand. Furthermore, Buru-Navi® can be used by people with visual impairments to provide directional cues for wayfinding. Our previous study with a fire department showed that 91% of participants with visual impairment were able to walk safely along a predefined route without any prior training [3]. In a fire emergency, smoke hinders the visual field of people at the scene. Thus, Buru-Navi® will be useful for sighted people as well as for people with visual impairments in such a situation.

After the experiment, we raised the following question: What is the most effective and efficient way for users to understand the force direction in the application? One may think that it would be helpful to always update the force direction and maintain the same direction in global coordinates depending on the orientation of the force display, much like a compass needle points to the direction of the magnetic north. It is unclear whether the active exploration of the direction of an illusory force by hand or arm movement improves the perception of the force direction. This is because some studies have reported that tactile processing is suppressed by hand movement [15], while others have reported that active touch sensing facilitates tactile performance [18]. We have shown that active manual movement in both the rotational and translational directions enhances the precise perception of the direction of an illusory force created by Buru-Navi® [10], which suggests that the active exploration of force direction by moving the arm or hand is a good strategy for understanding the direction in a pedestrian navigation application.

3 Haptic Displays Inducing Self-Motion Perception

Self-motion, one of the most frequent movements in daily life, is experienced when we walk or ride in a vehicle. In VR theme parks or VR amusement centers, some types of self-motion can be expressed while the user sits on a chair-like vehicle, such as driving simulators or motion seats. This section introduces three studies on perceptual illusions of self-motion using haptic displays with participants seated on a chair-like vehicle.

3.1 Change in Velocity Perception of Self-Motion

In chair-like vehicles, velocity information is detected by means of visual sensory cues, and acceleration and angular acceleration information is detected by means of mechanical ones, such as vestibular and tactile sensations. However, it is unclear whether the tactile information integrated with visual information alters the velocity perception of self-motion, because some similarities exist between the perceptual and neural mechanisms when motion stimuli of vision and touch are processed.

To investigate this, we examined whether the forward velocity of self-motion is altered by applying rapid tactile flows using a vibratotactile array on a seat pan. In the study, participants viewed optical flows while gazing at a fixation cross and sitting on a tactile stimulator on the seat pan. The brief tactile motion stimulus consisted of four successive rows of 200-ms vibration with a frequency of 50 Hz, and the inter-stimulus onset between the tactile rows was varied to change the velocity of the tactile motion. The experimental results showed that the forward velocity of self-motion is significantly overestimated for rapid tactile flows and underestimated for slow ones, compared with only optical flow or non-motion vibrotactile stimulation conditions. Furthermore, temporal tactile rhythm patterns (i.e., a train of taps) did not affect the perceived velocity of self-motion as much as tactile flow stimuli, especially when the inter-stimulus onset interval was appropriate for eliciting a clear sensation of tactile apparent motion, which indicates the importance of the spatiotemporal feature of tactile motion stimuli in modulating the velocity [8].

3.2 Change in Topographic Surface Perception

It has been reported that we often misperceive a surface topography simply based on visual cues at “magnetic hills”, where a slight downhill slope appears to be an uphill one due to the surroundings. We have investigated multimodal perception of a topographic surface induced by visual and body-tilt stimuli [12].

One remarkable study on haptic perception has shown that humans more strongly perceive the shape of an object during active touch from the force profile applied to the finger than from the position profile of the finger [21]. This implies that the shape perception could be induced by local changes in topographic information without vertical movement, although the shape perception by the finger and by the body will differ. To verify the hypothesis that shape perception could be induced by body tilt, we constructed an experimental system using a motion chair with two DOF in roll and pitch rotations and conducted a user study to classify the perceived shape based on visual and vestibular cues. Experimental results show that the vestibular shape cue contributed to making the shape perception larger than the visual one [12]. This result suggests that concave and convex surfaces can be expressed with only two-DOF rotating motion of the body without the participant’s moving in the vertical direction while sitting on a chair-like vehicle.

3.3 Change in Boundary of Peripersonal Space

We have found that vibration on the foot sole induces a clear walking sensation even when a person is seated on a chair. Furthermore, we found that the magnitude of the subjective sensation of walking affected the objective scores, such as the reaction times of a tactile detection task [11].

Previous studies have shown that a moving sound that gives an impression of a sound source approaching the body boosts tactile reaction times when it is presented close to the stimulated body part, that is, within and not outside the PPS [14]. For instance, the PPS representation of the chest expands in the direction of walking [20]. Based on these results, we presented several vibration patterns on the soles of the feet of seated participants to evoke a sensation of pseudo-walking and examined the change in reaction times to detect a vibrotactile stimulus on the chest while the participants listened to a looming sound approaching their body, which was taken as a behavioral proxy for the PPS boundary [11]. Results revealed that a cyclic vibration consisting of low-pass-filtered walking sounds presented at the soles that clearly evoked a sensation of walking decreased the reaction times, indicating that the PPS boundary was expanded forward by inducing a sensation of walking (Fig. 2).

Fig. 2.
figure 2

Sole vibration to evoke a sensation of pseudo-walking expands the boundary of peripersonal space. The seated participant received a vibration pattern on the sole. Tactile reaction times on the chest when listing to a looming sound approaching the body decreased when the vibration patterns were rated high for pseudo-walking were applied, indicating that the boundary was expanded forward.

Full size image

4 Conclusion

This paper introduced haptic displays based on perceptual illusions and multisensory stimuli. The sense of touch is very powerful for presenting a feeling of the existence of objects. Thus, it has been thought that there are few perceptual illusions in the haptic modality. However, several perceptual illusions in the haptic modality have been reported [16], and some have been implemented for novel information displays. This trend will continue because vision and touch work together to create a richer experience. Future information displays will ultimately utilize not only human perceptual aspects but also human perceptual flaws, such as sensory and perceptual illusions.