Abstract
With virtual reality now accessible to anyone through high-end consumer headsets and input devices, researchers are seeking cost-effective designs based on human perceptual properties for virtual reality interfaces. The author has been studying a sensory-illusion-based approach to designing human-computer interface technologies. This paper overviews how we are using this approach to develop force displays that elicit illusory continuous force sensations by presenting asymmetric vibrations and kinesthetic displays based on a cross-modal effect among visual, auditory, and tactile cues of self-motion.
Keywords
1 Introduction
After the several decades since the emergence of the virtual reality (VR) concept, the cost of the hardware for an immersive VR experience has drastically decreased [17]. Today, VR is accessible to everyone thanks to inexpensive high-end consumer-friendly head-mounted displays and input devices. However, a limited number of haptic interface displays, such as tactile or force feedback displays, have been adopted in VR systems compared with the audiovisual ones. This is mainly due to the technical difficulty in reproducing a haptic or somatosensory experience with inexpensive haptic displays.
On the other hand, innovative information displays can be designed that consider the characteristics of human perception and optimize information technologies for it. The perceptual limits on the human sensory system have often been considered to determine guidelines for designing audiovisual displays. Examples of these guidelines are video frame rates that produce smooth motion and perceptual coding in audio data compression for natural sound. In addition, human sensory illusions are often utilized to invent innovative audiovisual displays. Thus, some haptic interface displays can be built on the basis of the sensory-illusion-based approach without incurring much cost. This paper introduces a novel approach that exploits the nonlinearity of human perception and sensory integration for developing somatosensory and kinesthetic displays.
2 Haptic Displays Using Perceptual Illusions
2.1 Directed Force Perception by an Asymmetric Oscillation
Over the past two decades, a great number of force displays have been developed and studied. Most of them are grounded force displays, such as PHANToM and SPIDAR, which use mechanical linkages to establish a fulcrum relative to the ground. The fulcrum (grounding support) is required for grounded displays because of the action-reaction principle. However, since mobile devices lack a fulcrum, most conventional force display systems for mobile devices can produce neither a constant nor translational force; that is, they can generate only short-term rotational force (e.g., using the gyro effect [24] or angular momentum change [23]). Thus, in mobile devices, the haptic cues are usually limited to simple vibrotactile ones.
The author and his colleagues have succeeded in creating a force sensation of being pulled or pushed with various kinds of mobile apparatuses. The display, called Buru-Navi® [1, 2, 9], creates both a constant- and translational-force sensation by utilizing the nonlinear characteristics of human perception. The trick is to use different acceleration patterns for two directions to create a perceived force imbalance. A brief and strong force is generated in a desired direction, while a weaker one is generated over a longer period of time in the direction opposite to the desired one. Although the temporal average of the net force is physically zero (e.g., the average of the forces in each direction are the same), people who hold a device vibrating by the acceleration patterns feel as if they are being pulled to one direction because the amplitude of the weaker force is adjusted to be below a sensory threshold.
Over the past ten years, the author has been refining a method to create a sensory illusion of being pulled with a slider-crank mechanism [1,2,3,4,5] or spring-cam mechanism [6], and, during that time, he has developed various prototypes of ungrounded force displays as shown in Fig. 1. The author has succeeded in reducing the size and weight of the force display remarkably by using a linear electromagnetic actuator [7, 9]. The prototype was designed to be pinched by the fingers because the author focused on the finger pad, which is one of the most sensitive body surfaces. In the human finger pad, there are four major tactile mechanoreceptors, which are Pacinian corpuscles, Merkel disks, Meissner corpuscles, and Ruffini endings. The Pacinian corpuscles are sensitive to vibrations with high temporal frequencies from 100 to 300 Hz in the normal direction [13], but they seem to have nearly the same sensitivity to sliding directions tangential to the skin [22]. In contrast, SA-I and RA-I fibers, whose signals are thought to come mainly from the Merkel disks and Meissner corpuscles, respectively, are sensitive to vibrations with lower frequencies (less than 100 Hz), and some of them can clearly code the sliding or tangential force direction [18]. The SA-II fiber innervating Ruffini endings is also sensitive to skin stretch, particularly to tangential forces generated in the skin. Therefore, to create an illusory force sensation of being pulled, the author designed an asymmetrical oscillation pattern that contains the frequency components stimulating these receptors (i.e., less than 100 Hz) and contains the asymmetric magnitude which exceeds the thresholds of shearing displacement in one direction but not in the other.
Our previous studies on force direction discrimination of the asymmetric oscillation confirmed that almost all the participants felt a clear illusory force of being pulled or pushed persistently and that they were able to distinguish the force direction correctly.
2.2 Wayfinding by an Illusory Force
An illusory force of being pulled as introduced above has the potential to be applied for pedestrian navigation without visual or auditory information, as if someone is being led by the hand. Furthermore, Buru-Navi® can be used by people with visual impairments to provide directional cues for wayfinding. Our previous study with a fire department showed that 91% of participants with visual impairment were able to walk safely along a predefined route without any prior training [3]. In a fire emergency, smoke hinders the visual field of people at the scene. Thus, Buru-Navi® will be useful for sighted people as well as for people with visual impairments in such a situation.
After the experiment, we raised the following question: What is the most effective and efficient way for users to understand the force direction in the application? One may think that it would be helpful to always update the force direction and maintain the same direction in global coordinates depending on the orientation of the force display, much like a compass needle points to the direction of the magnetic north. It is unclear whether the active exploration of the direction of an illusory force by hand or arm movement improves the perception of the force direction. This is because some studies have reported that tactile processing is suppressed by hand movement [15], while others have reported that active touch sensing facilitates tactile performance [18]. We have shown that active manual movement in both the rotational and translational directions enhances the precise perception of the direction of an illusory force created by Buru-Navi® [10], which suggests that the active exploration of force direction by moving the arm or hand is a good strategy for understanding the direction in a pedestrian navigation application.
3 Haptic Displays Inducing Self-Motion Perception
Self-motion, one of the most frequent movements in daily life, is experienced when we walk or ride in a vehicle. In VR theme parks or VR amusement centers, some types of self-motion can be expressed while the user sits on a chair-like vehicle, such as driving simulators or motion seats. This section introduces three studies on perceptual illusions of self-motion using haptic displays with participants seated on a chair-like vehicle.
3.1 Change in Velocity Perception of Self-Motion
In chair-like vehicles, velocity information is detected by means of visual sensory cues, and acceleration and angular acceleration information is detected by means of mechanical ones, such as vestibular and tactile sensations. However, it is unclear whether the tactile information integrated with visual information alters the velocity perception of self-motion, because some similarities exist between the perceptual and neural mechanisms when motion stimuli of vision and touch are processed.
To investigate this, we examined whether the forward velocity of self-motion is altered by applying rapid tactile flows using a vibratotactile array on a seat pan. In the study, participants viewed optical flows while gazing at a fixation cross and sitting on a tactile stimulator on the seat pan. The brief tactile motion stimulus consisted of four successive rows of 200-ms vibration with a frequency of 50 Hz, and the inter-stimulus onset between the tactile rows was varied to change the velocity of the tactile motion. The experimental results showed that the forward velocity of self-motion is significantly overestimated for rapid tactile flows and underestimated for slow ones, compared with only optical flow or non-motion vibrotactile stimulation conditions. Furthermore, temporal tactile rhythm patterns (i.e., a train of taps) did not affect the perceived velocity of self-motion as much as tactile flow stimuli, especially when the inter-stimulus onset interval was appropriate for eliciting a clear sensation of tactile apparent motion, which indicates the importance of the spatiotemporal feature of tactile motion stimuli in modulating the velocity [8].
3.2 Change in Topographic Surface Perception
It has been reported that we often misperceive a surface topography simply based on visual cues at “magnetic hills”, where a slight downhill slope appears to be an uphill one due to the surroundings. We have investigated multimodal perception of a topographic surface induced by visual and body-tilt stimuli [12].
One remarkable study on haptic perception has shown that humans more strongly perceive the shape of an object during active touch from the force profile applied to the finger than from the position profile of the finger [21]. This implies that the shape perception could be induced by local changes in topographic information without vertical movement, although the shape perception by the finger and by the body will differ. To verify the hypothesis that shape perception could be induced by body tilt, we constructed an experimental system using a motion chair with two DOF in roll and pitch rotations and conducted a user study to classify the perceived shape based on visual and vestibular cues. Experimental results show that the vestibular shape cue contributed to making the shape perception larger than the visual one [12]. This result suggests that concave and convex surfaces can be expressed with only two-DOF rotating motion of the body without the participant’s moving in the vertical direction while sitting on a chair-like vehicle.
3.3 Change in Boundary of Peripersonal Space
We have found that vibration on the foot sole induces a clear walking sensation even when a person is seated on a chair. Furthermore, we found that the magnitude of the subjective sensation of walking affected the objective scores, such as the reaction times of a tactile detection task [11].
Previous studies have shown that a moving sound that gives an impression of a sound source approaching the body boosts tactile reaction times when it is presented close to the stimulated body part, that is, within and not outside the PPS [14]. For instance, the PPS representation of the chest expands in the direction of walking [20]. Based on these results, we presented several vibration patterns on the soles of the feet of seated participants to evoke a sensation of pseudo-walking and examined the change in reaction times to detect a vibrotactile stimulus on the chest while the participants listened to a looming sound approaching their body, which was taken as a behavioral proxy for the PPS boundary [11]. Results revealed that a cyclic vibration consisting of low-pass-filtered walking sounds presented at the soles that clearly evoked a sensation of walking decreased the reaction times, indicating that the PPS boundary was expanded forward by inducing a sensation of walking (Fig. 2).
4 Conclusion
This paper introduced haptic displays based on perceptual illusions and multisensory stimuli. The sense of touch is very powerful for presenting a feeling of the existence of objects. Thus, it has been thought that there are few perceptual illusions in the haptic modality. However, several perceptual illusions in the haptic modality have been reported [16], and some have been implemented for novel information displays. This trend will continue because vision and touch work together to create a richer experience. Future information displays will ultimately utilize not only human perceptual aspects but also human perceptual flaws, such as sensory and perceptual illusions.
References
Amemiya, T., Ando, H., Maeda, T.: Virtual force display: direction guidance using asymmetric acceleration via periodic translational motion. In: Proceedings of World Haptics Conference 2005, pp. 619–622 (2005)
Amemiya, T: Haptic interface using sensory illusion. In: Tutorial in IEEE Virtual Reality 2008, Integration of Haptics in Virtual Environments: from Perception to Rendering, Reno, NV (2008)
Amemiya, T., Sugiyama, H.: Orienting kinesthetically: a haptic handheld wayfinder for people with visual impairments. ACM Trans. Access. Comput. 3(2), 1–23 (2010). Article 6
Amemiya, T., Maeda, T.: Asymmetric oscillation distorts the perceived heaviness of handheld objects. IEEE Trans. Haptics 1(1), 9–18 (2008)
Amemiya, T., Maeda, T.: NOBUNAGA: multicylinder-like pulse generator for kinesthetic illusion of being pulled smoothly. In: Ferre, M. (ed.) EuroHaptics 2008. LNCS, vol. 5024, pp. 580–585. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-69057-3_75
Amemiya, T., Ando, H., Maeda, T.: Hand-held force display with spring-cam mechanism for generating asymmetric acceleration. In: Proceedings of World Haptics Conference 2007, Tsukuba, Japan, pp. 572–573 (2007)
Amemiya, T., Gomi, H.: Distinct pseudo-attraction force sensation by a thumb-sized vibrator that oscillates asymmetrically. In: Auvray, M. (ed.) EUROHAPTICS 2014. LNCS, vol. 8619, pp. 88–95. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44196-1_12
Amemiya, T., Hirota, K., Ikei, Y.: Tactile apparent motion on the torso modulates perceived forward self-motion velocity. IEEE Trans. Haptics 9(4), 474–482 (2016)
Amemiya, T.: Perceptual illusions for multisensory displays. Invited talk. In: Proceedings of the 22nd International Display Workshops (IDW 2015), Otsu, Japan, vol. 22, pp. 1276–1279 (2015)
Amemiya, T., Gomi, H.: Active Manual Movement Improves Directional Perception of Illusory Force. IEEE Trans. Haptics 9(4), 465–473 (2016)
Amemiya, T., Ikei, Y., Hirota, K., Kitazaki, M.: Vibration on the Soles of the Feet Evoking a Sensation of Walking Expands Peripersonal Space. In: Proceedings of IEEE World Haptics 2017, Munich, Germany, pp. 234–239 (2017)
Amemiya, T., Hirota, K., Ikei, Y.: Topographic surface perception modulated by pitch rotation of motion chair. In: Proceedings of 18th International Conference on Human-Computer Interaction (HCI International 2016), Toronto, Canada, pp. 483–490 (2016)
Bolanowski Jr., S.J., Gescheider, G.A., Verrillo, R.T., Checkosky, C.M.: Four channels mediate the mechanical aspects of touch. J. Acoust. Soc. Am. 84(5), 680–694 (1988)
Canzoneri, E., Magosso, E., Serino, A.: Dynamic sounds capture the boundaries of peripersonal space representation in humans. PLoS ONE 7(9), e44306 (2012)
Chapman, C.E., Bushnell, M., Miron, D., Duncan, G., Lund, J.: Sensory perception during movement in man. Exp. Brain Res. 68(3), 516–524 (1987)
Hayward, V.: A brief taxonomy of tactile illusions and demonstrations that can be done in a hardware store. Brain Res. Bull. 75(6), 742–752 (2008)
Hirose, M.: The second generation virtual reality technology. Keynote/invited Speech. In: Proceedings of 16th International Conference on Virtual Systems and Multimedia (VSMM 2010), Seoul, Korea (2010)
Lederman, S.J., Klatzky, R.L.: Hand movements: a window into haptic object recognition. Cogn. Psychol. 19(3), 342–368 (1987)
Maeno, T., Kobayashi, K., Yamazaki, N.: Relationship between the structure of human finger tissue and the location of tactile receptors. Bull. JSME Int. J. 41, 94–100 (1998)
Noel, J.-P., Grivaz, P., Marmaroli, P., Lissek, H., Blanke, O., Serino, A.: Full body action remapping of peripersonal space: the case of walking. Neuropsychologia 70, 375–384 (2015)
Robles-De-La-Torre, G., Hayward, V.: Force can overcome object geometry in the perception of shape through active touch. Nature 412(6845), 445–448 (2001)
Srinivasan, M.A., Whitehouse, J.M., Lamotte, R.H.: Tactile detection of slip: surface microgeometry and peripheral neural codes. J. Neurophysiol. 63(6), 323–332 (1990)
Tanaka, Y., Masataka, S., Yuka, K., Fukui, Y., Yamashita, J., Nakamura, N.: Mobile torque display and haptic characteristics of human palm. In: Proceedings of ICAT 2001, pp. 115–120 (2001)
Yano, H., Yoshie, M., Iwata, H.: Development of a non-grounded haptic interface using the gyro effect. In: Proceedings of HAPTICS 2003, pp. 32–39 (2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Amemiya, T. (2018). Haptic Interface Technologies Using Perceptual Illusions. In: Yamamoto, S., Mori, H. (eds) Human Interface and the Management of Information. Interaction, Visualization, and Analytics. HIMI 2018. Lecture Notes in Computer Science(), vol 10904. Springer, Cham. https://doi.org/10.1007/978-3-319-92043-6_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-92043-6_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-92042-9
Online ISBN: 978-3-319-92043-6
eBook Packages: Computer ScienceComputer Science (R0)