1 Introduction

Immersive VR environments now are often used for scientific research, product design, and vocational training. Most immersive VR environments use data gloves to track hand movements. However, traditional input devices such as physical keyboards are not easy to use when users have data gloves or head-mounted displays put on their bodies. In addition, physical keyboards are not easy to carry around in the immersive VR environments. As a result, different virtual keyboards have been developed to allow users to directly input data in the immersive VR environments.

Previous studies developed VR keyboards in attempts to enhance human-computer interaction in immersive VR. Study results showed that virtual keyboards without haptic feedback have higher typing error rates than virtual keyboards with haptic feedback [1]. In addition, haptic feedback has been proven to be able to improve work efficiency, work accuracy, and user pleasure [2, 3]. Therefore, virtual keyboards with haptic feedback can improve users’ overall experience in immersive VR.

Du and Charbon [4] developed a VR keyboard system based on multi-level feature matching. Their virtual keyboard system did not take tactile haptic feedback into consideration and thus may reduce typing accuracy. Kim and Kim [5] developed a VR keyboard which was overlaid on a real keyboard. Thus, their VR keyboard was limited by the physical keyboard, which in turn limited the workspace for some VR tasks.

Currently, many haptic feedback studies focus on mechanical tactile feedback. Mechanical tactile devices use actuators, such as linear motor, solenoid, vibration motor, piezoelectric motor, pneumatic, and speakers, to provide tactile feedback. For example, Lylykangas et al. [6] used piezoelectric actuators and different durations and delay times of the piezo vibration to provide button click sensations.

This study creates a haptic VR keyboard system to enhance human-computer interaction in immersive VR environments. The virtual keyboard system uses a pair of finger-wised data gloves. Realistic key clicking sensations are provided to each finger. The VR keyboard is not anchored on any physical objects. Therefore, the users can use the VR keyboard at any locations in the VR environments.

2 Haptic Feedback of Realistic Keystroke

2.1 Keystroke Measurement

To create realistic tactile haptic feedback, a high torque servo motor was used to stably press and release a real key of a real keyboard, a laser interferometer was used to measure the displacement and velocity of the key press. Figure 1 shows the key press device, and the measured displacement and velocity of the key.

Fig. 1.
figure 1figure 1

The key press device and the measured displacement and velocity

Full size image

Previous studies showed that the ‘clicks’ sensations that users feel when they press and release the key are caused by buckling and restitution forces [7]. In our research, the acceleration data of both buckling and restitution were found by calculating the first derivative of the measured velocity of the key. The results show that the buckling and restitution cause dramatic acceleration changes when users press and release of the key.

2.2 Method for Reproducing Actual Vibration Signals

In our study, actual vibration signals were reproduced by using different damped sine-wave signals to train a neural network to find the corresponding driving signals for the actuators. The neural network was used to adjust the vibrations for sensor, sensor installation, and actuator effects.

Equation (1) was used to create 300 different sine-wave signals by choosing different amplitudes (A), damping constants (α), frequencies (ω), and phases (θ). The sine-wave signals used are sufficient in training the neural network to provide realistic vibrations.

$$ {\text{I}}\left( {\text{t}} \right) \, = {\text{ Ae}}^{{ -\upalpha{\text{t}}}} ({ \cos }\left( {\upomega{\text{t }} - \,\uptheta} \right) \, - { \sin }\left( {\upomega{\text{t }} - \,\uptheta} \right) $$
(1)

The micro-speakers were used to create vibrations that matched the measured vibrations of the key. Figure 2 shows the measured vibrations of the key and the measured vibrations of the actuator. The results show that both amplitudes of the vibrations were about ± 1.5 G, and both the center frequencies of the vibrations were about 50 Hz.

Fig. 2.
figure 2figure 2

The measured vibrations of the key and actuator (micro-speaker)

Full size image

3 System Implementation

The system used an ASUS personal computer, a commercial program (Unity 3D), a 3D VR display device (HMD), 1 pair of P5 data gloves, and 10 micro-speakers (attached to each finger on the P5 data gloves) to create a virtual environment, display virtual objects, detect key presses, and create realistic tactile haptic feedback. Figure 3 shows an image of the testing environment.

Fig. 3.
figure 3figure 3

A testing environment

Full size image

4 User Test

4.1 Test Mode

During the test, the participants used the HMD and P5 data gloves to receive 3D images and track their hand positions. The participants were asked to adjust a virtual object’s parameters by entering values and type a string of predefined words. Three different test modes were tested. In mode 1, a physical keyboard was used. In mode 2, the virtual keyboard with a pair of P5 data gloves were used. Mode 3 was mode 2 with vibro-tactile feedback added.

4.2 User Test

During the user test, 25 participants were asked to use the three data input modes to change the dimensions of a virtual rectangular column, and type “block” in the text input field. Figure 4 shows the virtual environment. After the participants finished the three test modes, they were asked to fill out a questionnaire.

Fig. 4.
figure 4figure 4

Virtual environment

Full size image

The user test measured the performance of our haptic VR keyboard system using a seven-point Licker scale (1 = strongly disagree, 4 = neutral, 7 = strongly agree). The mean score and the standard deviation for each test mode are shown in Table 1. Participants generally prefer to use the virtual keyboard in a fully immersive virtual environment because they did not need to take off their HMD and data gloves to enter data. Furthermore, data gloves with haptic feedback received higher responses than that without haptic feedback.

Table 1. Average score for each test mode
Full size table

5 Conclusions

This study created an interactive haptic VR keyboard system for immersive VR environments. The system can create complex vibrations that match measured vibrations from real keyboards. The results of this study show that the advantages of the haptic VR keyboard are that users can use the VR keyboard when wearing HMDs (users do not need to remove HMDs to use the VR keyboard in immersive VR environments), the VR keyboard can pop-up display at any location, as needed (users do not need to go to a specific location to use a keyboard), and the VR keyboard can be used to provide realistic key click haptic feedback. Haptic feedback can enhance users’ interactions with virtual objects and allow users to control their actions more intuitively in the VR environments. As a result, the haptic virtual keyboard can be used to enhance human-computer interactions in the immersive VR environments.