Keywords

1 Introduction

Immersive Virtual Reality (VR) driving environments have certain advantages when it comes to testing new automated driving concepts. Modern computation technology and sensors enable fully automated cars. Thus the interest in this type of driving has been increased. With VR systems a safe, fully controlled and still high fidelity environment can be provided at a fraction of the costs of real driving studies. In other words, an environment can be created which is close to a realistic driving experience without the liability issues or high costs [1]. However, one of the disadvantages of the VR systems is simulation sickness. Simulation sickness is a form of motion sickness which is induced by virtual environments and is also referred to as VR sickness or cybersickness [2]. Simulation sickness is a condition of physiological discomfort felt during or after exposure to a virtual environment. According to cue conflict theory, the discrepancy between visual and motion cues is one of the assumed reasons for simulation sickness while using driving simulators [3]. A motion system might be able to replace the missing motion cues in the VR driving environment. In this paper, a pilot study is presented to assess the feasibility of a future experiment to evaluate simulation sickness in an autonomous VR driving environment. Automated driving is categorized into five levels. Fully automated cars have the highest level of automation and the driving system has full control over all driving tasks under all road conditions which are managed by a human driver in lowest level [4]. In particular, the effect of adding a motion platform to such a system is of interest. The study has two objectives: First, testing the experimental setup and the data collection methods and second, gathering preliminary results on the participants’ responses. The data was collected through subjective questionnaires and interviews, and objective physiological measurements.

In the next section a brief overview of virtual reality, simulation sickness, and related work will be provided. The methodology for the user study and the experimental setup will be described in Sect. 3, followed by the results of the trials. This paper concludes with a discussion of the results and shows a way towards future research.

2 Background

2.1 Virtual Reality

VR is a technology that enables a person to experience a computer-simulated environment and interact with it. These virtual worlds can be accessed through a Head-mounted Display (HMD) which provides access to these virtual environments. VR is a well-known technology since the 1960s [5]. Despite its half-century long existence, VR was neither widely adopted by consumers nor the industry. However, in recent years VR was rediscovered as a potential gaming and visualization platform. The year 2012 was a turning point for the technology when a Kickstarter project called Oculus Rift [6] released a user-friendly, low-cost HMD. This development expanded the interest and adoption of VR technology exponentially [7]. Oculus Rift was developed with the intention to serve the consumer electronics market. Despite being a low-cost device, the Rift offers key advantages over previous, much more expensive HMDs that have been used by the industry; a wide Field-Of-View (FOV), high-resolution display panels, affordable head-tracking technology, and a combination of hard- and software to ensure a minimal latency [8]. Besides gaming, VR technology is widely adopted by industry and academia in fields such as architecture, health, aerospace, and military. In the automotive sector, one of the applications for VR is driving simulation. It can be used for testing new interior concepts and interfaces in a realistic three-dimensional driving environment.

2.2 Simulation Sickness

Cue conflict theory is also known as sensory conflict, neural mismatch, and sensory rearrangement theory. It is the most widely adopted theory of the origins of simulation sickness. It was originally developed to explain motion sickness, later it was discovered that the theory is also applicable to simulation sickness. In (simulated) motion conditions a mismatch between visual, vestibular and muscular proprioceptive systems can arise [9]. The brain is confused by the received information and this mismatch can result in an immediate response. This can lead to physical discomfort like disorientation, nausea or eyestrain. For example, if a person is using a static driving simulator, the vestibular system indicates that the body does not move. The visual system sends opposing signals. Due to this cue conflict, the brain concludes that normal functions of the body are interrupted, in extreme circumstances such a mismatch can lead to emesis.

Poison theory, also known as evolutionary theory, suggests that unnatural movement can cause physical discomfort. The human body learned naturally how to move in the surrounding environment [10]. If a person is in a situation, which involves uncommon movements like walking in a virtual environment, the body misreads the sensory input information. Sensory systems send signals to the brain to bring awareness of the intake toxin. As a result, the person can feel disorientation, overall discomfort and ultimately an emesis response to dispense of the suspected toxins.

Decades later, Stoffregen disagreed with the cue conflict theory and suggested another theory on why people get sick known as the postural instability theory. The theory is based on the fact that the main purpose of the human body is to maintain postural stability, and when this balance is interrupted, the person feels discomfort (e.g. disorientation, nausea, and dizziness) [11].

The widest used measurement to quantify the level of sickness is the simulation sickness questionnaire (SSQ) [12]. The questionnaire consists of 16 questions where each question has four possible answers e.g. none, slight, average and severe. According to the SSQ scoring system, each item falls into one of three clusters: nausea, disorientation and oculomotor. Other measurements include physiological signals, such as electro-dermal activity (EDA) and electrocardiogram (ECG). Skin conductance levels, acquired by EDA, provides information about stress level which is related to simulation sickness outbreak [13]. The heart rate (HR), acquired by ECG, was related to simulation sickness onset in previous research [14].

2.3 Related Work

Research in VR driving simulation and, more specifically, driving simulation with HMDs has grown in the last years [15, 16]. There are a few studies focusing on VR driving simulator evaluation with HMD [1]. In a comprehensive study of simulation sickness in a virtual environment, Kolasinski [17] found that 42 factors are related to sickness outbreak. Factors, such as gender [18, 19], motion sickness history [20, 21], calibration [22], and latency [23, 24], have been investigated over the years. As there are many factors that contribute to simulation sickness, possible mitigation techniques are diverse and should be adapted to each specific use case. Regarding simulation sickness during simulated driving, research suggests that missing motion cues are one of the biggest problems of static driving simulators [25]. Missing motion cues are not the only contributing factors to simulation sickness. There is evidence that the low resolution HMDs in previous studies did not suffice to generate appropriate visual illusion [26]. This can contribute to simulation sickness and bring symptoms, such as blurred vision or disorientation. Hence, the visual quality of the simulation should be high to prevent these symptoms.

In a study comparing static and dynamic driving simulators, users who drove the motion driving simulator experienced fewer side effects. The results showed significantly reduced values of nausea, dizziness, eyestrain, and tiredness [25]. In a study, which compared a driver and a passenger exposed to the same motion simulation, it was reported that the participants in the driver’s group experienced less motion sickness. The authors of the study conclude that the results are caused by the driver’s concentration on the task [27]. These results suggest that users who ride a fully autonomous car will experience a higher level of sickness.

A proof-of-concept, presented in the next section, extends the use of VR driving environment for testing with addition of a fully autonomous driving environment. This concept could find application in future experiments in the domain of autonomous driving.

3 Methodology

3.1 Experimental Design and Procedure

A fully autonomous driving study with a within-subjects design was conducted. The participants were exposed to a scenario which did not require any intervention or monitoring from the user. Two conditions were presented: First, a scenario without motion platform and a scenario with additional motion. Each participant took part in both conditions with a 24 h gap between the sessions. Approximately 80% of the participants started with the static condition. The total duration of the driving simulation was around 11 min. The participants were instructed to sit comfortably on the driver’s seat and explore freely the interior of the car or the surroundings. The simulation contained a traffic environment with no other movable visual assets (i.e. no traffic situations with vehicles or pedestrians). The driving simulation started with a right turn to a terminal of an airport. There the car stopped for 5 s and afterwards continued towards a highway. Before reaching the highway, the participant experienced driving down under a bridge. During the highway driving, the car changed the lane from the left to the right side. After that, the participants experienced driving up and down a hill, followed by a right turn to a country road including two left and two right turns. The simulation stopped when the car reached a specific point where the participant could see a city in the far distance.

3.2 Virtual Reality Driving Environment

For a realistic VR simulation environment which supports the current consumer version HMDs, a game development environment is required. For the proof-of-concept reported in this paper, a game engine named Unreal Engine 4 was employed. This game engine is widely used by game developers, and it delivers high quality graphics that can be optimized for VR applications. High quality graphics are desirable elements in the development of VR driving simulation [28]. Unreal Engine 4 was used in the following study for creating and executing the driving scene and the highest level of automated driving simulation (i.e. automated level 5). The VR driving system consisted of an HMD with a tracking system, a motion platform with four pneumatic actuators, and a computer for rendering the driving scene (Fig. 1). As the HMD, the Oculus Rift CV1 with a FOV of 110° and a resolution of 2160 × 1200 pixels per eye was used.

Fig. 1.
figure 1

The concept of VR driving simulation with HMD.

Full size image

With a three Degrees of Freedom (3 DOF) the motion platform can simulate the most common movements during driving, namely, pitch, roll, and heave [29]. Each actuator has a maximum payload of 227 kg, a maximum acceleration of 1 g-force, a maximum angle of 15°, and a stroke of 152.4 mm. The four actuators are attached to the corners under the platform. Two actuators have one master box. Each box translates the signals from the Actuator Control Module (ACM) and controls the actuators. The ACM is connected to a high-end personal computer (PC), where a Motion code is executed. On the same PC, a 3D scene of the driving simulation is displayed. The tracking sensor was attached to the platform and therefore follows the platform’s movements. This ensured that the platform’s movements are not interpreted as head movements in the virtual scene. To create an immersive environment, a car seat was added to the platform for a more realistic experience. During the experiment, the 3D scene was shown on the display and the HMD (Fig. 2). This allowed the experiment supervisor to monitor the participants’ progress.

Fig. 2.
figure 2

A screenshot shows part of the VR environment used for the concept testing.

Full size image

3.3 Participants

Nine participants aged between 30 and 53 years (M = 35.67, SD = 7.04) took a part in the study. Only one female participated and therefore, gender as a variable was unequally distributed and could not be evaluated. Five of the participants described themselves as frequent drivers who drove more than 10 000 km in the past year. Regarding previous experience with driving simulation, 67% responded positively. Only one of the participants reported discomfort after being exposed for over 30 min to a driving simulator. Two participants reported that they play video games on a daily basis. In respect to previous experience with HMDs, 67% had used an HMD before this trial, one participant did not respond and one experienced simulation sickness during previous VR session. They had spent between 5 and 300 min (M = 73.3, SD = 111.8) in VR environments.

3.4 Measurements

For this pilot study, the objective physiological measurements ECG, EDA and subjective measurements, such as questionnaires were recorded. For measuring the physiological signals, 3-channel-ECG and skin conductance level data were recorded with medical sensors by g.Tec Medical Engineering GmbH with a sampling frequency of 512 Hz [30]. A baseline signal was recorded for 2 min in a resting position in the VR environment prior to the driving scenario.

Two subjective questionnaires were handed out before and two after each simulation. Prior to the trial, the participants were given a questionnaire asking basic demographic and biographical data. Also before the trial, a short version of the motion sickness susceptibility questionnaire (MSSQ) was used [31]. The post-questionnaires consisted of the SSQ [12] and the iGroup Presence Questionnaire (IPQ). A few questions regarding enjoyment [32] were included in the IPQ questionnaire to measure the users’ emotional reaction to the VR simulation. Additionally, interviews were developed to collect participants’ responses. The aim of these interviews was to acquire additional information about the trial.

3.5 Data Analysis

The study was designed as a pilot study to assess the feasibility and possible procedure for a future study. Due to the small sample size, the data was analyzed with the help of descriptive statistics. The HR was obtained from ECG signals [14]. Skin conductance level (SCL) was obtained from EDA signals and processed with the Ledalab software [33].

4 Results

The results of the subjective data collected through SSQ indicated that no clear trend could be observed between the motion and static conditions regarding simulation sickness. Only light to non-existent symptoms in both conditions were reported. The overall score indicated that the general discomfort and stomach awareness score was higher in the static condition. However, difficulty focusing and fullness of the head shows a trend towards higher scores in the motion platform condition. Figure 3 shows the calculated clusters’ scores based on the SSQ’s weight system. The symptoms of nausea cluster which showed changes in the responses are sweating, salvation increased and stomach awareness. In the static condition, one participant had a slight sweating and three participants had slight stomach awareness. In the motion platform condition, sweating and stomach awareness were not reported and one participant felt increased salivation.

Fig. 3.
figure 3

SSQ score divided into clusters for static and motion platform conditions.

Full size image

The results of the MSSQ-short questionnaire showed that two users had a high level, two had moderate, and five had a low level of susceptibility to motion sickness. From the users with high level, one showed a lower SSQ score in the static condition. One of the users with moderate level demonstrated a lower SSQ score in the static, while the third showed a lower SSQ score in the motion platform condition. Regarding the sense of presence, the answers to the question “I was involved in the virtual reality experience.” from IPQ indicated a tendency towards the motion platform condition. The question “I experienced a delay between my actions and expected outcome.” suggested that the static condition provoked a higher experienced delay compared to the motion platform condition. The last four questions regarding enjoyment showed mostly positive answers. No differences in HR and SCL could be observed (Figs. 4 and 5).

Fig. 4.
figure 4

Mean HR in static and motion platform conditions.

Full size image
Fig. 5.
figure 5

Mean SCL in static and motion platform conditions.

Full size image

The participants were asked to describe their overall experience from the VR autonomous driving. The experience was described as interesting and exciting, as one participant said: “It was very exciting because it was my first time to try out a driving simulation with virtual reality and at the same time it was a bit strange.” Another participant, responding to the question when the highest amount of discomfort was felt, said: “Breaking and acceleration in both conditions made you feel strange. The motion felt mostly like a vibration.” Other participants responded to the same question that the turns were the most unpleasant part of the simulation. One participant stated that “the simulation felt very artificial like nothing is happening there which got boring at some moment.” And another commented “It was much better than using only a screen.” One of the participants felt a light discomfort shortly after the trial which increased with time. One hour later, the participant got nauseous. According to the experiment’s notes, a second participant experienced similar discomfort, but after a short duration of time returned to a normal level. The overall response to the VR autonomous driving experience was positive and participants expressed the willingness to participate again.

A more detailed information for a particular participant is shown in Fig. 6. It shows that the HR and SCL were higher in the static condition. The last event, car drives up, has lower HR level than the previous event, car drives down, in the motion platform condition. This differs from the static condition where the same event had a higher HR level than the car drives down event.

Fig. 6.
figure 6

HR and SCL over time (in seconds) in static and motion platform condition during the following events: car stops, car drives down, car changes to the right lane, car drives down, and car drives up.

Full size image

5 Discussion and Future Work

A pilot study was conducted to test a proof-of-concept and collect preliminary results of the effect of autonomous VR driving simulation on simulation sickness. The findings have shown that a setup with HMD could be used for evaluating new interior concepts in an autonomous environment. No substantial simulation sickness was reported while using an autonomous car in a VR simulation whether a motion platform was used or not. A possible explanation for these findings is the low speed which was used. The speed was reduced in order to minimize the simulation sickness provocation.

Most of the participants had previous experience with virtual environments and were accustomed to wear HMDs repeatedly. Hence, the used sample is not representative of the general population. Literature suggests that a person has the ability to adapt to a virtual environment to some extent which can lead to a decrease of motion sickness symptoms [34]. Some of the participants felt less sick in the static condition. These findings may be due to the response time of the platform which was too high and therefore, the induced motion stimuli might be wrong. This could have made simulation sickness outbreak worse.

An interesting finding is that the frequency of general discomfort, difficulty focusing, and dizziness with eyes open was reduced in the motion platform condition. A possible reason for that could be that motion cues contributed to the overall known driving behavior which leads to a feeling of comfort while using an autonomous car. Intriguingly enough, the frequency of sweating and stomach awareness changed to zero in the motion platform condition. Sweating, and more specifically cold sweating is the body’s reaction to stress [35]. This indicates that participants in motion platform condition experienced less stress. However, this condition also induced fatigue and fullness of the head. The HR and SCL data differed in the static condition regarding the event car drives up in relation to the previous event car drives down. A possible explanation is that in the motion platform condition the discrepancy between the visual and motion cues is less during the car drives up event. However, this participant showed higher SSQ total score in motion platform condition. A possible reason for that could be factors such as stress from work, level of tiredness, and general well-being. These factors could not be easily controlled but could be taken into consideration in future experiments. No differences in HR and SCL could be observed. This might be caused limitations of the experimental design, a small sample size and an order of the scenarios that were not counterbalanced.

One of the reasons given by the participant for simulation sickness outbreak was the turning during the simulation. The left and right turns were felt sometimes unpleasant to the users especially in the static condition. One possible explanation is that during the steering of the automated car the discrepancy is higher which corresponds to cue conflict theory [36]. However, in the motion platform condition, discomfort might have occurred because the user’s driving style did not match completely with the style of the autonomous driving. The aftereffects which were experienced by two of the participants also have been reported in other studies [37]. The assumption that the higher SSQ score is associated with higher skin conductance level could not be observed in the sample.

The limitations of this study are the small sample size, unequal distribution across gender and age, the design was not counterbalanced, and there was no familiarization scenario. Also, subjects participated on two separate days, which might add more confounding factors like different daytime or different well-being state.

Considering the findings from the concept design, a follow-up study might include a different tilting angle and vibration frequency of the motion platform. That way, different motion levels, which would represent different speed, could be investigated. The rare occurrence of simulation sickness in the pre-study points to a larger sample size, a better balanced order of testing, and a longer driving time in future experiments. Furthermore, testing various environments might be further investigated. This study strengthens the idea that a VR environment might be used for experiments in the autonomous driving domain. An immersive environment can bring flexibility for testing concepts which are coming in the near future.