VR Evaluation of Motion Sickness Solution in Automated Driving

Abstract

The sensory conflict theory describes the occurrence of motion sickness caused by the discrepancy between the motion felt and the motion visually perceived. During driving, drivers monitor the environment while performing driving tasks, this enables them to get the visual perception of the motion felt. Visual cues help drivers to anticipate the direction of movement and thus, eliminate confusion, which could lead to anxiety, and thus motion sickness. Occupants of highly automated vehicles will have the luxury of performing activities such as reading or interacting with their mobile devices while the system performs the driving tasks. However, if the passenger takes his eyes off the surrounding traffic environment, sensory conflict is likely to occur. We implemented a concept in virtual reality to prevent motion sickness during automated driving based on a split screen technology. A part of the screen shows a video capture of the car surrounding in real time, while the other part is free to be used for individual applications. This additional data enables visual cues, which makes it possible to monitor the direction of movement of the vehicle. This minimizes sensory conflict and prevents motion sickness. An experiment was conducted with fourteen participants on a virtual reality automated driving simulator with an integrated motion platform. The result shows that the video streaming of the horizon presented to the passengers on a display helps them to feel comfortable and also reduced motion sickness during automated driving.

1 Introduction

Motion sickness (MS) is described as the conflict between the perceived motion of the body as detected by the vestibular system (VS) (an organ responsible for equilibrium) and the visual perception [1]. The brain gets different motion signals that do not correspond for the same stimulus, and thus gets confused, a phenomenon known as sensory conflict (SC) [2]. The human body’s evolutionary program then comes into effect and assumes that the body is poisoned, and responds to the restoration of health with symptoms such as nausea, headache, salivation, and in a worst case, vomiting. This is known as the “Poison theory” [3]. MS is frequent with passengers travelling by car, ship or by air, also known as travel sickness. The following three aspects characterize the critical factors for passenger-induced MS. Firstly, the discrepancy between the motion perceived by the VS and the visually perceived motion. Secondly, the loss of control of the direction of movement of the vehicle and inability to adjust to the possible direction of movement of the vehicle. Finally, MS-inducing activities [4].

One main benefit of automated driving (AD) is the time available for occupants to read or interact with mobile devices without having to deal with the traffic environment and the conventional driving tasks since the system takes over the driver’s role [5]. All drivers therefore assume the role of passengers, and thus the number of passengers will increase significantly as a result of ride sharing. This also increases the number of passengers who are likely to suffer from MS while travelling by car. If the vehicle assumes the driver’s role, this will increase the risk for all passengers of getting motion sick, especially those who are reading and not focusing on the surrounding traffic environment, but still perceive body motion. Activities such as reading a book or watching movies, which takes the eyes off the road are most likely to make users of AD sick [6].

Because the possibility to use the driving time effectively to perform other tasks other than driving is what makes AD attractive, MS will definitely be a big setback for the acceptance of the technology and will reduce AD benefits. Unfortunately, there are no concrete solutions on how to tackle MS effectively while travelling by road. Some sources recommend medications, acupressure wristbands which are targeted to particular groups of persons only [7]. Some sources recommend design measures to make the surrounding visible to all by avoiding tinting of rear windows, stabilizing mobile devices used while the car is in motion, encouraging all passengers to face forward, and many more [4]. These solutions still force the passengers to focus on the road most of the time, and limits in-vehicle-design, therefore not very beneficial for AD occupants.

Since most AD passengers are likely to spent most time with their smartphones and tablets, this work therefore suggests a concept evaluated in virtual reality (VR) automated vehicle simulation meant to prevent MS during AD based on a split screen technology, where a part of the screen presents the missing visual cues of the surrounding in real time, through a video capture of the surrounding traffic environment, while the other screens could be used for individual preferences (refer to Fig. 1). Passengers of AD can therefore use the available time efficiently for other activities without worrying about getting sick. Moreover, desired designs must not be eradicated in order to avoid SC (rear windows may remain tinted if so desired).

Fig. 1.

Tablet with split screen shows a text and the video capture of the traffic road during.

2 Related Works

There is no limit to what self-driving cars can do when compared to a human driver with technology innovation advancing in full speed. The advantages of AD if effectively implemented and inaugurated knows no limit e.g. traffic safety, reduced traffic congestion, free and safe time to work while the car is in motion, comfort and unlimited mobility for the elderly. The vehicle will become a safe, social and working platform to hold meetings, talk on the phone with hands off the steering or relax during congested traffic situations. However, concern about MS affecting occupants during AD is rising despite numerous studies carried out on this topic for decades. A study carried out by the University of Michigan showed that by relieving the driver’s role, around 30% of occupants of automated vehicles would carry out activities that could induce MS. Activities such as reading, working, watching movies and writing emails are critical factors that could lead to MS due to the opposing information of “movement” in the VS and the visual signal through focusing on a resting point [6]. Though it is not evident what really causes MS, it could be described as the presence of mental and physical discomfort (e.g. nausea, increased salivation and disorientation) due to the discrepancy between the perceived motion or missing motion and the actual motion felt [8, 9].

The VS, located in the inner ear and responsible for the sense of balance and spatial orientation is composed of the semi-circular canals. Each canal detects one of the following head movements; nodding up and down, shaking side by side or moving left and right. A fluid called endolymph moves through the canals when the head is rotated. The canals are connected by a component known as Ampulla which contains tiny hair cells with nerve impulses. When humans move, the fluid moves in the direction of the movement. The tiny hairs detect the movement of the fluid and sends nerve impulses to the brain, which helps for orientation. When the VS transmits conflicting signals from other sensory signals, this results to sensory conflict and explains why travelers get motion sick, most especially when passengers are reading while the car is in motion. In this case, the VS detects car motions while the eyes see only texts. The brain receives conflicting signals and this results to SC, which in turn causes discomforts.

Passengers are usually more affected than the driver because the driver can anticipate the next move and therefore knows what to expect and can readily adapt [10]. Passengers who are not always looking outside the car in the direction of movement, might be ignorant of the speed, direction of movement, and might not anticipate a bad curve. In order to adapt to the next move, they have to look out the window so that the visual and VS can detect and agree on the sense of motion. It is therefore recommended that passengers look at the horizon so that the eyes can detect the motion felts by the body. This will help to eliminate SC, though susceptibility to MS differs from person to person.

Many studies have been carried out and still ongoing on how to avoid MS during AD. Some studies recommend a constant speed to reduce MS. Other sources suggest the interior vehicle design modification such as avoiding rear-sitting arrangement and tinted glasses in order to make the horizon visible [8]. Another source recommends working devices such as displays or tablets to be positioned in such a way that the horizon is still visible to the eyes for possible visual cues [10]. AD will allow for more connectivity and interaction with mobile devices, it is therefore expected that passengers will most likely interact more with devices such as tablets and smartphones. Therefore, designers are recommended to focus on the stabilization of these devices for a steady information presentation and also consider user interface designs mentioned above that reduce MS [4]. Finally, another study investigated two different approaches to reduce MS while watching movies on onboard displays. The first approach used vertical stripes as a surrounding image which is perceived as a background, which induced circular vection to a certain degree. In the second approach, movies were displayed as if the plane of the screen rotated on a vertical axis, and circular vection was induced as in the first approach. And because circular vection reduces SC, MS was reduced [11]. While considering various design approaches to combat MS during AD, the approaches have to be validated for safety in order to avoid injury or risking the lives of the occupants [8]. Though all the approaches and solutions mentioned above could help to avoid MS during AD, they place some constraints which could reduce the benefits of AD.

3 Virtual Automated Vehicle Implementation

In order to investigate the impact of the visual cues presented to passengers during AD, an automated vehicle prototype was developed in VR environment using Unity 3D engine and Oculus Rift consumer version 1 (CV1) [12, 13]. The implemented solution is a virtual automated vehicle with a simple but effective takeover manoeuver which enables both manual and AD on a typical German highway with few traffic features. The simulation of a highly detailed real-world traffic elements and integration of VR technology is meant to improve the realism of the implemented traffic scenario [14]. Figures 2 and 3 show the road network layout and the simulated three-lane German highway respectively. A Tesla-like vehicle model was initially developed and textured. The vehicle interior was designed based on a previous Bosch automated vehicle human-machine-interaction concept [15].

Fig. 2.

Layout of the implemented road network.

Fig. 3.

The virtual 3-lane typical German highway.

3.1 Development Tools

VR applications require high computing intensity due to the increased total number of pixels, high refresh rate, and because 3D scene has to be rendered separately for each eye at the same time. Therefore, powerful components are needed to accomplish these tasks. This project was developed with a VR-Ready computer with the processor Intel Core i7 5960X with 8 cores and 16 threads, as well as clocking up to 3.5 GHz. The NVidia Titan X graphic card used offers a variety of functions e.g. simultaneous multi-projection, which is automatically rendered specifically for VR for a higher performance and a more detailed virtual environment to increase realism [16].

The Oculus CV1 used for this work offers a resolution of 2160 × 1200 pixels for both eyes, and a refresh rate of 90 Hz. The higher resolution as well as the refresh rate helps to achieve a higher level of immersion in the virtual environment and also reduces the occurrence of simulation sickness during user evaluation. Oculus CV1 was chosen because of ergonomic preferences (light-weight and comfortable) and the high-performing audio. It is also well supported by Unity 3D for a simple and fast integration.

Logitech electric steering wheels and pedals are used to reproduce the steering, accelerating and braking controls of the virtual vehicle, which is then passed on to the computing system. In this project, the Logitech G29 steering wheels, pedals and shifters were used. The red button on the steering wheel was implemented to help the drivers choose a profile, and also scroll through the texts while reading. Logitech offers a Unity plug-in as well that facilitates integration.

Atomic A3 of the Atomic Motion Systems (AMS) was integrated to the VR driving simulator as a cost-effective way to transfer motion from the virtual world to the real world. It offers the possibility to move the drivers on two axes, each with a 27° angle of incidence and a speed of approximately 72° per second. AMS provides native Unity support for Atomic A3. Simphynity software with which it is possible to run the platform, detects motions of the implemented vehicle dynamics and reproduces these forces in form of motion. The Atomic A3 2 degree of freedom motion simulator was integrated to the system in order to simulate the feeling of a moving vehicle (refer to Fig. 4).

Fig. 4.

Atomic A3 from Atomic Motion Systems.

4 Experiment

The study separated participants into two randomized groups with each group having equal number of male and female participants: a control group which only got displayed texts on the tablet, and the experimental group which got both texts and video streaming of the traffic environment on a split screen in real time (refer to Figs. 7 and 8). In order to collect and measure data, two questionnaires were used. The first questionnaire consisting of thirteen questions, was established to collect socio-demographic information, previous driving experience, and VR experience. The second questionnaire is the simulation sickness questionnaire (SSQ), which consists of 16 questions and was developed by Kennedy and colleagues in 1993 (refer to Fig. 5) [17].

Fig. 5.

Original version by Kennedy and colleagues [17].

Simulation sickness questionnaire.

4.1 Participants

The experiment comprised of fourteen participants (N = 14), eight males (Mage = 38.34, SD = 10.28) and six females (Mage = 34.83, SD = 5.42). The participants were aged between 28 to 55 years old. Two participants had previous cases of MS, seven were never affected while five were not sure if they suffered from MS before. Ten out of the fourteen participants had previous experience with driving simulators (refer to Fig. 6). More than 85% of the participants are frequent drivers. Participation was voluntary and no compensation was awarded to the participants. All participants have a university degree.

Fig. 6.

Most users never suffered from motion sickness before (left) and most had previous experience with driving simulators (right).

4.2 Procedure

The experimental group was presented with a multiple screen tablet, which displayed both texts and video streaming of the horizon in real time, while the control group was presented with a tablet with texts only to be read out loud for ten minutes during AD. Firstly, each participant completed the pre-questionnaire which mostly consisted of socio-demographical questions, previous experience with driving simulators, driving experience, MS history and previous experience with VR headset. Because most participants were working prior to the experiment, and most probably experienced some symptoms such as fatigue, the SSQ was completed before and after the experiment in order to effectively compare the initial and current state of wellbeing of the participants.

Secondly, instructions were giving on how to operate the system and navigate freely in the virtual environment. This was followed by a two-minute test drive in order to enable the users to get acquainted with the system and eliminate anxiety during the experiment. The experiment proper starts with the system requesting the users to drive manually for two minutes, after which they can perform a takeover and hand over control to the system. The system then controls the vehicle for approximately ten minutes. In the autonomous mode, a tablet appears with captivating African fairy tales which all participants read out for ten minutes. While reading, the participants were not allowed to look at the horizon or get any visual cue of the surrounding traffic environment. They could look around the vehicle inner design or stay focused on the reading assignment. Motion cues was provided with the motion simulator in order to give the feeling of driving on a real road. Participants of the multiple screen group were presented with the video capture of the surrounding on the lower section of the tablet (see Figs. 1 and 7) while participants in the control group got only texts displayed (see Fig. 8). Finally, the experiment ends with the participant completing the SSQ for the second time. Each test slot lasted between thirty and forty minutes depending on how fast the users completed the questionnaires. None of the participants dropped out nor stopped the experiment before the simulation ended.

Fig. 7.

User of the experimental group reading texts while viewing the displayed horizon.

Fig. 8.

Only texts presented to the control group during automated driving.

5 Results

This section presents the results of the experiment while displaying the total score of typical symptoms of MS based on the SSQ score for both groups. The result of the experiment revealed that participants with the split screen suffered less from those symptoms peculiar to MS, when compared to the control group who were only presented texts to read and no visual cues of the horizon during AD.

To learn how the SSQ score of each symptom is calculated, refer to [3, 17]. The score distribution for the symptom “General discomfort” for the experimental group was 9.5% before exposure as compared to only 4.7% after exposure. Meanwhile, the score distribution for the control group was 14.3% for “General discomfort, as compared to 9.5% before exposure (refer to Fig. 9). Therefore, the control group suffered more general discomfort. Most participants of the experimental group described the experience as relaxing while the control complained it was strenuous and boring. Likewise, the severity level for the symptom headache did not change for the experimental group, unlike the control group which suffered severe headache after exposure. The score distribution of the control group for the symptom “Headache” was registered as 4.7% before the experiment, as compared to 10% after exposure. No headache was recorded before and after exposure for the experimental group (see Fig. 10).

Fig. 9.

Before and after comparison of general discomfort maximum score for both groups.

Fig. 10.

Experimental group suffered no headache. Control group suffered severe headache.

Figures 11, 12, 13 and 14 show the results of the symptoms salivation increasing, stomach awareness, nausea and eye strain for both groups. Figure 11 shows that 9.5% maximum score was recorded for the experimental group before and after exposure for the symptom “Salivation increasing”. The control group suffered more with a 4.7% before as against 9.5% after the simulation. Meanwhile, no prior stomach awareness discomfort was recorded for both groups. Very high stomach awareness discomfort was recorded after exposure for the control group as compared to the experimental group (Fig. 12).

Fig. 11.

Before and after results of salivation increasing for both groups.

Fig. 12.

Before and after results of stomach awareness for both groups.

Fig. 13.

Before and after result of the symptom nausea for both groups.

Fig. 14.

Distribution score for eye strain for both groups.

Meanwhile, other symptoms which are not graphically represented here e.g. vertigo recorded a 0% before and 4.8% after exposure for both groups. Burping also recorded a 4.8% score distribution before and after exposure for both groups. The maximum score recorded for the symptom “sweating” for both groups remained the same before and after exposure. Symptoms such as “difficulty concentrating” showed a slight advantage for the experimental group when compared to the control group. On the other hand, the experimental group (before = 38.1%, after = 38.1%) was more affected by the symptom “fullness of the head” than the control group (before = 28.6%, after = 19%). Finally, the symptom “fatigue” was greatly reduced for the experimental group after the exposure (before = 43%, after = 29%) as compared to the control which was only slightly reduced (before = 29%, after = 24%). Though eye strain is peculiar to simulation sickness [18], the result was also considered as important because the experiment was carried out in a VR environment (refer to Fig. 14).

6 Discussion and Future Work

This work evaluated whether using a multiple screen display which streams the traffic environment in real time, could help reduce MS during AD. We implemented a VR solution for MS using Oculus rift CV1. The implemented solution is based on the projection of a video capture of the surrounding vehicle environment on a multiple screen device such as a tablet or vehicle onboard display, which enables the occupant to view the horizon while performing other activities during AD. The motion simulator was integrated to the system in order to simulate the feeling of a moving vehicle. An experiment was conducted with fourteen participants and the results show that occupants of automated vehicles require visual cues of the horizon while reading or carrying out activities which take their eyes off the road while being driven. This helps to reduce the discrepancy between the motion they feel and the motion they do not see while reading.

Typical symptoms of MS such as stomach awareness, headache and nausea were mostly higher for the control group which gives a cause for alarm for AD passengers. The occurrence of MS could greatly affect the acceptance of AD because the fear of MS could make drivers to avoid activities that could make them sick, or just focus on the road in order to avoid sickness. This will hinder the effective usage of the time available and the luxury of being driven, thereby greatly reducing the benefits of self-driving cars [8]. When the driver takes up the role of the passenger during AD, they should not be under any obligation to focus on the road at all times because of the fear of getting motion sick, this could greatly reduce the benefits of automated vehicles. However, it is important to get the visual perception of the motion that the body feels in order to avoid the brain getting mismatched motion signals. When this happens, the result is sensory conflict between the motion visually perceived (the eyes tell about the texts being read by the driver) and the motion the body perceives (the vestibular system tells another story: the person is moving). This could result in most people to MS.

Occupants of automated vehicles are most likely to indulge in activities that will warrant them to take their eyes off the road such as reading, and this might result to MS. This could be avoided by generating moving image signals and optical representation of the vehicle surrounding by a vehicle camera, and presenting these video data to the passenger through a display. The display provides multiple screen, one for the real time images of the moving vehicle environment, and the other screens for work or just interaction. This will enable passengers to fully enjoy the luxury of being driven and concentrate on other tasks not related to driving. Though the implemented VR solution shows an improvement and a potential solution to MS in automated vehicles, a thorough study with more participants on a real car should further solidify and demonstrate this solution.

Future work should focus on the implementation of a real physical prototype to be integrated and evaluated in a real car. A user experience evaluation with a significant number of participants is also relevant in order to gather sufficient data for an effective evaluation of the solution. The acquired results should help in facilitating the development and improvement of the final product.

AD will offer numerous benefits such as safety, reduced traffic congestion, flexible connectivity, comfort, and mobility to many without driver’s license and the elderly. It is therefore important to implement solutions that will eliminate potential limitations. MS is one limitation that should not be taken for granted during AD because this could drastically reduce the benefits, and it is a concern to all stakeholders [19].