Keywords

1 Introduction

As vehicles are equipped with an ever-increasing amount of Advanced Driver Assistance Systems (ADAS) and tend to act more and more autonomously, the importance of investigating the User Acceptance (UA) and User Experience (UX) in this context is increasing. Currently, there is a trend towards cars, which can drive parts of a route or even the entire route fully autonomously. Some functional prototypes are already available like the Google-CarFootnote 1 or the transportation network company Uber Technologies is currently testing multiple autonomous cars in city traffic [2].

When discussing autonomous cars one has to distinguish between different levels of autonomy. The National Highway Traffic Safety Administration (NHTSA) [4] defines five different levels (0 to 4) of autonomy, ranging from no automation at all to fully self-driving cars.Footnote 2 However, cars at Level 0, which means no automation, are not very common anymore. Level 1 provides function-specific automation with the purpose to aid the driver who still has to control the car at all time. Level 2 adds at least two more advanced automated control functions such as adaptive cruise control or lane centering. At Level 3 the car is able to drive fully autonomously in predefined situations for a limited time span, e.g., on highways. And finally, at Level 4, the vehicle is able to act fully autonomously for the entirety of a trip. These changes are likely to have a big influence on how the driver interacts with the car and therefore User Experience probably changes as well. Rödel et al. [7] have conducted a study to examine how different levels of autonomy affect User Acceptance and User Experience. For this purpose, an online-questionnaire was created and answered by 336 study participants. The results indicate that User Acceptance and User Experience are highest at levels of autonomy that have already been deployed in modern cars (i.e. levels 1 and 2 as defined by the NHTSA).

In our study we explore the effects of increased autonomy on UA and UX measures in a more immersive setup. For this we look at three different levels of autonomy (NHTSA levels 1, 3 and 4) using a Virtual Reality driving simulator.

2 Related Work

Our study design is based on the previously mentioned work of Rödel et al. [7] 336 participants with different levels of driving experience and experience with ADAS (automatic driving assistance systems) were asked to imagine five driving scenarios with different levels of autonomy. After each scenario, UA and UX factors were measured using standardized questionnaires. To determine user acceptance the authors used a variation of the Technology Acceptance Model (TAM) by Davis [1]. To determine UA in our study we used an alternative instrument the CTAM introduced by Osswald et al. [5] which will be described in greater detail below. The findings of Rödel et al. [7] suggest that the attitude towards driving systems decreases significantly with the level of autonomy. In addition, the perceived behavioral control is highest at the lowest level of autonomy. The authors conclude that people experience a higher UA and better UX if they are more familiar with the system.

The CTAM was originally developed to research drivers’ acceptance of in-car technology but it has also been used for research on autonomous vehicles, e.g. by Robertson et al. [6]. The model is based on the Unified Theory of Acceptance and User of Technology (UTAUT) but extends this model by adding dimensions like safety and anxiety. In contrast to the Technology Acceptance Model (TAM), which focusses on desktop-based systems, the CTAM takes car-related factors such as limited mental resources into consideration as well as the assistance the user gets while performing a driving task. The resulting determinants for the CTAM are performance expectancy, effort expectancy, attitude towards using technology, social influence, facilitating conditions, self-efficacy, anxiety, behavioral intention to use the system, and perceived safety. The questionnaire comprises 39 items that build up these eight dimensions.

Another paper we based our experiment on was published by Helldin et al. [3] that discusses the use of an interface to display uncertainty levels of an autonomous car system. What we found interesting in this study was the setup of the test to determine the users’ trust in the autonomous system in general. Therefore, they used a simulator setup where participants could interact with a fully functional cockpit while the environment was displayed on a big projector canvas surrounding them. The authors allowed the participants to get used to the simulator in a 3–5-min test session and presented them with tasks to follow a predefined route and instructions of how to interact with the system afterwards. After each task, the users answered the questionnaire assessing their trust in the system during the task. We followed this setup in our own experiment but added some free text questions and an immersion questionnaire to gain further information.

3 Methods

3.1 Participants

The study was conducted with 17 participants (6 male, 11 female) aged between 20 and 29 years. Most of them were university students and were in possession of a driver’s license (15 out of 17). None of the subjects had previous experience with vehicle automation systems but three had used driver assistance systems before.

3.2 Setup

The study was conducted in the Future Interaction Laboratory at the University of Regensburg using an Oculus Rift CV1, a Logitech G27 Steering Wheel, and a self-developed Simulator Application. We designed three tasks in which the participants had to navigate to a destination using the simulator system.

In each task, the participants drove the simulator car through a city landscape with either no automation, semi-automation or full automation. In the semi-automated scenario, the participant had to drive by himself until the car takes over control on the highway. In the fully automated scenario the car drives completely autonomously allowing no interaction from the driver.

Using a within-subjects design each participant had to complete each task in a counter balanced Latin square order to eliminate learning effects (Fig. 1).

Fig. 1.
figure 1

Top: The test setup with the Oculus and the racing chair. Bottom: Screenshot of our simulator from the tester’s perspective.

Full size image

3.3 VR Driving Simulator

We considered using OpenDSFootnote 3 as driving simulator for our study, as it comprises various features and is open source and can be extended with additional features like a multimedia panel and autonomous driving. However, after evaluating OpenDS, we found that it was not well suited for our purpose for several reasons. First, it is built for performance-specific tasks, like measuring reaction times. The design of the environment is very basic and just consists of a green plan with some roads on it and some square blocks as houses. This is not very suitable when trying to create an immersive experience. Second, OpenDS does not yet support the consumer version of Oculus Rift (Oculus Rift CV1), which we decided to use. For these reasons, we decided to implement our own driving simulator. We used UnityFootnote 4 as game engine for our simulator and the Unity Asset StoreFootnote 5 to get most of the assets. There is also an Oculus-SDK5 provided for Unity that can be integrated quickly.

We implemented basic vehicle physics and made the car controllable with a steering wheel controller. We added an infotainment system into the car, which can be controlled with buttons on the steering wheel. The infotainment system includes a media-player to playback music and videos and a dummy address book and a dummy hands-free telephone. The driving simulator guided the user towards navigation points with an arrow that is displayed on the front shield.

There was no need for complex traffic AI, because the user only drives each route once, which allowed for a very simple AI. The traffic just follows certain waypoints repeatedly and also reacts to other traffic and stops if someone else is in front of them. This AI concept had also the benefit that it could be reused for the autonomy functions of the user car. It either is triggered at a certain point on the route (NHTSA level 3) or controls the user car on the entire route (NHTSA level 4). There are audio hints for the user to inform him when the car will switch to autonomous driving or back to manual driving and some feedback when the user has reached his destination (Fig. 2).

Fig. 2.
figure 2

Top view of the city the participants had to navigate through. The dotted lines show the three different routes participant cars and AI controlled traffic followed in randomized order.

Full size image

3.4 After Task Questionnaire

After a brief introduction to the study setup, the participants provided basic demographic data on themselves and then executed three simulator tasks in a counter balanced Latin square order. All participants had to fill out the CTAM questionnaire after each task.

3.5 Final Interview

After filling out the CTAM for the last task a final interview was conducted with the participants consisting of two qualitative questions asking the participants about their subjective preference for one of the three autonomy levels and whether they could imagine driving an autonomous car in the future and the igroup Presence Questionnaire (IPQ)Footnote 6 which was designed for measuring the sense of presence in a virtual environment. The sense of presence is a variable of the user’s experience describing how immersed the user feels in the virtual environment. The IPQ consists of 14 items that include questions about the “General feeling of immersion”, the “Spatial Presence”, the “Involvement” and the “Experienced Realism”. These items are answered using a seven-point Likert scale. We used the IPQ to evaluate the immersion of our simulator.

4 Results

In this section, we present the quantitative results of the study followed up by a presentation of the qualitative data and a corresponding interpretation. We will give a thorough description of the evaluation of User Experience (UX) and User Acceptance (UA) factors described in the previous chapter after outlining the IPQ results on the immersiveness of our simulator environment.

We followed the IPQ guidelines for the questionnaire evaluation to arrive at the following results: the values of “Realism” (REAL) – so in how far the virtual experience feels like the reality – and “Involvement” (INV), which describes how much participants feel like being part of the experience, are almost identical to the values offered by the IPQ consortium as average values across several studies. Only the “Spacial Presence” (SP) value that describes how much the virtual surroundings feel like a real world, was slightly lower (our SP: 3.15; average IPQ SP: 3.75). These values suggest that the immersion generated by our driving simulator environment compares reasonably well against the collected data of evaluations contributed to the IPQ consortium (Fig. 3).

Fig. 3.
figure 3

Our IPQ results in comparison to the whole collection of IPQ answers.

Full size image

UA and UX for the various conditions were measured with a total of 120 items – 40 items repeated for each autonomy level – in the user experience and user acceptance questionnaires (CTAM) for each participant. Each question consists of a seven-point Likert scale. As suggested in the paper of Osswald et al. [5] each question item was labelled from the most positive rating as number one to the most negative rating as number seven (Table 1).

Table 1. Measured user acceptance and user experience factors.
Full size table

Looking at the individual user experience and user acceptance scores measured by the questionnaire we found that some factors are significantly affected by the level of automation provided in the different scenarios of the study. With a significance level of alpha = 0.05 Friedman’s Chi Square Test suggested a significant difference between the different automation levels of “Performance Expectancy” (χ2(2) = 12.644; p = 0.002), “Social Influence” (χ2(2) = 8.758; p = 0.013) and “Perceived Safety” (χ2(2) = 9.909; p = 0.007). Although none of the other factors differ significantly across conditions by Friedman’s Test, we will look in more detail at the different scenarios and the various acceptance dimensions in the following.

Dunn-Bonferroni post hoc tests were carried out and there were significant differences between no automation and full automation in all three categories: for the “Performance Expectancy” (p = 0.01) with mean ranks (M) that show an increasing rating from no automation (M = 2.62) to partial automation (M = 1.88) and a slightly decreased rating from partial to full automation (M = 1.50); the “Social Influence” (p = 0.004) with steadily decreasing mean ranks for no automation (M = 2.50), partial automation (M = 2.00) and full automation (M = 1.50) – as well as the “Perceived Safety” (p = 0.002) that also show decreasing mean ranks from no to full automation (M(level 1) = 2.50; M(level 2) = 2.06; M(level 3) = 1.44). There were no significant differences comparing levels 1 and 2 or 2 and 3 of those three categories or in any other variables of the remaining categories which are discussed below.

For “Effort Expectancy” the mean ranks are rather similar for all three scenarios with a value of 2.26 for no automation, 2.00 for part automation and 1.74 for full automation and results in a non-significant Chi square value of only 2.793 (χ2(2) = 2.793; p = 0.247).

Minor differences could be found for “Facilitating Conditions” with p = 0.627 (χ2(2) = 0.933) and “Behavioural Intention to use the System” (χ2(2) = 1.418; p = 0.492).

“Self-efficacy” shows the biggest differences of mean ranks between no automation (M = 2.38) and full automation (M = 1.71) which is not significant though (χ2(2) = 5.055; p = 0.080).

“Anxiety” does not show any significant differences between any of the three levels of automation. The mean ranks show similar differences between no automation (M = 1.65) and partial automation (M = 2.06) and partial and full automation (M = 2.29). But even the difference between no and full automation is not significant (χ2(2) = 3.875; p = 0.144).

Lastly the “Attitude towards using Technology” mean ranks are also rather similar but differ from the other values in so far as the biggest difference exists between partial (M = 2.15) and full automation values (M = 1.79) which again is not significant (χ2(2) = 1.279; p = 0.528).

Rödel et al. [7] could distinguish significant differences in all their factors measured by their questionnaire. In contrast to that, this study led only to significant difference in a few of the measured factors. Rödel et al. [7] using an online questionnaire were able to include more participants and had therefor less factors measured per participant.

In our simulator setting only a small sample size including rather young participants could be tested. These factors may in part explain our different result compared to Rödel et al. [7].

Furthermore our new approach to use virtual reality instead of just detailed scenarios as stimulus is also likely to have influenced the outcome. Perhaps our participants could immerse themselves more in using self-driving cars through VR than the participants of Rödel et al. [7] by reading texts.

As mentioned above we also asked the participants two qualitative questions about their personal impression of semi-autonomous and autonomous driving. The results of these give a rather positive impression as nine of the 17 participants named the autonomous scenario as the most pleasant one, five named the partially autonomous scenario and only three the fully manual scenario. This positive attitude was also confirmed by replies to the second question about whether test subjects could imagine driving an autonomous car in the future as 15 participants replied with “yes” and only two mentioned safety issues that made them unsure about their willingness to use this system. These qualitative findings indicate a high acceptance of autonomous driving in our study with a rather young group of participants.

5 Conclusion

Rödel et al. [7] found that UA and UX values are highest for non-autonomous scenarios and concluded that users felt more comfortable in situations they are already familiar with. In contrast our results indicate that Performance Expectancy, Perceived Safety and Social Influence are significantly higher for the fully automated condition than for no automation. The monotonically but insignificantly increasing value for “Anxiety” along the rising levels of autonomy may suggest that people do not experience markedly greater anxiety about future autonomous cars than about the manual driving mode. The insignificant differences in the dimension “Attitude towards using technology” are harder to interpret. Perhaps the autonomous driving scenarios and the interior design of the simulated levels of autonomy were too similar and therefore un-realistic, and did not quite match the user’s expectations, so that their attitude towards using the new technology in autonomous cars was adversely affected. The factor “Self-efficacy” shows, that users tend to think, that they could handle fully autonomous cars better in comparison to manual and partially autonomous cars. In this regard, autonomous driving is also perceived as more efficient than partially automated driving or manually driving.

In conclusion, investigating user acceptance of different autonomy levels in an immersive approach with a VR driving simulator delivered different results compared to previous research. Future work has to examine autonomous driving in VR with a more elaborated and perhaps more futuristic and at the same time more realistic driving simulator design as well as an increased sample size including participants of more varying age.