Keywords

1 Background

Nowadays, numerous machineries are being altered to enable them to operate autonomously using the rapid developments in information technology and artificial intelligence. The same is true for automobiles. For example, TESLA Inc., USA, has manufactured a full self-driving system. However, from a user perspective, some people have worries or doubts regarding the safety and reliability of self-driving cars, which may lead to anxiety while using such cars. It can cause worries not only to drivers but also to passengers, when those autonomous cars are used in the public transportation system. Especially passengers usually do not have sufficient knowledge about such technologies and the specific risks associated with them. As Peyre et al. [1] indicated, the question regarding whether riding automatically driven vehicles as a passenger can cause anxiety or not is an important issue [1].

Anxiety is a difficult concept to assess and it is often measured on a five-to seven-point scale in response to questions on subjective feelings of anxiety, e.g., the State-Trait Anxiety Inventory (STAI) [2]. However, such scales cannot capture anxiety as a real time index because of limitations related to the non-continuous style of data sampling and its retrospective reporting. Subjective anxiety has also been assessed by researches using biometric indexes, especially cardiac wave analysis, i.e., indexes of the parasympathetic nerves (high frequency cardiac waves: HF) and sympathetic nerves (low frequency cardiac waves divided by HF: LF/HF) [3, 4]. These methods can measure some aspects of anxiety continuously. However, it is hard to capture these indexes as direct values of the anxiety level because such biomedical indexes reflect different physiological states and not just anxiety. Additionally, there are latencies between subjective feelings of anxiety and the measurement of these indexes.

In this study, we have attempted to measure the subjective feeling of anxiety continuously from the time before vehicular movement starts to the time when the destination is reached using a new haptic device, which can slide forward and backward. It is possible to assess real-time subjective feelings, anxiety in this case, directly while a participant experiences the stimulus of riding an autonomous car. In order to test its effectiveness, we executed an experiment using computer graphics (CG) delivered through a head-mounted display to measure the anxiety, which a participant experienced while virtually riding in a new automated vehicle. To determine their sensitivity, the maximum speed of the vehicle was varied from 19 km/h to 320 km/h and patterns of acceleration/deceleration were introduced.

Another simulation variable was introduced during the experiment, which was aimed at decreasing the passenger’s anxiety. We used sound that varied according to the change in speed of the vehicle as a possible way to decrease the passenger anxiety in autonomous cars. The sound was in ascending/descending musical scale, in which the tone changed gradually in accordance with the speed of the train (as an example: Keihin electric Express Railway Co. Ltd, Japan). In our experiment, gradually changing sounds were presented for 6 s at the beginning of acceleration and another 6 s before stopping. The introduction of these sounds may be able to eliminate or diminish passenger anxiety because they can help the passenger to predict speed changes.

If this new method, which uses a handle-like device to ask and record subjective feelings of anxiety continuously, is effective for measuring anxiety as well as other feelings like fun and excitement, it can be useful in evaluating the design of motion control in autonomous cars in the future. We executed a psychological experiment to determine whether this new method would be able to measure passenger anxiety and to see the effectiveness of sound in decreasing passenger anxiety. This experiment is the first of its kind to evaluate the subjective anxiety of a passenger continuously.

2 Methods

2.1 Experimental Design

In order to assess the effectiveness of our new method in capturing the participant’s anxiety, we set three conditions of maximum speed (19 km/h, 160 km/h and 320 km/h) and two accelerate/decelerate patterns (linear and exponential). Those speeds were chosen from data obtained from pilot experiments, which were executed to determine the velocity needed for a participant to feel some anxieties with the CG simulation set used in our study.

In addition, experiments were performed with and without sound. Totally, there were 3 (levels of speed) × 2 (speed change patterns) × 2 (with and without sound) conditions, all of which were within-subject factors. This resulted in twelve experimental conditions. Each one in those twelve conditions were presented three times, thus participants had thirty-six trials, which were presented in random orders.

2.2 Stimuli and Devices

We created the handle device for measuring the participants’ subjective feelings of anxiety using a 3D printer. The handle device consisted of a T-shaped handle, which could move in the longitudinal direction and had a rectangular-shaped base (Fig. 1). The device included elastic that held the handle at the center. The more the handle was pushed (or pulled), the more was the reaction force to return to the center. This device was connected to a computer that tracked handle position at all times during the experiment. This data was used to capture the conditions under which the participants experienced anxiety.

Fig. 1.
figure 1

Image of the handle device. The base can be made to slide forward and backward by pulling or pushing the handle. The handle can move a length of 300 mm. Participants grasped the handle and were instructed to pull it when they experienced anxiety or push it when they felt relaxed.

Full size image

We used 3D scenery as an experimental stimulus that gave the participant the illusion of moving forward. This stimulus was developed using a computer graphics platform, Unity R (Unity technologies). The scenes consisted of a road with two lanes, a sidewalk, some human models, buildings, and poles (Fig. 2). The stimuli were automatically changed to make the participants experience a feeling of moving ahead on the straight road. The first ten seconds from the start consisted of gradually moving ahead with acceleration. After reaching the maximum speed, the participants experienced ten seconds of moving at a constant speed followed by ten seconds of moving with gradually decreasing speed until coming to a halt. The scenes moved along a longitudinal direction and were presented through a head-mounted display (Oculus Rift Development Kit 2, Oculus VR, Inc.).

Fig. 2.
figure 2

An example of the stimulus presented in the experiment. The stimulus consisted of computer graphics created by Unity and contained some human models, buildings, and poles. The stimulus was created for giving the participants the experience of virtually riding in an autonomous car, where the car automatically moved in the longitudinal direction. Participants watched the stimulus through a head-mounted display.

Full size image

We used two types of acceleration and deceleration patterns, namely linear pattern and exponential (non-linear) pattern; in the former, the speed during acceleration and deceleration was changed at a constant ratio (Fig. 3.a), which resulted in a linear pattern, while in the latter, the speed was changed at a non-linear or exponential ratio, which resulted in a non-linear pattern (Fig. 3.b). Further, the maximum speed was set to 19 km/h, 160 km/h, and 320 km/h in different test runs.

Fig. 3.
figure 3

Graphs indicating the speed patterns of the autonomous car with acceleration and deceleration. The X-axis indicates time while the Y-axis indicates the speed of the car. The solid lines show the speed of the car and dotted lines indicate the acceleration/deceleration ratio. Graph (a): linear pattern; Graph (b): exponential (non-linear) pattern.

Full size image

Two sound patterns were created for acceleration (ascending musical scale: sound A) (Fig. 4.a) and deceleration (descending musical scale: sound B) (Fig. 4.b) in the with-sound condition. There was no sound stimulus in the without-sound condition. Sound stimuli were presented through a headphone system (Sennheiser, HD 380 pro).

Fig. 4.
figure 4

Sounds presented during acceleration and deceleration. Upper (a): musical score indicating the sound presented during acceleration. Lower (b): musical score indicating the sound presented during deceleration. Both sound patterns lasted 6 s.

Full size image

Participants were also asked to wear a heart rate sensor, myBeat (UNION TOOL CO., WHS-2), to record their heart beat. This data was used for investigating the relationship between the continuous evaluation of subjective anxieties and biometrical indexes, which will be reported in another study.

2.3 Participants

Sixteen undergraduate or graduate students (of which eight were female) of the University of Tsukuba participated in this experiment voluntarily. All the participants had normal or corrected-to-normal vision and normal hearing.

2.4 Procedure

The experiment was executed separately for each participant. When a participant came the laboratory, s/he was explained the purpose of this experiment and s/he gave their consent to participate in the experiment in writing. Next, the participant was made to wear a heart rate sensor and asked to sit on a seat fitted with the handle device. S/he wore the head-mounted display and a headphone and was asked to grasp the device handle while watching the simulated scenes.

The participants could watch the scenes in all directions using the head-mounted display. They were instructed to grasp the handle while watching the scene and pull the handle when they felt anxious; the more anxiety they felt, the more they pulled the handle toward themselves. On the other hand, they were asked to push the handle when they felt relaxed; the more they felt relaxed, the more they pushed the handle forward. At the end of the experiment, the participants were questioned on their acceptability of the vehicle: “Would you wish to ride this vehicle if it was a kind of public transport?” They were asked to respond on a seven-point scale: “Absolutely not, I won’t” (1) to “Yes, I would” (7). The next test was started after s/he had answered the question. After two practice trials, the experiment was executed with a 5 min resting time between every nine tests. The experiment lasted 90 min approximately.

3 Results

The report in this study covers the anxiety levels as recorded by the handle on the handle device. The remaining data consisting of acceptability rating and biometric indexes will be reported in another paper. We analyzed the data relating to handle position and calculated the average handle position per second. We normalized the data for each participant and separated the data into three sections: the acceleration part, the part with constant speed, and the deceleration part. With this process, we obtained ten data corresponding to the three sections under each of the twelve conditions.

The acceleration and deceleration sections alone were analyzed in this study because the objective of this study was to assess the change in anxiety level with change in speed. The data corresponding to these 2 sections were analyzed individually by 3 × 2 × 2 × 10 analysis of variances (ANOVAs) with repeated measures corresponding to three maximum speeds (19 m/h, 160 km/h, or 320 km/h), two patterns for changing speed (linear vs. non-linear), two sound conditions (with sound vs. no sound), and ten divided sections (intervals of every 1 s from 1 to 10 s for acceleration and deceleration). The average of the handle position standardized for each participant is depicted in Fig. 5. The results of the ANOVA are summarized in Tables 1 and 2.

Fig. 5.
figure 5

The mean of the handle positon maintained by participants while they watched the simulation. The graphs on the left (a) indicate the handle positions during acceleration and the graphs on the right (b) indicate the handle positions during deceleration. The solid line with black circles indicates linear pattern of acceleration with no sound. The dotted line with black circles indicates linear pattern of acceleration accompanied by sound. The solid line with white circles indicates non-linear pattern of acceleration with no sound. The dotted line with white circles indicates non-linear pattern of acceleration with sound. The error bar shows the standard error.

Full size image
Table 1. The results of ANOVA of handle position during acceleration. This table lists the main effects, interactions, and significant post hoc multiple comparisons. Lower values in mean indicates higher anxiety, by pulling handles more. A single asterisk indicates a p-value lower than 5%, two asterisks indicate a p-value lower than 1%, and three asterisks indicate a p-value lower than 0.1%. A plus sign indicates a p-value lower than 10%. This statistical analysis was performed using IBM SPSS Statistics 24.
Full size table
Table 2. The results of ANOVA of handle position during deceleration. This table lists the main effects, interactions, and significant post hoc multiple comparisons. Lower values in mean indicates higher anxiety, by pulling handles more. A single asterisk indicates a p-value lower than 5%, two asterisks indicate a p-value lower than 1%, and three asterisks indicate a p-value lower than 0.1%. A plus symbol indicates a p-value lower than 10%. This statistical analysis was performed using IBM SPSS Statistics 24.
Full size table

During acceleration, the effect of the maximum speed was significant (F(2, 30) = 29.605, p < .001, \(\eta{^{\text{2}}_{p}}\) = .664; see Table 1). Participants pulled the handle more during the maximum speed of 320 km/h than during 160 km/h (p < .001). Similarly, the pulling movement on the handle was more at 160 km/h than at 19 km/h (p = .007) and more at 320 km/h than at 19 km/h (p < .001). These results indicate that our method could measure the subjective anxieties, which were dependent on the maximum moving speed.

There were significant interactions between the pattern of speed change and sound (F(1, 15) = 7.251, p = .017,  \(\eta{^{\text{2}}_{p}}\)= .326), speed changing pattern and divided sections (F(9, 135) = 24.818, p < .001, \(\eta{^{\text{2}}_{p}}\) = .623), and maximum speed and divided sections (F(18, 270) = 31.089, p < .001, \(\eta{^{\text{2}}_{p}}\) = .675). These results indicated that when the simulation was accompanied by sound, the participants pulled the handle more during linear acceleration than during non-linear acceleration (F(1, 15) = 6.091, p = .026, \(\eta{^{\text{2}}_{p}}\) = .298). During non-linear acceleration, participants pulled the handle more when the simulation was not accompanied by sound than when sound was present (F(1, 15) = 5.443, p = .034, \(\eta{^{\text{2}}_{p}}\) = .266). In addition, in the interval from 3 s to 5 s, the participants pulled the handle more during linear acceleration (ps < .05). On the other hand, in the interval from 8 s to 10 s, they pulled it more during non-linear acceleration than during linear acceleration (ps < .05). Furthermore, there were differences between the three conditions of maximum speed for nearly every divided section except at 2 s (ps < .05).

In addition, there were three-way interactions between the speed changing pattern, maximum speed, and the divided 10 s (F(18, 270) = 5.455, p < .001, \(\eta{^{\text{2}}_{p}}\) = .267), and between speed changing pattern, sound, and the divided 10 s (F(9, 135) = 3.504, p = .001, \(\eta{^{\text{2}}_{p}}\) = .189). There were no four-way interactions (F(18, 270) = 0.867, n.s., \(\eta{^{\text{2}}_{p}}\) = .055). For example, at 320 km/h with non-linear acceleration (lower left image in Fig. 5a.3), the handle position clearly varied across the divided 10 s. In the divided sections from 2 to 10 s, the handle position was significantly or marginally lower in the no-sound condition than with sound (ps < .10). In addition, the ascending sound, i.e., the gradual change in upper musical scale in accordance with the accelerating speed, showed decreasing feelings of anxiety in the non-linear acceleration condition alone with maximum speeds of 160 km/h and 320 km/h.

During deceleration (Table 2), there were significant and marginally significant effects of the speed changing pattern (F(1, 15) = 8.573, p = .010, \(\eta{^{\text{2}}_{p}}\) = .364), maximum speed (F(2, 30) = 54.993, p < .001, \(\eta{^{\text{2}}_{p}}\) = .786), sound (F(1, 15) = 3.775, p = .071, \(\eta{^{\text{2}}_{p}}\) = .201), and divided 10 s (F(9, 135) = 6.890, p < .001, \(\eta{^{\text{2}}_{p}}\) = .315) (Fig. 5(b)). Additionally, there were significant or marginally significant interactions between speed changing pattern and sound (F(1, 15) = 4.410, p = .053, \(\eta{^{\text{2}}_{p}}\) = .227), maximum speed and sound (F(2, 30) = 3.321, p = .050, \(\eta{^{\text{2}}_{p}}\) = .181), and speed changing pattern and divided 10 s (F(9, 135) = 10.676, p < .001, \(\eta{^{\text{2}}_{p}}\) = .416). There was three-way interaction between speed changing pattern, maximum speed, and divided 10 s (F(18, 270) = 4.119, p < .001, \(\eta{^{\text{2}}_{p}}\) = .215). The results relating to the deceleration section indicated that differences in simulation conditions with sound and without sound were significant only for the maximum speed of 160 km/h (F(18, 270) = 4.119, p < .001, \(\eta{^{\text{2}}_{p}}\) = .215). However, the subjective anxiety was higher for non-linear deceleration than for linear deceleration without sound (p < .001). In the case of non-linear deceleration, the subjective anxiety was higher without sound (p = .015). Moreover, we found that there was a significant difference between the first five seconds and last five seconds during non-linear deceleration and maximum speed of 19 km/h (ps < .05).

4 General Discussion

The present study investigated whether our new haptic device was effective in measuring real-time subjective anxiety caused by riding in an autonomous car. In the experiment, a participant who was wearing a head-mounted display watched a CG animation that simulated the feeling of riding in an autonomous car. The CG animation stimuli were operated using three variables: maximum speed (19 km/h, 160 km/h, or 320 km/h), acceleration/deceleration pattern (linear or non-linear), and with or without ascending/descending sounds (sound and no sound). While the participant was watching the simulation, s/he grasped the handle device and pulled when he/she experienced anxiety and pushed when they felt relaxed.

The results of the experiment demonstrated that the higher the stimulus of speed was, the higher was the anxiety the participants felt. Additionally, subjective anxiety was higher during non-linear changes in speed than during linear speed changes. The results indicated that our new method could measure the anxiety experienced by the participants in accordance to the speed at the time, while riding an automated vehicle. Moreover, we found that sounds can decrease feelings of anxiety during automated driving. The result showed the possibility that the sound might eliminate passenger anxiety related to riding an autonomous car if an appropriate sound was presented especially when the car started to accelerate.

Even though we got some interesting results here, it should be noted that this experiment is not “real” in some meaning, because there are much differences between the circumstances of an actual passenger riding an autonomous car and one who watches a CG simulation of the same. Although in some previous papers reported that people could perceive self-movement using CG simulation, by the sensation of self-motion or vector [5, 6], it is not clear that those sensation are same with those in the real riding. Those differences might be related to the fact that the maximum speed in the current study was 320 km/h. This was because most participants did not experience anxiety when they were shown a simulation where the speed was less than 320 km/h in our CG animation. The speed of 320 km/h cannot be a practical speed for an autonomous car, which implies that there was really a gap between the CG animation and the real-life situation. Those differences should be passengers’ feeling of the gravitational acceleration, vibrations due to the movement of the car, or airflows from the window, in a real-life situation. If we would like to use this method with simulations to evaluate the design of motion control actually, methods to fill those gaps might be important.

The experiment performed in current study showed that the handle device could measure the continuous subjective anxiety, which varied with factors in maximum speed, acceleration or deceleration pattern, and presence of sound in the simulation presented using a head-mounted display. The result indicated the possibility that using computer simulation with the handle device enabled the measurement of not only anxiety but also of other emotions such as “fun associated with riding.” Actually, even in current results, the data upper than the “anxiety 0” point, pushing the handle might mean a participant’s relaxing or enjoying the riding. Using the new device to evaluate more various KANSEI subjective feeling might be useful, to create more acceptable designs of motion control. Possibilities and limits for capturing real-time or on-time subjective feeling should be investigated in future.