Keywords

1 Introduction

Recent advances in audiovisual and virtual environment (VE) technologies have been employed to create high-accuracy virtual scenes. One of the aims of the VE technology is to provide a feeling of presence, defined as a feeling of “being there” [1].

VE technology is mainly based on the control of information input to a sensory organ. The control of visual input is most often used in VE technology because majority of information in daily life is visually obtained [2], and visual information affects human activity the most. Edwards reported that visual information has the most significant effects on human posture control and constitutes more than 50 % of all inputs [3]. Postural responses induced by visual information, such as motions or gradients, are called visually evoked postural responses (VEPRs) [4].

VEPRs that are attributed to both video contents and visual environments occur when subjects are in VEs, which mainly consist of videos [5–7]. Ohmi et al. [8] and Freeman [9] proposed that objective evaluations of the feeling of presence can involve VEPRs measurements. In addition, they reported that subjective and objective VEPR assessments exhibit the same trends in various viewing conditions.

Viewing movies in VEs favorably affects the presence, but it can also cause complex symptoms that are similar to motion sickness as a collateral effect, which are generally called visually induced motion sickness (VIMS) or cyber sickness. Stanney showed that 88 % of VE participants developed VIMS when viewing virtual reality movies for 1 h [10]. VIMS is generally explained by the sensory conflict theory that suggests the presence of conflicts among the visual system, labyrinthus vestibularis, and the experience of a subject [11, 12]. However, the detailed mechanisms underlying the occurrence of VIMS are unclear. Similar to the assessments of the feeling of presence, both subjective and objective evaluations of VIMS have been proposed. One of the representative subjective assessments is the simulator sickness questionnaire (SSQ) [13]. For the objective assessments of VIMS, changes in the autonomic nerve activity are detected by measuring physiological factors such as sudation, blood pressure, and respiration. On the other hand, Smart et al. [14] and Villard et al. [15] reported that measurements of body sway while subjects are viewing a movie can detect VIMS.

We develop an interest in VEPRs when a human watches a global motion movie. In particular, we have investigated the change in VEPRs to better represent the feeling of presence or symptom of VIMS. In our previous study, we verified the degree of feeling, which is represented by VEPRs, by simultaneous measurement of both the feeling of presence, symptom of VIMS, and body sway by changing the viewing distance (viewing angle) [16]. Hence, we investigated the effect of stereoscopic viewing on the change in the degree of feeling and VEPRs using the same measurement procedure.

2 Materials and Methods

2.1 Stimulus and Apparatus

The visual stimulus that we employed consisted of a movie created by 3ds Max 2015 computer graphics software (Autodesk, Inc., San Rafael, CA, USA). A screenshot of the movie used in this study is shown in Fig. 1. The basic construction of the movie consisted of a large number of balls shown at random positions and a green cross shown at the center position as the reference point.

Fig. 1.
figure 1

Screenshot of the movie used in this study. A large number of balls were located at random positions, and the green cross was located at the center position as a point of reference in the movie space. The motion in the movies was sinusoidal at 0.25 Hz in the depth direction.

Full size image

The motion in the movies was sinusoidal at 0.25 Hz in the depth direction (Z-direction) and was generated by moving camera-simulated ocular globes (the balls themselves did not move). The amplitude of the sinusoidal motion was set to 150 as the software setting.

To present the movie, it was projected onto a transmissive screen 200 cm in front of a standing subject using a domestic three-dimensional (3D) projector (EH-TW5100, Seiko Epson Corporation, Suwa, Japan). The maximum dive and maximum pull distances from the ocular globes in the 3D movies were 85 cm (parallactic angle, 2.5°) and 296 cm (parallactic angle, 1.8°), respectively, when the viewing distance was 200 cm. The subjects watched both the experimental 3D and two-dimensional (2D) movies, and 3D glasses (ELPGS03, Seiko Epson Crporation, Suwa, Japan) were used during the 3D movie viewing. To enable objective evaluation of the body sway, the subject stood on a Wii Balance Board (Nintendo Co., Ltd., Kyoto, Japan) adopting a Romberg’s posture. To continuously measure the position of the center of pressure (COP), the FitTri ver. 1.1c self-build stabilometry software for the Wii Balance Board, which was created by Yoshimura, was used.

2.2 Procedure and Design

Eleven paid volunteers [five males and six females, age range: 20–22 years; motion sickness susceptibility questionnaire-short (MSSQ-short [17]) adult score: 7.20 (average) ± 5.83 (S.D.); total score: 16.3 (average) ± 12.2 (S.D.)] were employed in the experiment. They have no vision or equilibrium problem. Through the MSSQ-short accomplished by the subjects before examination, we confirmed that the distribution of participants did not have a sensitivity bias attributed to sensory conflict. The study was approved by the Research Ethics Committee of Nagoya University. Written consent was obtained from the participants after the purpose and significance of the study and the nature and risk of the measurements were explained both orally and in writing. In addition, the study was conducted in line with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.

We performed the measurements in a controlled environment (illuminance: 5.9 lx, screen brightness 131.6 cd/m2). The protocol followed is shown in Fig. 2. First, a subject watched a static (nonmoving) movie for 60 s as a pretest. Next, the subject watched a sinusoidally motion movie for 180 s. Finally, the initial static movie was shown again for 60 s. By treating this 240-task (except for the pretest) as one trial, two trials (3D and 2D movie viewing) were performed in a random sequence to avoid order effect. In the duration of the trials, body sway was continuously recorded. The trial interval was set to more than 5 min. Each subject watched all test movies in a day.

Fig. 2.
figure 2

Study protocol. A subject watched a static (nonmoving) movie for 60 s as a pretest. Next, the subject watched a sinusoidal moving movie for 180 s and then a static (nonmoving) movie for 60 s. By treating this 240-s task (except for the pretest) as one trial, two trials (2D and 3D movie viewing) were performed in a random sequence.

Full size image

For the subjective measurements, the feeling of presence and VIMS symptoms were measured. The assessment of the presence in this study used words that precisely represented the presence because presence is a relatively unfamiliar construct to most nonexperts [18]. We expected that the viewing of the global motion movie would provide the viewer with the illusion of self-motion. Considering that presence is also a sensation caused by the motion characteristics in the movie, the degree of sensation of self-motion can serve as an assessment of the presence, which was limited to the viewing of the motion movie. Thus, in this study, the subjects orally reported their feeling of body sway every 30 s using a visual analog scale (VAS) that ranged from 0 to 100 after the sensory scale was explained (0: quiet stance to 100: rollover).

Next, for the VIMS symptom assessment, the subjects completed the SSQ, which has been used in a number of previous studies, before and after each motion movie viewing.

2.3 Analysis

The COP measurements were sampled at 100 Hz using the basic setting of the Wii Balance Board. In addition, a clinical test of the body sway was performed using a stabilometer at 20 Hz. The COP data were downsampled at 20 Hz and low-pass filtered at 10 Hz to adapt to the clinical standard.

The continuous COP data were separated by intervals of 60 s of viewing time to analyze each time segment. We reported the position of the COP moves in synchrony with the phase of the motion movie when the subjects watched movies with low-frequency global motion [19]. Thus, to evaluate the synchronization accuracy of the movie viewing, each separate data unit underwent a frequency analysis using a fast Fourier transform with a Hamming window. Moreover, the total locus length (TLL), area and the standard deviation of the COP data for the motion direction of the movie (SDz), which are general indexes of the body sway, in each separate data unit were calculated. Then, to examine the relationships of each of the body-sway indexes and the VAS value of the presence, the Pearson product–moment correlation coefficients were calculated for all time segment results.

For the SSQ, the total score and three subscores (oculomotor discomfort, disorientation, and nausea) under each condition were calculated (see [13] for the SSQ calculation methods). Then, the Pearson product–moment correlation coefficients were similarly calculated.

3 Result

The temporal changes in the indexes of the body sway and the VAS value of the feeling of presence while viewing each movie were analyzed, as shown in Fig. 3. The VAS value and all body-sway indexes showed similar results, and the VAS value and the amplitude component at 0.25 Hz (Amp. at 0.25 Hz) obtained from the frequency analysis also exhibited similar results. First, the increases in the motion movie viewing time increased all body-sway indexes, including the amplitude component obtained at 0.25 Hz from the frequency analysis and the VAS value. Second, when the motion movie viewing was stopped, all the index values decreased. Third, these tendencies increased when viewing the 3D movie compared with that when viewing the 2D movie.

Fig. 3.
figure 3

Temporal changes in the body-sway indexes and the subjective measurements that configured the VAS of the presence. The solid line in each graph represents the average value, and the error bars represent the standard error.

Full size image

Table 1(a) and (b) list the correlation coefficients between the body-sway indexes and the VAS value of the presence, including all time segment results. All body-sway indexes were strong and were significantly positively correlated (0.76 > r > 0.98, P < 0.01). In addition, difference in the correlations was not found in the viewing of the movie type. However, the relationship between the VAS value and the body-sway indexes differed from the relationships between each body-sway index. For the 3D movie viewing, the relationships between the VAS value and the body-sway indexes were significantly and positively correlated. On the other hand, for the 2D movie viewing, the correlation coefficient between the VAS value and the body-sway indexes show weak and positive correlations compared with that when viewing the 3D movie.

Table 1. Pearson product–moment correlation coefficients of the relationship between the body-sway indexes and the VAS value of the presence. (a) 3D movie viewing. (b) 2D movie viewing
Full size table

For the correlations between the subjective evaluation of VIMS (difference between the SSQ scores after and before viewing) and the objective evaluation (difference between the body-sway index value calculated in the 120- to 180-s time segment to that in the pretest), no significant correlation was found under each viewing condition listed in Table 2.

Table 2. Pearson product–moment correlation coefficients of the relationships between the difference in the postscore and prescore SSQ scores and the difference in the body-sway index values between the 120- and 180-s time segment and that in the pretest.
Full size table

4 General Discussion

With regard to the body-sway index during the motion movie viewing, the area, TLL, standard deviation of the movie motion direction (indexes of instability), and amplitude component at 0.25 Hz (calculated with the frequency analysis as an index of synchronization acuity to the phase of the motion of the movie) were calculated. The results of this study agreed with those of our previous study [19] in which the subjects watched sinusoidal reciprocating motion movies at 0.3 Hz. Both studies reported the same trends: the instability (increased index values) increased with the increases in the viewing time, and the stability (decreased index value) increased when the motion in the movie stopped Moreover, the instability while viewing the 3D movie was higher than that while viewing the 2D movie.

The relationships of the VAS value of the presence and the body-sway indexes indicated different correlation coefficients depending on the movie types. When the 3D motion movie was viewed, this relationship was significantly and positively correlated (0.61 < r < 0.67; P < 0.01). In contrast, the correlation coefficients during the 2D movie viewing were consistently lower than that while viewing the 3D movie (0.43 < r < 0.45; P < 0.01). Ideally, it is preferable that all the correlation coefficients are the same regardless of the movie types. However, the results of this study showed that the correlations between the subjective and objective evaluations were reduced under conditions of low instability for the body sway and low synchronization accuracy. This trend were also observed in our previous study [16].

With regard to the symptoms attributed to VIMS during the movie viewing, the subjects evaluated VIMS through the SSQ, and the body sway was measured as an objective assessment. SSQ is currently considered as the gold standard for the subjective evaluation of VIMS. For instance, Solimini [20] and Naqvi et al. [21] investigated the effects of stereoscopic viewing on VIMS using the SSQ.

For the correlations between the subjective evaluation of VIMS (difference between the SSQ score after and before viewing the movie) and the objective evaluation (difference between the body-sway index value calculated in the 120- to 180-s time segment and that in the pretest), no significant correlation was found in both 3D and 2D movie viewing (Table 2). The results in this study suggested that the relationships between each SSQ score and the body-sway index were small. Therefore, the relationships between the degree of symptoms induced by VIMS and the body-sway indexes were also small. These results were similar to our previous study results [16]. We have reported that VEPR is a response to correct the conflict conditions between the visual and equilibrium senses [22]. Moreover, we also reported that humans who are prone to motion sickness based on the sensory conflict theory are prone to body sway [22]. Thus, these findings suggested that we can only treat the occurrence of VEPRs as an indicator of the occurrence of VIMS. However, the VEPRs cannot directly represent the degree of subjective symptoms because the VEPRs are simply a conflict correction response.

5 Conclusion

We have verified the degree of feeling, which is represented by VEPRs, by simultaneous measurement of both feeling of presence, symptom of VIMS, and body sway while viewing 3D and 2D movies. The following results were demonstrated:

  1. 1.

    Significantly positive correlations between the body-sway indexes and the VAS value of the presence were recognized. However, when the subject watched the 2D movie, the correlations between the subjective and objective evaluation decreased.

  2. 2.

    No significantly positive/negative correlation between the body-sway indexes and the SSQ scores was found. Therefore, performing objective assessments of the degree of subjective symptoms induced by VIMS by measuring the body sway was difficult.