1 Introduction
Walking is one of the most basic and essential actions for immersive experiences in virtual reality (VR). However, the real environment where one walks in VR is limited compared with the virtual environment of an enormous space. Therefore, studying techniques of moving freely in an immense-sized virtual space in a finite-size real space is important. Omnidirectional treadmills [
62] and walking-in-place [
50] have been studied for this purpose. However, providing accurate kinesthetic, proprioceptive, and vestibular sensations using these techniques is challenging because they do not follow the natural human walking motions [
49]. In this context, redirected walking (RDW) is designed to support moving in a larger virtual space in a narrow physical space using natural human gait motions [
54].
RDW is a technology that changes the user’s path by mismatching the user’s movement in real space with that in virtual space [
54]. It assists users to walk seamlessly in a virtual space by changing their path to prevent collisions with obstacles and walls with visual manipulations. For example, RDW technology can create an arc shape of the user’s path in the real world through visual manipulation that continuously rotates the map to avoid obstacles when a user enters a virtual environment. The user can freely navigate the infinite virtual space because these RDW processes are provided in such a manner that the user is oblivious to them. However, when the user’s direction must be changed to a broader angle to efficiently avoid obstacles, visual manipulations must be increased to a level so that the user might perceive these manipulations; thus, RDW reaches the limit of visual–vestibular inconsistency [
71]. Modulated visual information is inconsistent with the user’s self-acceptance and vestibular senses. If this inconsistency increases, the user notices the manipulation; the immersion and presence decrease, and simulation sickness increases [
1,
9]. Therefore, a limitation in the RDW technique’s ability for manipulation is called the detection threshold (DT) [
71]. The limitations are manipulating the user’s path and the range of the expandable space [
4,
33,
34,
35].
This study introduces a method to expand DT by alleviating visual–vestibular inconsistency, a significant cause of DT, through various vestibular stimuli that have minimally been studied. We expanded the DT using electrical (i.e., noisy galvanic vestibular stimulation (noisy GVS) and directional galvanic vestibular stimulation (directional GVS)) and non-electrical stimulations (i.e., bone-conduction vibration (BCV) and caloric vestibular stimulation (CVS)) (Figure
1). Furthermore, we verified the vestibular stimulation RDW in a game environment that is similar to the user’s actual VR experience. Most RDW studies used only a simple DT measurement environment [
5,
40,
71]; however, this environment differs from the content that users can experience. We implemented a game in which users played using their visual and auditory senses to locate objects. We investigated whether the user is affected by vestibular stimulation RDW in the following aspects: immersion and presence, task performance, simulator sickness, and discomfort.
In previous studies, noise was added to the vestibular organ using galvanic current [
40] or inducing vestibular stimulation in a specific direction using a directional current [
28,
64] to expand DT using vestibular stimulation. These studies expanded DT using a simple device. In addition, vestibular stimulation has the advantage that it can be used independently of the content compared to auditory or haptic manipulation [
23,
41,
43] methods [
40]. Among the various vestibular stimulation methods, those using galvanic currents have been mostly used although vestibular stimulation has been proven to be a promising technique to expand DT. Previous studies were limited in that they did not suggest an alternative or solution to the side effects of electrical stimulation of the skin (e.g., skin stimulation, retina stimulation, and eye muscle stimulation) [
37,
76]. Research on non-electrical alternatives for people prone to these side effects is required for application to a broader range of users. To the best of our knowledge, this is the first study to explore the possibility of using non-electrical stimulations: BCV and caloric stimulation in RDW as an alternative to the side effects of these electrical stimulations. GVS, BCV, and caloric stimulation stimulate vestibular organs in different ways to modulate vestibular information. We confirmed that this stimulation method significantly affects DT expansion performance and user usability. This study shows that providing a non-electrical vestibular stimulation RDW technique for people suffering from side effects of electrical stimulation is possible.
We also investigated gait stability to confirm the safety of RDW using vestibular stimulation. The phenomenon in which the user’s gait stability is lowered in the RDW situation has been proven through experiments [
27,
29,
46], which can be a significant risk for the safety of users blindfolded with a head mounted display (HMD). We measured gait instability using foot pressure data to determine the effects of various vestibular stimulations on gait stability in the RDW. The first extensive comparison of GVS, BCV, and CVS in this study is expected to contribute to the non-electrical use of RDW with vestibular stimulation.
We fabricated a wearable device that provides a set of vestibular stimulations, including noisy GVS, directional GVS, BCV, and CVS. We conducted experiments in two environments to compare the modalities using the manufactured device. The first environment investigated quantitative factors to determine whether each vestibular stimulus can be used with the RDW technique, such as the DT expansion performance of each vestibular stimulus, degree of simulator sickness induced, gait stability, and stimulus discomfort. In the second environment, each vestibular stimulus was applied to a simple game environment to investigate user experience using the vestibular stimulus RDW in an actual game. We investigated immersion and presence, task performance, simulator sickness, and stimulus discomfort over time in a simple ball game environment. The research questions of this study are as follows:
•
How far can BCV and CVS extend the DT through vestibular stimulation instead of electrical stimulation?
•
How do different vestibular systems affect gait stability and user experience?
•
Which stimulation method guarantees immersion in-game environment?
4 Experiment 1. Quantitative RDW Performance Verification of GVS, BCV, and CVS (E1)
This study investigated the possibility of using four vestibular stimuli in RDW: noisy GVS, directional GVS, BCV, and CVS. The following were hypothesized:
•
H1: Non-electrical stimulations (BCV and CVS) can extend the curvature DT instead of electrical stimulation.
•
H2: Direction-based stimulation (directional GVS) makes the user walking more stable than noise-based stimulations (noisy GVS, BCV and CVS).
•
H3: Participants believe that non-electrical stimulations (BCV and CVS) are more uncomfortable than electrical stimulations (noisy GVS and directional GVS), and this interferes with the user’s immersion and experience in the game environment.
H1 was set based on the MLE model of multisensory integration described in Section 2.3. When visual and vestibular information are conflicting, vestibular noise caused by BCV and CVS lowers the weight in the multisensory integration of vestibular information. Therefore, the visual–vestibular inconsistencies can be alleviated because the conflict is resolved owing to the relative predominance of visual information; thus, the curvature DT can be expanded.
H2 was set based on previous studies on gait stability presented in Section 2.5. Noise-based stimulations caused unstable gait because they restricted the use of vestibular information in gait in previous studies [
77]. Because direction-based stimulation provides vestibular information modified to suit visual information, unlike noise-based stimulations, it will enable relatively more stable walking.
H3 was set based on the vestibular stimulation method of non-electrical stimulations. Non-electrical stimulations stimulate the vestibular systems by using physical stimuli, such as vibrations or cold fluids. These physical stimuli can interfere with the user’s immersion and experience. The experiment to verify the hypothesis was conducted in two stages. All experiments in this study were approved by the Institutional Review Board (IRB 20220628-HR-67-18-04).
In the first experiment (E1), quantitative factors, such as DT expansion performance, induced simulator sickness, gait stability, and stimulus discomfort, were tested when each vestibular stimulus was applied to the RDW. Each factor was measured by changing the curvature gain to 0, ±
π/180, ±
π/90, ±
π/60, and ±
π/45. The experimental environment was created based on the DT-measurement environment proposed by Steinicke et al [
70] (Figure
9). In the second experiment (E2), a simple ball-search game environment with RDW was built. E2 was conducted on how users feel about RDW using vestibular stimulation in a realistic game environment, such as immersion and presence, task performance, simulator sickness, and stimulus discomfort over time. Based on the DT measured through E1, RDW was applied using the steer-to-center algorithm, which continuously redirected the user toward the center of the room [
54].
Device wear and personalization. In all experiments and cases, participants wore the device in the same way so that the experimental results were not affected by the fit of the stimulation device. Before the experiment, all vestibular stimuli were applied once to each participant for personalization of stimulations. Two GVS stimuli were applied to the participant; if the participant felt extremely uncomfortable, the experiment was immediately terminated. In the case of BCV, the 15-step vibration personalization method introduced in Section 3.2 was used to determine the vibration intensity suitable for each participant. Finally, CVS stimulation was applied at a flow rate of 5 L/min. If the participant felt highly uncomfortable, stimulation was applied at a flow rate of 4 L/min. Before all experiments, the wearing condition of the participants was checked. If there was interference between devices (particularly in the electrodes of the GVS and the BCV vibrator), the position of the interfering stimulation part was modified such that the stimulation for the experimental case could be received well. The details of each experiment are as follows
4.1 Experiment Setup
In E1, we measured DT, gain stability, simulation sickness, and discomfort when vestibular stimulation was used for the RDW. The experimental environment was configured based on the DT-measurement environment proposed by Steinicke et al [
71]. A detailed description of the experimental environment is presented in this section. In E1, each participant experienced four types of vestibular stimulation (noisy GVS, directional GVS, BCV, and CVS), and a control condition in which no stimulation was applied for a total of five cases. To exclude unexpected results from wearing the equipment, the equipment was worn under all conditions. We randomized the experimental sequence of all stimulation cases using Latin squire. The participant experienced nine different curvature gains (±
π/180, ±
π/90, ±
π/60, ±
π/45, and 0) in the experimental environment. Each curvature gain was randomly applied 45 times (five times per case). The total number of participants recruited in E1 was 20 (
N=20, age range: 20-–28,
M = 23.05,
SD = 2.33). No participant had a history of epilepsy, vestibular system disorders, migraine, brain injury, cardiovascular disorders, central-nervous-system abnormalities, or sensitive skin. Pregnant women and people with pacemakers were excluded from this study. Moreover, no participant had motion-sickness symptoms for 3D games, and there were no problems with normal walking while wearing the HMD.
Questionnaire. We used the 2-AFC questionnaire used by Steinicke et al. to measure DT in this experiment [
71]. Participants answered left or right to “Is the physical path bent left or right?” on the in-game VR per trial. We used the simulator sickness questionnaire (SSQ) to measure the simulation sickness once for each stimulus [
30]. Furthermore, simulator sickness was measured once in the initial state before all the experiments. To measure the user’s discomfort in the game, we used Fernandes and Feiner’s discomfort score measurement questionnaire [
18]. The participant answered, “On a scale of 0–-10, zero being how you felt coming in, ten is that you want to stop, where are you now?” for discomfort with the VR controller.
Procedure. The participants changed shoes with pressure sensors to measure gait stability and answer the SSQ. After completing the questionnaire, the participants wore the stimulation device and calibrated the strength of the BCV. When the experiment started after taking sufficient rest after calibration, the participants started from the green disk shown in Figure
9 (a) and moved forward along the green path. All curvature gains were simultaneously applied after a straight path of 1.5 m to prevent the participants from noticing the gain. The participants walked along the path where a curvature gain of 5 m was applied (a total of 6.5 m walking) and stopped when the questionnaire screen appeared. The questionnaire comprised the 2-AFC questionnaire to measure DT, and Fernandes and Feiner’s discomfort score to measure discomfort. After the survey, the participants returned to the initial point indicated by the green disk shown in Figure
9 (a) and repeated the same task. When one case was completed, the participants removed the HMD, answered the SSQ for the case, and freely wrote about whether or not the stimulus was uncomfortable and the reason for discomfort. After the questionnaire, the participants were provided sufficient rest before conducting the experiment on the next stimulus case.
4.2 Result of E1
4.2.1 Curvature DT of each vestibular stimulus.
The curvature DT-measurement results of each vestibular stimulus were analyzed using the 2-AFC questionnaire. The results are as follows: Figure
10 shows the measured probability of the participant’s left response at nine different curvature gains (±
π/180, ±
π/90, ±
π/60, and ±
π/45, 0). Each graph is marked with a standard error. Each measured value was fitted using a sigmoid function [
71], and the calculated DT values are listed in table
1. The pooled DT region was the largest in directional GVS and expanded in the order of BCV, CVS, and noisy GVS. Furthermore, the pooled DT area between the lower and upper DT was the smallest in the control condition and expanded by 6.42%, 26.61%, 20.18%, and 18.35% in the noisy GVS condition, directional GVS condition, BCV condition, and CVS condition, respectively.
The above results confirm that the user did not notice the direction change when the directional GVS stimulus was applied. Furthermore, the user’s movement could be significantly changed. Directional GVS stimulation was the most advantageous in terms of spatial expansion or obstacle avoidance among all the conditions. In contrast, the noisy GVS stimulus exhibited the most negligible DT expansion, which was slightly different from the experimental results of Matsumoto et al. (12%–16.4%) because of the difference in the magnitude of the voltage used. However, additional verification is required (Matsumoto et al. used 100 V, whereas this study used 15 V).
4.2.2 Gait stability - Anterior/Posterior (A/P) gait stability.
The results of A/P gait instability measured using plantar pressure data are as follows (Figure
11). Under all conditions, the absolute values of skewness and kurtosis did not exceed 3.0 and 10.0, respectively; thus, normality is satisfied [
32]. To analyze the change in A/P gait instability according to the type of vestibular stimulation, a one-way analysis of variance (ANOVA) was used. The analysis showed a significant difference in A/P gait instability for the five different vestibular stimuli. F(4, 895) = 8.747, and
p < 0.001.
Post-hoc comparisons were performed using Bonferroni correction for five different vestibular stimuli. The A/P gait in the control condition was significantly stable than that in the noisy GVS and BCV conditions. Furthermore, the A/P gait of the directional GVS was significantly stable compared to that of the noisy GVS and BCV, similar to the control condition. Although CVS was not statistically significant, A/P gait instability was higher than that in the control and directional GVS, and gait instability was lower than that of noisy GVS and BCV.
4.2.3 Gait stability - Medio/Lateral (M/L) gait stability.
The results of the M/L gait instability measured using plantar pressure data are as follows (Figure
12). Normality was satisfied because the absolute values of skewness and kurtosis of all conditions did not exceed 3.0 and 10.0, respectively. One-way ANOVA was used to examine the effect of vestibular stimulation type on the M/L instability. The M/L instability for the five different vestibular stimuli showed a significant change (F (4, 895) = 12.139,
p < 0.001).
The Bonferroni correction was used for post-hoc comparison of vestibular stimulation. The results for M/L gait instability were similar to those for A/P gait instability. In the control and directional GVS conditions, the M/L gait was significantly stable compared to those of the noisy GVS, BCV, and CVS conditions. Although not statistically significant, BCV had the most unstable M/L gait among GVS, BCV, and CVS. Additionally, although insignificant, directional GVS was the most stable during M/L walking. Thus, the noisy GVS, BCV, and CVS, which add noise to the vestibular information, had higher gait instability in the M/L direction than that of the directional GVS, which provides vestibular information suitable for visual information.
4.2.4 Simulator sickness.
The total severity scores measured using the SSQ are as follows (Figure
13). The simulator sickness was measured once in the initial state before the experiment and once for each of the five conditions. For all the conditions, the absolute values of skewness and kurtosis did not exceed 3.0 and 10.0, respectively; therefore, normality was satisfied. We performed one-way ANOVA to determine the effect of each vestibular stimulus on simulator sickness. However, the simulator sickness for the five experimental conditions and one pre-question was not statistically significant, F(5, 114) = 3.371,
p = 0.007, but
\(\eta ^2_p\) = 0.129, indicating moderate significance.
Post-hoc comparisons of each condition were analyzed using the Bonferroni correction. The analysis confirmed that the simulator sickness significantly increased in the CVS condition compared to the pre-question and control conditions. While no significant difference was found in other conditions, directional GVS showed the least simulator sickness among the vestibular stimulations. Thus, the simulator sickness felt by the participant under the vestibular stimulation condition, except for CVS, showed a statistically insignificant change compared to the pre-condition and control conditions.
4.2.5 Discomfort.
The results of the discomfort of each stimulus collected during the game are as follows (Figure
14). The questionnaire was designed to assess current discomfort on a scale of 0–10. The skewness and kurtosis of the data collected from all stimuli satisfied normality considering their absolute values did not exceed 3.0 and 10.0, respectively. We performed a one-way ANOVA to analyze whether each type of vestibular stimulation affected discomfort. The discomfort results according to the four types of stimuli showed significant differences: (F (3, 716) = 16.564,
p < 0.001).
The effect of each vestibular stimulus on discomfort was evaluated by post-hoc comparison using the Bonferroni correction. The analysis showed that the noisy GVS was the most comfortable, and the discomfort was 50% lower than for all other vestibular stimuli. Although directional GVS, BCV, and CVS did not show a statistically significant difference, on average, users were most uncomfortable with BCV and least uncomfortable with directional GVS. Therefore, among the four vestibular stimuli, users felt uncomfortable with noisy GVS and other vestibular stimuli, except for noisy GVS.
4.3 Summary of E1
Through the E1 experiment, we confirmed the DT expansion performance, gait stability, simulator sickness, and discomfort felt by the users for each stimulus when each vestibular stimulus was used in the RDW. First, noisy GVS had the lowest user discomfort. However, the DT-expansion performance of noisy GVS was the worst; therefore, expecting a significant effect in space expansion or obstacle avoidance using RDW is difficult. In addition, noisy GVS has high gait instability, which may cause safety problems for the users. In contrast, directional GVS has higher user discomfort than noisy GVS. However, the DT-expansion performance was the best; therefore, using RDW to perform well in space expansion and obstacle avoidance can be expected. Furthermore, directional GVS has the lowest gait instability; hence, it can support a user in stably and safely walking in a virtual environment. Next, BCV has the advantage that it can be used as a non-electrical alternative for people who cannot use GVS, which is an electrical stimulus. A promising RDW performance can be expected because the DT-expansion performance of BCV is also high. However, similar to other methods of providing noise to the vestibular system, there is a high gait instability, which may cause safety problems for the user and has a disadvantage of high discomfort felt by the user. Finally, similar to BCV, CVS can be used as a non-electrical alternative to GVS, an electrical stimulation, but it induces significantly higher simulator sickness compared to other conditions. In addition, good usability and safety cannot be assured using CVS because of high discomfort and gait instability. We analyzed these shortcomings of CVS based on a survey of participants, which will be dealt with in the Discussion section. Based on the E1 results, the answers to H1 and H2 were obtained. For H1, we confirmed our hypothesis as BCV and CVS extended the curvature DT by 20.18% and 18.35%, respectively. Moreover, for H2, our hypothesis was confirmed because direction-based stimulation produced a more stable gait than noisy GVS, BCV in A/P gait stability, and all noise-based stimulations in M/L gait stability.
5 Experiment 2. Usability Evaluation in Realistic Game Scenario of RDW System Using GVS, BCV, and CVS (E2)
Immersion and presence, task performance, simulator sickness, and discomfort were investigated over time when a user played a game to which RDW was applied using each vestibular stimulus. Based on the results of E1, the range of RDW curvature gain in a ball search game (Figure
15) environment of E2 was set. In E2, an experiment was designed to test the user’s game experience according to the characteristics of each stimulus by unifying the range-of-curvature gains for all cases. The range-of-curvature gain used in this experiment is the average value of the DT obtained through E1, [ −
π/52.360,
π/62.086].
5.1 Experimental Setup
In this experiment, the experimental environment was constructed based on the environment used in the live-user experiment of RDW by Sun et al. and Hodgson et al [
26,
72]. Each study used a game of finding a stick or a ball. Thus, the user could freely move around as much as possible in an environment where RDW was applied. Previous studies used fog or adjusted the transparency of the ball to encourage the user’s free movement. In the present study, the same method as that of Sun et al. was used to control the transparency of the ball [
72]. The transparency of the ball decreases in proportion to the distance between the user and the ball. Therefore, in a place far away from the ball, the ball is transparent, and the user has to randomly search the map to find the ball, moving around the map in various ways. In addition, we made the ball make a sound so that the user could enjoy the game using various modalities. The ball provides a stereo sound according to the location, and the user can search the ball by inferring the location where the sound is heard. The user walked freely in a virtual space of 5 m X 5 m in total and searched for a ball for three minutes per trial. The remaining time and the number of balls found are displayed on the UI at the bottom of the participant, encouraging participants to actively participate in the game. The total number of balls found in each condition was recorded as task performance and was used to indicate how well the participant performed the task required in the game. Participants experimented with each of the four vestibular stimuli and one non-stimulus control condition, and all trials were randomized to a Latin square. A total of 27 participants were recruited in E2 (
N = 20, age range 20–36,
M = 23.89,
SD = 3.17). As in E1, no participant had history of epilepsy, vestibular system-related migraine, brain injury, cardiovascular disorder, central nervous system abnormality, or sensitive skin. Pregnant women and pacemaker wearers were excluded from recruitment. No participant had motion-sickness symptoms related to 3D games and VR, and there was no problem with walking for a long time while wearing an HMD.
Questionnaire. In this experiment, the following questionnaire was used to measure the immersion and presence, simulator sickness, and discomfort over time. For immersion and presence, the Igroup Presence Questionnaire (IPQ) was used, and three sub-scales (spatial presence, involvement, realism) were used [
57]. Spatial presence measures the sense of being physically in the virtual environment. Involvement indicates the attention devoted to the VE and the involvement experienced. Experienced realism indicates how much the participant’s experience in the virtual environment appear like the real one. Simulator sickness was measured using SSQ. IPQ and SSQ were measured once after the experiment for each condition. Moreover, SSQ was measured once before all experiments to measure the initial state. We used Fernandes and Feiner’s discomfort score measurement questionnaire every 30 seconds in-game to measure discomfort over time. The participant submitted an answer to "On a scale of 0–10, zero being how you felt coming in, ten is that you want to stop, where are you now?" about their discomfort while finding the ball. The timer for the remaining time during the questionnaire was stopped so the participant could think enough about the questionnaire.
Procedure. Participants first answered the simulator sickness in the initial state through the SSQ questionnaire. After the participant wore the vestibular stimulation device, the BCV strength was calibrated. After providing sufficient rest time after calibration, the participant stood in the middle of the room and waited for the experimental environment to start. After the experiment started, the participant explored the virtual environment using sight and sound to find as many balls as possible within the 3 min time limit. When a discomfort questionnaire appeared every 30 s during navigation, the participant stopped, answered the questionnaire, and then immediately resumed the game. After the time limit, the participant took off the HMD, answered questions, such as IPQ and SSQ, and had sufficient rest time. After sufficient rest, the participants proceeded with the experiment under the following conditions.
5.2 Result of E2
5.2.1 Immersion and Presence.
Immersion and presence were measured on three subscales (spatial presence, involvement, and realism). The measurement results are shown in Figure
16. All the conditions satisfied normality because absolute skewness and kurtosis values did not exceed 3.0, and 10.0, respectively. A one-way ANOVA was performed to determine whether each vestibular stimulus affected immersion and presence. The resulting spatial presence (F(4, 130) = 0.266,
p = 0.899), involvement (F(4, 130) = 0.133,
p = 0.970), realism (F(4, 130) = 0.985,
p = 0.418), and the total score (F(4, 130) = 0.278,
p = 0.892) were not significantly different. These results indicated that the imposition of vestibular stimulation on users does not impair immersion or presence. Therefore, in terms of immersion and presence, vestibular stimulation RDW is a system that does not harm the user’s experience.
5.2.2 Task performance.
The following are the results of task performance depending on the type of vestibular stimulation. Participants in this experiment had to find as many balls as possible within the 3 min time limit, and task performance was measured using the total number of balls found in each trial. The results are shown in Figure
17. In all cases, normality was confirmed by the absolute values of skewness and kurtosis. The ANOVA of the task-performance results for the five stimulus conditions was F(4, 115) = 6.918,
p < 0.001, confirming statistical significance. As a result of post-hoc analysis using the Bonferroni correction, no statistical significance was found in any condition except CVS. However, the CVS had a significantly poorer task performance than the other stimulation conditions. Thus, we confirmed that vestibular stimulation, except for CVS, did not affect the user’s ability to achieve the game goal using sight and hearing. However, CVS significantly hindered users from achieving the goal of the game, and the reason for this could be confirmed through the survey. Over 90 % of the users answered that the CVS wind noise interfered with the auditory experience of the game. Additionally, other opinions included difficulty concentrating and dizziness. Although BCV had auditory noise, there was no significant effect on the task performance compared to CVS noise. It is presumed that the noise of the CVS is wind noise and is distributed over the entire frequency range; however, the noise of the BCV is a single frequency of 500 Hz. Therefore, it was assumed that the auditory noise of the BCV did not interfere with the sound of the ball.
5.2.3 Simulator sickness.
The following are the total severity scores felt by users in the actual game to which the RDW using each vestibular stimulus measured by the SSQ was applied. The simulator sickness was measured under six conditions. The absolute values of skewness and kurtosis did not exceed 3.0 and 10.0, respectively, in all conditions, thereby satisfying normality. The results are shown in Figure
18. The effect of each vestibular stimulus on simulator sickness was investigated using one-way ANOVA. As a result of the analysis, simulator sickness for the six conditions was F(5, 146) = 4.493,
p < 0.001, which was statistically significant.
The Bonferroni correction was used for post-hoc comparison of all conditions. The analysis confirmed that BCV caused significantly higher simulator sickness than the pre-question and control conditions. Although not statistically significant under other conditions, the noisy GVS showed an average low simulator sickness compared to other vestibular stimuli. Therefore, each vestibular stimulus, except BCV, showed a statistically insignificant increase in the simulator sickness in the virtual game environment compared to the pre-question and control conditions. Among them, the increase in noisy GVS was the smallest. Thus, we could confirm that noisy GVS and BCV caused the least and most simulator sickness, in the E2 environment, respectively.
5.2.4 Discomfort.
For Fernandes and Feiner’s discomfort score measurement questionnaire, the participants responded in-game with a score of 0–-10 for their discomfort every 30 s. We performed a one-way ANOVA on six measurements for each vestibular stimulus to investigate whether there was a difference in the discomfort score with time for each vestibular stimulus. In all vestibular-stimulation conditions, no significant change in discomfort was observed with time, and the results were as follows: discomfort score over time with noisy GVS, F(5, 156) = 0.131. p = 0.985; directional GVS: F(5, 154) = 0.241, p = 0.944; BCV: F(5, 156) = 0.026, p = 1.000; CVS: F(5, 156) = 0.067, p = 0.997;
Therefore, we conducted a one-way ANOVA of the mean discomfort to investigate the differences in discomfort for each vestibular stimulus. The results are shown in Figure
19. For all conditions, normality was confirmed using skewness and kurtosis. The mean discomfort score for the four vestibular stimuli was F(3, 642) = 31.831,
p < 0.001, which indicated a significant difference. We used the Bonferroni correction for post-hoc analysis. BCV showed significantly higher discomfort than all other stimuli. The directional GVS and CVS were lower than the BCV but induced higher discomfort than the noisy GVS. Finally, the noisy GVS significantly induced the lowest discomfort. Therefore, we confirmed that the noisy GVS and BCV were most comfortable and uncomfortable in the E2 environment, respectively. Most participants answered that the reason for the discomfort of BCV was vibration and the accompanying sound. They reported that the vibration of the BCV was itchy or painful and that a constant sound of 500 Hz was highly uncomfortable.
5.3 Summary of E2
We used RDW with vestibular stimulation in a game scenario similar to the environment experienced by real VR users in the E2 experiment, distinct from the E1 experiment. We implemented a game in which users find objects using their visual and auditory senses to assume a situation in which users experience actual VR content. We investigated users’ immersion and presence, task performance, simulator sickness, and discomfort with vestibular stimulation RDW in an actual game environment. We confirmed that vestibular stimulation did not impair users’ sense of immersion.
First, the noisy GVS showed excellent task performance and reduced simulator sickness and discomfort. Next, the directional GVS showed a statistically similar value to the noisy GVS in terms of task performance and simulator sickness; however, user discomfort was significantly higher than in the noisy GVS. Next, BCV induced the most simulator sickness and discomfort in the game environment. We confirmed that the vibration and sound of the BCV caused significant discomfort to the users. However, BCV did not interfere with the user’s achievement of the game’s goal, nor did it interfere with the sense of immersion. Therefore, improving the vibration and noise of the BCV is expected to reduce BCV discomfort. Finally, although CVS performed poorly on the task, it was comparable to directional GVS in terms of simulator sickness and discomfort. However, characteristic wind noise of the CVS prevents the user from achieving the game’s goal using hearing. It will be possible to use the CVS as an immersive RDW system if the wind noise of the CVS can be effectively reduced.
H3 was concluded from the results of E2. Non-electrical stimulations indicated relatively high discomfort and motion sickness. However, the vibration stimulation of BCV did not cause low task performance, and BCV and CVS did not diminish users’ sense of immersion. Directional GVS induced a relatively high sense of motion sickness and discomfort among the electrical stimulations; therefore, our hypothesis H3 was false. This is analyzed in detail in the Discussion section.
6 Discussion
In this section, we divide the vestibular stimuli into groups and analyze new aspects based on the characteristics of each group, unlike the above sections, which compared various aspects of all vestibular stimuli. The vestibular stimuli were classified according to two criteria. First, we divided vestibular stimulation into electrical stimulation (noisy GVS, directional GVS) and non-electrical stimulation (BCV, CVS). Second, we divided vestibular stimulation into direction-based (directional GVS) and noise-based stimulation (noisy GVS, BCV, and CVS). The following are the conclusions that we arrived at for each group.
6.1 Electrical versus non-electrical stimulations
We first compared electrical stimulation (noisy GVS, directional GVS) with non-electrical stimulation (BCV, CVS), and the most significant difference between the two groups was the side effects of electrical stimulation of the skin (e.g., skin, retina stimulation, eye muscle stimulation) [
37,
76]. Non-electrical stimulation is a technique that can be considered for multiple users who are prone to side effects when electrical stimulation is applied to their skin [
12]. Non-electrical stimulation uses vibration and cold air instead of electrical stimulation, which the participants primarily perceive as auditory noise. The BCV applies a vibration of 500 Hz within the human audible frequency. The CVS blows wind into the external auditory meatus, causing the vibration of the eardrum. Therefore, the participant experienced noise from all non-electrical stimulations, which significantly affected discomfort.
Figure
20 shows the discomfort scores for each vestibular stimulus in E1 and E2. The graphs demonstrate the difference in discomfort between E1 and E2. We observed that discomfort in the E2 condition was higher than that in the E1 condition for all non-electrical stimulations. While additional analysis is required to determine whether this is significantly attributable to the sample difference in the two conditions, the data collected in each condition represent the entire population because statistically they satisfy normality [
45], and the results still have informative implications in terms of showing the difference in user experience according to the environment. We analyzed this cause as auditory noise produced by the BCV and CVS. E2 was designed to use both sight and hearing to set up a more general VR gaming environment compared with E1. Therefore, users must continuously use what they hear to enjoy the game in E2. In this situation, non-electrical stimulations continuously provide noise to users, which induces audio interferences. Humans exhibit a high cognitive load when auditory elements with multiple pieces of information are simultaneously reproduced [
78], and they experience a decrease in concentration and comfort [
39,
79]. These audio interferences in the E2 environment of this experiment increased the cognitive load and caused discomfort. This can be verified by the user’s answer to the reason for discomfort. The users felt “tingling” as discomfort caused by electrical stimulation, which was mostly the same for E1 and E2. However, the answers regarding the discomfort due to non-electrical stimulations were different in E1 and E2. Constant sound in E1 and other non-electrical stimulations, such as the vibration of BCV, were the primary contributors to discomfort. However, in E2, most participants answered that the sound was aurally uncomfortable while enjoying the game. This indicates interference between the auditory elements of the game and the noise induced by non-electrical stimulations, and the cognitive load induced by this interference caused the discomfort.
Audio interferences could be used to investigate why non-electrical stimulations cause relatively high discomfort at E2. However, we observed higher discomfort in the directional GVS in E2 although it was an electrical stimulation. This is because of the difference between direction- and noise-based stimulations, as described in the next section.
6.2 Direction-based versus noise-based stimulations
In this study, we alleviated the visual–vestibular inconsistencies in two ways: 1) Direction-based stimulations (directional GVS) provide vestibular information consistent with visual information, and 2) noise-based stimulations (noisy GVS, BCV, CVS) add noise to the vestibular information. The two methods exhibited a difference in gait stability, as discussed in Sections 4.2.2–4.2.3. Noise-based stimulations reduce the reliability of vestibular information, thereby alleviating inconsistency by reducing the weight of decision-making. This property restricts the use of vestibular information in gait, and, similar to previous studies, it was possible to confirm the result of inducing an unstable gait. In this section, we focus on how the presence or absence of directionality affects the discomfort and simulator sickness experienced by users in E1 and E2.
First, we analyzed discomfort. As can be observed in Figure
20, the discomfort to directional GVS, which is a direction-based stimulation, was higher in E2 than in E1. However, the reason for the discomfort answered by the user was the same, “tingling,” in both E1 and E2. Therefore, we interpreted this increase in discomfort as an increase in the tingling frequency. The participant experienced a constant curvature gain during one trial in E1. Therefore, the direction and amount of visual manipulation felt by the user during each trial were constant. By contrast, E2 constantly changed the amount and direction of the curvature gain to redirect the user to the center of the room. Therefore, the user experienced considerable changes in the direction and intensity of visual manipulation during one trial, and the change in the direction of current also occurred frequently following visual manipulation. Furthermore, the current direction changed more frequently because, in E2, the user had to turn their head in various directions to find the ball compared to the game in E1. Therefore, the current direction in E2 changed more frequently than in E1, and the user would experience more frequent tingling because of the frequently changing current direction.
Next, we analyzed the changes in simulator sickness at E1 and E2 with direction- and noise-based stimulations. As can be seen in the Figure
21, all noise-based stimulations decreased simulator sickness in E2 compared with E1, but the direction-based stimulations were almost the same. We believe that E2 had lower simulator sickness than E1 in most situations because of the difference in the time over which users experienced visual–vestibular inconsistency in RDW. The average time with RDW was 10–20 min for E1 and 3–4 min for E2. Therefore, the participants were less exposed to visual–vestibular inconsistency in E2 and showed relatively low simulator sickness compared to E1. However, direction-based stimulation induced relatively high simulator sickness at E2. We interpreted this to be because of frequent switching of the curvature gain at E2. There is a disadvantage in that it is challenging to determine how much current must be utilized to offer accurate vestibular information consistent with visual information [
80]. A slight difference between the vestibular information modulated by directional GVS and visual information can significantly affect a participant’s performance and comfort [
37]. Although the direction of the curvature gain is the same as that of the vestibular information applied by the directional GVS, it may cause sensory inconsistency because the intensity changes continuously in E2. Therefore, direction-based stimulation is more susceptible to simulator sickness than noise-based stimulation. However, the alleviation of simulator sickness can be expected if the intensity of direction-based stimulations can be changed to match the curvature gain.
7 Conclusion & Future works
We implemented an RDW system using four vestibular stimuli: noisy GVS, directional GVS, BCV, and CVS, which successfully resolved visual–vestibular inconsistency. The RDW system was tested in two environments, and all the systems successfully verified the possibility of spatial expansion and obstacle avoidance performance without impairing immersion and presence. Furthermore, we could show the advantages and disadvantages of each type of stimulation (electrical versus non-electrical stimulation) and the modulation method of vestibular information (direction-based versus noise-based stimulation).
We succeeded in increasing the curvature DT by an average of 19.27% using non-electrical stimulations, demonstrating that non-electrical stimulations can be used as an alternative for people suffering from the side-effects of electrical stimulations. Although non-electrical stimulation can be used as an alternative for people experiencing side effects to electrical stimulation, it could destabilize gait; thus, it is not a completely safe alternative as it can increase the user’s risk of falling. In contrast, electrical stimulations had significant advantages in the game environments; they did not generate acoustic noise; hence, they were free from the audio interference and induced relatively low discomfort. Moreover, the results indicated that the acoustic noise of non-electrical stimulations could cause an audio interference and induce discomfort in the gaming environment. In particular, the acoustic noise of CVS directly interferes with the auditory elements of the game.
Direction-based stimulations were found to enable stable gait compared to noise-based stimulation. Stable walking can protect a user from a dangerous situation where the user falls or loses the center of gravity. However, direction-based stimulations resulted in high discomfort and simulator sickness in a game environment where the curvature gain frequently changed. In contrast to direction-based stimulations, noise-based stimulation induced relatively low discomfort and simulator sickness in the game environment. We analyzed the cause of the frequent changes in the direction of stimulation and the difficulty in providing vestibular stimulation with the correct intensity.
The findings of this study provide a foundation for creating bespoke stimulation techniques tailored to the needs of both the user and the situational context. For instance, implementing BCV stimulation may be possible for individuals who experience adverse effects from electrical stimulation but can stably walk or to utilize simultaneous direction-based and noise-based stimulation to achieve a high DT without incurring user discomfort when visual manipulation is frequently changing. This study specifically focuses on examining the efficacy and user experience of various types of vestibular stimulations in RDW and does not delve into the combinations of such stimulation techniques. However, future research has the potential to address the limitations of certain stimulation methods, such as the risk of falls or discomfort, by combining different stimulation techniques.
In this study, we confirmed that the four vestibular stimulation RDW systems could successfully support users’ seamless walking in VR by checking DT expansion, gait stability, induced simulator sickness, immersion and presence, discomfort, and task performance. We preemptively verified the DT for the curvature gain of several vestibular stimuli. However, further studies could investigate vestibular stimulation in other RDW techniques, such as rotation and translation gain and using an attractor/distractor. In future studies, we intend to use our vestibular stimulation RDW system in more diverse environments. RDW technology can be used not only in games but also in numerous other VR scenarios. Inspired by this research, we intend to study users’ experience of our RDW system in various VR environments. The environmental comparison in this work was between different participant groups; therefore, we will conduct research in various environments targeting the same participant groups in a follow-up study. A deeper understanding of vestibular stimulation RDW technology can be achieved by examining the effects of vestibular-stimulation RDW technology on users with more rigorous measurements and more diverse environments. In addition, we plan to conduct a follow-up study on our system’s effect on users in various environments and interactions based on previous studies on the expansion of DT due to game elements and distraction in various environments [
6,
48,
67]. In addition, to accurately measure the user’s fall risk in various situations, we plan to develop an algorithm that can distinguish whether the user’s gait instability is caused by each stimulation activated for RDW or is naturally induced by the user’s interaction with the media in the application while walking. In addition, it is necessary to verify the performance of spatial expansion by quantifying how much each stimulation extends the perceived virtual space.
Finally, we will improve the way vestibular stimulation is applied. Noisy GVS could overcome the low DT scalability through voltage and current optimizations, whereas directional GVS could reduce user discomfort and simulated sickness by synchronizing stimulus intensity and curvature gain in RDW. BCV and CVS can make the RDW experience more comfortable and safer by reducing the acoustic noise (e.g., using cold metal for CVS). Reducing the discomfort of each vestibular stimulation would be an essential improvement and key step toward realizing the application of stimulation to various RDW techniques. Currently, our vestibular stimulation systems apply stimulation, which is sometimes uncomfortable, to the user following the modulation of visual information. Unlike RDW technology, where visual modulation occurs continuously (as in this study), vestibular stimulation can be a significant addition to overt redirection techniques (e.g., rotation gains with interventions and freeze-and-turn resetting).