Error Monitoring and Correction Related to the Sense of Presence in Virtual Environments

The sense of presence is an important aspect of interaction with virtual environments. It refers to the subjective illusion of the “realness” of virtual reality [1, 2]. Achieving a high level of presence is an important goal in the development of virtual reality applications as it may promote the effectiveness of user interaction in virtual environments. Presence is a multidimensional construct determined by a host of factors [3]. Quite obviously, presence is strongly influenced by technological factors like frame rate and immersion level. However, presence is a psychological phenomenon and today’s research focus shifts to the study of the psychological determinants of presence [4]. These includes demographic, cognitive and personality variables. In this study, we assess the influence of error monitoring and correction functions on the sense of presence in virtual environments.

Error monitoring and correction is a subsystem within human cognitive control [5]. Cognitive control is a set of meta-cognitive process responsible for the tuning of specialized cognitive processes towards achieving a specific goal in a given context [6]. Within cognitive control, error monitoring and correction is responsible for the detection of conflicts and for the subsequent re-configuration of cognitive processes with the aim of reducing the conflict. A conflict is conceptualized as the deviation of the expected cognitive activity from the actual cognitive activity. In the famous Stroop paradigm, conflict is produced in incongruent trials when the font color does not match the meaning of the color word. In this case, the error monitoring subsystem detects the conflict and activates cognitive control system which inhibits the inappropriate response and reduces conflict. An error (faulty actions) is a form of conflict as it clearly implies a difference between the expected and the observed. Errors are therefore detected and corrected by the same system which is employed for the control of conflicts [5].

Error monitoring and correction may be involved in the emergence of presence as virtual reality exposure clearly has strong conflict generating potential. Virtual reality is based on an illusion and presents the user with artificially constructed perceptual inputs which substitute for the “real” reality. Simulator sickness observed in many cases is an obvious consequence of the conflict between the virtual sensory input and the sensory-motoric expectancies of the users habituated to the real environment. On a more abstract level, error monitoring signals to the user that the virtual environment is not real even if the perceptual evidence strongly advocates for the contrary. Thus, error monitoring and correction is an important component in the process of presence formation. This study aims to research the relationship between error monitoring and correction and subjective feeling of presence. To this end, we assess error monitoring and correction functions in a sample of VR users and their feeling of presence during interaction with a virtual environment.

Error (conflict) monitoring and correction is a subsystem within cognitive control – a set of meta-cognitive processes responsible for the fine-tuning of regular cognitive processes aimed at achieving of specific goals in a given context [6]. The error monitoring and correction subsystem is responsible for the detection of errors committed by the human and for the subsequent reorganization of regular cognitive processes in order to increase the quality of cognitive activity [5]). Neuroscientific data clearly associates error monitoring and correction with a part of medial frontal cortex – the anterior cingulate cortex (ACC). This brain structure is thought to be responsible for the detection of conflicts – the difference between the expected and the observed – with errors being a particularly strong special case of conflict. According to the dominating “control loop” hypothesis [5, 7], error monitoring in the ACC detects conflicts and errors and activates prefrontal cortex associated with cognitive control which in turn modulates the workings of regular cognitive processes.

2.1 Post-error Slowing

The workings of error monitoring and correction is reflected in two behavioral effects: post-error slowing and conflict adaptation. Post-error slowing [8] refers to the increase of reaction time in the correct probe which follows an erroneous probe. This increase in reaction time may be quite substantial (30–50% of average reaction time) and it dissipates gradually over several subsequent probes. This increase in the reaction time is usually interpreted as reflecting the intervention of the error monitoring and correction system as a reaction to the perceived error. This interpretation is supported by the observation that post-error slowing is reduced via the increase of the inter-trial interval. Thus, the size of post-error slowing reflects the efficiency of error monitoring and – more importantly – error correction.

2.2 Conflict Adaptation

Conflict adaptation is another behavioral effect of error monitoring and correction. Conflict adaptation refers to the decrease of conflict interference in incongruent probes if the previous probe also was an incongruent one [9]. This effect means that if the error (conflict) monitoring and resolution system was already activated for the resolution of the conflict in the previous probe, it may more efficiently resolve the conflict in the current probe. Conflict adaptation is observed in several cognitive tasks where the performance in conflict inducing incongruent probes is contrasted with the performance in no-conflict congruent probes. On the whole, conflict adaptation is interpreted as indicative of strategically operating conflict resolution associated with the workings of error monitoring and correction subsystem of cognitive control.

Below, we study how error monitoring and correction is related to the sense of presence by correlating aspects of presence with indices of post-error slowing computed for an error-inducing task (the letter-digit switching task) and conflict adaptation computed for a conflict-inducing cognitive task (the Eriksen flanker task).

3.1 Subjects

39 students of Moscow State University, 20 females, aged 18–23, participated in the study in exchange for a course credit.

3.2 Virtual Scenario

Subjects had to explore a virtual space (20 × 20 m) and to traverse a series of randomly placed digits in correct numerical order. The presentation medium was either a full-immersion CAVE system or a standard 19-inch computer display. The CAVE system has four large flat screens (Barco ISpace 4), which were connected into one cube consisting of three walls and a floor. The length of each screen side was about 2.5 × 2.2 m. Shutter eye glasses were made by Volfoni. Projection system was based on BarcoReality 909. The projector’s matrix resolution was 1280 × 1024 with 100 Hz update frequency. Tracking system produced by ArtTrack2, include Flystick, Fingertracking, Motion Capture. Graphic Cluster for 5 PC was based Quadro FX 5800. 3Dvia Virtools was used for software developing. There was a training session (with digits from 1 to 5) and two experimental session (with digits from 1 to 9). The total exposure to the virtual scenario was about 10 min.

3.3 Presence Inventory

Aspects of presence were assessed via a Russian version of the ITC-SOPI inventory. ITC-SOPI is a cross-media presence inventory with four dimensions. These are Spatial Presence, Naturalness of Virtual Scenario, Emotional Involvement, and Negative Effects [3].

3.4 Post-error Slowing

Post error slowing was computed for a demanding cognitive task involving task switching (the letter-digit task). The task involved the random presentation of a letter-digit pair with the task being either an odd/even judgement task or a vowel/consonant judgement task. The task was specified by a color cue aligned with stimulus onset. The subjects had thus to switch between tasks on approximately the half of the trials. Post-error slowing was computed individually as the ratio between average RT on correct trials following an incorrect trial to average RT in correct trials following a correct trial. Higher values of this variable are indicative of effective error monitoring and correction.

3.5 Conflict Adaptation

Conflict adaptation was computed for a conflict inducing selective attention task (the Eriksen flanker task). The task required the subjects to identify the direction of an arrow surrounded by four arrows (two arrows on each side). The flanker arrows could be either congruent or incongruent to the central arrow but should in any case be ignored by the subject. Conflict adaptation was computed as the ratio of average RT in incongruent trials following a congruent trial to average RT in incongruent trials following an incongruent trial. Lower values of this variable are indicative of effective conflict monitoring and resolution.

3.6 Procedure

The subjects first performed the cognitive tasks within a computerized battery in a group session. Virtual scenario was presented individually with the order of CAVE and display presentations counterbalanced across participants. After the presentation of the virtual scenario in each medium, the subjects completed the presence inventory.

Pearson correlation coefficients were computed between poste-error slowing and conflict adaptation and dimensions of presence for both presentation mediums (Table 1).

For the full-immersion CAVE environment two significant correlations were found. First, post-error slowing was significantly inversely related to negative somatic effects. This indicates that efficient error monitoring and correction leads to less experience of negative vestibular symptoms which may arise as a consequence of exposure to virtual reality. A result like this has a simple explanation as negative effects arise from sensory-motor errors during virtual exposure and effective error correction may be an obvious remedy for this. Second, conflict adaptation was significantly and inversely related to perceived virtual scenario naturalness. This indicates that more effective conflict resolution is related to more scenario naturalness. Such a result may be related to the fact that scenario naturalness may be compromised by scenario’s inconsistencies and violations of user’s expectations which are conflict-inducing experiences. Effective conflict suppression my counteract this, leading to more presence.

For the low-immersion display environment three significant correlations were observed. Post-error slowing was inversely related to negative effects, and conflict adaptation was inversely related to virtual scenario naturalness. These results are exact replicas of that obtained for the high-immersions CAVE environment indicating a sufficient generalizability of the positive relationship between presence and efficient error monitoring and correction. Additionally, post-error slowing is inversely related to emotional involvement. This means less presence with effective error monitoring and correction indicating that it may have different effects on the sense of presence at least in low-immersion environments. Specifically, effective error monitoring is related to a stronger critical stance towards virtual scenario which may undermine the illusion of “realness” and prevent emotionally comforting interaction with the virtual environment.

This study was aimed at exploring the role of error (conflict) monitoring and correction meta-cognitive system in the emergence of the sense of presence during exposure to a virtual environment. It was found that error monitoring and correction is indeed related to the subjective sense of presence.

First, post-error slowing was inversely related to the emergence of negative somatic effects which reflect sensory-motor discoordination typical of virtual environments. Effects like nausea emerge in virtual environment because visual input is to some extent misaligned with proprioceptive input. This clearly produces a sort of error, and it is not surprising that effective error correction (as indicated by post-error slowing) may be involved in effectively combating this error. Effective error control is thus a remedy for negative effects produced by virtual environment. The absence of negative effects promotes presence by inducing more subjective comfort and preventing negative symptoms from distracting attention from the interaction with the virtual environment. Thus, effective error monitoring and correction has the potential to increase presence. This effect was found both in high- and low-immersion environments which suggest generality of this result.

Second, conflict adaptation was consistently found to be related to virtual scenario naturalness in both high- and low-immersion scenarios. The naturalness factor of the ITC-SOPI refers to the extent to which the virtual scenario is understandable, predictable, and free of contradictions. If a virtual environment is considered as “natural” is allows for the construction of a mental model on which interactions with the virtual environment and interpretations of virtual stimuli and events may be based. The importance of mental models in the emergence of presence has been long acknowledged in presence research [10, 11]. The construction of a coherent mental model for a virtual environment may be hindered by the inconsistences of the virtual scenario and its high “artificiality”. These inconsistences may be effectively suppressed by an efficient mechanism of conflict resolution which may be the reason why empirical correlations between conflict adaptation and scenario naturalness are observed. For instance, a specific mechanism of the link between efficient conflict suppression and scenario naturalness may be the increased ability to “suspend disbelief” (which is a strong determinant of presence, [12]) in people with increased conflict processing abilities. Thus effective conflict monitoring and control may advance presence via better conflict resolution, corroborating the results observed for the negative somatic effects.

Third, it was found that in low-immersion conditions post-error slowing may be inversely related to the emotional involvement dimension of presence. We see this as an indication that strongly developed error detection may have differential effects on presence. While – as seen from the discussion above – effective error control may suppress errors and conflicts induced by a virtual environment – it may also undermine presence due to increased sensitivity to conflicts/errors. Specifically, effective error monitoring and control may be related to critical thinking. The ability to think rationally would clearly suggest the user of a virtual environment that the displayed reality is only an illusion. This would be especially strong if inadequacies of the virtual environment are clearly perceptible. Sensitivity to conflicts may thus promote a more critical stance towards virtual environments and thus diminish presence. This is especially true of low-immersion virtual environments where the sophistication of the virtual environment is severely limited due to technological factors.

The results obtained above have consequences for the development and usage of virtual environments. These include:

• User selection for better cognitive control

• Understanding of presence mechanisms related to cognitive control

• Maintaining appropriate level of cognitive control

• Objectively measuring presence

User selection procedures may include error monitoring and correction assessment in order to select users more capable of developing a strong sense of presence in a virtual environment. Given the above result, selecting potential users based on post-error slowing data may lead to a decrease of negative somatic symptoms experienced during VR exposure. This is especially relevant for dynamic virtual scenarios with a lot of simulated motion. User selection for better conflict adaptation may be especially relevant for optimizing presence in highly artificial virtual scenarios in which it is important to suspend disbelief and to suppress conflicts.

Understanding the mechanics of the relationship between error monitoring and correction and presence may also help to develop more sophisticated virtual environments. For instance, our results make a strong case for the reduction or error and conflicts produced by virtual environments. This refers, for instance, to the conflicts between visual and proprioceptive sensory inputs (via better visual stimulation or appropriate proprioceptive feedback). On a more conceptual level, this refers to making virtual scenarios as coherent as possible avoiding the need for conflict suppression.

The individual level of error monitoring and correction is the product of complex long-term development and cannot be directly manipulated. Still, some factors may influence the efficiency of error monitoring and correction. For instance, cognitive control has been shown to depend on cognitive resources [13] which may be depleted in the state of fatigue or illness. Exposure to virtual environments in a less than optimal functional state may lead to less effective cognitive control, less ability to compensate for errors and conflicts, and less presence. Thus, it is advisable to refrain from employing user in non-optimal functional states as they will not be able to exert full cognitive control over their interactions with a virtual environment.

Finally, the present results may be used to drive the development of new objective measures for presence. Specifically, error monitoring and correction is related to the activity of the ACC. It is now widely believed that ACC activity has a distinct electrophysiological index – the fronto-medial theta rhythm [14]. Thus, measuring bursts of fronto-medial theta during virtual exposure may be used to assess aspects of presence.

Error monitoring and correction is a subsystem within human cognitive control. Better errors/conflicts processing was related to less negative somatic effects during virtual reality exposure and more naturalness of a virtual scenario both in high- and low-immersion virtual settings. In a low-immersion environment, error processing was also related to less emotional involvement which reflects a critical stance towards virtual stimulation. The explication of cognitive mechanisms of presence can be used to better recruit users and to better develop virtual reality applications. This work was supported by Russian Foundation for Basic Research, grant no. 15-06-08998.