KR102024794B1 - Apparatus and method for experiencing fire of main control room - Google Patents

Abstract

The present invention relates to a fire situation experience, and more particularly, to an apparatus and method for experiencing a fire situation in a main control room of a nuclear power plant, comprising: an output unit for outputting virtual reality content according to a selected experience scenario, and at least one of When the sensing unit for detecting a bio-signal and the virtual reality content is output, the user's immersion degree to the virtual reality content is measured based on the detected bio-signal, and the user's attention is based on the measured immersion degree. And a control unit for generating an additional event for venting.

Description

Apparatus and method for simulation of fire in the main control room {APPARATUS AND METHOD FOR EXPERIENCING FIRE OF MAIN CONTROL ROOM}

The present invention relates to a fire situation experience, and more particularly, to an apparatus and a method for experiencing a fire situation in a main control room of a nuclear power plant.

Since a nuclear power plant includes a main facility such as a nuclear reactor, it corresponds to a building having a fireproof structure, and thus, a main facility such as a main control room may be located in a fireproof designed facility. However, since the occurrence of fire cannot be predicted, the fireproof structure and the fireproof design alone cannot provide sufficient protection against the fire. Accordingly, the operation of the nuclear power plant in the nuclear power plant requires normal operation of the nuclear power plant under such a fire situation.

Thus, in order for a driver to operate a nuclear power plant normally even in a fire situation such as a main control room, the driver's experience training for such a fire situation is necessary. However, due to the nature of the disaster situation of fire, it is very dangerous to experience it, and there is a problem that requires a large cost, such as the need to make a similarly formed experience site to experience a more realistic fire situation.

On the other hand, various measures are emerging to solve the risk and cost of fire experience. One of them is the use of recently developed virtual reality technology. An example of such a virtual reality technology may include a head mounted display (HMD) that can be worn on the head. In this case, the HMD provides a user with a virtual 360 degree image of a specific situation. Can be simulated as interacting with the actual surroundings or environment.

In addition, such a virtual reality technology can provide a user with a higher sense of reality using a variety of devices, for example, Wearable device, 3D Display, 4D dynamic chair.

However, the virtual reality up to now is mainly applied to the realistic element using the optical illusion effect of the user, so it is mainly applied to the virtual game or the virtual training. Therefore, most of the technologies that deal with the sensory elements among the domestic and international patents are realistic technologies for games.

On the other hand, even when using such a virtual reality device (for example, HMD), due to differences in the characteristics or constitution of each operator, the degree of realism felt by each operator may be different. Accordingly, even in the same fire experience training, the degree of training and skill in handling a fire situation of each operator may be different. In addition, this difference has a problem that can cause great damage due to inadequate operation of the main control room due to the immature training of a specific operator in the actual fire situation.

It is an object of the present invention to solve the above-mentioned problems and other problems, and to provide a simulation apparatus and method for allowing an operator to experience a fire situation in a nuclear power plant main control room in advance using virtual reality.

In addition, another object of the present invention, the operator who experiences a fire situation using the virtual reality, to measure the degree of immersion in the virtual reality and to improve the immersion degree of the operator when the measured immersion level is less than a predetermined level. It is to provide a simulation apparatus and method that can be.

According to an aspect of the present invention to achieve the above or another object, the experience simulation apparatus according to an embodiment of the present invention, the output unit for outputting the virtual reality content according to the selected experience scenario, and at least one bio-signal from the user And a sensing unit for detecting a virtual reality content, when the virtual reality content is output, measuring the immersion degree of the user with respect to the virtual reality content based on the detected biosignal, and drawing the user's attention based on the measured immersion degree. It characterized in that it comprises a control unit for generating an additional event for.

In an embodiment, the biosignal is at least one of an electrocardiogram and an electroencephalogram of a user, and the controller measures a change amount of the biosignal measured at a predetermined time and compares the change amount measured at a predetermined time with each other. Based on the calculated change rate, the immersion degree of the user with respect to the virtual reality content is measured.

In one embodiment, the control unit, based on at least one change rate of the change rate calculated from each of the electrocardiogram and the electroencephalogram is less than a predetermined time for a predetermined time is output the content through the additional event It is characterized by determining.

The virtual reality content may include visual content and auditory content, and the controller may be configured to determine whether the rate of change for which the state is maintained for less than a predetermined time for a predetermined time is one of the ECG rate and the EEG rate. According to whether the rate of change in the electrocardiogram and the rate of change in the electroencephalogram, one of the visual content and the auditory content, or the visual content and the auditory content are all output through the additional event.

In one embodiment, the control unit, if the at least one of the visual content and audio content is further output through the additional event to re-measure the user's immersion, and further generates the additional event according to the re-measurement result, When the additional event is further generated, the content having the increased output level is output through the further generated additional event.

According to an embodiment, the virtual reality content may be virtual reality content for experiencing a fire situation, and the visual content may include a flame image or an explosion image having a different size and brightness depending on an output level. The content may include explosion sounds or warning sounds having different volumes depending on the output level.

According to an aspect of the present invention to achieve the above or another object, the experience simulation method according to an embodiment of the present invention, the step of outputting the virtual reality content according to the preset experience scenario based on the user's input, and the user, Detecting at least one biosignal from the user, measuring a user's commitment to the virtual reality content based on the detected at least one biosignal, and based on the measured commitment Generating an additional event to ventilate.

In one embodiment, the biosignal is at least one of an electrocardiogram and an electroencephalogram of the user, and the measuring of the immersion degree of the user may include measuring a change amount of the biosignal measured at a predetermined time, and at every predetermined time. It is characterized in that the step of measuring the user's commitment to the virtual reality content on the basis of the change rate calculated as a result of comparing the measured change amount with each other.

In one embodiment, the step of measuring the user's immersion degree, based on at least one change rate of the change rate calculated from each of the electrocardiogram and electroencephalogram is maintained for a predetermined time less than a predetermined time And determining the content output through the event.

According to an embodiment, the virtual reality content includes visual content and audio content, and the content output through the additional event has a change rate in which a state of being maintained for a predetermined time for a predetermined time is less than the ECG change rate and the EEG. According to any one of the rate of change or the rate of change in the electrocardiogram and the rate of change in the electroencephalogram, any one of the visual content and the auditory content or both the visual content and the auditory content.

In an embodiment, the generating of the additional event may further include outputting at least one of the visual content and the audio content through the generated additional event, re-measuring a user's immersion degree, and The method may further include generating the additional event according to the re-measurement result, wherein generating the additional event further includes increasing an output level of the content to be output.

According to an embodiment, the virtual reality content may be virtual reality content for experiencing a fire situation, and the visual content may include a flame image or an explosion image having a different size and brightness depending on an output level. The content may include explosion sounds or warning sounds having different volumes depending on the output level.

Referring to the effects of the main control room fire experience apparatus and the experience method according to the present invention.

According to at least one of the embodiments of the present invention, the present invention allows the operator to experience in advance the main control room situation of the fire state by using virtual reality, so that the operator can normally operate the main control room without embarrassment even in the actual fire situation. It has the advantage of being able to drive.

In addition, according to at least one of the embodiments of the present invention, the present invention by measuring the virtual reality immersion of the operator having a fire experience using the virtual reality, by causing the additional event is generated based on the measured immersion degree, The advantage is that the operator's virtual reality immersion can be maintained above a certain level.

1A is a block diagram illustrating a structure of a fire experience simulation apparatus according to an embodiment of the present invention.
1B is an exemplary diagram illustrating an HMD as an example of a fire experience simulation apparatus according to an embodiment of the present invention.
FIG. 1C is an exemplary diagram illustrating an example of biosignal sensors for measuring virtual reality immersion of an HMD wearer in an HMD according to an embodiment of the present invention.
2 is a flowchart illustrating an operation process of performing a fire experience simulation by the fire experience simulation apparatus according to an exemplary embodiment of the present invention.
FIG. 3 is a flowchart illustrating a process of generating an additional event for improving immersion of an operator during the operation of FIG. 2.

Hereinafter, exemplary embodiments disclosed herein will be described in detail with reference to the accompanying drawings, and the same or similar components will be given the same or similar reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used in consideration of ease of specification, and do not have distinct meanings or roles from each other. In addition, in describing the embodiments disclosed herein, when it is determined that the detailed description of the related known technology may obscure the gist of the embodiments disclosed herein, the detailed description thereof will be omitted. In addition, the accompanying drawings are intended to facilitate understanding of the embodiments disclosed herein, but are not limited to the technical spirit disclosed herein by the accompanying drawings, all changes included in the spirit and scope of the present invention. It should be understood to include equivalents and substitutes.

First, FIG. 1A is a block diagram illustrating a structure of a fire experience simulation apparatus according to an embodiment of the present invention. Meanwhile, the fire experience simulation apparatus is a device capable of outputting a virtual reality image and an audio signal according to a selected fire scenario, and may be a device such as an HMD, a smart glass, or a smart lens worn by a user. However, in the following description, for the convenience of description, the fire experience simulation apparatus according to the embodiment of the present invention will be described as an example of HMD. However, it is a matter of course that the present invention is not limited thereto.

Referring to Figure 1a, the fire experience simulation apparatus according to an embodiment of the present invention, that is, the HMD 100 is a wireless communication unit 110, power supply unit 120, input unit 130, sensing unit 140, output unit ( 150, the memory 160, the controller 170, and the like. The components shown in FIG. 1A are not essential to implementing the HMD 100 in accordance with an embodiment of the present invention, such that the HMD 100 described herein is more or less than the components listed above. It may have components.

More specifically, among the above components, the wireless communication unit 110 may be one or more modules that enable wireless communication between the HMD 100 and various peripheral devices, for example, a preset controller device or another HMD or an external server. It may include. It may also include one or more modules that connect the HMD 100 to one or more networks.

The power supply unit 120 receives power from an external power source and an internal power source under the control of the controller 170 to supply power to each component included in the HMD 100. The power supply unit 120 may include a battery, and the battery may be a built-in battery or a replaceable battery.

On the other hand, the input unit 130 is for inputting image information (or signal), audio information (or signal), data, or information input from a user, and a microphone 132 for inputting an audio signal or information from a user. It may include a user input unit (for example, a touch key, a mechanical key, etc.) for receiving the input. In this case, the user input unit may be formed in a separate device that can be connected to the HMD 100, that is, a controller, in order to receive a user input to the HMD 100. In this case, various inputs sensed through the controller may be applied as a user input to the HMD 100.

Meanwhile, the user's input applied through the controller may vary. For example, the user's input may be applied from at least one key included in the controller. Alternatively, various gestures of the user sensed through the controller may be applied as the user's input. To this end, the controller may separately include one or more sensors such as an acceleration sensor or a gyroscope to identify various gestures that the user takes while holding the controller.

Meanwhile, the combination of the above-described specific key input and the specific gesture may be applied as the input of the specific user. For example, the controller 170 may be configured according to the user's wrist snap while the user is inputting a specific key provided in the controller or when a preset drag input such as a swipe input is applied. When the controller is moved in a specific direction more than a predetermined angle, it may be determined that the user's input is a user's preset input related to the function of the HMD 100. That is, when a gesture of the user is detected, the controller 170 may respond to the gesture to display a menu relating to various functions of the HMD 100, for example, playback of content for a fire experience scenario, of the HMD 100. It may be displayed on the display unit 152.

In addition, the input unit 130 may further include a camera unit 120 to input image information. The camera unit 120 may include at least one camera. The camera unit 120 may process image information obtained by the image sensor as an input for another application program or another function. The processed image information may be displayed on the display unit 152 of the HMD 100 or stored in the memory 170. Therefore, the gesture of the user sensed through the camera unit 120 may be input as a user input.

On the other hand, the sensing unit 140 may include one or more sensors for detecting the head movement of the user wearing the HMD (100). For example, the sensing unit 140 may include an acceleration sensor 141 and a gyro sensor 142. Here, the acceleration sensor 141 and the gyro sensor 142 may detect the acceleration and the angular velocity according to the movement of the head of the user.

In addition, the sensing unit 140 may further include an eyetracking sensor 146. For example, the eye tracking sensor 146 reads the position of the pupil center from the pupil image of the user, and based on the read pupil center position and the user's eye modeling information, the eye tracking sensor 146 detects a position in the virtual reality image corresponding to the user's gaze. The point can be recognized. In addition, information about the recognized point may be transmitted to the controller 170.

In addition, the sensing unit 140 may include at least one biosignal sensor 148. Here, the biosignal sensor 148 may be used to measure how immersed the user is in the virtual reality content that is currently provided through the HMD 100. For example, the biosignal sensor 148 may be an electrocardiogram (ECG) sensor or an electroencephalogram (EEG) sensor for measuring an electrocardiogram of a user. In this case, the control unit 170 may calculate the ECG change rate or the EEG change rate of the user as a result of the measurement of the biosignal sensor 148, and measure the current state of immersion of the current user based on at least one of the calculated ECG change rate or the EEG change rate. can do.

The output unit 150 is used to generate an output related to visual, auditory or tactile senses, and may include a display unit 152 and a sound output unit 154. The display unit 152 may be installed at a position corresponding to both eyes of the user when the user wears the HMD 100 so as to provide a larger sized image to the user. In addition, the sound output unit 154 is in the form of a headphone (headphone) that can be in close contact with both ears of the user when the user wears the HMD 100 so that the sound signal related to the content for the fire experience scenario to be reproduced can be delivered. It can be formed as.

In addition, the output unit 150 may include an olfactory information output unit 156, and the olfactory information output unit 156 may generate a bullet (for example, a phenol odor) such as an odor of smoke generated in an actual fire. Can be. For example, the olfactory information output unit 156 may be formed to include a storage unit for storing the fragrance or fragrance water, which may generate the smell of smoke or coal, etc., and the degree of opening of the storage unit under control of the controller 170. By controlling the intensity of the smoke smell or bullet can be different from each other.

In addition, the output unit 150 may include a haptic output unit 158. The haptic output unit 158 generates a vibration or heat related to the fire experience scenario content that is currently playing to the user, thereby allowing the user to experience the fire situation more realistically.

In addition, the memory 160 stores data supporting various functions of the HMD 100. The memory 160 may store a plurality of application programs (applications) or applications that are driven by the HMD 100, data for operating the HMD 100, and instructions. At least some of these applications may be downloaded from an external server via wireless communication. In addition, at least some of these applications may be used for the basic functions of the HMD 100 (for example, the reproduction of the virtual reality content and the output of video signals and sound signals of the reproduced virtual reality content, etc.) from the time of shipment from the HMD (100). ) May be present. The application program may be stored in the memory 160 and installed on the HMD 100 to be driven by the controller 170 to perform an operation (or function) of the HMD 100.

On the other hand, the controller 170 typically controls the overall operation of the HMD 100 in addition to the operation associated with the specific application program. The controller 170 may provide or process information or a function appropriate to a user by processing signals, data, information, and the like input or output through the above-described components or driving an application program stored in the memory 170.

In addition, the controller 170 may control at least some of the components described with reference to FIG. 1 to drive an application program stored in the memory 170. In addition, the controller 170 may operate by combining at least two or more of the components included in the HMD 100 to drive the application program.

1B is an exemplary view showing an example of the HMD 100 according to an embodiment of the present invention.

Referring to FIG. 1B, FIG. 1B illustrates an example of a user wearing an HMD according to an embodiment of the present invention. As shown in FIG. 1B, when the user wears the HMD 100, the controller 170 based on the detection values of the sensors provided in the sensing unit 140, the head of the user wearing the HMD 100. The direction 102 facing the front side can be identified. When the virtual reality content is reproduced, an image of one region corresponding to the identified front direction 102 may be displayed on the display unit 152 from the virtual space image of the reproduced virtual reality content.

The reproduced virtual reality content may be based on a scenario for experiencing a fire situation and may be stored in the HMD 100 or provided from an external server connected to the HMD 100. When the virtual reality content is provided from an external server, the external server may provide the same virtual reality content to a plurality of HMDs, and the same virtual reality content may be reproduced from the plurality of HMDs. .

FIG. 1C is an exemplary diagram illustrating an example of biosignal sensors for measuring virtual reality immersion of an HMD wearer in an HMD according to an embodiment of the present invention.

Referring to FIG. 1C, the HMD 100 according to the embodiment of the present invention may be formed to include a plurality of electrocardiogram sensors or brain wave sensors at a plurality of points in close contact with the head of the user when worn. Therefore, when the user wears the HMD 100, the EKG and the EEG of the user may be measured at the same time as the user wears the HMD 100.

Meanwhile, the electrocardiogram and the electroencephalogram may be simultaneously measured. The rate of change may be calculated every time interval or whenever a specific auditory event or time event occurs. The immersion degree of the user with respect to the currently provided virtual reality content may be measured based on the calculated change rate.

For example, the controller 170 of the HMD 100 according to an exemplary embodiment of the present invention may determine that the user is in an unstable state or in a tense state when the measured change rate of the electrocardiogram or the electroencephalogram is a predetermined level or more. Then, the controller 170 may determine that the user is immersed in the content currently being provided.

On the other hand, if the measured rate of change in the electrocardiogram or electroencephalogram is less than a predetermined level, the controller 170 may determine that the user is in a stable state. As such, when it is determined that the user is in a stable state, the controller 170 may determine that the user is not sufficiently immersed in the currently provided virtual reality. This is because the virtual reality currently being provided is for the main control room of the nuclear power plant in which the fire situation occurred, which may be a testimony that the user is not immersed in the fire situation.

On the other hand, if it is determined that the user is not immersed in the virtual reality of the fire situation, the control unit 170 may generate an event that can call the user's attention without affecting the progress of the currently selected fire scenario. Can be. For example, the controller 170 additionally outputs a relatively loud sound signal such as a fire alarm sound or an explosion sound (adds an acoustic event), or additionally outputs a visual image related to flame or smoke to a panel of the main control room ( By creating additional visual events, the user can focus on the virtual reality that is currently being provided.

In addition, the output level of the additional event provided may vary according to the measurement result of the user's commitment. Therefore, the user may be more immersed in the presently provided virtual reality through additional events occurring according to the degree of being immersed in the virtual reality, and thus fire experience training may be performed more efficiently.

Hereinafter, embodiments related to a control method that may be implemented in the fire experience simulation apparatus 100 configured as described above will be described with reference to the accompanying drawings. It is apparent to those skilled in the art that the present invention can be embodied in other specific forms without departing from the spirit and essential features of the present invention.

2 is a flowchart illustrating an operation process in which the fire experience simulation apparatus 100 according to an embodiment of the present invention performs a fire experience simulation.

Referring to FIG. 2, the fire experience simulation apparatus 100 according to an embodiment of the present invention may output virtual reality content for simulating a virtual fire situation according to a fire experience scenario selected by a user (S200).

The virtual reality content may include visual content and audio content for simulating the virtual fire situation. In addition, the virtual reality content may include olfactory content and tactile content.

The visual content may be an image displayed through the display unit 152. For example, the visual content may include a disaster situation of a currently selected experience scenario, that is, a fire situation and an image related to a foreground of a specific place. That is, if the currently selected fire experience scenario is a 'fire situation' occurring in the main control room of the nuclear power plant, the visual content may include a plurality of control panels (eg, a monitoring control panel panel) corresponding to the main control room view of the nuclear power plant. Image). The image may include images of a flame or smoke for simulating the 'fire situation', and an image for visually displaying an alarm state, such as an alarm or a warning for visually displaying a fire alarm.

On the other hand, the image of the flame may be displayed differently according to the intensity, that is, the output level according to the brightness and size, and an image such as a warning light or an alarm may be displayed immediately after the display of the flame image.

On the other hand, the smoke image may be displayed after a predetermined time elapses after the flame is generated, that is, after the flame image is displayed on the foreground image of the main control room. The transparency of the smoke image may be adjusted according to the intensity, that is, the output level. In this case, the image of the foreground of the main control room may be covered by the smoke image, thereby generating an effect of limiting the viewing distance of the user.

The audio content may be sound displayed through the sound output unit 154. The audio content may include an alarm sound (including an alarm sound of a strobe alarm) so that a user may recognize an occurrence of a fire by hearing, and may be output simultaneously with the display of the alarm image. In addition, the audio content may further include noise of surrounding power sources such as yelling, screaming, and urgent sound as situation noise. In addition, the situation noise may further include an explosion sound due to fire, a noise (burning sound, etc.) in the fire.

The olfactory content may include various smells of smoke. For example, the olfactory content may include a scent like a bullet. The output level of the olfactory content may be adjusted according to the control of the controller 170. For example, the control unit 170 may adjust the degree of opening of the storage unit in which the fragrance substance capable of generating the coal is stored according to the output level, thereby controlling the extent to which the olfactory content is emitted, that is, the divergence intensity of the coal.

The tactile content may be vibration or heat provided through the haptic output unit 158. For example, the controller 170 may simulate the heat due to the flame generated in the main control room by increasing the temperature of the part where the user's skin is in contact with the haptic output unit 158. For example, the controller 170 may allow the temperature to rise differently according to the distance between the position where the flame is generated and the user.

In addition, the control unit 170 may control the haptic output unit 158 so that the vibration is output in conjunction with when the flame occurs, or when the explosion sound is output to allow the user to feel the vibration generated by the flame and explosion have.

The controller 170 may output a virtual reality content including at least one of the visual content, the audio content, the olfactory content, and the tactile content in operation S200 to simulate a virtual fire situation according to the currently selected experience scenario. .

Meanwhile, in the state of outputting the virtual reality content, the controller 170 may detect a user's input. In operation S202, the currently selected experience scenario may be processed according to the detected user input.

The controller 170 may detect a user input based on a gesture of the user detected through the input unit 130. For example, the user's input may be for a driver to recognize a fire and to make an initial report by wire. To this end, the controller 170 may identify a situation in which the user reports a fire situation based on the voice or gesture of the user input through the microphone 132 or the camera 134.

In addition, after the flame and smoke is generated from the fire ignition point, it is possible to detect the fire suppression behavior for operating the fire extinguishing device (eg, extinguishing gas secretion) from the user's gesture, and determine the operation of the fire extinguishing device accordingly can do. And it is possible to simulate the operation of the fire extinguishing device according to the determination result of the fire extinguishing device operation.

Meanwhile, the controller 170 may determine whether the fire is extinguished according to the experience scenario when the fire extinguishing device is operated, and may simulate the fire extinguishing situation of the next step according to the determination result. For example, when the fire is not suppressed, the control unit 170 may simulate the act of manually stopping the reactor in the main control room according to the user's input (voice or gesture), and transferring control from the measurement control facility room to the remote stop room. Can be.

To this end, the control unit 170 may detect an operation in which the user stares (or pushes a catch button) the fire extinguisher for the fire suppression action, and the user moves the fire extinguisher and the fire extinguishing point to suppress the fire. It can be detected. In addition, it is possible to detect the act of staring at (or pushing the open button) the door to move from the main control room to the measurement control equipment room, and staring at the control transfer switch located in the control cabinet of the measurement control equipment room (or pushing the switch). It can detect actions that can be activated.

On the other hand, if the fire experience scenario proceeds according to the user's input, the controller 170 may measure the user's immersion degree to the currently provided virtual reality content (S204). The immersion degree of the user may be measured using at least one biosignal sensor 148. The at least one biosignal sensor 148 may be an electrocardiogram (ECG) sensor or an electroencephalogram (EEG) sensor.

The controller 170 may estimate the mental state of the user from the electrocardiogram and the electroencephalogram measured by the electrocardiogram sensor and the electroencephalogram sensor. To this end, the control unit 170 may calculate the rate of change by comparing the average of the amount of change calculated every predetermined time with the average of the amount of change calculated at a previous time, as shown in Equation 1 below.

For example, if the measured rate of change in the electrocardiogram and electroencephalogram is maintained for a predetermined level or more, the controller 170 may determine that the user's mental state is in a tense state or an anxiety state. If the user is in a tense or anxious state as described above, the user may be determined to be fully immersed in the fire situation virtual reality that is currently provided. If it is determined that the user is fully immersed in the virtual reality, the control unit 170 determines whether the experience scenario is terminated (S210), and if not, proceeds to step S202 again and proceeds to step S202 to determine the virtual reality content according to the user's input. You can continue the currently selected experience scenario by providing.

On the other hand, if the measured rate of change in the electrocardiogram and EEG is less than a predetermined level, the controller 170 may determine that the user's mental state is in a stable state. In this stable state, it may be determined that the user is not sufficiently immersed in the fire situation virtual reality currently being provided.

Then, the controller 170 may generate an additional event to call the user's attention to improve the user's immersion (S208). For example, the additional event may include visual content such as an additional flame image or explosion image, or additional auditory content such as an additional warning sound or explosion sound. Meanwhile, the additional visual content or audio content may be further displayed in the dummy area currently in progress, and the dummy area may be an area not related to the current experience scenario.

That is, the dummy area may be an area excluding an area of interest in which an alarm for displaying a current fire warning, a control panel required for controlling a manual stop of a nuclear reactor and a transfer of control right, a door required for a user's movement, and the like are displayed. In this case, the ROI may include a region where a user input is input for the progress of the experience scenario.

The additional event may be variously generated according to a result of determining the immersion degree of the user. For example, as a result of the measurement of the biosignal sensor 148, a different additional event may be generated when only the rate of change of the electrocardiogram is kept below a certain level and when only the rate of change of the electroencephalogram is kept below a certain level. As described above, an operation process of step S208 to generate different additional events according to the result of determining the immersion degree of the user according to the biosignal measurement result in step S208 will be described in more detail with reference to FIG. 3.

On the other hand, if it is determined that the user is fully immersed in the virtual reality according to the generation of the additional event, the control unit 170 may proceed to step S210 to determine whether the experience scenario has ended. If the experience scenario is not finished, the process proceeds to step S202 again to provide virtual reality content according to a user's input, and to continue the currently selected experience scenario.

FIG. 3 is a flowchart illustrating a process of generating an additional event for improving immersion of an operator during the operation of FIG. 2.

Referring to FIG. 3, the control unit 170 of the HMD 100 according to an embodiment of the present invention maintains a state in which at least one of the rate of change of the electrocardiogram or the rate of change of the electroencephalogram measured by the user is less than a predetermined level for a predetermined time or more. In the case of S208, the process of generating an additional event for improving the immersion of the user may be performed.

Herein, the control unit 170 may allow different additional events to be generated according to the measurement value in which a state below a predetermined level is maintained for a predetermined time or more. To this end, the control unit 170 may first determine a biosignal of the user whose state below a predetermined level is maintained for a predetermined time or more (S300).

As a result of the determination of step S300, if the rate of change of the electrocardiogram is maintained below a predetermined level for more than a predetermined time, the controller 170 may further generate an auditory event (S302). The auditory event may be an event in which additional auditory content such as an additional warning sound or an explosion sound is output. Meanwhile, the controller 170 may further control the haptic output unit 158 so that the vibration corresponding to the explosion sound is output together with the output of the additional auditory content such as the explosion sound. That is, the tactile content may be further output together with the added auditory content. In addition, the user may be alerted through the output of the additional auditory event so that the user can focus on the virtual reality content currently provided.

On the other hand, if the determination result of step S300, if the rate of change of the EEG is maintained below a predetermined level for more than a predetermined time, the controller 170 may further generate a visual event (S304). The auditory event may be an event in which visual content such as an additional flame image or an explosion image is output. The controller 170 may further control the haptic output unit 158 to generate heat corresponding to the added flame image together with the output of the additional visual content such as the flame image. In addition, as the visual content is further displayed, the smell of smoke, that is, the olfactory content may be further output.

That is, at least one of the tactile content and the olfactory content may be further output in addition to the added visual content. In addition, the user may be alerted by outputting such additional visual events, thereby allowing the user to focus on the virtual reality content currently provided.

On the other hand, if the determination result of step S300, if the rate of change of the electrocardiogram and the rate of change in the electroencephalogram are both kept below a predetermined level for a predetermined time or more, the control unit 170 may further generate both the auditory event and the visual event ( S306). Therefore, step S306 may be a step of outputting both the additional auditory content and visual content. In addition to the addition of the added auditory content and visual content, tactile content or olfactory content related to each content may be further output.

Meanwhile, step S306 may be a step in which the additional auditory content and the additional visual content are alternately output at predetermined time intervals.

On the other hand, if the additional content is further output through any one of the step S302, S304 or S306, the control unit 170 can measure the user's immersion again (S308). Here, step S308 may be a step of checking whether at least one of the rate of change of the electroencephalogram or the electrocardiogram measured by the user is less than a predetermined level for a predetermined time or more, similarly to the step S204 of FIG. 2.

If the immersion degree remeasured in step S308 is greater than or equal to a predetermined level, the controller 170 proceeds to step S210 of FIG. 2 to determine whether the experience scenario is finished, and proceeds to step S202 again according to the determination result. You can continue the selected experience scenario.

On the other hand, if the immersion degree remeasured in step S308 is less than the predetermined level, the controller 170 may increase the output level of the added event (S312). That is, the controller 170 may increase an output level, for example, a volume, of the auditory content when the event that is additionally generated is an auditory event in which the auditory content is output. Alternatively, when the additionally generated event is a visual event in which visual content is output, the brightness or size of the visual content may be increased (output level increase). If the event that is additionally generated is both an auditory event and a visual event, the controller 170 may increase both the output level of the auditory content and the output level of the visual content.

In addition, the controller 170 may proceed to step S300 again to determine a biosignal of the user whose state below a predetermined level is maintained for a predetermined time or more. The audio content or the visual content may be further output again according to the determined result (when the process proceeds to step S302 or S304). Alternatively, both the auditory content and the visual content may be additionally output (in step S306). In this case, the additionally output contents may be output according to the increased output level in step S12.

Therefore, the present invention can measure the degree of immersion of the user experiencing the fire situation using the virtual reality, and can be immersed in the virtual reality content of the fire situation by drawing attention to the user according to the measured result.

Meanwhile, in the above description, the user's immersion degree is measured while the user experiences the fire situation experience scenario. However, the generation of the additional event according to the user's immersion level measurement and the progress of the experience scenario are separately performed. Of course, it may be performed. In this case, the controller 170 separately performs a process of proceeding with the experience scenario according to the user's input and a process of outputting additional content according to the measurement result of the user's immersion degree, regardless of the progress of the experience scenario. Of course, additional content may be output.

On the other hand, on the contrary, the control unit 170 may proceed in conjunction with the process of proceeding the experience scenario according to the user's input, and the process of outputting additional content in accordance with the user's immersion measurement results. In this case, when the user's immersion degree is not more than a predetermined level, the controller 170 repeats the process of FIG. 3 and prevents the next process according to the experience scenario from being performed. Of course, the degree of immersion can be maintained above a certain level.

On the other hand, the above description has been described taking only the experience scenario for the fire experience as an example, of course, the present invention is not limited thereto. That is, the present invention, which enables the user's attention to be called on the basis of the measurement result of the immersion degree of the user, may be applied to various virtual reality experiences related to various disaster situations as well as fire.

The present invention described above can be embodied as computer readable codes on a medium in which a program is recorded. The computer-readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include hard disk drives (HDDs), solid state disks (SSDs), silicon disk drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and the like. This also includes implementations in the form of carrier waves (eg, transmission over the Internet). In addition, the computer may include a controller 170 of the terminal. Accordingly, the above detailed description should not be construed as limiting in all aspects and should be considered as illustrative. The scope of the invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the invention are included in the scope of the invention.

100: HMD 110: wireless communication unit
120: power supply unit 130: input unit
132: microphone 134: camera
140: sensing unit 142: acceleration sensor
144: Gyro Sensor 146: Eye Tracking Sensor
148: biosignal sensor 150: output unit
152: display unit 154: sound output unit
156: olfactory information output unit 158: haptic output unit
160: memory 170: control unit

Claims (12)

An output unit for outputting virtual reality content according to the selected experience scenario;
Sensing unit for detecting at least one biological signal from the user;
When the virtual reality content is output, the control unit for measuring the user's commitment to the virtual reality content based on the detected bio-signal, and generates an additional event to call the user's attention based on the measured commitment Including;
The control unit,
Outputting content according to the additional event to a dummy area not related to the progress of the experience scenario,
The dummy area is,
And an area within the virtual reality except for an area of interest including an area in which a user input is input for the progress of the experience scenario. The method of claim 1,
The bio signal is,
At least one of the user's ECG and EEG,
The control unit,
Experience measuring the amount of immersion of the user to the virtual reality content based on the rate of change calculated as a result of measuring the amount of change of the bio-signals measured at a certain time, and the amount of change measured at each time is compared with each other Simulation device. The method of claim 2, wherein the control unit,
And the content output through the additional event is determined based on at least one change rate of the change rates calculated from the electrocardiogram and the electroencephalogram, respectively, for a predetermined time. The method of claim 3,
The virtual reality content,
Includes visual and audio content,
The control unit,
Any one of the visual content and the auditory content, or the visual content, depending on whether the rate of change maintained for less than a predetermined time for a predetermined time is either the rate of change of the electrocardiogram and the rate of change of the electroencephalogram or both of the rate of change of the electrocardiogram and rate of change of the electroencephalogram Experience simulation apparatus, characterized in that to output both the content and the auditory content through the additional event. The method of claim 4, wherein the control unit,
If at least one of the visual content and the audio content is further output through the additional event, the user's immersion degree is re-measured, and the additional event is further generated according to the re-measurement result.
And when the additional event is further generated, outputting the content having the increased output level through the further generated additional event. The method of claim 4, wherein
The virtual reality content,
Virtual reality content to experience the fire situation,
The visual content is,
Includes flame or explosion images that vary in size and brightness depending on the output level,
The auditory content,
Experience-experimental device, characterized in that the volume includes a different explosion sound or warning sound depending on the output level. Outputting virtual reality content according to a preset experience scenario based on a user input;
Detecting at least one biosignal from a user;
Measuring a user's commitment to the virtual reality content based on the detected at least one biosignal; And,
Generating an additional event to call attention of the user based on the measured immersion degree,
Generating the additional event,
Outputting content according to the additional event to a dummy area not related to the progress of the experience scenario;
The dummy area is,
And an area within the virtual reality except for an area of interest including an area in which a user input is input for the progress of the experience scenario. The method of claim 7, wherein
The bio signal is,
At least one of the user's ECG and EEG,
Measuring the immersion of the user,
Measuring the amount of change of the bio-signal measured at a certain time, and measuring the user's immersion degree to the virtual reality content based on a rate of change calculated as a result of comparing the measured amount of change at each time. Experience simulation method. The method of claim 8, wherein measuring the immersion of the user comprises:
Determining the content output through the additional event based on at least one change rate of the change rates calculated from each of the electrocardiogram and the electroencephalogram, which is maintained for less than a predetermined time for a preset time; Experience simulation method. The method of claim 9,
The virtual reality content,
Includes visual and audio content,
The content output through the additional event,
Any one of the visual content and the auditory content, or the visual content, depending on whether the rate of change maintained for less than a predetermined time for a predetermined time is either the rate of change of the electrocardiogram and the rate of change of the electroencephalogram or the rate of change of the rate of change of the electrocardiogram and the rate of electroencephalogram. And the auditory content of all of the experience simulation method. The method of claim 10, wherein generating the additional event,
Outputting at least one of the visual content and audio content through the generated additional event;
Re-measuring the user's commitment; And,
Further generating the additional event according to the re-measurement result;
The additional event is further generated,
And increasing the output level of the content to be output. The method of claim 10,
The virtual reality content,
Virtual reality content to experience the fire situation,
The visual content is,
Includes flame or explosion images that vary in size and brightness depending on the output level,
The auditory content,
Experience simulation method characterized in that the volume includes different explosion sounds or warning sounds depending on the output level.

Images