KR20200043318A - Eye movement and emotion based HMD - Google Patents

Abstract

The present invention is worn on the head to cover the user's eyes, the headset is provided with an eyepiece positioned in front of the eye; A display unit provided on the headset to be seen through the eyepiece; A speaker provided to output audio to the headset; An electrode unit formed of a plurality of electrodes attached to the skin around the eyes to the headset, and configured to measure EOG signals according to eye movements through the electrodes; At least one camera provided to face the eye of the headset and acquiring an image of the eye and its surroundings by manipulation of a photographing button; Based on the current EOG signal measured through the electrode portion and a reference signal according to the movement of the eyeball, an ocular momentum measurement unit for measuring ocular momentum by checking whether ocular movement occurs; An emotion classifying unit for acquiring feature data of the eye from the image obtained through the camera and classifying emotions by comparing it with reference data of the eyes for each emotion; A memory unit for storing content for guiding the user's eye movement and content corresponding to the user's emotion type; An operation unit for inputting an operation signal to select contents and operations stored in the memory unit; A microcomputer that performs control for outputting corresponding content and performing an operation according to an operation signal of the manipulation unit; And a power supply unit that provides power required for operation.

Description

Eye movement and emotion based HMD}

The present invention relates to an eye movement and emotion-based HMD, and more particularly, to an eye movement and emotion-based HMD that increases the efficiency and ease of eye movement and benefits the emotion.

In general, eye movements play an important role in maintaining eye health through prevention of vision loss and presbyopia. On the other hand, one of the bio-signals, an electrooculogram (hereinafter referred to as “EOG”) signal, can objectively measure ocular momentum using a potential difference between the cornea and the retina, even if the eyes are closed or glasses are used.

As a technique related to the measurement of eye momentum by the conventional EOG signal, there has been proposed "Method for measuring eye movement using reference voltage feedback-based eye conduction signal" of Korean Patent Registration No. 10-1297330, which is transferred using the eye conduction signal. A method of measuring eye movement using a reference voltage feedback-based eye conduction signal that enables measurement of eye movement amount without being affected by movement by changing the reference voltage using the measured signal. Measuring, measuring the current EOG signal according to the eye movement, determining whether the eye movement occurs based on the current EOG signal and the reference, and the previous EOG signal previously measured according to whether the eye movement occurs And correcting the reference voltage. Accordingly, by changing the reference voltage using the previously measured signal using the eye conduction signal, it is possible to measure the amount of eye movement without affecting the movement even if the user moves the head severely.

In addition, the eyes express emotions, and people express their emotions in a variety of ways, such as language, voice, gestures, and vision. Unlike other emotion expression methods, the expression of emotion by vision is also controlled through training, etc. It is difficult to do, so it is a very useful tool to infer the exact feelings of a person.

As a technique for classifying emotions using conventional vision, Korean Patent Registration No. 10-1734845, “Emotion classification apparatus using visual analysis and its method,” has been proposed, which differs in emotion classification methods using emotion classification apparatus. Providing an image including a plurality of stimulus screens corresponding to emotion to a subject, measuring the eye movement of the subject while the image is being provided, analyzing the measured eye movement to open and close the eye of the subject, Generating eye feature data including a pupil size change rate and gaze position, and performing data averaging to integrate units of the eye feature data, and generating a three-dimensional emotion model using the averaged eye feature data as a dimensional axis. It includes the step of generating, and further, the image including the stimulus screen, the plurality The stimulus screens and the non-stimulation screen are repeatedly arranged as many as a predetermined number, a stimulus screen corresponding to the calm emotion is arranged on the first screen of the image, and generating the eye feature data includes: In the data generated for the opening and closing time of the eye through the difference value between the time when the subject closed the eyes and the time when the subject closed the eyes on the stimulus screen corresponding to the serenity emotion, Data on the gaze position is generated through a distance between a center of the upper n regions where the subject gazes for a long time and a predetermined object center of the plurality of stimulus screens among the set regions.

However, all of these conventional technologies are related to the eyes, and there is a problem in that it is inconvenient to use them, and the usability is reduced by not using them properly, thereby limiting the actual contribution.

In order to solve the problems of the prior art as described above, the present invention is to increase the efficiency and ease of eye movement through the measurement of the amount of eye movement through the EOG signal with the output of the content for eye movement, emotion through vision Classification can be applied to emotions by matching the emotions or outputting content that matches the current mood, and furthermore, it is possible to select locations according to desired emotions through statistics on emotions for each location. The purpose is to help.

Other objects of the present invention will be easily understood through the description of the following examples.

In order to achieve the above object, according to an aspect of the present invention, the headset is worn on the head to cover the user's eyes, the eyepiece positioned in front of the eye is provided; A display unit provided on the headset to be seen through the eyepiece; A speaker provided to output audio to the headset; An electrode unit formed of a plurality of electrodes attached to the skin around the eyes to the headset, and configured to measure EOG signals according to eye movements through the electrodes; At least one camera provided to face the eye of the headset and acquiring an image of the eye and its surroundings by manipulation of a photographing button; Based on the current EOG signal measured through the electrode portion and a reference signal according to the movement of the eyeball, an ocular momentum measurement unit for measuring ocular momentum by checking whether ocular movement occurs; An emotion classifying unit for acquiring feature data of the eye from the image obtained through the camera and classifying emotions by comparing it with reference data of the eyes for each emotion; A memory unit for storing content for guiding the user's eye movement and content corresponding to the user's emotion type; An operation unit for inputting an operation signal to select contents and operations stored in the memory unit; A microcomputer that performs control for outputting corresponding content and performing an operation according to an operation signal of the manipulation unit; And a power supply unit that provides power required for operation.

The electrode portion, each of the plurality of electrodes on the inner surface of the headset above and below any one of the eyepiece, the right side of the eyepiece corresponding to the right eye, the left side of the eyepiece corresponding to the left eye, the user's Each may be provided at a position corresponding to the forehead, and may be provided to be positioned at an extension extending downward from the headset so as to contact under the user's ear.

In the headset, the ends of the side supports that are provided on both sides are connected to the elastic parts between the ends of the rear supports that are supported behind the user's head, so that when the pull is pulled forward and separated from the user's face by the extension of the elastic parts, the extension parts When stretched to the maximum, in a state where the camera is stopped at a position suitable for acquiring the image of the eye and the surroundings of the image, the user presses the shooting button to view the image of the eye and the surroundings through the camera. Can be acquired.

Further comprising a GPS for obtaining the location information for the headset, the microcomputer counts the location and the classification result, respectively, when classifying the emotion classification unit to calculate the ratio of emotion types for each location, and upon request through the manipulation unit The ratio of emotion types for each location may be output through the display unit.

Further comprising a communication unit for performing wireless communication with the external content providing system, the micom, the content corresponding to the emotion type and content for guiding the eye movement by communication with the content providing system through the communication unit By being provided and stored in the memory unit, it is possible to control the content to be output through the display unit or the speaker upon request through the manipulation unit.

According to the eye movement and emotion-based HMD according to the present invention, it is possible to increase the efficiency and ease of eye movement through measuring the amount of eye movement through the EOG signal along with the output of the content for the eye movement, and to classify emotion through vision By corresponding to the emotion or by outputting the content that matches the current mood, it can be beneficial to the emotion, and furthermore, it is possible to select the location according to the desired emotion through statistics on the location-specific emotion to help the emotion change Can be given.

1 is a perspective view showing an eye movement and emotion-based HMD according to an embodiment of the present invention.
2 is a block diagram showing an eye movement and emotion-based HMD according to an embodiment of the present invention.
Figure 3 is a side view for explaining the motion of the eye movement and emotion-based HMD according to an embodiment of the present invention.

The present invention can be applied to various changes, and may have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and should be understood as including all modifications, equivalents, and substitutes included in the technical spirit and scope of the present invention, and to be modified in various other forms. May be, the scope of the present invention is not limited to the following examples.

Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings, and the same reference numerals will be assigned to the same or corresponding components regardless of reference numerals, and redundant description thereof will be omitted.

1 is a perspective view showing an eye movement and emotion based HMD according to an embodiment of the present invention, and FIG. 2 is a configuration diagram showing an eye movement and emotion based HMD according to an embodiment of the present invention.

1 and 2, the eye movement and emotion-based HMD 100 according to an embodiment of the present invention includes a headset 110, a display 120, a speaker 121, an electrode 130, a camera It may include a 140, the ocular momentum measurement unit 150, the emotion classification unit 160, the memory unit 170, the operation unit 180, the microcomputer 190 and the power supply unit 210.

The headset 110 is worn on the head to cover the user's eyes, and an eyepiece 111 positioned in front of the eyes is provided. The eyepiece 111 may be formed in a pair, as in this embodiment.

The display unit 120 is provided in the headset 110 to be viewed through the eyepiece 111.

The speaker 121 is provided to output audio to the headset 110, for example, may be configured in a small size in consideration of transmitting audio to the user's ear, or configured to output audio through a hole as in this embodiment, or a user It can be configured as an earphone to be inserted into the ear.

The electrode unit 130 is made of a plurality of electrodes 131 attached to the skin around the eyes on the headset 110, and measures the EOG signal according to the movement of the eyeball through the electrodes 131, as in this embodiment. It can consist of six.

The electrode unit 130 includes a plurality of electrodes 131, each of which is above and below one of the eyepieces 111 on the inner surface of the headset 110, and the right and left eyes of the eyepiece 111 corresponding to the right eye. Each of the eyepieces 111 corresponding to the left and the positions corresponding to the user's forehead, respectively, is provided to be located in the extension 115 extending downward from the headset 110 so as to contact under the user's ear Can be prepared. The electrode 131 of the electrode unit 130 may correspond to, for example, a reference electrode when it is in contact with the forehead, and may correspond to a ground electrode when it is contacted under the ear. The EOG signal is obtained by contacting the left and right sides of the eye by the electrodes located on the top and bottom of one eye by the electrodes located above and below the eyepiece 11, and by the electrodes located on the left and right sides of the eyepiece 111. It may include a horizontal signal. In addition, the electrode 131 may be provided with a connector 132 for connecting a signal line to the microcomputer 190, respectively.

The camera 140 is provided to face the eye of the headset 110, and acquires the image of the eye and its surroundings by the operation of the shooting button 142, and consists of at least one, in this embodiment, two It is done. Lights 141 may be provided for each of the cameras 140 to provide lighting to the photographing area of the camera 140.

1 to 3, the headset 110 is connected to the expansion and contraction parts 114 between the ends of the rear support 113 supported by the user's head, the ends of the side support 112 provided on each side respectively. When the front and rear (A) is pulled to be separated from the user's face by the extension (a) of the extension and contraction portion 114, when the extension and contraction portion 114 is stretched to the maximum, the camera 140 displays the image of the eye and its surroundings. In a state where it is stopped at a position suitable for acquisition, the user may acquire an image of the eye and surroundings through the camera 140 by pressing the photographing button 142. As such, the HMD needs to secure an appropriate shooting distance of the camera 140 due to the nature of the eyepiece 111 being in close contact with the eye. The structure of the elastic part 114 and the acquisition of the image for the eye are limited by the limited elasticity. You can make it more convenient. The elastic part 114 may be formed of, for example, a rubber band.

The ocular momentum measurement unit 150 measures ocular momentum using the EOG signal measured through the electrode unit 130. The eye movement measurement unit 150 converts the EOG signal measured from the electrode unit 130 into a digital signal when the user sees and follows the eye movement content displayed through the display unit 120. To measure the ocular momentum from the EOG signal from which noise is removed, the horizontal eye movement is detected through the horizontal signal obtained by the electrodes located on the left and right sides of the eyepiece 111, and the electrodes located above and below the eyepiece 11 To detect the vertical movement of the eye. The eye movement measurement unit 150 measures the user voltage for the EOG signal, and then measures the current EOG signal according to the eye movement, and determines whether the eye movement occurs based on the current EOG signal and the reference signal according to the eye movement. It is possible to measure the ocular momentum by a confirmation process, etc., which is an example, according to the prior art presented in the present invention, or other well-known techniques can be applied, of course.

The emotion classifying unit 160 obtains characteristic data of the eyes from the image obtained through the camera 140, and classifies emotions by comparing the reference data of the eyes for each emotion. The emotion classifying unit 160 provides, for example, an image including a plurality of stimulus screens corresponding to different emotions to the subject, measures the subject's eye movement while the image is provided, and analyzes the measured eye movement Creates eye feature data including the subject's eye opening / closing time, pupil size change rate, and gaze position, performs data averaging to integrate the unit of eye feature data, and uses the averaged eye feature data as a three-dimensional axis Emotion models can be created. The emotion classification by the emotion classification unit 160 is an example, and according to the prior art presented in the present invention, other well-known techniques may be applied. Here, the reference data of the eyes for each emotion may be data representing the eyes and surrounding features when joy, surprise, sadness, anger, interest, boredom, fear, pain, and neutrality, respectively. The eye opening / closing time, pupil size change rate, gaze position, as well as eyelid shape, wrinkle shape, and eyebrow shape may be added to these features.

The memory unit 170 may store content for guiding the user's eye movement and content corresponding to the user's emotion type, and further, may store various data and programs necessary for the operation.

The operation unit 180 allows input of an operation signal for selection of contents and operations stored in the memory unit 170, and may be made of a plurality of buttons or switches, or various input devices, such as a touch panel, may be used.

The microcomputer 190 is configured to perform control for output and operation of the corresponding content according to the operation signal of the operation unit 180. That is, when the content for eye movement is selected according to the manipulation signal of the manipulation unit 180, the micom 190 causes the display to be output through the display unit 120 by reproducing the selected content, and performs eye movement according to the reproduced content. In accordance with the operation signal of the manipulation unit 180, the measurement of the ocular momentum by the ocular momentum measurement unit 150 is measured by the measurement of the EOG signal by the electrode unit 130 according to the manipulation signal of the manipulation unit 180, and the display is performed. In addition to output through the unit 120, it is stored in the memory unit 170 and controlled so that it can be checked again as necessary.

When the microcomputer 190 acquires the image of the eye and the surroundings from the camera 140 by pressing the shooting button 142 of the camera 140 by the operation as shown in FIG. 3, without additional manipulation or the operation unit 180 ), The emotions classified by the emotion classification unit 160 can be sorted and displayed through the display unit 120, and stored contents suitable for the emotion according to the operation signal of the operation unit 180, such as music or video Alternatively, when one of the music videos is selected, it can be controlled to be output through the display unit 120 and the speaker 121. At this time, the emotion classification unit 160 adopts an image with excellent clarity from among a plurality of cameras 140 to classify emotions, or in the process of classifying emotions for each image, when comparing with reference data of eyes for each emotion You can also adopt feelings of high fitness.

The power supply unit 210 provides power necessary for operation. For example, it may provide power supplied from a battery or power supplied from a rechargeable battery. When using the power of a rechargeable battery, power for charging the rechargeable battery is provided. It may additionally have a charging part to be supplied, a charging connector, and the like.

The headset 110 may be provided with a GPS 230 for obtaining location information for the headset 110. The GPS 230 receives GPS information from a GPS satellite to generate GPS data for the current location of the headset 110 and provides it to the microcomputer 190. At this time, the microcomputer 190 calculates the ratio of emotion types for each location by counting the location and the classification result, respectively, when classifying the emotion classification unit 160, and upon request through the manipulation unit 180, determines the emotion type ratio for each location It can be output through the display unit 120. Here, the emotion type ratio may be represented by a graph indicating that it occupies a few percent of the total emotions, which may be configured to be displayed at a corresponding location on the map.

The headset 110 may further include a communication unit 220 to perform wireless communication with the external content providing system 300. The microcomputer 190 receives the content corresponding to the emotion type and the content for guiding the eye movement by communication with the content providing system 330 through the communication unit 220, thereby allowing the memory unit 170 to store the content. When requested through 180, the corresponding content may be controlled to be output through the display unit 120 or the speaker 121. Content corresponding to the emotion type may be stored in the memory unit 170 to be classified by emotion type. Here, the content providing system 300 not only distributes or provides the content free of charge through the Internet, but also distributes and provides paid content, and not only provides such content in the form of files, but also streams by providing a streaming path. Can also be provided.

According to the eye movement and emotion-based HMD according to the present invention, it is possible to increase the efficiency and ease of eye movement by measuring the amount of eye movement through the EOG signal along with the output of the content for the eye movement.

In addition, according to the present invention, it is possible to output the content corresponding to the emotion or to match the current emotion by sorting emotions through vision, thereby benefiting the emotion.

In addition, according to the present invention, it is possible to assist in switching emotions by enabling location selection according to desired emotions through statistics on emotions for each location.

Although the present invention has been described with reference to the accompanying drawings, various modifications and variations can be made without departing from the technical spirit of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the claims described below, but also by the claims and equivalents.

110: headset 111: eyepiece
112: side support 113: rear support
114: expansion and contraction 115: extension
120: display unit 121: speaker
130: electrode portion 131: electrode
132: connector 140: camera
141: Lighting 142: Shooting button
150: eye momentum measurement unit 160: emotional classification
170: memory unit 180: operation unit
190: microcomputer 210: power supply
220: communication unit 230: GPS
240: emotional map calculation unit 300: content provision system

Claims (5)

A headset worn on the head to cover the user's eyes and provided with an eyepiece positioned in front of the eyes;
A display unit provided on the headset to be seen through the eyepiece;
A speaker provided to output audio to the headset;
An electrode unit formed of a plurality of electrodes attached to the skin around the eyes to the headset, and configured to measure EOG signals according to eye movements through the electrodes;
At least one camera provided to face the eye of the headset and acquiring an image of the eye and its surroundings by manipulation of a photographing button;
Based on the current EOG signal measured through the electrode portion and a reference signal according to the movement of the eyeball, an ocular momentum measurement unit for measuring ocular momentum by checking whether ocular movement occurs;
An emotion classifying unit for acquiring feature data of the eye from the image obtained through the camera, and classifying emotions by comparing it with reference data of each eye;
A memory unit for storing content for guiding the user's eye movement and content corresponding to the user's emotion type;
An operation unit for inputting an operation signal to select contents and operations stored in the memory unit;
A microcomputer that performs control for outputting corresponding content and performing an operation according to an operation signal of the manipulation unit; And
A power supply unit that provides power required for operation;
Including, eye movement and emotion based HMD. The method according to claim 1,
The electrode portion,
Each of the plurality of electrodes corresponds to the top and bottom of any one of the eyepieces on the inner surface of the headset, the right side of the eyepiece corresponding to the right eye, the left side of the eyepiece corresponding to the left eye, and the user's forehead. Eye movement and emotion-based HMD, each provided at a location and provided to be located in an extension extending downward from the headset so as to contact under the user's ear. The method according to claim 1,
The headset,
When the ends of the side supports that are provided on both sides are connected to the elastic parts between the ends of the rear supports supported behind the user's head, so as to be pulled forward so as to be spaced apart from the user's face by the extension of the elastic parts, when the elastic parts are fully extended , In a state where the camera is stopped at a position suitable for acquiring the image of the eye and the surroundings of the image, allowing the user to acquire the image of the eye and the surroundings through the camera by pressing the shooting button, Eye movement and emotion based HMD. The method according to any one of claims 1 to 3,
Further comprising a GPS for obtaining the location information for the headset,
The microcomputer,
When classifying the emotion classification unit, the position and the classification result are counted to calculate a ratio of emotion types for each location, and upon request through the manipulation unit, the ratio of emotion types for each location is output through the display unit. . The method according to any one of claims 1 to 3,
Further comprising a communication unit for performing wireless communication with the external content providing system,
The microcomputer,
By receiving the content for guiding eye movement and content corresponding to the emotion type by communication with the content providing system through the communication unit and storing the content in the memory unit, the corresponding content is displayed on the display unit when requested through the manipulation unit Or controlled to be output through the speaker, eye movement and emotion-based HMD.

Images