JP6746033B1 - Information processing device, information processing method, video presentation device, and information processing program - Google Patents

Abstract

The information processing device (100) is a device that causes a video display device (500) to display a video based on video information, and a learning video determination unit (110) that determines a learning video (511) and a user's consciousness. While letting the user perceive the learning image (511) unconsciously while looking at the actual target region (810) to be perceived or showing the target region image (521) taken by the imaging device to the user. And a video generation unit (120) for generating video information.

Description

The present invention relates to an information processing device, an information processing method, a video presentation device including the information processing device, and an information processing program.

There are presentations of research on learning that is performed unconsciously. See Non-Patent Document 1, for example. According to this document, when different images are presented to the left and right eyes of the user, the images presented to one eye can be made continuous flash images so that the images presented to the other eye are not conscious. The phenomenon Continuous Flash Suppression (CFS) is described. This document describes that it is possible to pre-learn unconsciously by using CFS.

Hiroaki Suzuki et al., “Unconscious nature of insight problem solving: A study using subthreshold priming with continuous flash suppression”, Cognitive Studies Vol. 20 No. 3, pp. 353-367, September 2013

However, the above document does not assume that the work perceived by the consciousness and the learning perceived by the unconsciousness are simultaneously performed. In other words, the above document does not assume that the user unconsciously learns while attracting consciousness to the target area in which the work at hand is performed.

An object of the present invention is to allow a user to perceive a learning video unconsciously while attracting attention to a target area.

An information processing apparatus according to an aspect of the present invention is an apparatus that causes a video display device to display a video based on video information, and includes a learning video determination unit that determines a learning video and a reality that the user wants to perceive while attracting consciousness. while showing a region of interest image obtained by photographing the object area in the image pickup device to the user, the image information to be perceived unconsciously under the learning image to the user possess a video generation unit that generates, and the image information The image based on the image includes a first image and a second image presented to the user's first eye and second eye, respectively, and the image generation unit generates a consciousness attracting stimulus, A process of calculating a consciousness attraction amount indicating a degree of consciousness attraction for each of the first image including the learning image and the second image including the target region image and the consciousness attraction stimulus. Using the consciousness attraction amount, a process of determining whether or not a consciousness is attracted to the target area video and a consciousness is not attracted to the learning video is established, and the situation is established. Repeat until it is determined .

The information processing method according to another aspect of the present invention were taken to a method for displaying a video based on the video information to the video display device, the target area of the real to be perceived while attracting users consciousness imaging device object while showing an area image to the user, the image information to be perceived unconsciously under the learning image to the user possess a step that generates, wherein the video based on the video information, a first of said users In the step of generating the image information, which includes the first image and the second image presented to the eyes and the second eye, respectively, the step of generating the consciousness-stimulating stimulus, and the first image including the learning image And a process of calculating a consciousness attraction amount indicating the degree of consciousness attraction for each of the target region image and the second image including the consciousness attraction stimulus, and using the calculated consciousness attraction amount. The process of determining whether or not the situation in which the consciousness is attracted to the target area image and the consciousness is not attracted to the learning image is established until it is determined that the situation is established .

According to the present invention, the user can perceive the learning image unconsciously while attracting consciousness to the target area.

It is a figure which shows roughly the structure of the information processing apparatus and video presentation apparatus which concern on Embodiment 1 of this invention. (A) And (B) is a figure which shows the example of the image shown to the left and right eyes, and the example of the image perceived consciously regarding Embodiment 1, respectively. (A) And (B) is a figure which shows the example of the switching of the learning image perceived unconsciously, and the example of the image perceived consciously regarding Embodiment 1, respectively. 5 is a flowchart showing the operation of the information processing device according to the first embodiment. (A) And (B) is a figure which shows the example of the image shown to the right and left eyes, and the example of the image perceived consciously regarding a comparative example. (A) to (C) are diagrams showing an example of a method of evaluating the amount of consciousness attraction in the first embodiment. (A) to (C) are diagrams showing another example of the consciousness attraction amount evaluation method according to the first embodiment. FIG. 3 is a diagram showing an example of a hardware configuration of an information processing device and a video presentation device according to the first embodiment. It is a figure which shows roughly the structure of the information processing apparatus and video presentation apparatus which concern on Embodiment 2 of this invention. (A) to (C) relate to the second embodiment, showing an example of an image presented to the left and right eyes, an example of an image perceived with consciousness, and an image perceived with actual hand work and consciousness. It is a figure which shows each example. 7 is a flowchart showing the operation of the information processing device according to the second embodiment. (A) to (C) are diagrams showing an example of a method of evaluating the amount of consciousness attraction in the second embodiment. FIG. 6 is a diagram showing an example of a hardware configuration of an information processing device and a video presentation device according to the second embodiment. It is a figure which shows roughly the structure of the information processing apparatus and video presentation apparatus which concern on Embodiment 3 of this invention. FIG. 16 is a diagram showing an example of a target area image and an example of a learning image presented to a user regarding the third embodiment. 9 is a flowchart showing the operation of the information processing device according to the third embodiment. FIG. 16 is a diagram showing an example of an evaluation method of the amount of consciousness attraction in the third embodiment. FIG. 16 is a diagram showing an example of an evaluation method of the amount of consciousness attraction in the third embodiment. FIG. 16 is a diagram showing an example of an evaluation method of the amount of consciousness attraction in the third embodiment. FIG. 16 is a diagram showing an example of an evaluation method of the amount of consciousness attraction in the third embodiment. FIG. 16 is a diagram showing an example of an evaluation method of the amount of consciousness attraction in the third embodiment. FIG. 16 is a diagram showing an example of a method of adjusting the amount of consciousness attraction in the third embodiment. FIG. 16 is a diagram showing an example of a hardware configuration of an information processing device and a video presentation device according to the third embodiment. It is a figure which shows roughly the structure of the information processing apparatus and video presentation apparatus which concern on Embodiment 4 of this invention. FIG. 16 is a diagram showing an example of a target area viewed by a user and an example of a learning video according to the fourth embodiment. 16 is a flowchart showing the operation of the information processing device according to the fourth embodiment. FIG. 16 is a diagram showing an example of a hardware configuration of an information processing device and a video presentation device according to the fourth embodiment. It is a figure which shows the structure of the information processing apparatus and video presentation apparatus of a modification.

Hereinafter, an information processing device, an information processing method, a video presentation device, and an information processing program according to an embodiment of the present invention will be described with reference to the drawings. In the drawings, the same or similar components are designated by the same reference numerals. The following embodiments are merely examples, and various modifications can be made within the scope of the present invention.

Embodiment 1.
FIG. 1 is a diagram schematically showing the configurations of the information processing device 100 and the video presentation device 1 according to the first embodiment. The information processing device 100 is a device capable of implementing the information processing method according to the first embodiment. The video presentation device 1 includes an information processing device 100, a head mounted display (HMD) 500 as a video display device, and an imaging device (that is, a camera) 530 that captures a target area 810. The target area 810 is, for example, an area near the user 800. FIG. 1 shows a situation where a user 800 is handling an article 820 by hand. The information processing apparatus 100 causes the HMD 500 to display a video based on the video information. Although the imaging device 530 is provided on the front surface of the HMD 500 in FIG. 1, the imaging device 530 may be arranged at a position away from the HMD 500.

The image display device may be a device other than the HMD 500 as long as it is a device that can provide different images to the first eye and the second eye of the user 800. The image presentation device 1 may include a 3D projector or a 3D display device that projects a three-dimensional (ie, 3D) image instead of the HMD 500.

For example, the first eye and the second eye are the left eye and the right eye, respectively. Alternatively, the first eye and the second eye may be the right eye and the left eye, respectively. The information processing apparatus 100 according to the first embodiment provides the HMD 500 with a left-eye image 510 presented to the left eye and a right-eye image 520 presented to the right eye.

The information processing device 100 includes a learning video determination unit 110 and a video generation unit 120. The learning image determination unit 110 determines the learning image provided to the HMD 500. For example, the learning video determination unit 110 selects a learning video from a learning video database prepared in advance. As shown in FIG. 1, the learning video determining unit 110 includes, for example, a learning video dividing unit 111, a work situation recognizing unit 112, and a learning video switching unit 113. The operation of these configurations will be described later.

The video generation unit 120 generates video information that allows the user 800 to unknowingly perceive the learning video while showing the target area video of the target area 810 captured by the imaging device 530 to the user 800. As illustrated in FIG. 1, the video generation unit 120 includes, for example, a consciousness-attracting video generation unit 121, a consciousness-attracting video evaluation unit 122, and a video output unit 123. The operation of these configurations will be described later.

2A and 2B are diagrams showing an example of an image presented to the left and right eyes and an example of an image perceived with consciousness, respectively. As shown in FIG. 2A, in the first embodiment, left-eye image 510 includes learning image 511, which is an image that user 800 wants to learn. As shown in FIG. 2A, in the first embodiment, the right-eye image 520 includes a hand image 521 as a target range image and a continuous flash suppression (CFS) image 522 as a consciousness-stimulating stimulus image. .. The hand image 521 is generally a moving image.

The learning image 511 is an image that is presented to the left eye of the user 800, but is not noticed by the user 800 due to the CFS stimulation that is the action of the CFS image 522. That is, the learning image 511 is an image that is presented to the left eye of the user 800 but is not perceptually perceived by the user 800 due to CFS stimulation. The learning video 511 is generally a still image. In this way, the CFS stimulation allows the user 800 to learn the learning image 511 below the threshold.

The CFS stimulus is an example of a consciousness attracting stimulus. The consciousness-stimulating stimulus is also called a "consciousness-induced stimulus". The consciousness attracting stimulus is, for example, an image that randomly changes. The consciousness-stimulating stimulus may include one or more of a CFS stimulus, a blinking icon, a blinking of the target area image, and a repeating change of the size of the target area image. The consciousness-stimulating stimulus may be a combination of two or more of a CFS stimulus, a blinking icon, blinking of the target area image, and repeated changing of the size of the target area image.

In FIG. 2A, the hand image 521 is surrounded by the CFS image 522. By the CFS stimulation of the CFS image 522, the consciousness of the user 800 is attracted to the hand image 521. That is, the hand image 521 is consciously perceived by the user 800 by the CFS stimulus.

The user 800 perceives the image as shown in FIG. 2B by the CFS stimulus. That is, the user 800 consciously perceives the right-eye image 520 and does not consciously perceive the left-eye image 510. However, the user 800 unconsciously perceives the learning image 511 shown in FIG. 2A while perceiving the image shown in FIG. 2B and concentrating his/her attention on the work at hand. .. In other words, the user 800 is learning the sub-threshold learning video 511, for example, while keeping his or her focus on the work at hand.

FIGS. 3A and 3B are diagrams showing an example of switching of the learning image 511 perceived unconsciously and an example of the image perceived unconsciously, respectively. As shown in FIG. 3A, the learning video determination unit 110 switches the learning video 511 based on the detection information obtained by the sensor that detects the status of the work currently performed by the user 800 in the target area 810. .. For example, the learning video determination unit 110 switches the learning video 511 based on the video captured by the imaging device 530. For example, the learning video 511 shows a procedure of work performed after the work currently performed by the user 800 in the target area 810.

FIG. 4 is a flowchart showing the operation of the information processing device 100 according to the first embodiment. The learning video determination unit 110 reads learning data from a storage device or a learning video database on a network (step S101). Next, the learning video determination unit 110 extracts a key frame from the learning data as a learning video (step S102). These processes are performed, for example, by the learning video dividing unit 111 shown in FIG.

Next, the learning video determination unit 110 recognizes the current progress status of the work at hand by the user 800 (step S103), and acquires the key frame of the next work based on the recognized progress status (step S104). These processes are performed by, for example, the work status recognition unit 112 shown in FIG. The learning image determination unit 110 recognizes the current progress status of the work at hand based on, for example, a video image captured by the imaging device 530 or a detection signal from a sensor that detects the progress status.

Next, the learning video determination unit 110 determines a learning video based on the key frame and switches the left-eye video 510 to a new learning video 511 (step S105). These processes are performed by the learning video switching unit 113 shown in FIG. 1, for example. The learning video determination unit 110 repeats the processing of steps S103 to S105.

Next, the image generation unit 120 generates a CFS stimulus (step S106) and performs a process of combining the CFS image 522, the hand image 521, and the learning image 511 (step S107). These processes are performed by, for example, the consciousness attracting image generation unit 121 shown in FIG.

Next, the image generator 120 calculates the consciousness attraction amount for each of the left-eye image and the right-eye image (step S108). Next, the image generation unit 120 uses the consciousness attraction amount to determine whether or not a situation is established in which the consciousness is attracted to the hand image 521 and the consciousness is not attracted to the learning image 511 (step S109). If the determination is NO, the process returns to step S107, and if the determination is YES, the process proceeds to step S110. These processes are performed by, for example, the consciousness attracting image evaluation unit 122 shown in FIG.

When the process proceeds to step S110, the video generation unit 120 outputs the composite video generated at step S107 to the HMD 500. This processing is performed by the video output unit 123 shown in FIG. 1, for example.

FIGS. 5A and 5B are diagrams showing an example of an image presented to the left and right eyes and an example of an image perceived with consciousness, regarding an unfavorable combined image generated by the comparative example. As shown in FIG. 5A, when the left-eye image 510 includes the learning image 511 and the right-eye image 520 includes the hand image 521 and the CFS image 522, generally, the above-described FIG. As shown in (B), the hand image 521 and the CFS image 522 are perceived consciously, and the learning image 511 is not perceived consciously. However, as shown in FIG. 5B, the user 800 consciously perceives an image in which the image of FIG. 2B and the learning image 511 of FIG. A situation slightly perceived by the user may occur due to individual differences of the user 800. Therefore, as described below, a difference D (that is, a difference in the amount of consciousness attraction) obtained by subtracting the amount of consciousness attraction of the left-eye image 510 from the amount of consciousness attraction of the right-eye image 520 is predetermined. It is desirable that it is larger than the threshold T.

6A to 6C are diagrams showing an example of an evaluation method of the amount of consciousness attraction. The image generating unit 120 subtracts the consciousness attraction amount indicating the degree of consciousness attraction of the left eye image 510 from the consciousness attraction amount indicating the degree of consciousness attraction of the right eye image 520, based on the difference obtained between the left and right images. Generate eye images.

As shown in FIG. 6A, the image generation unit 120 acquires the learning image 511 as the left-eye image 510, and the hand image 521 and the CFS image 522 as the right-eye image 520.

As shown in FIG. 6B, the image generation unit 120 calculates the consciousness attraction amount Cl of the learning image 511 and the consciousness attraction amount Cs of the hand image 521 and the CFS image 522. The consciousness attracting amounts Cs and Cl are, for example, the consciousness attracting amounts Vsc and Vlc depending on the contrast of the image, the consciousness attracting amounts Vsb and Vlb depending on the brightness, the consciousness attracting amounts Vsd and Vld depending on the movement change amount, and the consciousness attracting amounts Vss and Vls depending on the saturation. (That is, the consciousness attraction amounts Vsh and Vlh based on the sharpness) and the consciousness attraction amounts Vsv and Vlv based on the level of visibility are calculated. The consciousness attracting amounts Cs and Cl are, for example, consciousness attracting amounts Vsc and Vlc due to image contrast, consciousness attracting amounts Vsb and Vlb depending on lightness, consciousness attracting amounts Vsd and Vld due to motion change amount, and consciousness attracting amounts Vss due to saturation, It is calculated as an addition value or a weighted addition value of one or more consciousness attraction amounts of Vls (that is, consciousness attraction amounts Vsh and Vlh based on sharpness). For example, when the consciousness attracting amounts Cs and Cl are weighted addition values of the consciousness attracting amounts Vsc and Vlc due to the image contrast, the consciousness attracting amounts Vsb and Vlb depending on the brightness, and the consciousness attracting amounts Vsd and Vld depending on the movement change amount, The consciousness attracting amounts Cs and Cl are calculated by the following equations (1) and (2).
Cs=kc*Vsc+kb*Vsb+kd*Vsd (1)
Cl=kc*Vlc+kb*Vlb+kd*Vld (2)
Here, kc, kb, and kd are weighting factors of the consciousness attraction amounts Vsc and Vlc by the contrast, the consciousness attraction amounts Vsb and Vlb by the brightness, and the consciousness attraction amounts Vsd and Vld by the movement change amount, respectively.

As shown in FIG. 6C, the difference D in the amount of consciousness attraction is calculated by D=Cs-Cl. When the difference D is larger than the threshold value T that is predetermined by experiments or the like, the mixing of the work video and the learning video as shown in FIG. 5B does not occur. The threshold value T may be determined by performing calibration for each user 800.

In addition, screen feature amounts such as visibility and saliency may be used as the consciousness attraction amount. The consciousness attraction amount based on the visibility is described in Non-Patent Document 2, for example, and the consciousness attraction amount based on the saliency is described in, for example, Non-Patent Document 3. Further, as the consciousness attraction amount, one or more of the consciousness attraction amount by contrast, the consciousness attraction amount by brightness, the consciousness attraction amount by movement change amount, and the consciousness attraction amount by saturation, and the visibility, An addition value or a weighted addition value with the consciousness attraction amount based on the screen feature amount such as saliency may be used.

Valero Laparra and 2 others, "Division Normalization Image Quality Metric Revisited" Image Processing Laboratory, University of Valencia, 27, 4 (10). Masatoshi Yoshida, "Application of Saliency Map to Visual Search Analysis", Journal of Japanese Neural Network Society, Vol. 21, No. 1 (2014), pp. 3-12

7A to 7C are diagrams showing another example of the method for evaluating the amount of consciousness attraction. The image generation unit 120 divides the learning image 511 into a plurality of small areas, divides the hand image 521 and the CFS image 522 into a plurality of small areas, as shown in FIG. As shown in, the consciousness attraction amount of the small area of the learning image 511 (for example, Cl11, Cl12,...) and the consciousness attraction amount of the small area of the combined image of the hand image 521 and the CFS image 522 (for example, Cs11, Cs12). ,...) may be used to calculate the difference. As shown in FIG. 7B, the difference Dxy is obtained by Dxy=Csxy-Clxy. x indicates the horizontal position of the small area, and y indicates the vertical position of the small area. For example, x=1, 2, 3 and y=1, 2, 3. However, the maximum values of x and y are not limited to the illustrated example. As shown in FIG. 7C, the learning image 511, the hand image, and the CFS image are generated so that Dxy>Txy is satisfied in all the small areas. Txy is a threshold value of the small area specified by the positions x and y.

FIG. 8 is a diagram illustrating an example of a hardware configuration of the information processing device 100 and the video presentation device 1 according to the first embodiment. As shown in FIG. 8, the video presentation device 1 includes an information processing device 100, an HMD 500, and an imaging device 530. The video presentation device 1 may include a storage device 930 such as a hard disk device. The information processing device 100 has a memory 920 that stores a program and a processor 910 that executes the program. The program can include an information processing program for implementing the information processing method according to the first embodiment. All or part of the functions of the learning image determination unit 110 and the image generation unit 120 illustrated in FIG. 1 can be realized by the processor 910 that executes a program. All or part of the functions of the learning image determining unit 110 and the image generating unit 120 shown in FIG. 1 may be realized by a semiconductor integrated circuit.

As described above, by using the apparatus and method according to the first embodiment, the user 800 can perceive the learning image 511 unconsciously while attracting attention to the target area 810. In other words, the user 800 can perform the sub-threshold learning (also referred to as “while learning”) on the learning video 511 while concentrating the consciousness at hand.

Embodiment 2.
FIG. 9 is a diagram schematically showing the configurations of the information processing device 200 and the video presentation device 2 according to the second embodiment. The information processing device 200 is a device capable of implementing the information processing method according to the second embodiment. The video presentation device 2 has an information processing device 200, a 3D projector 600 as a video display device, and an imaging device 530. The imaging device 530 may be a sensor (for example, a depth sensor) that detects the progress of work. The information processing device 200 causes the 3D projector 600 to display a video based on the video information. The image display device may be a device other than the 3D projector 600 as long as it is a device capable of providing different images to the first eye and the second eye of the user 800.

The information processing device 200 according to the second embodiment provides the 3D projector 600 with the left-eye image 510 presented to the left eye and the right-eye image 520 presented to the right eye. The information processing device 200 includes a learning video determination unit 110 and a video generation unit 220. The learning image determination unit 110 determines the learning image provided to the 3D projector 600.

The image generation unit 220 generates image information that allows the user 800 to perceive the learning image 511 unconsciously while watching the actual target area 810 that the user 800 wants to perceive while attracting consciousness. As illustrated in FIG. 9, the video generation unit 220 includes, for example, a consciousness-attracting video generation unit 121, a consciousness-attracting video evaluation unit 122, a video output unit 123, and a perceptual video estimation unit 221. The operation of these configurations will be described later.

FIGS. 10A to 10C are diagrams showing an example of an image presented to the left and right eyes and an example of an image perceived with consciousness. As shown in FIG. 10A, in the second embodiment, left-eye image 510 includes learning image 511, which is an image that user 800 wants to learn. As shown in FIG. 10A, in the second embodiment, right-eye image 520 does not include the hand image as the target range image, but includes CFS image 522 as the consciousness-stimulating stimulus image.

The learning image 511 is an image that is presented to the left eye of the user 800, but is not noticed by the user 800 due to the CFS stimulation that is the action of the CFS image 522. That is, the learning image 511 is an image that is presented to the left eye of the user 800 but is not perceptually perceived by the user 800 due to CFS stimulation. In this way, the CFS stimulation allows the user 800 to learn the learning image 511 below the threshold. Consciousness attraction stimuli may include CFS stimuli, repeated movements of areas of CFS stimulation, and repeated size changes of areas of CFS stimulation.

As shown in FIG. 10(A), the entire right-eye image 520 is a CFS image 522, and as shown in FIG. 10(B), the user 800 consciously perceives and learns the CFS image 522. The image 511 is not perceived consciously. Therefore, the CFS stimulus of the CFS image 522 attracts the consciousness of the user 800 to the target area 810. That is, as shown in FIG. 10C, the hand range 810 is perceived by the user 800 consciously by the CFS stimulation.

The user 800 perceives the situation as shown in FIG. 10C by the CFS stimulus. That is, the user 800 consciously perceives the right-eye image 520 and does not consciously perceive the left-eye image 510. However, the user 800 perceives the image shown in FIG. 10(B) and unconsciously perceives the learning image 511 shown in FIG. 10(A) while concentrating his/her attention on the actual work at hand. In other words, the user 800 is learning the sub-threshold learning video 511, for example, while keeping his or her focus on the work at hand.

FIG. 11 is a flowchart showing the operation of the information processing device 200 according to the second embodiment. The operation of the learning image determination unit 110 is the same as that described with reference to FIG.

The image generation unit 220 generates a CFS stimulus (step S106), and performs a process of combining the CFS image 522 including the CFS stimulus and the learning image 511 (step S201). These processes are performed by, for example, the consciousness attracting image generation unit shown in FIG.

Next, the image generator 220 estimates an image that will be perceived by the left and right eyes of the user 800. That is, since the user is directly looking at the real work at hand, the real work at hand and the learning video 511 are presented to the left eye, and the real work at work and the CFS video 522 are presented to the right eye. .. The video generation unit 220 estimates a video perceived by the left eye and a video perceived by the right eye based on the video captured by the imaging device 530, the learning video 511, and the CFS video 522 (step S202). Next, the image generation unit 220 calculates the consciousness attraction amount for each of the left-eye image and the right-eye image (step S108). Next, the image generation unit 220 determines whether or not the situation in which the consciousness is attracted to the hand image 521 and the consciousness is not attracted to the learning image 511 is established using the consciousness attraction amount (step S109). If the determination is NO, the process returns to step S201, and if the determination is YES, the process proceeds to step S110. These processes are performed, for example, by the perceptual image estimation unit 221 and the consciousness-attracting image evaluation unit 122 illustrated in FIG. 9.

When the process proceeds to step S110, the video generation unit 220 outputs the composite video generated at step S201 to the 3D projector 600. This processing is performed by the video output unit 123 shown in FIG. 1, for example.

12A to 12C are diagrams showing an example of an evaluation method of the amount of consciousness attraction. The image generation unit 220 is obtained by subtracting the consciousness attraction amount Cl of the combination of the real target region 810 and the left eye image 510 from the consciousness attraction amount Cs of the combination of the real target region 810 and the right eye image 520. A right-eye image 520 (CFS image 522 in the second embodiment) is generated based on the difference D. That is, the image generation unit 220 generates the right-eye image 520 (CFS image 522 in the second embodiment) so as to satisfy D=Cs-Cl>T. The difference between the small areas as shown in FIGS. 7A to 7C may be used as the difference in the consciousness attraction amount.

The image generation unit 220 can use the same method as in the case of the first embodiment as a method of obtaining the consciousness attraction amount Cl and the consciousness attraction amount Cs.

FIG. 13 is a diagram illustrating an example of the hardware configuration of the information processing device 200 and the video presentation device 2 according to the second embodiment. As shown in FIG. 13, the video presentation device 2 includes an information processing device 200, a 3D projector 600, an imaging device 530, and a storage device 930. The information processing device 200 includes a memory 920 that stores a program and a processor 910 that executes the program. The program can include an information processing program for implementing the information processing method according to the second embodiment. All or part of the functions of the learning image determination unit 110 and the image generation unit 220 illustrated in FIG. 9 can be realized by the processor 910 that executes a program. All or part of the functions of the learning image determining unit 110 and the image generating unit 220 may be realized by a semiconductor integrated circuit.

As described above, by using the device and method according to the second embodiment, the user 800 can perceive the learning image 511 unconsciously while attracting consciousness to the actual target area 810. That is, the user 800 can study the learning video 511 below the threshold while working while focusing his or her attention on the hand.

Other than the above, the second embodiment is the same as the first embodiment.

Embodiment 3.
FIG. 14 is a diagram schematically showing the configurations of the information processing device 300 and the video presentation device 3 according to the third embodiment. The information processing device 300 is a device capable of implementing the information processing method according to the third embodiment. The video presentation device 3 includes an information processing device 300, an HMD 500 as a video display device, and an imaging device 530.

The information processing device 300 according to the third embodiment provides a video presented by the HMD 500. The information processing device 300 includes a learning video determination unit 110 and a video generation unit 320. The learning image determination unit 110 determines the learning image provided to the HMD 500.

The image generation unit 320 generates image information that causes the user 800 to perceive the learning image unconsciously while showing the target region image obtained by capturing the target region 810 with the imaging device to the user 800. As illustrated in FIG. 14, the video generation unit 320 includes, for example, a consciousness-attracting video generation unit 121, a consciousness-attracting video evaluation unit 122, a video output unit 123, a gaze estimation unit 321, and a video region dividing unit 322. have. The operation of these configurations will be described later.

FIG. 15 is a diagram showing an example of an image presented to the left and right eyes of the user 800 and an example of an image perceived with consciousness. As shown in FIG. 15, in the third embodiment, the same image is presented to the left and right eyes of user 800. As illustrated in FIG. 15, the image generation unit 320 of the information processing device 300 causes the HMD 500 to present the center image 521 that is the target region image within the central visual field range 830 that is the range including the line of sight of the user 800, and the center of the image. By presenting the learning image 511 in the peripheral visual field range 840, which is a range outside the visual field range 830, the user 800 unconsciously perceives the learning image 511 while allowing the user 800 to perceive the target region image. ..

FIG. 16 is a flowchart showing the operation of the information processing device 300 according to the third embodiment. The operation of the learning image determination unit 110 is the same as that described with reference to FIG.

The image generation unit 320 estimates the line of sight (step S301) and determines the positions of the hand image 521 and the learning image 511 (step S302). These processes are performed by, for example, the line-of-sight estimation unit 321 and the consciousness attraction image generation unit 121 illustrated in FIG. 14.

Next, the image generation unit 320 divides the visual field range of the left and right eyes of the user 800 into a plurality of small regions, calculates the amount of consciousness attraction for each small region, and uses the amount of consciousness attraction to the consciousness of the hand image 521. Is determined, and it is determined whether or not the situation in which the consciousness is not attracted to the learning video 511 is established (step S109). If the determination is NO, the process returns to step S201, and if the determination is YES, the process proceeds to step S110. These processes are performed by, for example, the image area dividing unit 322 and the consciousness-attracting image evaluation unit 122 illustrated in FIG. 9.

When the process proceeds to step S110, the video generation unit 320 outputs the video generated at step S302 to the HMD 500. This process is performed by the video output unit 123 shown in FIG. 14, for example.

17 to 21 are diagrams showing an example of an evaluation method of the amount of consciousness attraction in the third embodiment. First, as shown in FIG. 17, the image generation unit 320 divides the visual field range of the user 800 into a plurality of small areas. FIG. 17 shows an example in which the visual field range is divided into 25 small areas of 5 rows and 5 columns. However, the number of small areas is not limited to 25, and the shape of the small areas is not limited to a rectangle.

Next, as shown in FIG. 18, the image generation unit 320 calculates the consciousness attraction amounts Vc11, Vc12, Vc13,... Of the small areas. As in the case of the first embodiment, the consciousness attracting amount of the small area is, for example, the consciousness attracting amount Vc based on the image contrast, the consciousness attracting amount Vb based on the brightness, the consciousness attracting amount Vd based on the movement change amount, and the consciousness attracting based on the saturation. One or more of the amount Vs (that is, the amount of consciousness attraction Vh due to sharpness) and the amount of consciousness attraction Vv depending on the level of visibility. However, the amount of consciousness attraction in the small area may include the amount of consciousness attraction due to another image feature amount (for example, saliency).

Next, as shown in FIG. 19, the image generation unit 320 calculates the attention amount Vg based on the line-of-sight position 850, with the line-of-sight position 850 at which the line of sight of the user 800 intersects the visual field range as a reference. As shown in FIG. 19, the attention amount Vg is calculated for each small area. In the example of FIG. 19, Vg=1 is the maximum value in the small region including the line-of-sight position 850, and the attention amount Vg of the small region apart from the line-of-sight position 850 is small. The attention amount of FIG. 19 is an example, and the value of the attention amount is not limited to that of FIG.

Next, as shown in FIG. 20, the image generation unit 320 calculates the consciousness attraction amounts C11, C12, C13,... Of the small areas. The consciousness attraction amount of the small area is, for example, the same as in the first embodiment, for example, the consciousness attraction amount Vc by the contrast of the image, the consciousness attraction amount Vb by the brightness, the consciousness attraction amount Vd by the motion change amount, and the consciousness attraction by saturation. It is an addition value and a weighted addition value of one or more of the amount Vs (that is, the amount of consciousness attraction Vh due to sharpness) and the amount of consciousness attraction Vv depending on the level of visibility. The consciousness attraction amounts C11, C12, C13,... Of the small area are calculated by, for example, the following formula (3).
C=kc*Vc+kb*Vb+kd*Vd (3)

Next, as shown in FIG. 21, the image generation unit 320 causes the consciousness attracting amounts C22, C23, C24, C32, C33, C34, C42, C43 of the area in which the hand image 521 is displayed among the small areas. Cs is calculated using C44. The image generation unit 320 calculates Cl using the consciousness attraction amount C55 of the region in which the learning image 511 is displayed among the small regions. Cs and Cl are calculated, for example, by the following equations (4) and (5).
Cs=(C22+C23+C24+C32+C33+C34+C42+C43+C44)/9 (4)
Cl=(C55)/1 (5)

The image generation unit 320 determines the positions of the hand image 521 and the learning image 511 based on the difference D obtained by subtracting the consciousness attraction amount Cs from the consciousness attraction amount Cs.

FIG. 22 is a diagram showing an example of a method of adjusting the amount of consciousness attraction in the third embodiment. The image generation unit 320 may display the consciousness attracting stimulus 542 around or near the hand image 521. As the consciousness attracting stimulus 542, the same one as in the first and second embodiments can be used.

FIG. 23 is a diagram illustrating an example of the hardware configuration of the information processing device 300 and the video presentation device 3 according to the third embodiment. As shown in FIG. 23, the video presentation device 3 has an information processing device 300, an HMD 500, an imaging device 530, and a storage device 930. The information processing device 300 includes a memory 920 that stores a program and a processor 910 that executes the program. The program can include an information processing program for implementing the information processing method according to the third embodiment. All or part of the functions of the learning image determination unit 110 and the image generation unit 320 illustrated in FIG. 14 can be realized by the processor 910 that executes a program. All or part of the functions of the learning image determining unit 110 and the image generating unit 220 may be realized by a semiconductor integrated circuit.

As described above, by using the device and the method according to the third embodiment, the user 800 can perceive the learning image 511 unconsciously while attracting the consciousness to the hand image 521 of the target area 810. That is, the user 800 can learn the learning image 511 below the threshold while working while focusing his or her attention on the hand.

Other than the above, the third embodiment is the same as the first or second embodiment.

Fourth Embodiment
FIG. 24 is a diagram schematically showing the configurations of the information processing device 400 and the video presentation device 4 according to the fourth embodiment. The information processing device 400 is a device capable of implementing the information processing method according to the fourth embodiment. The video presentation device 4 includes an information processing device 400, a projector 700 as a video display device, and an imaging device 530. The imaging device 530 may be a sensor that detects the progress of work.

The information processing device 400 according to the fourth embodiment provides a video presented by the projector 700. The information processing device 400 has a learning video determination unit 110 and a video generation unit 420. The learning image determination unit 110 determines the learning image provided to the projector 700.

The video generation unit 420 generates video information that allows the user 800 to perceive the learning video 511 unconsciously while watching the real target area 810 that the user 800 wants to perceive while attracting attention. As shown in FIG. 24, the video generation unit 420 includes, for example, a consciousness-attracting video generation unit 121, a perceptual video estimation unit 221, a consciousness-attractive video evaluation unit 122, a video output unit 123, and a line-of-sight estimation unit 421. , And a video area dividing unit 422. The operation of these configurations will be described later.

FIG. 25 is a diagram showing an example of an image presented to the left and right eyes of the user 800 and an example of an actual target area perceived under consciousness. As shown in FIG. 25, in Embodiment 4, the same video is presented to the left and right eyes of user 800. As shown in FIG. 25, the image generation unit 420 of the information processing device 400 causes the projector 700 to be within the peripheral visual field range 840, which is a range outside the central visual field range 830, which is a range including the user's 800 gaze position 850. By presenting the learning image 511, the user 800 unconsciously perceives the learning image 511 while causing the user 800 to perceive the actual target area unconsciously.

FIG. 26 is a flowchart showing the operation of the information processing device 400 according to the fourth embodiment. The operation of the learning image determination unit 110 is the same as that described with reference to FIG.

The image generation unit 320 estimates the line-of-sight position 850 (step S401) and determines the position of the learning image 511 (step S402). These processes are performed, for example, by the line-of-sight estimation unit 421 and the consciousness-attracting image generation unit 121 illustrated in FIG.

Next, the image generation unit 420 divides the visual field range of the left and right eyes of the user 800 into a plurality of small areas, estimates an image that the user 800 may perceive (step S403), and recognizes each small area. The amount of attraction is calculated (step S404), and it is determined whether or not the situation is such that the consciousness is attracted to the actual target area 810 at hand and the learning image 511 is not attracted using the consciousness attraction amount. (Step S109). If the determination is NO, the process returns to step S201, and if the determination is YES, the process proceeds to step S110. These processes are performed by, for example, the perceptual image estimation unit 221, the image area dividing unit 422, and the consciousness-attracting image evaluation unit 122 illustrated in FIG.

When the process proceeds to step S110, the image generation unit 420 outputs the learning image displayed at the position generated in step S402 to the projector 700. This process is performed by the video output unit 123 shown in FIG. 25, for example.

The image generation unit 420 divides the central visual field range 830 and the peripheral visual field range 840 into a plurality of small areas, and determines a plurality of peripheral visual field ranges 840 from the amount of consciousness of the actual area of the plural small areas of the central visual field range 830. The learning image 511 is generated based on the difference D obtained by subtracting the consciousness attraction amount of the combination of the learning image 511 of the small region and the actual region overlapping the learning image. Here, the generation of the learning video 511 includes the determination of the position of the learning video 511.

FIG. 27 is a diagram showing an example of the hardware configuration of the information processing device 400 and the video presentation device 4 according to the fourth embodiment. As shown in FIG. 27, the video presentation device 4 includes an information processing device 400, a projector 700, an imaging device 530, and a storage device 930. The information processing device 400 has a memory 920 that stores a program and a processor 910 that executes the program. The program can include an information processing program for implementing the information processing method according to the third embodiment.

As described above, by using the device and method according to the fourth embodiment, the user 800 can perceive the learning image 511 unconsciously while attracting the work in the target area 810. That is, the user 800 can perform sub-threshold learning on the learning video 511 while concentrating the consciousness at hand and working.

Other than the above, the fourth embodiment is the same as any of the first to third embodiments.

Modified example.
28: is a figure which shows the structure of the information processing apparatus 450 and the video presentation apparatus 5 of a modification. The information processing device 450 has a video generation unit 460. The image generation unit 460 differs from the image generation unit 120 of FIG. 1 in that it has a dominant eye estimation unit 461. The information processing apparatus 450 estimates the dominant eye or acquires the dominant eye information of the user 800, presents the hand image 521 and the CFS image 522 that should be consciously perceived by the dominant eye, and the eyes other than the dominant eye. The learning video 511 is presented. In this case, the probability that the learning video 511 can be perceived unconsciously while focusing on the hand video 521 increases. The image generation unit 460 may correct the generated image based on the pre-acquisition information including one or more of the visual acuity, biological reaction, and dominant eye of the left and right eyes of the user 800.

Note that the correction of the image based on the pre-acquisition information of one or more of the left and right eyes' visual acuity, biological reaction, and dominant eye can be applied to any of the first to fourth embodiments.

1 to 5 video presentation device, 100, 200, 300, 400, 450 information processing device, 110 learning video determination unit, 111 learning video division unit, 112 work situation recognition unit, 113 learning video switching unit, 120, 220, 320, 420, 460 video generation unit, 121 consciousness attracted video generation unit, 122 consciousness attracted video evaluation unit, 123 video output unit, 221 perceptual video estimation unit, 321, 421 gaze estimation unit, 322, 422 video region division unit, 461 dominant eye Estimator, 500 HMD, 510 left-eye image, 511 learning image, 520 right-eye image, 521 hand image, 522 CFS image, 530 imaging device, 542 consciousness attraction image, 600 3D projector, 700 projector, 800 user, 810 target area 830 central visual field range, 840 peripheral visual field range, 850 line-of-sight position, 910 processor, 920 memory.

Claims (24)

An information processing device for displaying a video based on video information on a video display device,
A learning image determination unit that determines the learning image,
While showing a region of interest image taken by the user by the imaging device an object area of the real to be perceived while attracting awareness to the user, image generation that generates the image information to be perceived unconsciously under the learning image to the user Department,
Have a,
The video based on the video information includes a first video and a second video presented to the user's first and second eyes, respectively.
The image generation unit generates a consciousness-stimulating stimulus, the first image including the learning image, and the second image including the target region image and the consciousness-stimulating stimulus. The process of calculating the amount of consciousness attraction indicating the degree of consciousness, and using the calculated amount of consciousness attraction, whether or not a situation has been established in which consciousness is attracted to the target area image and consciousness is not attracted to the learning image. An information processing apparatus that repeats the process of determining whether or not the above condition is satisfied . The information processing apparatus according to claim 1 , wherein the consciousness attracting stimulus is a randomly changing image displayed so as to surround the target region image. The information according to claim 1 or 2 , wherein the consciousness-stimulating stimulus includes one or more of a continuous flash suppression stimulus, a blinking icon, a blinking of the target area image, and a repetition of changing the size of the target area image. Processing equipment. Claim wherein the image generator is based on the difference obtained by subtracting the awareness attracting amount of the first video from awareness attracted amount of the second image to generate the second video The information processing apparatus according to any one of 1 to 3 . The image generation unit,
Each of the first image and the second image is divided into a plurality of first small areas and a plurality of second small areas,
A consciousness indicating the degree of consciousness of the second small area with respect to the corresponding first small area and second small area among the plurality of first small areas and the plurality of second small areas. based from the amount attracted to the difference obtained by subtracting the consciousness attract quantity indicating a degree of consciousness attracting said first small area, any one of claims 1-3 for generating said second video The information processing device according to 1. The video display device, an information processing apparatus according to any one of claims 1 to 5 is a projector for projecting a head-mounted display or 3-dimensional image. An information processing device for displaying a video based on video information on a video display device,
A learning image determination unit that determines the learning image,
An image generation unit that generates the image information that causes the user to perceive the learning image unconsciously while looking at the actual target area that the user wants to perceive while attracting consciousness.
Have
The video based on the video information includes a first video and a second video presented to the user's first and second eyes, respectively.
The image generation unit generates a consciousness attraction stimulus, and a consciousness attraction amount indicating a degree of consciousness attraction for each of the first image including the learning image and the second image including the consciousness attraction stimulus. And a process of using the calculated consciousness attraction amount to determine whether or not a situation in which consciousness is attracted to the actual target area and consciousness is not attracted to the learning image is established. Is repeated until it is determined that the above condition is established.
Information processing device. The information processing apparatus according to claim 7 , wherein the consciousness-stimulating stimulus is a randomly changing image projected so as to overlap the actual target area. The consciousness attracted stimulation, continuous flash suppression stimulation repetition of movement of the continuous flash suppression stimulation region, and according to claim 7 comprising one or more of the repetition of changing of the size of the continuous flash suppression stimulation region Information processing equipment. The image generation unit determines, from the consciousness attraction amount indicating the degree of consciousness attraction of the combination of the second image and the real area overlapping the second image, the reality overlapping the first image and the first image. information processing based on the difference obtained by subtracting the consciousness attract quantity indicating a degree of consciousness attracted combination of regions, according to any one of claims 7 to 9 for generating the second video apparatus. The image generation unit,
Each of the first image and the second image is divided into a plurality of first small areas and a plurality of second small areas,
Regarding the first small area and the second small area corresponding to each other among the plurality of first small areas and the plurality of second small areas, the second image and the second image in the second small area Awareness of the combination of the real regions overlapping the second video, from the consciousness attraction amount indicating the degree of the consciousness attracting, the consciousness of the combination of the first video and the real region overlapping the first video in the first small region. based on the difference obtained by subtracting the consciousness attract quantity indicating the degree of attracting, the information processing apparatus according to any one of claims 7 to 9 to generate the second image. An information processing device for displaying a video based on video information on a video display device,
A learning image determination unit that determines the learning image,
An image generation unit that generates the image information that allows the user to perceive the learning image unconsciously while showing the user a target region image captured by an imaging device of a real target region that the user wants to perceive while attracting consciousness. ,
Have
The video generation unit causes the video display device to present the target area video within a central visual field range that is a range including the line-of-sight position of the user, and within a peripheral visual field range that is outside the central visual field range. wherein by presenting learning images, while perceived the target area image before SL user in conscious, is perceived in unconscious the learning image to the user,
The image generation unit divides the central visual field range and the peripheral visual field range into a plurality of small areas, and determines the peripheral visual field from the consciousness attraction amount indicating the degree of consciousness attraction of the plurality of small areas of the central visual field range. A process of calculating a difference by subtracting a consciousness attraction amount indicating a degree of consciousness attraction of the plurality of small regions of the range; and a process of determining the positions of the target region video and the learning video based on the difference. , A process of determining whether or not a situation in which the consciousness is attracted to the target area video and the consciousness is not attracted to the learning video is established using the difference, and it is determined that the situation is established. Information processing device that repeats until it is told . An information processing device for displaying a video based on video information on a video display device,
A learning image determination unit that determines the learning image,
An image generation unit that generates the image information that causes the user to perceive the learning image unconsciously while looking at the actual target area that the user wants to perceive while attracting consciousness.
Have
The video generation unit causes the video display device to present the learning video to the user by presenting the learning video within a peripheral visual field range that is outside a central visual field range that is a range that includes the user's line-of-sight position. Make the image perceive unconsciously ,
The image generation unit divides the central visual field range and the peripheral visual field range into a plurality of small areas, and an amount of consciousness attraction indicating a degree of consciousness attraction of an actual area of the plurality of small areas of the central visual field range. From the process of calculating the difference by subtracting the consciousness attraction amount indicating the degree of consciousness attraction of the combination of the learning image of the plurality of small regions of the peripheral visual field range and the real region overlapping the learning image, A process of determining the position of the learning video based on the difference, and using the difference, whether or not a situation is established in which the consciousness is attracted to the real target area and the consciousness is not attracted to the learning video is determined. An information processing apparatus that repeats the determination process until it is determined that the situation is established . The amount of consciousness attraction is one of an amount of consciousness attraction by contrast, an amount of consciousness attraction by brightness, an amount of consciousness attraction by amount of change in image movement, an amount of consciousness attraction by saturation, and an amount of consciousness attraction by visibility level. The information processing apparatus according to any one of claims 4 , 5 , 10 , and 11 , which is calculated by the above addition value or weighted addition value. The information processing apparatus according to claim 14 , wherein the consciousness attraction amount is corrected based on pre-acquired information including one or more of the user's visual acuity, biological reaction, and dominant eye. The learning video determination unit switches the learning video based on detection information obtained by a sensor that detects a situation of work currently performed by the user in the target area.
The information processing device according to any one of claims 1 to 15. The sensor is an imaging device
The information processing apparatus according to claim 16. The learning video shows a procedure of work performed after the work currently performed by the user in the target area.
The information processing apparatus according to claim 16 or 17. An information processing apparatus according to any one of claims 1 to 18 ,
The video display device,
Video presentation device having a. An information processing method for displaying an image based on image information on an image display device,
Determining a learning video,
Generating a while showing a region of interest image taken by the user by the imaging device an object area of the real to be perceived while attracting awareness to the user, the image information to be perceived unconsciously under the learning image to the user ,
Have a,
The video based on the video information includes a first video and a second video presented to the user's first and second eyes, respectively.
In the step of generating the image information, a process of generating a consciousness attracting stimulus, the first image including the learning image, and the second image including the target region image and the consciousness attracting stimulus are included. Using the process for calculating the amount of consciousness attraction indicating the degree of consciousness attraction for each, and using the calculated amount of consciousness attraction, a situation is established in which consciousness is attracted to the target area image and consciousness is not attracted to the learning image. An information processing method in which the processing of determining whether or not the above is performed is repeated until it is determined that the situation is established . An information processing method for displaying an image based on image information on an image display device,
Determining a learning video,
A step of generating the image information that causes a user who is looking at a real target region to be perceived while attracting consciousness to perceive the learning image unconsciously;
Have
The video based on the video information includes a first video and a second video presented to the user's first and second eyes, respectively.
In the step of generating the image information, a process of generating a consciousness-attracting stimulus and a degree of consciousness-attracting for each of the first image including the learning image and the second image including the consciousness-attracting stimulus are determined. By using the process of calculating the amount of consciousness attraction shown, whether or not a situation in which the consciousness is attracted to the actual target area and the consciousness is not attracted to the learning image is established. Repeat the judgment process until it is judged that the above condition is established.
Information processing method. An information processing method for displaying an image based on image information on an image display device,
Determining a learning video,
A step of generating the image information that allows the user to perceive the learning image unconsciously while showing the user a target region image captured by an imaging device of a real target region that the user wants to perceive while attracting consciousness;
Have
In the step of generating the video information, the video display device is caused to present the target area video within a central visual field range which is a range including the line-of-sight position of the user, and a peripheral area which is outside the central visual field range. By making the user perceive the target area video unconsciously by causing the user to perceive the learning video unconsciously by presenting the learning video within the visual field range,
In the step of generating the image information, a process of dividing the central visual field range and the peripheral visual field range into a plurality of small areas, and an amount of consciousness attraction indicating a degree of consciousness attraction of the plurality of small areas of the central visual field range. A process of calculating a difference obtained by subtracting the consciousness attraction amount indicating the degree of consciousness attraction of the plurality of small regions in the peripheral visual field range, and the target region video and the learning video based on the difference. And a process for determining whether or not a situation is established in which the consciousness is attracted to the target area video and the consciousness is not attracted to the learning video, using the difference. Repeat until it is determined that
Information processing method. An information processing method for displaying an image based on image information on an image display device,
Determining a learning video,
A step of generating the image information that causes a user who is looking at a real target region to be perceived while attracting consciousness to perceive the learning image unconsciously;
Have
In the step of generating the image information, by causing the image display device to present the learning image in a peripheral visual field range that is a range outside a central visual field range that is a range including the line-of-sight position of the user, Let the user perceive the learning video unconsciously,
In the step of generating the image information, a process of dividing the central visual field range and the peripheral visual field range into a plurality of small areas, and a degree of consciousness attraction of an actual area of the plurality of small areas of the central visual field range are determined. The difference is calculated by subtracting the amount of consciousness attraction indicating the degree of consciousness attraction of the combination of the learning image of the plurality of small areas in the peripheral visual field range and the actual region overlapping the learning image from the indicated consciousness attraction amount. Processing, processing for determining the position of the learning video based on the difference, and using the difference, consciousness is drawn to the actual target area, and a situation is established in which consciousness is not drawn to the learning video. Repeat the process of determining whether or not there is the above condition until it is determined that
Information processing method. An information processing program for causing a computer to execute a process of displaying an image based on image information on an image display device ,
The information processing program for executing the information processing method according to any one of claims 20 to 23 before Symbol computer.

Images