JP2015126281A - Projector, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology for a projector having an imaging unit to properly photograph an imaging object.SOLUTION: A projector having an imaging unit includes: determination means for determining whether or not an imaging object is present within the projection range of the projector; changing means for changing projection display to be projected onto the imaging object, of projection displays projected from the projector, when the determination means has determined that the imaging object is present within the projection range; and imaging means for photographing the imaging object through the imaging unit while the projection display changed by the changing means being projected on the imaging object.

Description

  The present invention relates to a projector, an information processing method, and a program.

Conventionally, a projector with a built-in camera has been proposed.
The projector described in Patent Document 1 includes an imaging device that captures an image of a card pattern held in a predetermined direction around the projector body. The projector controls operation states such as power on / off switching, volume increase / decrease, and input signal switching of the projector body based on the analysis result obtained by analyzing the video detected by the imaging device.

JP 2006-293123 A

However, in the projector described in Patent Document 1, when the user views the projection screen of the projector in a dark room, the imaging target cannot be appropriately captured due to insufficient illuminance, and it is difficult to decipher the imaging target. There's a problem. In addition, when a flash is used to improve the illuminance, the room is dark and it is very dazzling for the user, the user is blinded, and the presentation (hereinafter referred to as a presentation) cannot be seen for a while. There is also a problem.
An object of the present invention is to provide a technique for a projector having an imaging unit to appropriately capture an imaging target.

  Therefore, the present invention is a projector having an imaging unit, wherein a determination unit that determines whether or not an imaging target exists in the projection range of the projector, and the imaging unit exists in the projection range by the determination unit When judged, the changing means for changing the projection display projected on the imaging target among the projection displays projected from the projector, and the projection display changed by the changing means being projected on the imaging target And imaging means for imaging the imaging object via the imaging unit.

  ADVANTAGE OF THE INVENTION According to this invention, the technique for the projector which has an imaging part to image an imaging target appropriately can be provided.

FIG. 2 is a diagram illustrating an example of a hardware configuration of a projector according to a first embodiment. It is a figure which shows an example of the scene assumed in Embodiment 1. FIG. 6 is a flowchart illustrating an example of processing according to the first exemplary embodiment. It is a figure which shows an example of a projection display. FIG. 6 is a diagram illustrating an example of a hardware configuration of a projector according to a second embodiment. 10 is a flowchart illustrating an example of processing according to the second exemplary embodiment. It is a figure which shows an example of a depth map. FIG. 10 is a diagram illustrating an example of a hardware configuration of a projector according to a third embodiment. It is a figure which shows an example of the scene assumed in Embodiment 3. 10 is a flowchart illustrating an example of processing according to the third exemplary embodiment.

The best mode for carrying out the present invention will be described below with reference to the drawings.
<Embodiment 1>
FIG. 1 is a diagram illustrating an example of a hardware configuration of the projector 10 according to the present embodiment.
The image capturing unit 11 is a camera having an image sensor or the like built in the projector 10. The image capturing unit 11 may have a function capable of capturing not only a still image but also a moving image (video). Note that still image data and moving image data acquired by the image capturing unit 11 are examples of captured image data.
The storage unit 12 is a storage area for storing captured images and various data including predetermined data required for the processing of the projector 10.
The video analysis unit 13 analyzes a video composed of a plurality of continuous images picked up by the image pickup unit 11 and detects a motion of an object (image pickup target). That is, the video analysis unit 13 detects a moving object based on the captured image captured by the image capturing unit 11. The video analysis unit 13 in the present embodiment detects a moving object using a background difference method. Note that the video analysis unit 13 may detect a motion of the object by analyzing a moving image captured by the image capturing unit 11. The same applies to the subsequent processing.

The material estimator 15 estimates (specifies) characteristics such as the material of the target object by analyzing a video formed by the captured image captured by the image capturing unit 11.
The object detection unit 16 analyzes the video received from the image capturing unit 11 to detect the shape of the object and performs various controls in the projector 10. In the following processing, the object is described as having a rectangular shape, but other shapes may be used.
The motion detection unit 17 captures the movement of the rectangular object detected by the object detection unit 16 (hereinafter referred to as a rectangular object) by tracking the movement detected by the video analysis unit 13.
The video output unit 18 projects the projection content (projection display) indicated by the projection data stored in the storage unit 12 onto a screen (projection screen) or the like. Also, the video output unit 18 changes the projection display of the position (projection position) of the rectangular object detected by the object detection unit 16 in the projection range of the projector 10 to white display (hereinafter referred to as partial white display). be able to. In addition, when outputting a video in which the projection display at the projection position of the rectangular object is changed to the partial white display, the video output unit 18 adds a predetermined margin to the peripheral part of the rectangular object to display the partial white. Output. In addition, the projection range here means the space where the projector 10 can project the projection content.
The partial exposure adjustment unit 20 calculates the exposure of the rectangular object that is the target detected by the object detection unit 16, and feedback corrects the brightness (illuminance) of the partial white display to be projected.

  Note that the video analysis unit 13, the material estimation unit 15, the object detection unit 16, the motion detection unit 17, the video output unit 18, and the partial exposure adjustment unit 20 described above may be configured by software. That is, when the CPU of the projector 10 executes the program stored in the storage area, the video analysis unit 13, the material estimation unit 15, the object detection unit 16, the motion detection unit 17, the video output unit 18, and the partial exposure adjustment unit. Twenty functions may be realized.

In the present embodiment, a case will be described in which the user holds the picture 21 of the paper medium in his hand 22 and inserts it in the projection range of the projector 10 in parallel with the screen (perpendicular to the projection light) as shown in FIG.
FIG. 2 is a diagram illustrating an example of a scene assumed in the first embodiment.
On the screen 23, the picture 21 inserted in the projection range and the finger of the hand 22 are displayed as shadows.
In the present embodiment, it is assumed that the viewer is watching the screen in a darkened room when the user makes a presentation using the projector 10. In a room, the amount of light other than the projection light projected on the screen is so small that the imaging target cannot be imaged in a visible manner, and there are often no other light sources. Therefore, unless the user presents a paper medium in front of the projector 10, that is, within the projection range, the projector 10 cannot capture and recognize the contents of the picture 21 with the image capturing unit 11 that is a built-in camera. However, when the projector 10 illuminates the picture 21 of the paper medium with the entire projection display as a white display, the viewer watching the screen suddenly becomes brighter because the screen on which the presentation material was copied becomes brighter. You will feel dazzled or dazzled.
Therefore, the projector 10 executes the processing of FIG. 3 described later when the paper medium picture 21 is inserted into the projection range parallel to the screen.

FIG. 3 is a flowchart illustrating an example of processing (information processing) in the first embodiment.
In S100, the object detection unit 16 captures a projection range at a predetermined interval (hereinafter referred to as “periodically”) via the image capturing unit 11, and acquires coarse image data (hereinafter simply referred to as an image). Then, the object detection unit 16 records the image acquired via the image capturing unit 11 in the storage unit 12. In addition, the rough image here means an image having a resolution lower than a predetermined resolution.
Note that the object detection unit 16 may acquire an image thinned out by the image sensor itself. Further, the object detection unit 16 reads out all the pixels with an image sensor, and then obtains a rough image generated by performing a process of roughening the image such as a reduction process, a process of extracting only the luminance component, or a process of DCT conversion. It may be. DCT is an abbreviation for Discrete Course Transform. In addition, the object detection unit 16 captures all the pixel images via the image capturing unit 11, and then the video analysis unit 13 performs the above-described processing for roughening the image on the captured all pixel images. Also good.

In step S <b> 101, the video analysis unit 13 analyzes and monitors the movement (change) of an object that appears in an image from a video composed of images that are periodically recorded in the storage unit 12 using a background difference method.
In S102, the video analysis unit 13 determines whether there is a moving object in the projection range from the analysis result in S101. If it is determined that the moving object exists, the process proceeds to S103. If it is determined that there is no moving object, the process is performed. Return to S101. If the video analysis unit 13 determines that a moving object is present in the projection range, the video analysis unit 13 detects the movement and size of the target object from the analysis result, and notifies the object detection unit 16 of the detection result. As described above, the video analysis unit 13 can determine whether or not an imaging target exists in the projection range based on whether or not a moving object exists in the projection range.
In S103, the object detection unit 16 analyzes the shape of the object from the video, determines whether or not the object is a rectangular shape, and proceeds to S104 if it is determined that the object is a rectangular shape. If it is determined that the shape is not rectangular, the process returns to S101. In S103, the object detection unit 16 determines based on whether or not the shape of the object is a rectangular shape. However, the object detection unit 16 may determine based on another predetermined shape. .

In S104, the motion detection unit 17 tracks the movement of the rectangular object that is the target from the analysis result by the video analysis unit 13, determines whether or not the rectangular object is stationary, and is stationary. If so, the process proceeds to S105. On the other hand, when the motion detection unit 17 determines that the rectangular object is not stationary, the process of S104 is repeated. As a result, the projector 10 can capture an image of the rectangular object in a stationary state, so that the image can be captured in focus.
In S105, the material estimation unit 15 estimates the material of the rectangular object. Here, the material estimation unit 15 estimates the material of the rectangular object using an estimation method such as object surface material estimation based on continuous images using a reflection model that is a known technique. More specifically, the material estimation unit 15 includes luminance information acquired from a plurality of consecutive captured images of the rectangular object stored in the storage unit 12 and reflection model information stored in advance. Analyze it and estimate the material.
In S106, the object detection unit 16 determines whether or not the rectangular object is a self-luminous object that emits light spontaneously from the estimation result in S105. If it is determined that the rectangular object is not a self-luminous object, the process proceeds to S107. The details of the processing when the object detection unit 16 determines in S106 that the rectangular object is a self-luminous object will be described in Embodiment 3. An example of the self-luminous object here is a liquid crystal tablet or the like. An example of a non-self-luminous object that does not emit light is the paper medium picture 21 described above.

In S107, the video output unit 18 adds a margin of a predetermined width to the periphery of the rectangular object based on the shape and size of the rectangular object analyzed by the background difference method described above. Projection data for projecting a white display (hereinafter referred to as a rectangular white display) is created. More specifically, the video output unit 18 creates projection data for projecting a rectangular white display 31 as shown in FIG. FIG. 4 is a diagram illustrating an example of the projection display.
In S108, the video output unit 18 synthesizes the rectangular white display indicated by the projection data newly created in S107 into a projection display indicating the original presentation content. That is, the video output unit 18 changes and outputs the original projection display based on the projection data created in S107. Thereby, the video output unit 18 can change the projection display at the projection position of the imaging target in the projection range according to the imaging target. Moreover, the user's eyes are not dazzled by glare as compared with the case where the video output unit 18 changes the display of the entire projection range to white.

In S109, the partial exposure adjustment unit 20 adjusts the brightness of the rectangular white display.
The partial exposure adjustment unit 20 cuts out the portion of the rectangular white display 31 in FIG. 4 from the image captured by the image capturing unit 11 and calculates a partial histogram. The partial exposure adjustment unit 20 compares the extracted information obtained by extracting the histogram features of the partial histogram with the stored storage information of the histogram features learned and stored in the storage unit 12 in advance, and the partial exposure adjustment unit 20 is close to overexposed or underexposed. Judgment is made on the exposure. Then, based on the result of the exposure determination, the partial exposure adjustment unit 20 projects the video output unit 18 so that the white brightness of the portion of the rectangular white display 31 in FIG. Automatically adjusts the whiteness of the projected display by applying feedback. Thereby, the partial exposure adjustment part 20 can adjust appropriately the brightness by the white projection display with respect to exposure of the captured image when the said rectangular object is zoomed and imaged by the image imaging part 11. FIG. That is, the projector 10 can acquire a captured image that is easier for the user to visually recognize via the image capturing unit 11.
The partial exposure adjustment unit 20 adjusts the self-tuning of the white brightness of the projection display output from the video output unit 18 based on the reflected luminance value acquired from the estimated reflectance table of the paper medium. May be.

In S110, the object detection unit 16 zooms the image capturing unit 11 according to the rectangular object, and focuses the rectangular object.
In S <b> 111, the object detection unit 16 captures a still image of the rectangular object that is the target by the image capturing unit 11.
In S112, the video output unit 18 stops the output of the combined projection display after being changed in S108.
In S113, the video output unit 18 projects the captured image of the still image captured in S111 on the entire projection screen of the projector 10. Thereby, the video output unit 18 can project and display an imaging target imaged appropriately.

  As described above, according to the present embodiment, the projector 10 can determine whether or not an imaging target exists in the projection range, and partially changes the projection display according to the material (characteristic) of the imaging target. be able to. Thereby, the projector 10 can appropriately capture an imaging target and project a captured image with good visibility without dizzying the user's eyes.

<Embodiment 2>
FIG. 5 is a diagram illustrating an example of a hardware configuration of the projector 10 according to the present embodiment.
Only the differences from the hardware configuration of the projector 10 described in FIG. 1 will be described below.
The distance sensor unit 14 measures the distance from the projector 10 to the object and creates depth information. Here, the depth information refers to information indicating a range in which the projector 10 can capture a captured image with a focus that is allowed via the image capturing unit 11.
In the present embodiment, a detection method based on a depth information analysis method is used as a detection method for moving object detection by the video analysis unit 13. In addition, a sensor using various distance measuring methods can be used as the distance sensor unit 14.

The projector 10 according to the present embodiment captures the situation where an infrared dot pattern other than visible light, for example, is irradiated with an infrared camera as the image capturing unit 11 and analyzes the image.
Hereinafter, an example of a depth information analysis method in which the distance sensor unit 14 extracts depth information will be described.
It is assumed that the projector 10 according to the present embodiment and the image pickup apparatus that picks up images with visible light connected to the image pickup unit 11 are in a substantially integrated position. Therefore, the distance between the image capturing apparatus and the imaging target can be regarded as the same distance that can be measured by the distance sensor unit 14.
Here, similarly to FIG. 2 described in the first embodiment, the processing of the projector 10 in the case where the user holds the picture 21 of the paper medium in the hand 22 and inserts it in parallel on the projection surface of the projector 10 will be described with reference to FIG. I will explain.

FIG. 6 is a flowchart illustrating an example of processing in the second embodiment.
In S <b> 200, the distance sensor unit 14 periodically creates a depth information map (hereinafter referred to as a depth map) as shown in FIG. 7 and records it in the storage unit 12. FIG. 7 is a diagram illustrating an example of a depth map. More specifically, the distance sensor unit 14 represents the difference in distance from the projector 10 to each of the projector screen 211, the inserted rectangular object 212, and the user's finger 213 supporting the rectangular object 212. The depth map 210 expressed by the above is created. Then, the distance sensor unit 14 sequentially records the created depth map in the storage unit 12.
In step S <b> 201, the video analysis unit 13 monitors the movement in the depth map periodically recorded in the storage unit 12. More specifically, the video analysis unit 13 monitors the movement (change) in the depth map from a plurality of depth maps periodically recorded in the storage unit 12.
In S202, the video analysis unit 13 determines whether there is a moving object in the projection range from the monitoring result in S201. If it is determined that the moving object exists, the process proceeds to S203. If it is determined that there is no moving object, the process is performed. Return to S201. When the video analysis unit 13 determines that a moving object exists in the projection range, the video analysis unit 13 notifies the object detection unit 16 that a moving object exists. Thereby, the video analysis unit 13 can determine whether or not an imaging target exists in the projection range based on whether or not a moving object exists in the projection range.

In S203, the object detection unit 16 analyzes the shape of the moving object from the depth map recorded in the storage unit 12, determines whether the moving object has a rectangular shape, and determines that the moving object has a rectangular shape The process proceeds to S204. On the other hand, if the object detection unit 16 determines that the moving object is not rectangular, the process returns to S201. In S203, the object detection unit 16 makes the determination based on whether or not the moving object has a rectangular shape. However, the object detection unit 16 may make the determination based on another predetermined shape.
In S204, when the motion detection unit 17 detects a predetermined operation (gesture) such as a case where the user makes the moving object stand still after shaking vertically, the process proceeds to S205, and when the detection is not detected, The process ends.
The processing from S205 to S213 is the same as the processing from S105 to S113 in FIG. However, when projecting a partial white display, the video output unit 18 uses information on the size and shape of the rectangular object analyzed by the above-described depth information analysis method instead of the background difference method described in the first embodiment. Use.

  As described above, according to the present embodiment, the projector 10 can determine whether or not an imaging target exists in the projection range based on the depth map, and partially according to the material (characteristic) of the imaging target. The projection display can be changed. Thereby, the projector 10 can appropriately capture an image of the imaging target and project a captured image with good visibility without dizzying the user's eyes.

<Embodiment 3>
FIG. 8 is a diagram illustrating an example of a hardware configuration of the projector 10 according to the present embodiment.
Only the differences from the hardware configuration of the projector 10 described in FIGS. 1 and 2 will be described below.
The video output unit 18 projects an appropriate projection display according to the size and material of the object on the position (projection position) of the rectangular object that is the object detected by the object detection unit 16. When the rectangular object is a self-luminous object that emits light such as a liquid crystal tablet, the video output unit 18 in the present embodiment projects a black projection display (hereinafter referred to as partial black display) at the projection position of the rectangular object. To do. At that time, the video output unit 18 adds a margin of a predetermined width to the peripheral part of the rectangular object and outputs it in a partial black display.
The hand detection unit 19 detects the human hand part and the movement of the hand part from the depth map described in the second embodiment. The hand detection unit 19 may be configured by software. That is, the function of the hand detection unit 19 may be realized by the CPU of the projector 10 executing the program stored in the storage area.

In the present embodiment, a detection method based on a depth information analysis method similar to that of the second embodiment is used as a moving object detection detection method by the video analysis unit 13.
In addition, the imaging target in the present embodiment is assumed to be a self-luminous object such as a liquid crystal tablet instead of the paper medium described in the first and second embodiments.
FIG. 9 is a diagram illustrating an example of a scene assumed in the third embodiment.
FIG. 9 shows an example in which the user holds the liquid crystal tablet 91 in the hand 92 and inserts it in the projection range of the projector 10 in parallel with the screen. The self-luminous object need not be limited to the liquid crystal tablet 91, and may be an organic EL or LED backlight, a display medium with a front light, or the like.

Hereinafter, the processing of the projector 10 in the present embodiment will be described with reference to FIG.
FIG. 10 is a flowchart illustrating an example of processing in the third embodiment.
The processing from S300 to S306 is the same as the processing from S200 to S206 of FIG. However, the object is assumed to be a liquid crystal tablet 91 rather than a paper medium picture 21. Further, the predetermined gesture detected by the motion detection unit 17 in S304 may be an operation of the user pinching the liquid crystal tablet or the like. Further, as described above, since the liquid crystal tablet 91 is assumed as the imaging target in the present embodiment, the object detection unit 16 determines that the imaging target is a self-luminous object in S306, and advances the processing to S307. When the imaging target is not a self-luminous object such as the liquid crystal tablet 91, the object detection unit 16 executes the processing after S107 in FIG. 3 described in the first embodiment.
In S307, the video output unit 18 adds a margin of a predetermined width to the periphery of the rectangular object based on the size and shape of the rectangular object analyzed by the depth information analysis method. Projection data for projecting a black display is created. Thereby, the video output unit 18 can improve the imaging environment by projecting a black projection display to an imaging target that emits light like the liquid crystal tablet 91.
In S308, the video output unit 18 synthesizes the rectangular black display indicated by the projection data created in S307 into a projection display indicating the original presentation content.
The processing from S309 to S312 is the same as the processing from S210 to S213 in FIG.

  As described above, according to the present embodiment, when there is an imaging target in the projection range, the projector 10 appropriately displays a partial projection according to whether or not the imaging target self-lights, that is, the characteristics of the imaging target. (Color) can be changed. Thereby, the projector 10 can appropriately capture an image of the imaging target and project a captured image with good visibility without dizzying the user's eyes.

<Other embodiments>
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.
In addition, as a moving object detection method in each of the above-described embodiments, a method based on a background difference method or a depth information analysis method is used, but any method that can detect a moving object such as a motion vector analysis method may be used. It may be the method.

  As mentioned above, according to each embodiment mentioned above, the technique for the projector which has an imaging part to image an imaging target appropriately can be provided.

  The preferred embodiment of the present invention has been described in detail above, but the present embodiment is not limited to the specific embodiment, and various modifications can be made within the scope of the gist of the present invention described in the claims.・ Change is possible.

Claims (11)

A projector having an imaging unit,
Determination means for determining whether or not an imaging target exists in the projection range of the projector;
Changing means for changing the projection display projected to the imaging target among the projection displays projected from the projector when the determination means determines that the imaging target is present in the projection range;
An imaging unit that images the imaging target via the imaging unit in a state where the projection display changed by the changing unit is projected on the imaging target;
Projector.   The projector according to claim 1, wherein the determination unit determines whether an imaging target exists in the projection range based on whether a moving object exists in the projection range. It further has an acquisition means for acquiring the imaging data of the projection range at a predetermined interval via the imaging unit,
The said determination means recognizes the said moving body as the said imaging target, and determines with the said imaging target existing in the said projection range, when the presence of the said moving body is detected from the change of the said imaging data acquired by the said acquisition means. Item 3. The projector according to Item 2. Further comprising creation means for creating depth information about the focus of the imaging unit in the projection range at predetermined intervals;
When the determination unit detects the presence of the moving object from the change in the depth information generated by the generation unit, the determination unit recognizes the moving object as the imaging target, and determines that the imaging target exists in the projection range. Item 3. The projector according to Item 2. Based on the imaging data of the imaging target acquired through the imaging unit, further includes a specifying unit that specifies the characteristics of the imaging target,
The said change means changes the projection display projected on the said imaging target among the projection displays projected from the said projector according to the characteristic of the said imaging target specified by the said specification means. Item 1. The projector according to item 1. The specifying unit specifies the characteristics of the imaging target based on luminance information of the imaging target acquired from the imaging data and reflection model information stored in advance,
The said change means changes the projection display projected on the said imaging target in white among the projection displays projected from the said projector, when the said imaging means specifies that the said imaging target is a characteristic which does not self-light-emit. Item 6. The projector according to Item 5. The specifying unit specifies the characteristics of the imaging target based on luminance information of the imaging target acquired from the imaging data and reflection model information stored in advance,
The said changing means changes the projection display projected on the said imaging target to black among the projection displays projected from the said projector, when the said imaging | photography target is specified as the characteristic which self-light-emitting by the said specific means. Item 6. The projector according to Item 5.   The projector according to claim 1, further comprising an adjusting unit that adjusts the illuminance of the projection display changed by the changing unit based on a histogram of illuminance related to imaging of the imaging target.   The changing unit newly creates projection data related to the projection display projected on the imaging target, and synthesizes the newly created projection data with the original projection data and projects the projection data, thereby projecting from the projector. The projector according to any one of claims 1 to 8, wherein a projection display projected on the imaging target is changed. An information processing method executed by a projector having an imaging unit,
A determination step of determining whether or not an imaging target exists in the projection range of the projector;
When it is determined in the determination step that the imaging target is present in the projection range, a change step of changing the projection display projected on the imaging target among the projection displays projected from the projector;
An imaging step of imaging the imaging target via the imaging unit in a state where the projection display changed in the changing step is projected on the imaging target;
An information processing method including: To a computer having an imaging unit,
A determination step of determining whether or not an imaging target exists in the projection range of the computer;
When it is determined in the determination step that the imaging target is present in the projection range, a change step of changing the projection display projected on the imaging target among the projection displays projected from the computer;
An imaging step of imaging the imaging target via the imaging unit in a state where the projection display changed in the changing step is projected on the imaging target;
A program for running