KR20110003891A - Space detecting apparatus for image pickup apparatus using auto focusing and the method thereof - Google Patents

Abstract

PURPOSE: A space sensing apparatus of an image pickup apparatus using an automatic focusing control for reducing a depth generation time is provided to improve the accuracy of the image data about the space sensing area by generating depth map. CONSTITUTION: An image pickup unit(110) outputs the depth information of video images and the video image of a photographed space sensing target area. An image signal processing unit(120) generates partial depth map through the depth information. An automatic focusing control unit(130) outputs a focus control signal to the image pickup unit.

Description

SPACE DETECTING APPARATUS FOR IMAGE PICKUP APPARATUS USING AUTO FOCUSING AND THE METHOD THEREOF}

The present invention relates to space sensing, and more particularly, to a space sensing apparatus and a space sensing method of an image photographing apparatus using auto focusing, which enables to detect a change of an imaging object quickly and accurately.

Analog cameras using films of the prior art are mostly replaced by digital cameras that generate and store image signals by CCD or CMOS image sensors.

Such a digital camera converts an optical signal including an image into an electrical image signal using a CCD or CMOS image sensor. Therefore, an image signal processor (ISP) is mounted to perform image processing to maintain the best image quality.

The ISP may maintain the best image quality by performing a function of compensating for distortion generated by the lens and the sensor and improving image clarity and color.

In addition, the ISP performs image signal processing for generating a 3D image and detecting a space.

In order to detect a space using a camera among the functions of the ISP, it is necessary to generate a depth map having depth information of two-dimensional image images. The depth map is generated for all regions of the video image, so as to perform spatial sensing for detecting a representation of a 3D image, movement of an object in the video image space, and the like.

However, the depth map is generated for the entire image image, and when a change of the image image occurs, the depth map must be generated again from the entire image image area. Therefore, image processing in the ISP takes a long time, and the CPU configured in the ISP itself requires high performance.

In order to solve such a delay in generating a depth map and a high-performance ISP request of the prior art, Korean Laid-Open Patent Publication No. 2009-55803 discloses a multi-view depth map generation method.

In the conventional multi-view depth map generation method, a plurality of image images and depth information are obtained using a plurality of cameras and a depth camera, respectively. Subsequently, the coordinates of the plurality of image images of the same point are estimated, and then the adjacent regions of the same point are searched and compared to obtain a shift value in each of the plurality of images of the same point to generate a depth map. It is composed.

However, even in the prior art, since a variation value is obtained after selecting an arbitrary point on a multiview image to generate a depth map for the multiview image, time for generating the depth map is required.

In addition, in the prior art, since the entire process is repeatedly performed for space sensing, the efficiency of shortening the depth map generation time is reduced when applied to space sensing.

In addition, the prior art does not provide a method of extracting a detection target region for space detection, and requires a lot of time when generating a depth map for space detection, which still requires an ISP equipped with a high-performance CPU. .

This limitation of ISP also acts as a cause of difficulty in miniaturization when implementing two-dimensional space sensing.

Accordingly, the present invention for solving the above-described problems of the prior art provides a space sensing apparatus and a space sensing method that can reduce the generation time of the depth map for the spatial sensing for the generation of a video image when taking an image.

The present invention also provides a space sensing apparatus and a space sensing method for increasing the accuracy of the target area data for space sensing.

In addition, the present invention provides a space sensing apparatus and a space sensing method for reducing the data of the space sensing target area to perform a quick and accurate space sensing even in a low performance image signal processor.

According to an aspect of the present invention, there is provided an apparatus for detecting a space using an autofocus control function. An image signal processor configured to output spatial sensing target region information of the video image, generate a partial depth map using the spatial image and the depth information of the spatial sensing target region, and generate a changed image image; And an auto focusing control unit for outputting a focus adjustment signal for focusing on a spatial sensing target area of the image signal processing unit to the image capturing unit.

The image signal processor may generate a changed image image by replacing a partial depth map generated by the space sensing target region with a global depth map of the entire imaging region of the imaging unit.

The image signal processor may include a depth map generator configured to generate a global depth map and a partial depth map using the image image and depth information input from the image pickup unit; A space sensing object setting unit which outputs the space sensing target area information to the auto focusing controller; A space sensing unit configured to sense a change in movement of the space sensing target region by using the global depth map and the partial depth map; And an image processor configured to generate a changed image image by using the image image and the partial depth map information.

In order to achieve the above object, the spatial sensing method using the auto focusing function of the present invention is a spatial sensing method using a spatial sensing device including an imaging unit, an image signal processing unit, and an auto focusing control unit. Creating a global depth map for generating a global depth map for the region; And a spatial sensing process of generating a video image including the changed portion by comparing the partial depth map of the spatial sensing target area automatically focused by the auto focusing control unit with the global depth map. do.

The space sensing method may include: a space sensing target area setting step of setting a space sensing target area of the imaging area before performing the space sensing process; And a partial depth map generation process of generating a partial depth map using the image image and the depth information of the spatial sensing target region obtained according to the auto focus control signal of the auto focusing control unit.

The space sensing target area may be set by detecting the change of the image image. In addition, the space sensing target area may be set by a user.

The space sensing target area is set as a partial area of the entire captured image area including an area in which auto focus is focused by the auto focusing control unit. Accordingly, the data for generating the depth map is reduced, and the depth map generation time is also significantly reduced.

The partial depth map may include an image image and depth information within a predetermined range including a depth of an object on which the motion change is performed.

That is, the partial depth map is generated only for the adjacent range of the depth region in which the spatial sensing object is located, thereby significantly reducing the depth map generation time and not requiring the high specification of the image signal processing unit for image processing. .

According to the present invention, a video image data of a spatial sensing target region is acquired by using an auto focus function and a depth map is generated for a partial region including the corresponding region. It increases the accuracy of the image data for the spatial sensing target area acquired for spatial sensing for generating the changed image image during image capturing.

In addition, the present invention significantly reduces the depth map generation time by removing the data unnecessary for generating the depth map by generating a depth map by acquiring only the image image of the spatial sensing target region.

In addition, the present invention shortens the time required for generating the depth map, so that a quick and accurate space detection can be performed even in a low specification image signal processing processor, thereby reducing the system cost for space detection.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

1 is a partial block diagram of a space sensing apparatus 100 according to an embodiment of the present invention.

As shown in FIG. 1, the space sensing apparatus 100 includes an image capturing unit 110, an image signal processing unit 120, and an auto focusing control unit 130.

The imaging unit 110 includes a camera having a plurality of 2D cameras and an infrared depth camera, a binocular camera, a 3D video camera, and the like.

The imaging unit 110 outputs depth information for each pixel region for generating an image image and a depth map of a spatial sensing target region to the image signal processor 120.

In addition, the imaging unit 110 outputs an image image and depth information included in a depth range of a predetermined range in the region having a changed focus under the control of the auto focusing controller 130 when the space is detected to the image signal processing unit 120. do.

The image signal processor 120 includes a depth map generator 121, a space sensing object setting unit 122, a space sensing unit 123, and an image processing unit 124.

The depth map generator 121 generates a depth map by assigning depth information for each image image input from the imaging unit 110.

In detail, the depth map generator 121 generates a depth map (hereinafter, referred to as a global depth map) for the entire image image region that is initially photographed.

The depth map generator 121 may also use the captured image and depth information within a predetermined depth range including a region to be focused according to the auto focusing controller 130 after the sensing target region is set when detecting the space. To create a depth map (hereinafter referred to as a partial depth map).

The depth map generation process of the depth map generator 121 may include various methods known in the art, such as a contrast comparison method, a method of using distance information to an imaging target by an infrared depth camera, and a stereo matching method. Can be applied.

The space sensing target setting unit 122 selects a space sensing target region according to user input information.

The method for setting a space sensing target region may be a method of setting coordinate information of a sensing target image image region included in a previously generated global depth map or a window mask including a sensing target region.

In addition, the method for setting a space sensing target region may be a method of setting a region where a change occurs by an image change sensing scheme such as edge change detection or contrast contrast as a space sensing target region.

The space detecting unit 123 detects a change in a sensing object according to the presence or absence of a depth change by comparing depth information for each image image of the partial depth map generated by the depth map generator 121 and depth information for each image image of the global depth map. do.

The image processing unit 124 changes the image image of the spatial sensing object detected by the spatial sensing unit 123 according to the depth map information from the previously photographed image image to change the image image including the change of the spatial sensing object. Create and print

The auto focusing controller 130 performs auto focus on the sensing target region set by the space sensing target setting unit 122 to focus the imaging unit 110 according to the movement of the sensing target.

The spatial sensing device 100 of FIG. 1 includes a depth including a depth at which a spatial sensing object is positioned while performing auto focus using the imaging unit 110, the image signal processing unit 120, and the auto focusing control unit 130. Spatial detection is performed to detect changes in the movement of the object by generating only a partial depth map for the depth range.

That is, the space sensing apparatus 100 significantly reduces the depth map generation data for space sensing, thereby reducing the depth map generation time required for space sensing (motion change detection). Accordingly, the image signal processing unit 120 having a low specification CPU can perform a fast and reliable space sensing.

FIG. 2 is a flowchart illustrating a process of a space sensing method applied to the space sensing apparatus 100 of FIG. 1.

As shown in FIG. 2, for sensing the space of the present invention, the space sensing target setting unit 122 designates a sensing target for motion sensing.

In this case, the motion detection target may be set as in the method of setting a window mask including the coordinate information or the detection target region of the detection target image image area included in the depth map data previously generated by the user. This can be done in a variety of ways.

Alternatively, the portion where the change is made by the edge change detection, contrast contrast, etc., in the entire image image captured may be set as the space sensing target region (spatial sensing target setting process: S10).

After the spatial sensing object is set, the image signal processor 120 uses the depth map generator 121 to display the entire image photographed using the image image input by the imaging unit 110 and data for generating the depth map. Generate a global depth map having depth information about the global map (Global depth map generation process: S20).

When the area corresponding to the image image area in which the spatial sensing target area is changed by the image change comparison method is set as the spatial sensing target area, the sympathy sensing target setting step S10 is performed by the global depth map generation step S20. It may be performed later.

As described above, after the global depth map is generated, the partial depth map is generated using the image image and the depth information within a predetermined depth range including a position focused on the spatial sensing target region.

That is, after the global depth map is generated, the image signal processing unit 120 outputs the information on the spatial sensing target area set in the sympathy sensing target setting process S10 to the auto focusing controller 130.

The auto focusing controller 130 outputs, to the imaging unit 110, a focus adjustment signal that is automatically adjusted to the sympathy detection target region according to the received spatial sensing target region information.

The imaging unit 110 captures an image image having a predetermined depth range including a position focused on a spatial sensing object according to a focus signal output from the auto focusing controller 130, and then performs an image signal processing unit 120 together with depth information. )

The image signal processor 120 generates a partial depth map including the image image and depth information photographed according to the focus of the auto focusing controller 130.

Accordingly, the time for generating the partial depth map is significantly shortened (partial depth map generation process: S30).

When the partial depth map is generated, the space sensing unit 123 compares the global depth map with the newly created partial depth and determines whether there is a change, thereby detecting a change in the position of the image pickup object in the space sensing target area. The detection is performed (space detection process: S40).

If a change occurs in the spatial sensing object in the space sensing process (S40), the image processor 124 regenerates the global depth map information by updating the changed image image and the depth information with the global depth map information. Then, a video image including the changed part is generated according to the reproduced global depth map. That is, the motion of the object is changed in the video image (motion change process: S50).

After that, when the above-described space sensing process is not finished, the process returns to the partial depth map generation process (S20) of generating a depth map for the space sensing target region again and repeats the process.

3 is a diagram illustrating a space sensing target area in which a partial depth map is generated in the space sensing process of the present invention.

As shown in FIG. 3, when only a user's hand moves within a valid range in a specific person's movement, the auto focusing controller 130 focuses on the user's hand. Accordingly, the image capturing unit 110 generates an image image and depth information of a predetermined range of depth areas including the position of the user's hand and outputs the image information to the image signal processing unit 120.

The image signal processor 120 generates a partial depth map having an image image and depth information of only a predetermined depth area including the position of the user's hand.

Therefore, the data for generating the depth map applied to the spatial sensing (motion change detection) is significantly reduced, thereby reducing the overall depth map generation time.

In addition, by acquiring a video image by focusing the spatial sensing target area by auto focusing, the accuracy of the data for generating the spatial sensing target video image and the depth map is significantly increased, thereby increasing the reliability of the spatial sensing. It also rises.

The present invention may be variously modified or changed within the scope of the technical spirit of the present invention without being limited by the embodiments of the present invention described above.

The present invention can be applied to an image photographing apparatus for generating an image image signal.

1 is a partial block diagram of a space sensing apparatus 100 according to an embodiment of the present invention,

FIG. 2 is a flowchart illustrating a process of a space sensing method applied to the space sensing apparatus 100 of FIG. 1.

FIG. 3 is a diagram illustrating an image of a space sensing target region in which a partial depth map is generated in the space sensing process of the present invention.

DESCRIPTION OF THE RELATED ART [0002]

100: space sensing device 110: imaging unit

120: image signal processing unit 130: auto focusing control unit

Claims (11)

An image capturing unit configured to output an image image of a spatial sensing target region and depth information of the image images photographed according to a focus control signal; An image signal processor configured to output spatial sensing target region information of the video image, generate a partial depth map using the spatial image and the depth information of the spatial sensing target region, and generate a changed image image; And, And an auto focusing control unit for outputting a focus control signal for focusing on a spatial sensing target area of the image signal processing unit to the image capturing unit. The method of claim 1, wherein the image signal processing unit, And a partial depth map generated by the space sensing target region to generate a changed image image in the global depth map of the entire image capturing region of the image capturing unit. The method of claim 1, wherein the image signal processing unit, A depth map generator for generating a global depth map and a partial depth map by using the image image and depth information input from the imaging unit; A space sensing object setting unit which outputs the space sensing target area information to the auto focusing controller; A space sensing unit configured to sense a change in movement of the space sensing target region by using the global depth map and the partial depth map; And, And an image processor configured to generate a changed image image by using the image image and the partial depth map information. The method of claim 1, wherein the partial depth map, And a video image and depth information within a predetermined range including depth of the object on which the motion change is performed. The apparatus of claim 1, wherein the space sensing target area is set by a user. The spatial sensing apparatus of claim 1, wherein the spatial sensing target region is set by detecting a change in the image image. A spatial sensing method of a spatial sensing device comprising an imaging unit, an image signal processing unit, and an auto focusing control unit, A global depth map generation process of generating a global depth map of the entire imaging area of the imaging unit; And, And a spatial sensing process of generating a video image including the changed part by comparing the partial depth map of the spatial sensing target area automatically focused by the auto focusing control unit with the global depth map. Spatial Sensing Method of Imaging Device Using Auto Focus. The method according to claim 7, Before performing the space detection process, A space sensing target area setting step of setting a space sensing target area of the imaging area; And, And a partial depth map generating step of generating a partial depth map using the image image and the depth information of the spatial sensing target region obtained according to the auto focusing control signal of the auto focusing control unit. Spatial Sensing Method of Imaging Apparatus Using Digital Camera. The method of claim 7, wherein the partial depth map comprises a video image and depth information within a predetermined range including a depth of an object on which the motion change is performed. The space sensing method of claim 7, wherein the space sensing target area is set by a user. 8. The method of claim 7, wherein the space sensing target area is set by detecting a change in the video image.

Images