KR101599434B1 - Space detecting apparatus for image pickup apparatus using auto focusing and the method thereof - Google Patents

Abstract

The present invention discloses a spatial sensing apparatus for an image sensing apparatus using autofocus. An image pickup unit for outputting a video image of a space sensing target area and depth information of the video images photographed according to a focus adjustment signal; An image signal processing unit for outputting spatial sensing target region information of the video image, generating a partial depth map using the video image of the spatial sensing target region and the depth information, and generating a changed video image; And an autofocusing controller for outputting a focus adjustment signal for focusing on a space sensing target area of the image signal processing unit to the image sensing unit. Therefore, the spatial sensing target image is extracted by the automatic focus adjustment, and the depth map is generated only for the spatial sensing target area of a certain depth range in order to generate the changed image image. Thus, accuracy and reliability of the spatial sensing are improved, Thereby shortening the generation time. Accordingly, it is possible to perform rapid and accurate spatial sensing without having a high-performance image signal processing processor.

Spatial sensing, depth map, autofocus, image signal processing processor

Description

TECHNICAL FIELD [0001] The present invention relates to a space sensing device and a space sensing method for an image sensing device using an auto focus control method,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to a space sensing apparatus, and more particularly, to a space sensing apparatus and a space sensing method of an image sensing apparatus using an auto focus control that enables rapid and accurate sensing of a change in an object to be sensed.

An analog camera using a conventional film is mostly replaced by a digital camera that generates and stores a video signal by a CCD or CMOS image sensor.

Such a digital camera converts an optical signal including an image into an electrical image signal using a CCD or CMOS image sensor. Therefore, an image signal processor (ISP) (Image Signal Processor) is installed to perform image processing for maintaining the image quality at the best.

The ISP compensates for distortions caused by the lens and the sensor, and improves the sharpness and color of the image. Thus, the ISP can maintain the best image quality.

In addition, the ISP performs image signal processing for generation of three-dimensional image and spatial sensing.

Among the functions of the ISP, in order to detect a space using a camera, it is necessary to generate a depth map having depth information for 2D image images. The depth map is generated for all areas of the image image, and it is possible to perform a 3D image detection, a space detection for detecting the movement of the object in the image image space, and the like.

However, the generation of the depth map is performed for the entire image, and when a change in the image occurs, the depth map is again generated from the entire image region. Therefore, it takes a long time to process images on the ISP, and the CPU configured on the ISP itself requires high performance.

In order to solve the time delay in generating the depth map and the requirement of the high-performance ISP of this prior art, Korean Patent Laid-Open Publication No. 2009-55803 discloses a method of generating a multi-viewpoint depth map.

In the conventional method for generating a multi-viewpoint depth map, a plurality of cameras and a depth camera are used to acquire a plurality of image images and depth information, respectively. Then, the coordinates in the plurality of image images for the same point are estimated, and the neighboring regions of the same point are searched and compared to obtain a variation map for each of the plurality of images for the same point to generate a depth map .

However, in the case of the above-described conventional techniques, since a variation point is obtained after selecting an arbitrary point on a multi-view image image to generate a depth map for a multi-view image, time for generating a depth map is required.

In addition, the conventional art has a problem that the efficiency of shortening the depth map generation time is deteriorated when it is applied to the space sensing because the entire processing is repeated to perform the space sensing.

In addition, the prior art does not provide a method of extracting a sensing object region for sensing a space, and requires a long time to generate a depth map for sensing space, and thus still requires an ISP equipped with a high-performance CPU .

These ISP constraints also cause difficulties in miniaturization in implementing two-dimensional spatial sensing.

SUMMARY OF THE INVENTION Accordingly, the present invention has been made to solve the above-mentioned problems occurring in the prior art, and it is an object of the present invention to provide a space sensing apparatus and a space sensing method for reducing a generation time of a depth map for space sensing for generating a video image.

In addition, the present invention provides a space sensing apparatus and a space sensing method that can increase the accuracy of object area data for space sensing.

In addition, the present invention provides a space sensing device and a space sensing method that can perform space sensing quickly and accurately even in a low-performance image signal processor by reducing data of a space sensing target area.

According to an aspect of the present invention, there is provided an apparatus for sensing a space using an auto-focus adjusting function, the apparatus comprising: an image sensing unit for outputting a sensed image of a space sensed in accordance with a focus adjustment signal and depth information of the sensed images; An image signal processing unit for outputting spatial sensing target region information of the video image, generating a partial depth map using the video image of the spatial sensing target region and the depth information, and generating a changed video image; And an autofocusing controller for outputting a focus adjustment signal for focusing on a space sensing target area of the image signal processing unit to the image sensing unit.

The image signal processing unit generates a modified image image by replacing the partial depth map generated by the space sensing target area with the global depth map for the entire imaging region of the imaging unit.

Wherein the image signal processing unit comprises: a depth map generator for generating a global depth map and a partial depth map using the image information and the depth information input from the imaging unit; A space sensing unit configured to output the space sensing target area information to the autofocusing controller; A space sensing unit sensing a change in motion of the space sensing target area using the global depth map and the partial depth map; And an image processor for generating a modified image using the image and partial depth map information.

According to another aspect of the present invention, there is provided a method of sensing a space using an auto-focus adjustment function, the method comprising: sensing a space using an image sensing unit, an image signal processing unit, A global depth map generation process that generates a global depth map for the region; And a space sensing step of comparing the partial depth map of the space sensing target area automatically focused by the auto focusing controller with the global depth map to generate a video image including the changed portion. do.

The space sensing method may include: a space sensing target area setting process of setting a space sensing target area in the sensing area before performing the space sensing process; And a partial depth map generation step of generating a partial depth map using the image information and the depth information of the space sensing target area obtained according to the auto focus control signal of the auto focusing control unit.

The space sensing target area may be set by detecting a change in the video image. Also, the space sensing target area may be set by a user.

The space sensing target area is set as a partial area of the entire shooting image area including an area to be auto-focused by the auto focusing control part. Accordingly, the data for generating the depth map is reduced, and the depth map generation time is also significantly reduced.

The partial depth map is characterized by comprising an image image and depth information within a certain range including the depth of the object on which the motion change is performed.

That is, since the partial depth map is generated only for the adjacent range of the depth region where the spatial sensing object is located, the depth map generation time is significantly reduced and the specification of the image signal processing unit for image processing does not require high specification .

The present invention is characterized in that the generation of the changed image in the image capturing is changed by acquiring the image image data of the spatial sensing target area by using the auto focus function and generating the depth map for the partial area including the corresponding area The accuracy of the image data of the acquired space sensing target area is increased to detect the space for generating the changed image image at the time of shooting the image.

Also, according to the present invention, a depth map is generated by acquiring only a video image of a space-sensing subject area, thereby eliminating unnecessary data for generating a depth map, thereby significantly reducing a depth map generation time.

In addition, the present invention shortens the time required to generate the depth map, thereby enabling rapid and accurate spatial sensing even in a low-specification image signal processing processor, thereby reducing system cost for space sensing.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

1 is a partial block diagram of a spatial sensing apparatus 100 according to an embodiment of the present invention.

1, the spatial sensing apparatus 100 includes an image sensing unit 110, an image signal processing unit 120, and an auto focusing control unit 130.

The image sensing unit 110 includes a camera having a plurality of 2D cameras and an infrared depth camera, a binocular camera, a 3D image camera, and the like.

The image sensing unit 110 outputs the image information of the spatial sensing target area and the depth information of each pixel area for generating the depth map to the image signal processing unit 120.

In addition, the image sensing unit 110 outputs image images and depth information included in a depth region of a predetermined range in a region having a changed focus to the image signal processing unit 120 under the control of the auto focusing controller 130 at the time of sensing the space do.

The image signal processing unit 120 includes a depth map generating unit 121, a space sensing unit 122, a space sensing unit 123, and an image processing unit 124.

The depth map generator 121 generates a depth map by giving depth information for each video image input from the image pickup unit 110. [

Specifically, the depth map generating unit 121 generates a depth map (hereinafter, referred to as a global depth map) for the entire image image region to be initially photographed.

The depth map generating unit 121 may also use the captured image image and the depth information in the predetermined depth range including the focused area according to the auto focusing controller 130 after the sensing target area is set at the time of sensing the space, (Hereinafter referred to as a " partial depth map ").

The depth map generation process of the depth map generation unit 121 may include various methods known in the art, such as a confidence comparison method, a method using distance information to an object to be imaged by an infrared depth camera, and a stereo matching method Can be applied.

The space sensing unit 122 selects the spatial sensing target area according to the input information of the user.

The space sensing target area setting method may be a method of setting coordinate information of the sensing target image area included in the created global depth map or a window mask including the sensing target area.

In addition, the space sensing target area setting method may be a method of setting a region where a change is made by the image change sensing method such as edge change detection, contrast control, etc. as a space sensing target area.

The space sensing unit 123 compares the depth information of the partial image of the partial depth map generated by the depth map generator 121 with the depth information of the global image of the global depth map, do.

The image processing unit 124 changes a video image of the space sensing object in which the change is detected in the space sensing unit 123 according to the depth map information in the previously captured video image, And outputs it.

The autofocusing controller 130 performs autofocus on the sensing target area set by the space sensing unit 122 and focuses the sensing unit 110 according to the movement of the sensing target.

The spatial sensing apparatus 100 of FIG. 1 performs automatic focusing using the image sensing unit 110, the image signal processing unit 120, and the auto focusing control unit 130, And generates a partial depth map for the depth range to perform spatial sensing to detect movement change of the object.

That is, the space sensing apparatus 100 significantly reduces the depth map generation data for sensing the space, thereby reducing the depth map generation time required for the spatial sensing (motion change sensing). Accordingly, the image signal processing unit 120 having a low-specification CPU can perform quick and reliable spatial sensing.

2 is a flowchart illustrating a process of a space sensing method applied to the spatial sensing apparatus 100 of FIG.

As shown in FIG. 2, in order to detect the space of the present invention, the space-space restoration unit 122 designates a detection target for motion detection.

In this case, as shown in FIG. 1, the setting of the motion sensing object may include coordinate information of the sensing object image area included in the created depth map data or a method of setting a window mask including the sensing object area Can be performed in various ways.

Alternatively, a portion where a change has occurred by a method of edge change detection, Contrast contrast, or the like may be set as a space detection target area in a whole image to be photographed (S10).

After the space sensing object is set, the image signal processing unit 120 uses the depth map generating unit 121 to generate a depth map using the image data input by the image sensing unit 110, A global depth map having depth information is generated (global depth map generation process: S20).

In the case where the area corresponding to the image image area changed by the image change comparing method is set as the space sensing target area, the symmetry sensing target setting step (S10) May be performed later.

After the global depth map is generated as described above, the partial depth map is generated using the image information and the depth information within the predetermined depth range including the focused position on the space sensing target area.

That is, after the global depth map is generated, the image signal processing unit 120 outputs information on the space sensing target area set in the sympathetic sensing target setting process S10 to the auto focusing control unit 130. [

The autofocusing controller 130 outputs a focus adjustment signal automatically adjusted to the sympathetic sensing target area to the imaging unit 110 according to the received space sensing target area information.

The image sensing unit 110 senses an image of a predetermined depth range including a focused position on a space sensing object according to a focus signal output from the auto focusing controller 130, .

The image signal processing unit 120 generates a partial depth map including a photographed image and depth information according to the focus of the autofocusing controller 130.

Accordingly, the time for generating the partial depth map is significantly shortened (partial depth map generation step: S30).

When the partial depth map is generated, the space sensing unit 123 compares the global depth map with a newly generated partial depth to discriminate whether or not there is a change, thereby detecting a space change in the space sensing target area (Space detection process: S40).

When the spatial sensing object changes in the spatial sensing step S40, the image processing unit 124 regenerates the global depth map information by updating the changed image image and the depth information to the global depth map information. And generates a video image including the changed portion according to the regenerated global depth map. That is, the motion of the object is changed in the video image (motion change process: S50).

Thereafter, when the above-described spatial sensing process is not terminated, the process returns to the partial depth map generating process (S20) for generating a depth map for the spatial sensing target area again, and the process is repeated.

3 is a view showing a space sensing target area where a partial depth map is generated in the space sensing process of the present invention.

As shown in FIG. 3, when only the user's hand moves within a valid range in the movement of a specific person, the auto focusing control unit 130 focuses on the user's hand. Accordingly, the image sensing unit 110 generates image information and depth information for a depth region including a position of the user's hand, and outputs the image image and depth information to the image signal processing unit 120.

The image signal processing unit 120 generates a partial depth map having depth information and a video image of only a predetermined depth region including the position of the user's hand.

Therefore, the data for generating the depth map applied to the spatial sensing (motion change detection) is significantly reduced, and the entire depth map generation time is shortened.

In addition, since the image sensing is performed by focusing the space sensing subject area by focusing, the accuracy of data for generating the space sensing image image and the depth map obtained for space sensing is significantly increased, Also increases.

The present invention is not limited to the above-described embodiments of the present invention, but can be variously modified or changed within the technical scope of the present invention.

The present invention can be applied to an image capturing apparatus that generates a video image signal.

1 is a partial block diagram of a spatial sensing apparatus 100 according to an embodiment of the present invention,

FIG. 2 is a flowchart showing a process of a space sensing method applied to the space sensing apparatus 100 of FIG. 1,

FIG. 3 is a view showing an image of a space sensing target area in which a partial depth map is generated in the space sensing process of the present invention.

DESCRIPTION OF THE RELATED ART [0002]

100: Space sensing device 110:

120: image signal processing unit 130: auto focusing control unit

Claims (11)

An image pickup unit for outputting a video image of a space sensing target area photographed according to a focus adjustment signal and depth information of the video images; An image signal processing unit for outputting spatial sensing target region information of the video image, generating a partial depth map using the video image of the spatial sensing target region and the depth information, and generating a changed video image; And And an autofocusing control unit for outputting a focus adjustment signal for focusing on a space sensing target area of the image signal processing unit to the image sensing unit Wherein the image signal processor comprises: After generating the partial depth map, only the partial depth map generated by the space sensing target area is compared with the global depth map for the entire imaging area of the imaging unit, and the global depth map including the changed portion obtained by the comparison And generating an image by using the updated global depth map. delete The image processing apparatus according to claim 1, A depth map generator for generating a global depth map and a partial depth map using a video image and depth information input from the imaging unit; A space sensing unit configured to output the space sensing target area information to the autofocusing controller; A space sensing unit sensing a change in motion of the space sensing target area using the global depth map and the partial depth map; And And a video processor for generating a modified video image using the video image and the partial depth map information. The method of claim 3, Wherein the partial depth map includes an image within a certain range including the object subjected to the motion change and depth information within the predetermined range. The apparatus of claim 1, wherein the spatial sensing target area is set by a user. The apparatus of claim 1, wherein the spatial sensing target area is set by detecting a change in the image. A space sensing method of a space sensing device including an image sensing unit, an image signal processing unit, and an auto focusing control unit, A global depth map generation step of generating a global depth map for the entire imaging area of the imaging unit; A space sensing target area setting step of setting a space sensing target area in the sensing area; A partial depth map generating step of generating a partial depth map using the image information and the depth information of the space sensing target area obtained according to the auto focusing signal of the auto focusing control unit; And Updating the global depth map including the changed portion obtained by comparing only the partial depth map of the space sensing target area automatically focused by the autofocusing controller on the global depth map, and using the updated global depth map And a spatial sensing process for generating a spatial image by performing a spatial filtering process. delete 8. The method of claim 7, Wherein the partial depth map includes at least one of a video image in a certain range including a depth of the object subjected to the motion change and a video image using autofocus including depth information, A method of sensing a space in an imaging device. 8. The method of claim 7, wherein the spatial sensing target area is set by a user. The method of claim 7, wherein the spatial sensing target area is detected by detecting a change in the image.

Images