JP2941139B2 - Parallax image creation method and device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for creating a parallax image, and more particularly to a method and apparatus capable of determining corresponding points of a plurality of parallax images at once without performing feature point extraction processing. Performs a simple process, non-contact and passively obtains the three-dimensional shape of the measured object with occlusion, projects this three-dimensional shape on a two-dimensional plane in any direction, and creates a new parallax in any direction. BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a parallax image generating method and apparatus capable of generating an image, having extremely high versatility, and suitable for stereoscopic display in which various subjects are assumed.

[0002]

2. Description of the Related Art In recent years, various methods for displaying a three-dimensional image using a hologram and other methods have been proposed. Among these, the parallax panoramagram method of displaying a stereoscopic image by combining a large number of parallax images can observe different images depending on the observation position,
Since a natural stereoscopic effect can be obtained, it has been applied to stereoscopic vision using a holographic stereogram lenticular lens, and has received a very excellent evaluation.

However, in such a method, taking a holographic stereogram as an example, in order to obtain a natural stereoscopic effect, the number of parallax images must be as follows.
A large number of 50 to 100 parallax images are required.
When the number of the parallax images is large, the shooting of the parallax image becomes large, and it takes a long time to shoot, and the size of the shooting device is also large. For this reason, it is very difficult to shoot a fast moving subject or a large subject such as a building.

Therefore, it is conceivable to obtain a three-dimensional shape of a subject from a small number of parallax images and obtain a new parallax image based on the three-dimensional shape. The conventional stereo image method, which is one of the methods, has the following various problems in creating a parallax image.

That is, in the three-dimensional shape measurement by the stereo image method, a corresponding point on each parallax image is determined.
The position of the corresponding point is determined by the principle of triangulation. At this time,
If the corresponding point to be searched is hidden by an object in front of the corresponding point, or if so-called occlusion has occurred, the corresponding point determination process cannot be performed, and the position of the corresponding point cannot be obtained. In principle, when three or more parallax images are used, if the corresponding point is present in at least two parallax images, the position of the corresponding point can be obtained using the parallax image including the corresponding point. .

However, in the conventional method, corresponding points are searched a plurality of times two times, and processing is performed by tracking corresponding points. In the processing for occlusion, prediction of occlusion is performed in successively performed corresponding point tracking. Need to be done. In such a way,
The occlusion prediction processing involves errors and has a disadvantage that the processing becomes complicated, so that the accuracy of the three-dimensional shape measurement is deteriorated and the processing takes time.

In these methods, a feature point is generally extracted by a feature extraction process, and a process is performed using the feature point as a corresponding point. This feature extraction process needs to be changed depending on the brightness and complexity of the original image, and the versatility is impaired.

Further, when the occlusion exists in the curved surface portion, the feature points of each parallax image do not correspond, which causes an error.

[0009]

As described above, in the conventional method of creating a parallax image by the stereo image method, not only the process of extracting feature points is necessary, but also errors in determining corresponding points are accumulated. If the measured object has occlusion, the corresponding point determination process cannot be performed.
Furthermore, it is not suitable for hardware, processing is complicated,
There is a problem that the method is poor in versatility and is not a suitable method for stereoscopic display in which various subjects are assumed.

The present invention has been made in order to solve the above-mentioned problem, and it is possible to determine the corresponding points of a plurality of parallax images at a time without performing feature point extraction processing. The non-contact and passive determination of the three-dimensional shape of the object to be measured having occlusion is performed by performing simple processing, and this three-dimensional shape is projected onto a two-dimensional plane in an arbitrary direction to generate a new parallax image in an arbitrary direction. It is an object of the present invention to provide a method and an apparatus for generating a parallax image, which is extremely versatile and suitable for stereoscopic display in which various subjects are assumed.

[0011]

To achieve the above object, according to the first aspect of the present invention, in a method for creating a parallax image, an object to be measured is first photographed from different positions. From the plurality of parallax images obtained, a continuous stereo image of the measured object is created, and then the continuous stereo image is decomposed into directional components, the depth of a portion without occlusion is calculated, and the portion having the determined depth is excluded. By repeating the process of calculating the depth again from the continuous stereo image, the depth of the occlusion part is calculated, and then the depth data is reconstructed to generate three-dimensional voxel data. Direction 2
By projecting on a dimensional plane, a new parallax image in an arbitrary direction is created.

According to the second aspect of the present invention, in the apparatus for generating a parallax image, a continuous stereo image of the measured object is generated from a plurality of parallax images obtained by photographing the measured object from different positions. A continuous stereo image generating means, and a continuous stereo image generated by the continuous stereo image generating means is decomposed into directional components to calculate a depth of a portion having no occlusion, and a continuous stereo image excluding a portion having the determined depth. Depth determining means for calculating the depth of the occlusion portion by repeating the process of calculating the depth again from the image, and three-dimensional voxel data generating means for generating three-dimensional voxel data by reconstructing the depth data calculated by the depth determining means And the three-dimensional voxel data generated by the three-dimensional voxel data generation means By projecting the original plane is constituted by a parallax image producing means for producing a new parallax image at an arbitrary direction.

[0013]

Therefore, according to the parallax image generating method and apparatus of the present invention, a continuous stereo image of the measured object is generated from a plurality of parallax images obtained by photographing the measured object from different positions. The depth of the occlusion portion is calculated by repeating the process of calculating the depth again from the continuous stereo image excluding the portion where the depth is determined, excluding the portion where the depth has been determined, and calculating the depth of the occlusion portion. Is reconstructed to obtain three-dimensional voxel data. By projecting the three-dimensional voxel data onto a two-dimensional plane in an arbitrary direction, a new parallax image in an arbitrary direction is created.

Thus, the continuous stereo image is decomposed into directional components, and the depth is determined by giving a depth corresponding to the direction, so that it is not necessary to perform a feature point extraction process on the original image or the continuous stereo image. In addition, since the corresponding points of a plurality of parallax images can be determined at a time, the versatility is high and the three-dimensional shape of the measured object can be easily obtained.

Further, even if occlusion occurs, the directional component decomposition process of the continuous stereo image is repeated except for the portion where the depth is determined, so that the object to be measured is extremely versatile, simple and accurate, and This is a method suitable for creating a parallax image.

Furthermore, since only a process of decomposing a continuous stereo image into directional components and calculating a depth is required, a three-dimensional image can be obtained directly from a continuous stereo image. Can do it.

[0017]

DESCRIPTION OF THE PREFERRED EMBODIMENTS First, the concept of the present invention will be described.

Continuous stereo images (Epipolar-P)
The lane image (EPI) is obtained by arranging the horizontal cutting lines of each parallax image in the shooting order, and the trajectory of the corresponding point of the parallax image on the continuous stereo image is a straight line having an inclination corresponding to the distance from the camera. Appears as.

Therefore, the continuous stereo image is decomposed into directional components, the depth of the portion having no occlusion is obtained, and the process of obtaining the depth again from the continuous stereo image excluding the portion where the depth is determined is repeated. , The depth data is reconstructed to generate three-dimensional voxel data, and the three-dimensional voxel data is projected onto a two-dimensional plane in an arbitrary direction to form a new parallax image in an arbitrary direction. Can be created.

Hereinafter, an embodiment of the present invention based on the above concept will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing an example of the overall configuration of a parallax image creating apparatus according to the present invention. That is, as shown in FIG. 1, the parallax image creating apparatus of the present embodiment includes a plurality of (four in the figure) cameras 1 arranged at regular intervals.
1, 12, 13, 14 and continuous stereo image creating means 2
, Depth determining means 3, three-dimensional voxel data generating means 4, and parallax image creating means 5.

Here, the cameras 11, 12, 13, 14
Is to obtain a plurality of (four in the figure) parallax images by photographing the measured object 6 from different positions.

Further, the continuous stereo image creating means 2 arranges the horizontal cutting lines of the respective parallax images from the four parallax images obtained by the cameras 11, 12, 13, and 14 in the photographing order, thereby obtaining an image. A continuous stereo image of the measurement object 6 is created.

Further, the depth determining means 3 decomposes the continuous stereo image created by the continuous stereo image creating means 2 into directional components, calculates the depth of the portion without occlusion, and calculates the depth determined portion. The processing of calculating the depth again from the removed continuous stereo images is repeated to calculate the depth of the occlusion portion.

In this case, as a method of determining the depth by decomposing the directional components of the continuous stereo image, for example, processing is performed by a method using the variance of the density of a straight line on the continuous stereo image. This variance method is a process of determining the variance of the density on various straight lines on a continuous stereo image and determining that an object exists at a depth corresponding to the slope of the straight line when the variance is below a certain reference value. Is the way.

On the other hand, three-dimensional voxel data generating means 4
Is for reconstructing the depth data calculated by the depth determining means 3 to generate three-dimensional voxel data.

The parallax image generating means 5 projects the three-dimensional voxel data generated by the three-dimensional voxel data generating means 4 onto a two-dimensional plane in an arbitrary direction to generate a new parallax image in an arbitrary direction. Is what you do.

Next, a parallax image forming method in the parallax image forming apparatus of the present embodiment configured as described above will be described with reference to a flowchart shown in FIG.

First, the object to be measured 6 is photographed from different positions by the cameras 11, 12, 13, and 14, respectively.
Obtain two parallax images.

Next, a continuous stereo image on the Y-th line of the measured object 6 is generated by the continuous stereo image generating means 2 from the four parallax images obtained by the cameras 11, 12, 13, and 14 (step S1). S1, step S2).

Next, the continuous stereo image created by the continuous stereo image creating means 2 is decomposed by the depth determining means 3 into continuous stereo image direction components by taking dispersion on straight lines having various inclinations. Since the inclination and the depth of the straight line indicating the locus of the corresponding point correspond, the depth can be obtained. Here, the depth of the portion where occlusion exists cannot be determined because the object in front blocks the locus of the corresponding point. That is, in the first depth determination process, the depth of the portion without occlusion is determined (step S3).

Next, the portion whose depth is determined is removed from the previously created continuous stereo image by the continuous stereo image creating means 2 (step S5).

Then, the depth is determined by the depth determining means 3 by again taking various variances on the straight line of the inclination. At this time, the part whose depth has already been determined (the part removed by the above processing) is not used for the variance calculation. With this processing, the depth of the part hidden by the part whose depth has been determined by the first processing can be obtained (step S3).

By repeating the above steps S3 and S5, the depth of the point where occlusion occurs can also be obtained (step S4).

By calculating these processes for all horizontal lines, the depth of the measured object 6 can be obtained (steps S6 and S7).

Next, the three-dimensional voxel data generating means 4 reconstructs the depth data calculated by the depth determining means 3 to obtain three-dimensional voxel data.

Finally, the parallax image generating means 5 projects the three-dimensional voxel data generated by the three-dimensional voxel data generating means 4 on a two-dimensional plane in an arbitrary direction by using a well-known computer graphics technique. Thus, a new parallax image in an arbitrary direction can be created (step S8).

In the above description, the depth may not be determined in a portion where there is no change in density. However, when a depth error occurs in a portion where there is no change in density, pixels having the same density are not displayed on the parallax image. Only the replacement is performed, and the image quality does not deteriorate at all, and there is no particular problem.

FIG. 5 is a schematic diagram showing an example of a case where a new parallax image is created from a small number (four) of parallax images by the parallax image creating apparatus of this embodiment.

Next, a method for obtaining the depth of the occlusion portion will be described more specifically with reference to FIGS.

More specifically, as shown in FIG. 3, a continuous stereo image is scanned with straight lines having various inclinations, and the variance on the straight lines is obtained. When the variance is equal to or less than a certain reference value, it can be considered that an object exists at a depth corresponding to the direction of the straight line.

In this case, since the locus of the corresponding point is interrupted in the occlusion portion, the density change at the interrupted portion becomes large, so that the depth cannot be obtained. Therefore, as shown in FIG. 4, the portion excluding the region whose depth is determined in the above-described processing from the continuous stereo image is again scanned with various straight lines, and the region hidden by the portion whose depth has already been determined. Can be determined.

As described above, the parallax image creating apparatus according to the present embodiment obtains four parallax images by photographing the measured object 6 from different positions.
And a continuous stereo image creating means 2 for creating a continuous stereo image of the measured object 6 from the four parallax images obtained by the cameras 11, 12, 13, and 14, and the variance of the density of a straight line on the continuous stereo image The continuous stereo image created by the processing continuous stereo image creating means 2 is decomposed into directional components to calculate the depth of the portion without occlusion, and the continuous stereo image excluding the portion where the depth is determined is used. Depth determining means 3 for calculating the depth of the occlusion portion by repeating the process of calculating the depth again from the image; and three-dimensional voxel data for generating the three-dimensional voxel data by reconstructing the depth data calculated by the depth determining means 3 Generating means 4 and three-dimensional voxel data generated by three-dimensional voxel data generating means 4 in any direction. By projecting dimension plane, which is constituted from the parallax image generating unit 5 for creating a new parallax image at an arbitrary direction.

Therefore, the continuous stereo image is decomposed into directional components, and the depth is determined by giving the depth corresponding to the direction. Therefore, it is not necessary to perform the feature point extraction processing on the original image or the continuous stereo image. In addition, since the corresponding points of a plurality of parallax images can be determined at a time, the versatility is high and the three-dimensional shape of the measured object 6 can be easily obtained.

Further, since a new parallax image in an arbitrary direction can be created by projecting the three-dimensional shape of the measured object 6 on a two-dimensional plane in an arbitrary direction, the versatility is extremely high. It is suitable for stereoscopic display in which various subjects are assumed.

Further, even if occlusion occurs, the directional component decomposition processing of the continuous stereo image is repeated except for the part where the depth is determined, so that the object to be measured is extremely versatile, simple and accurate. 5, which is a method suitable for creating a parallax image.

In other words, according to the conventional method, in the portion where occlusion has occurred, the depth of the object to be measured can be accurately obtained because the object in front blocks the trajectory of the corresponding point and the trajectory of the corresponding point is interrupted. However, in this embodiment,
Since the process of decomposing the continuous stereo image into directional components and determining the depth is performed a plurality of times, and the depth is determined in order from the object in front, the depth of the portion where occlusion occurs can be obtained. In this case, since the calculation between a plurality of parallax images can be performed at once, the three-dimensional shape of the subject can be obtained by a relatively fast operation, which is a method suitable for creating a parallax image.

Furthermore, since it is only necessary to decompose the continuous stereo image into directional components and calculate the depth, a three-dimensional image can be obtained directly from the continuous stereo image, and extremely simple processing by hardware is performed. It becomes possible.

It should be noted that the present invention is not limited to the above embodiment, but can be implemented as follows.

(A) In the above embodiment, four cameras 1
The case where four parallax images of the measured object 6 are obtained by installing 1, 12, 13, and 14 has been described. However, the present invention is not limited to this. 4), the object 6 to be measured may be photographed while moving on a straight line to obtain four parallax images.

(B) In the above-described embodiment, a case has been described where the method of determining the depth by decomposing the directional component of the continuous stereo image uses the method of dispersing the density of straight lines on the continuous stereo image. However, the present invention is not limited to this. Instead, for example, a method based on a Fourier transform or a method based on a Hough transform may be used to decompose the direction component.

[0052]

As described above, according to the present invention, when creating a parallax image, a continuous stereo image of the measured object is obtained from a plurality of parallax images obtained by photographing the measured object from different positions. By decomposing this continuous stereo image into directional components, calculating the depth of the portion without occlusion, and repeating the process of calculating the depth again from the continuous stereo image excluding the portion where the depth is determined, Calculate the depth of the occlusion part, reconstruct this depth data to generate three-dimensional voxel data,
By projecting the three-dimensional voxel data onto a two-dimensional plane in an arbitrary direction, a new disparity image in an arbitrary direction is created, so that a plurality of disparity images can be handled without performing feature point extraction processing. The point can be determined at one time, and extremely simple processing by hardware is performed, and the three-dimensional shape of the object to be measured having occlusion is obtained in a non-contact and passive manner, and the three-dimensional shape is two-dimensionally determined in an arbitrary direction. It is possible to create a new parallax image in an arbitrary direction by projecting the parallax image on a plane, and it is possible to provide a parallax image creating method and apparatus which are extremely versatile and suitable for stereoscopic display in which various subjects are assumed.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of a parallax image creating apparatus according to the present invention.

FIG. 2 is a flowchart for explaining the operation in the embodiment.

FIG. 3 is an exemplary diagram for explaining a method of obtaining a depth of an occlusion portion in the embodiment.

FIG. 4 is an exemplary diagram for explaining a method of obtaining a depth of an occlusion portion in the embodiment.

FIG. 5 is an exemplary diagram illustrating an example of creating a new parallax image from a small number of parallax images in the embodiment.

[Explanation of symbols]

11, 12, 13, 14 ... camera, 2 ... continuous stereo image creation means, 3 ... depth determination means, 4 ... three-dimensional voxel data creation means, 5 ... parallax image creation means, 6 ... object to be measured.

──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Susumu Takahashi 1-5-1, Taito, Taito-ku, Tokyo Toppan Printing Co., Ltd. (72) Masahiro Yamaguchi 4-1-1-502 Kamata, Setagaya-ku, Tokyo ( 72) Inventor Katsuo Honda 1913-12 Kitahasaku-cho, Midori-ku, Yokohama-shi, Kanagawa Prefecture (72) Inventor Nagaaki Oyama 2-3-9 Kannon, Kawasaki-ku, Kawasaki-shi, Kanagawa Prefecture (72) Inventor Torou Iwata, Taito-ku, Taito-ku, Tokyo JP-A-5-37965 (JP, A) JP-A-6-74731 (JP, A) JP-A-62-25385 (JP, A) Hei 6-195447 (JP, A) (58) Fields investigated (Int. Cl. ⁶ , DB name) G06T 7/00 G01C 3/06 G06T 17/00

Claims (2)

(57) [Claims] 1. A method of creating a parallax image, comprising: creating a continuous stereo image of the measured object from a plurality of parallax images obtained by photographing the measured object from different positions; Decomposing the continuous stereo image into directional components, calculating the depth of the portion without occlusion, and repeating the process of calculating the depth again from the continuous stereo image excluding the determined portion of the depth, the depth of the occlusion portion Then, the depth data is reconstructed to generate three-dimensional voxel data. Thereafter, by projecting the three-dimensional voxel data onto a two-dimensional plane in an arbitrary direction, a new one in an arbitrary direction is obtained. A parallax image creating method, wherein a parallax image is created. 2. A device for creating a parallax image, comprising: a continuous stereo image creating means for creating a continuous stereo image of the measured object from a plurality of parallax images obtained by photographing the measured object from different positions; Decomposing the continuous stereo image created by the continuous stereo image creating means into directional components, calculating the depth of the portion without occlusion, and calculating the depth again from the continuous stereo image excluding the determined portion of the depth Depth determining means for calculating the depth of the occlusion portion by repeating the process of performing, the three-dimensional voxel data generating means for generating three-dimensional voxel data by reconstructing the depth data calculated by the depth determining means, The three-dimensional voxel data generated by the voxel data generation means is converted into a two-dimensional plane in an arbitrary direction. It projected to parallax image creating apparatus characterized by comprising and a parallax image producing means for producing a new parallax image at an arbitrary direction.