KR20240115631A - System and method for generating high-density point cloud - Google Patents

Abstract

The present invention relates to a method and system for generating high-density point cloud information. More specifically, the present invention relates to a method and system for generating high-density point cloud information. More specifically, the present invention relates to a method and system for generating high-density point cloud information, and more specifically, to generate a multi-view image from a 360-degree camera image and to reconstruct a fused depth image by backtracking the camera position through SFM. It relates to a system and method for generating point cloud information.
A high-density point cloud information generation system according to an embodiment of the present invention includes a multi-view image generator that acquires a 360-degree image of a three-dimensional space and generates a multi-view image of the three-dimensional space from the 360-degree image; a depth estimation unit that traces camera position information from the multi-view image and generates depth information of a plurality of feature points included in the multi-view image; And generating point cloud information for the three-dimensional space using a first depth map generated by applying the depth information to a patch-based stereo technique and a second depth map generated by applying the position information of the camera to a vision-based converter. It may include a point cloud generating unit.
According to an embodiment of the present invention, high-density point cloud information can be generated by constructing a multi-view image from a 360-degree camera image, estimating the depth for each pixel set, and fusing it.

Description

System and method for generating high-density point cloud information

The present invention relates to a method and system for generating high-density point cloud information. More specifically, the present invention relates to a method and system for generating high-density point cloud information. More specifically, the present invention relates to a method and system for generating high-density point cloud information, and more specifically, to generate a multi-view image from a 360-degree camera image and to reconstruct a fused depth image by backtracking the camera position through SFM. It relates to a system and method for generating point cloud information.

The 3D point cloud representation used in 3D object recognition is used as a powerful tool in fields such as autonomous driving, machine vision, and virtual reality, and various methods for this are being proposed.

Generally, the SFM (Structure from motion) algorithm is used as a method for precise 3D reconstruction. The SFM algorithm consists of the process of extracting feature points of images, matching feature points between images, and calculating the matched feature points through triangulation to restore 3D points. At this time, various types of SFM algorithms exist depending on the detailed differences in each process. However, 360-degree images have lower resolution than the information contained in the image, making it difficult to create a dense 3D point cloud.

Accordingly, patch-based depth estimation technology is being developed as a method for generating high-density 3D point clouds. However, the image matching pipeline of patch-based depth estimation generally backtracks the internal and external parameters of the camera to reduce the amount of calculation, determines the relationship between multi-views, and then estimates depth only for adjacent viewpoints, so objects are generally textured. There is a problem that information is lost due to occluded areas occurring on surfaces without teeth or complex shapes.

KR 10-2399955 B1

In order to solve the above-mentioned problems, the present invention improves the patch-based depth estimation result by fusing the patch-based depth estimation result and a vision-based converter with a 360-degree camera image as input, and multi-view images from the 360-degree camera image. The purpose is to provide a system and method for generating high-density point cloud information through hybrid depth estimation that generates, backtracks (SFM) camera position information based on feature points, and estimates depth through patch-based depth estimation and vision-based converter. Do it as

As an embodiment of the present invention, a high-density point cloud information generation system is provided.

A high-density point cloud information generation system according to an embodiment of the present invention includes a multi-view image generator that acquires a 360-degree image of a three-dimensional space and generates a multi-view image of the three-dimensional space from the 360-degree image; a depth estimation unit that traces camera position information from the multi-view image and generates depth information of a plurality of feature points included in the multi-view image; And generating point cloud information for the three-dimensional space using a first depth map generated by applying the depth information to a patch-based stereo technique and a second depth map generated by applying the position information of the camera to a vision-based converter. It may include a point cloud generating unit.

In the high-density point cloud information generation system according to an embodiment of the present invention, the multi-view image generator generates a plurality of multi-view images by slicing the 360-degree image for each of a plurality of viewpoints, and the plurality of multi-view images can be created to have an overlapping area between each viewpoint.

In the high-density point cloud information generation system according to an embodiment of the present invention, the depth estimation unit includes: a feature point extraction unit that extracts features that appear overlapping in a plurality of multi-view images as feature points; a matching pair forming unit that matches feature points included in the multi-view image to form a matching pair, and calculates a correspondence relationship between the multi-view images using a geometric relationship between the matching pairs; and a depth information generator that generates location information of the camera and depth information of the feature point using the geometric relationship between the matching pairs and the correspondence relationship between the multi-view images.

In the high-density point cloud information generating system according to an embodiment of the present invention, the point cloud generating unit includes a dividing unit that divides the first depth map and the second depth map into a plurality of pixel sets; And when the pixels included in the first depth map are divided into a pixel set in the second depth map that is different from the pixel set divided in the first depth map, the depth value of the corresponding pixel is divided into the estimated value of the first depth map and the It may further include a correction unit that corrects using the estimated value of the second depth map.

As an embodiment of the present invention, a method for generating high-density point cloud information is provided.

The method for generating high-density point cloud information according to an embodiment of the present invention includes acquiring a 360-degree image of a three-dimensional space by a multi-view image generator, and generating a multi-view image of the three-dimensional space from the 360-degree image. A multi-view image generation step; A depth estimation step in which camera position information is traced back from the multi-view image by a depth estimation unit and depth information of a plurality of feature points included in the multi-view image is generated; And using a first depth map generated by applying the depth information to a patch-based stereo technique by a point cloud generator and a second depth map generated by applying the position information of the camera to a vision-based converter, the three-dimensional space It may include a point cloud generation step in which point cloud information is generated.

In the method for generating high-density point cloud information according to an embodiment of the present invention, in the multi-view image generation step, the plurality of multi-view images are sliced for each of a plurality of viewpoints, and an overlap area between each viewpoint is formed. It can be created to have .

In the method for generating high-density point cloud information according to an embodiment of the present invention, the depth estimation step includes a feature point extraction step in which features that appear overlapping in the plurality of multi-view images are extracted as feature points by a feature point extractor; A matching pair forming step in which a matching pair is formed by matching feature points included in the multi-view image by a matching pair forming unit, and a correspondence relationship between the multi-view images is calculated using a geometric relationship between the matching pairs; And it may further include a depth information generation step in which position information of the camera and depth information of the feature point are generated by a depth information generator using a geometric relationship between the matching pairs and a correspondence relationship between the multi-view images.

In the method for generating high-density point cloud information according to an embodiment of the present invention, the point cloud generating step includes: dividing the first depth map and the second depth map into a plurality of pixel sets by a dividing unit; And when the pixel included in the first depth map is divided into a pixel set in the second depth map by the correction unit that is different from the pixel set divided in the first depth map, the depth value of the pixel is determined by the first depth map. It may further include a correction step of correcting through the estimated value of and the estimated value of the second depth map.

As an embodiment of the present invention, a computer-readable recording medium on which a program for implementing the above-described method is recorded is provided.

According to an embodiment of the present invention, high-density point cloud information can be generated by constructing a multi-view image from a 360-degree camera image, estimating the depth for each pixel set, and fusing them.

Additionally, according to an embodiment of the present invention, a robust depth map can be generated in an occluded area.

The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

Figure 1 is a block diagram of a high-density point cloud information generation system according to an embodiment of the present invention.
Figure 2 is a diagram for explaining a multi-view image generator according to an embodiment of the present invention.
Figure 3 is a block diagram of a depth estimation unit according to an embodiment of the present invention.
Figure 4 is a diagram for explaining a depth estimation unit according to an embodiment of the present invention.
Figure 5 is a block diagram of a point cloud generator according to an embodiment of the present invention.
Figure 6 is a diagram for explaining a division unit according to an embodiment of the present invention.
Figure 7 is a diagram for explaining a correction unit according to an embodiment of the present invention.
Figure 8 is a diagram for explaining the effect of the high-density point cloud information generation system according to the present invention.
Figure 9 is a flowchart of a method for generating high-density point cloud information according to an embodiment of the present invention.

Below, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts unrelated to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

The terms used in this specification will be briefly explained, and the present invention will be described in detail.

The terms used in the present invention are general terms that are currently widely used as much as possible while considering the functions in the present invention, but this may vary depending on the intention or precedent of a person working in the art, the emergence of new technology, etc. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the relevant invention. Therefore, the terms used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than simply the name of the term.

When it is said that a part "includes" a certain element throughout the specification, this means that, unless specifically stated to the contrary, it does not exclude other elements but may further include other elements. Additionally, terms such as “unit” and “module” used in the specification refer to a unit that processes at least one function or operation, which may be implemented as hardware or software, or as a combination of hardware and software. In addition, when a part is said to be "connected" to another part throughout the specification, this includes not only the case where it is "directly connected," but also the case where it is connected "with another element in between."

Hereinafter, the present invention will be described in detail with reference to the attached drawings.

Figure 1 is a block diagram of a high-density point cloud information generation system 10 according to an embodiment of the present invention.

Referring to FIG. 1, the high-density point cloud information generation system 10 according to an embodiment of the present invention will include a multi-view image generator 100, a depth estimation unit 200, and a point cloud generator 300. You can.

Figure 2 is a diagram for explaining the multi-view image generator 100 according to an embodiment of the present invention.

Referring to FIG. 2, the multi-view image generator 100 acquires a 360-degree image of a three-dimensional space and generates a multi-view image of a three-dimensional space from the 360-degree image.

At this time, the 3-dimensional space may be a space composed of one or more 3-dimensional objects. For example, the three-dimensional space may include not only outdoor spaces such as roadsides and parks, but also indoor spaces such as inside apartments and buildings.

A 360-degree image may be an image taken in all directions of 360 degrees in a three-dimensional space based on the horizontal direction. For example, a 360-degree image may be a panoramic image as shown in FIG. 2(a). At this time, the 360-degree image may be a spherical panoramic image or a cylindrical panoramic image, but is not necessarily limited thereto.

Meanwhile, a 360-degree image may be an image captured by a 360-degree camera, or may be a combination of multiple images of a three-dimensional space captured by one or more cameras. However, the above-described 360-degree image generation method is only an example, and 360-degree images may be generated through various methods.

Depending on the embodiment, a plurality of multi-view images may be generated by slicing a 360-degree image for each of a plurality of viewpoints, as shown in FIG. 2(b). At this time, a plurality of multi-view images may be generated to have overlapping areas between each viewpoint.

Preferably, the overlap area between the plurality of multi-view images may be 35% of the total multi-view image.

Figure 3 is a block diagram of the depth estimation unit 200 according to an embodiment of the present invention.

The depth estimation unit 200 backtracks the position information of the camera from the multi-view image and generates depth information of a plurality of feature points included in the multi-view image.

Depending on the embodiment, the depth estimator 200 may backtrack the location information of the camera from the multi-view image using the SFM algorithm. At this time, the SFM (Structure From Motion) algorithm is an algorithm that uses the motion information of a two-dimensional image to trace the position and direction of the camera and then structures the relationship between the image and the camera.

At this time, the pipeline of the SFM algorithm can largely have a Correspondence Search step to find feature-based correspondences, and an Incremental Reconstruction step to reconstruct the depth by estimating and correcting the camera position.

As shown in FIG. 3, the depth estimation unit 200 according to an embodiment of the present invention will include a feature point extraction unit 210, a matching pair forming unit 220, and a depth information generating unit 230. You can.

The feature point extraction unit 210 extracts features that appear overlapping in a plurality of multi-view images as feature points.

As shown in FIG. 4, a plurality of multi-view images are created with overlapping areas between each viewpoint, so the appearance of a specific object may appear overlapping in multiple multi-view images.

That is, the feature point extractor 210 may extract features (e.g., bed head, etc.) for a specific object that appear overlappingly in each multi-view image as feature points (a to e in FIG. 4).

At this time, SIFT, which is particularly robust to the size and rotation of the image and has immutable features, may be used for the feature points extracted from each image.

The matching pair forming unit 220 matches feature points included in the multi-view image to form a matching pair, and calculates a correspondence relationship between the multi-view images using the geometric relationship between the matching pairs.

As shown in FIG. 4, a matching pair can be formed by matching feature points extracted through the feature point extractor 210 for the same object. For example, in Figure 4, feature points a and c correspond to the left edge of the bed head, forming one matching pair, and feature points b, d, and e correspond to the right edge of the bed head, forming one matching pair. can be formed. Depending on the embodiment, the matching pair forming unit 220 may increase the matching rate through RANSAC.

At this time, the geometric relationship between matching pairs (Epipolar Geometry) represents the relationship between matching pairs formed when the same object is acquired from different viewpoints.

The depth information generator 230 generates camera position information and depth information of the feature point using the geometric relationship between matching pairs and the correspondence relationship between multi-view images.

Depending on the embodiment, the depth information generator 230 may generate depth information of the matching pair by utilizing the geometric relationship (Epipolar Geometry) and triangulation between the matching pairs. At this time, if there are more than 9 matching pairs in the two multi-view images, the corrected translation, rotation, and depth can be obtained through RANSAC.

That is, the depth estimation unit 200 according to an embodiment of the present invention extracts a specific point from each multi-view image, calculates the correspondence between the multi-view images by matching the relationship between the feature points, and calculates the correspondence between the multi-view images. Use to determine the location of the camera.

Figure 5 is a block diagram of the point cloud generator 300 according to an embodiment of the present invention.

The point cloud generator 300 uses a first depth map generated by applying depth information to a patch-based stereo technique and a second depth map generated by applying the position information of the camera to a vision-based converter in the three-dimensional space. Generate point cloud information for

According to the embodiment, the point cloud generator 300 generates a segmentation (class) for two pairs of depth maps, corrects the depth map for each class, and then fuses each class to generate the final depth map. It can be implemented.

The patch-based stereo (MULTI-VIEW STEREO WHICH SEMANTIC PRIORS) technique is a technique used to reconstruct high-density 3D images, starting from a randomly initialized position from a combination of matching pairs of sparse feature points obtained through the SFM algorithm. This is a technology that repeatedly calculates depth through spatial propagation.

Vision Transformer (ViT) is a depth estimation technology using artificial intelligence and includes an encoder and decoder.

At this time, the vision-based converter divides the input image into patches, converts them into one time series data, transfers the divided patches to linear embedding, and uses the resulting image as the input value of the transformer. In other words, each patch is considered a token of NLP.

In other words, the point cloud generator 300 according to the present invention corrects the occluded area created in the image generated by the patch-based stereo technique using the image generated through the vision-based converter, thereby causing the viewpoint mismatch between the multi-view images. Occurring occluded areas or hole areas can be interpolated.

Referring to FIG. 5, the point cloud generator 300 according to an embodiment of the present invention may be configured to further include a division unit 310 and a correction unit 320.

The division unit 310 divides the first depth map and the second depth map into a plurality of pixel sets. For example, as shown in FIG. 6, the division unit 310 divides objects in the first depth map and the second depth map into a plurality of meaningful units ((a), (b), and (c) in FIG. 6. ), (d)).

When a pixel included in the first depth map is divided into a pixel set different from the pixel set classified in the second depth map, the correction unit 320 calculates the depth value of the pixel into the estimated value of the first depth map and the second depth. Calibrate using the estimated value of the map.

For example, as shown in FIG. 7, the correction unit 320 compares the pixel set of the first depth map (FIG. 7(a)) and the pixel set of the second depth map (FIG. 7(b)) to determine the difference. If there is, correction may be performed on the corresponding pixel set. At this time, the correction unit 320 may perform correction on the entire pixel set or on the portion corresponding to the difference (i.e., a specific pixel) among the pixel set.

Depending on the embodiment, the correction unit 320 may, for a specific pixel set, calculate the pixel value of the first depth map (FIG. 7(a)) and the pixel value of the second depth map (FIG. 7(b)) belonging to the pixel set. It can be implemented as a model that minimizes the sum of the residuals.

At this time, the correction unit 320 uses pixel values of the first depth map (FIG. 7(a)) and the second depth map (FIG. 7(a) and (b)) to fuse the first and second depth maps (FIG. 7(a) and (b)). Model that minimizes the pixel value of Figure 7(b)) ( ) is estimated. That is, the residual between the pixel values of the first depth map and the second depth map for each pixel set ( ) Estimate the parameters (a,b) that minimize the sum of ).

At this time, a model that minimizes the sum of residual pixel values can be implemented as shown in [Equation 1] below.

(Here, r is , d is the estimated value of the first depth map, d' is the estimated value of the second depth map.)

When the number of pixels in the pixel set is n, the parameters a and b that minimize the difference between the two depth maps (d, d') are expressed as [Equation 2] below.

If the above [Equation 2] is expressed as a determinant, it becomes the following [Equation 3].

X in [Equation 3] is obtained as in [Equation 4] below.

That is, the point cloud generator 300 according to the present invention calculates parameters for each pixel set, approximates the difference for each pixel set, and fuses them with the first depth map to create the final depth map (FIG. 7(c)). can be created.

Figure 8 is a diagram for explaining the effect of the high-density point cloud information generation system 10 according to the present invention.

Figure 8(a) is a 3D image generated through point cloud information obtained using only a patch-based stereo technique, and Figure 8(b) is a 3D image generated through point cloud information obtained using only a vision-based converter. Figure 8(c) is a 3D image generated through point cloud information obtained using the high-density point cloud information generation system 10 according to the present invention.

Referring to FIG. 8(a), it can be seen that when point cloud information is acquired using only the patch-based stereo technique, a number of occluded areas occur in the 3D image. Referring to FIG. 8(b), it can be seen that when point cloud information is acquired using only a vision-based converter, the accuracy of the 3D image deteriorates.

On the other hand, referring to Figure 8(c), it can be seen that when point cloud information is generated by fusing depth values obtained through a patch-based stereo technique and a vision-based converter, it is possible to generate a high-density 3D image that is robust to occluded areas. .

That is, when point cloud information is acquired through the point cloud information generation system 10 according to the present invention, a 3D image with higher density can be generated compared to when point cloud information is acquired using only a patch-based stereo technique.

Regarding the method according to an embodiment of the present invention, the contents of the above-described system may be applied. Therefore, description of the same contents as the above-described system in relation to the method will be omitted.

Figure 9 is a flowchart of a method for generating high-density point cloud information according to an embodiment of the present invention.

Referring to FIG. 9, the method for generating high-density point cloud information according to an embodiment of the present invention may include a multi-view image generation step (S100), a depth estimation step (S200), and a point cloud generation step (S300).

In the multi-view image generation step (S100), a 360-degree image of a three-dimensional space is acquired by the multi-view image generator 100, and a multi-view image of the three-dimensional space is generated from the 360-degree image.

Depending on the embodiment, in the multi-view image generation step (S100), a plurality of multi-view images may be generated by slicing a 360-degree image for each of a plurality of viewpoints and having an overlapping area between each viewpoint.

In the depth estimation step (S200), the position information of the camera is traced back from the multi-view image by the depth estimation unit 200, and depth information of a plurality of feature points included in the multi-view image is generated.

Depending on the embodiment, the depth estimation step (S200) may further include a feature point extraction step, a matching pair forming step, and a depth information generating step.

In the feature point extraction step, features that appear overlapping in the plurality of multi-view images are extracted as feature points by the feature point extraction unit 210.

In the matching pair forming step, a matching pair is formed by matching feature points included in the multi-view image by the matching pair forming unit 220, and a correspondence relationship between the multi-view images is established using the geometric relationship between the matching pairs. It is calculated.

In the depth information generation step, the depth information generator 230 generates camera position information and depth information of the feature points using the geometric relationship between the matching pairs and the correspondence relationship between the multi-view images.

In the point cloud generation step (S300), a first depth map generated by applying the depth information to a patch-based stereo technique by the point cloud generator 300 and a second depth generated by applying the position information of the camera to a vision-based converter Using the map, point cloud information for the three-dimensional space is generated.

Depending on the embodiment, the point cloud generation step (S300) may further include a division step and a correction step.

In the division step, the first depth map and the second depth map are divided into a plurality of pixel sets by the division unit 310.

In the correction step, when the pixels included in the first depth map are classified by the correction unit 320 into a pixel set in the second depth map that is different from the pixel set classified in the first depth map, the depth value of the corresponding pixel is It is corrected using the estimated value of the first depth map and the estimated value of the second depth map.

Meanwhile, the above-described method can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. Additionally, the data structure used in the above-described method can be recorded on a computer-readable recording medium through various means. The computer-readable recording media includes storage media such as magnetic storage media (e.g., ROM, RAM, USB, floppy disk, hard disk, etc.) and optical read media (e.g., CD-ROM, DVD, etc.). Includes.

The description of the present invention described above is for illustrative purposes, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential features of the present invention. will be. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. For example, each component described as single may be implemented in a distributed manner, and similarly, components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. do.

10: Point cloud information generation system
100: Multi-view image generation unit
200: depth estimation unit
210: Feature point extraction unit
220: Matching pair forming unit
230: Depth information generation unit
300: Point cloud generation unit
310: division part
320: Correction unit

Claims (9)

a multi-view image generator that acquires a 360-degree image of a three-dimensional space and generates a multi-view image of the three-dimensional space from the 360-degree image;
a depth estimation unit that traces camera position information from the multi-view image and generates depth information of a plurality of feature points included in the multi-view image;
Generating point cloud information for the three-dimensional space using a first depth map generated by applying the depth information to a patch-based stereo technique and a second depth map generated by applying the position information of the camera to a vision-based converter. Including a point cloud generation unit,
High-density point cloud information generation system. According to claim 1,
The multi-view image generator,
The 360-degree image is sliced for each of a plurality of viewpoints to generate a plurality of multi-view images,
The plurality of multi-view images are generated to have overlapping areas between each viewpoint,
High-density point cloud information generation system. According to claim 1,
The depth estimation unit,
a feature point extraction unit that extracts features that appear overlapping in a plurality of multi-view images as feature points;
a matching pair forming unit that matches feature points included in the multi-view image to form a matching pair, and calculates a correspondence relationship between the multi-view images using a geometric relationship between the matching pairs;
It further includes a depth information generator that generates location information of the camera and depth information of the feature point using the geometric relationship between the matching pairs and the correspondence relationship between the multi-view images.
High-density point cloud information generation system. According to claim 1,
The point cloud generator,
a dividing unit that divides the first depth map and the second depth map into a plurality of pixel sets;
When pixels included in the first depth map are divided into a pixel set in the second depth map that is different from the pixel set divided in the first depth map, the depth value of the corresponding pixel is divided into the estimated value of the first depth map and the second depth map. 2 Further comprising a correction unit that corrects using the estimated value of the depth map,
High-density point cloud information generation system. A multi-view image generation step of acquiring a 360-degree image of a three-dimensional space by a multi-view image generator and generating a multi-view image of the three-dimensional space from the 360-degree image;
A depth estimation step in which camera position information is traced back from the multi-view image by a depth estimation unit and depth information of a plurality of feature points included in the multi-view image is generated;
Using a first depth map generated by applying the depth information to a patch-based stereo technique by a point cloud generator and a second depth map generated by applying the position information of the camera to a vision-based converter, in the three-dimensional space Including a point cloud generation step in which point cloud information is generated,
Method for generating high-density point cloud information. According to claim 5,
In the multi-view image generation step,
The plurality of multi-view images are,
The 360-degree image is sliced for each of a plurality of viewpoints, and is generated to have an overlapping area between each viewpoint,
Method for generating high-density point cloud information. According to claim 5,
The depth estimation step is,
A feature point extraction step in which features that appear overlapping in the plurality of multi-view images are extracted as feature points by a feature point extraction unit;
A matching pair forming step in which a matching pair is formed by matching feature points included in the multi-view image by a matching pair forming unit, and a correspondence relationship between the multi-view images is calculated using a geometric relationship between the matching pairs; and
Further comprising a depth information generation step in which position information of the camera and depth information of the feature point are generated by a depth information generator using the geometric relationship between the matching pairs and the correspondence relationship between the multi-view images,
Method for generating high-density point cloud information. According to claim 5,
The point cloud generation step is,
A division step of dividing the first depth map and the second depth map into a plurality of pixel sets by a division unit; and
If the pixel included in the first depth map is divided into a pixel set in the second depth map by the correction unit that is different from the pixel set divided in the first depth map, the depth value of the corresponding pixel is that of the first depth map. Further comprising a correction step of correcting through the estimated value and the estimated value of the second depth map,
Method for generating high-density point cloud information. A computer-readable recording medium on which a program for implementing the method of any one of claims 5 to 8 is recorded.