CN108010123A - A kind of three-dimensional point cloud acquisition methods for retaining topology information - Google Patents

Abstract

The present invention provides a three-dimensional point cloud acquisition method that retains topological information. First, the camera is used to acquire images through surround or low-altitude aerial photography, and data preprocessing such as grayscale, Gaussian denoising, and photo alignment is performed on the images; secondly, Extract and match feature points that retain topological information; then, solve the 3D point cloud and map the 2D topological relationship to 3D space, and the obtained point cloud data results can be used to build a 3D model. Compared with the current conventional sequence image-based 3D point cloud acquisition method, the present invention has the advantages of uniform point cloud distribution and self-contained 3D topological information, and can significantly improve the precision of building a 3D model.

Description

A 3D Point Cloud Acquisition Method Preserving Topological Information

technical field

The invention relates to image processing in the field of computer vision and three-dimensional point cloud reconstruction based on sequence images, in particular to a three-dimensional point cloud acquisition method that preserves topological information.

Background technique

In computer vision, 3D reconstruction refers to the process of reconstructing 3D information from single-view or multi-view images. Since the information of a single view is incomplete, 3D reconstruction needs to utilize empirical knowledge. However, multi-view 3D reconstruction (similar to human binocular positioning) is relatively easy. The method is to first calibrate the camera to obtain the internal camera parameters, and then calculate the motion parameters of the camera through the matched feature point pairs. The two can be combined. Calculate the relationship between the image coordinate system of the camera and the world coordinate system, and finally use the information in multiple two-dimensional images to reconstruct three-dimensional information.

3D point cloud acquisition is a key technology and difficulty based on multi-view 3D reconstruction. The quality of 3D point cloud determines the accuracy of subsequent 3D model construction. The existing 3D point cloud acquisition steps: image preprocessing, feature point extraction and matching, and 3D point cloud solution. The feature point extraction and matching is the most resource-consuming part, and it is also the research that relevant scholars are committed to optimization and improvement. hotspot. The existing feature point extraction algorithms include SIFT, SURF, ORB, etc. These algorithms have achieved good results in overcoming image scale and rotation changes, illumination changes, and image deformation. However, the feature points extracted by various methods currently have defects such as redundancy, uneven distribution, and lack of two-dimensional topological information, which affect the success or failure and accuracy of the subsequent construction of the three-dimensional model.

Contents of the invention

Purpose of the invention: Aiming at the limitations of the above-mentioned prior art, the purpose of the present invention is to provide a 3D point cloud acquisition method that retains the contour texture topological information, and solve the problems of point cloud data redundancy, uneven distribution and no contour texture topological information Defects are used to subsequently build a 3D model with topological constraints to improve model accuracy.

Technical solution: A 3D point cloud acquisition method that retains topological information. The main process of the method is as follows: First, use the camera to acquire images through surrounding or low-altitude aerial photography, and then grayscale the images, Gaussian denoising, photo alignment, etc. Data preprocessing; secondly, extract and match feature points that retain topological information; then, solve the 3D point cloud and map the 2D topological relationship to 3D space, and the obtained point cloud data results can be used to build a 3D model. In the process of acquiring a three-dimensional point cloud based on sequence images, the method provided by the present invention includes the following steps:

1. Camera calibration, obtain the internal parameters of the camera, and save them in the form of a matrix;

2. Obtain the image data of the target area by surrounding or low-altitude aerial photography;

3. Perform image grayscale, Gaussian denoising, and photo alignment preprocessing on the acquired image data. The specific photo alignment steps are as follows:

3.1. Use the FAST operator to extract feature points and calculate descriptors;

3.2. Use FLANN to achieve efficient feature point matching and bidirectional matching technology to reduce mismatching;

3.3. Use RANSAC to filter mismatches;

3.4. Use the feature point pair obtained in step 3.3 to solve the basic matrix by using the 8-point method, and combine the existing camera calibration matrix in step 1 to obtain the essential matrix of the image pair;

3.5. Estimate the matching images of all images using the essential matrix of the image pair obtained in step 3.4:

3.5a. By decomposing the essential matrix of the image pair into two parts of rotation and translation, the position transformation relationship between the two cameras can be obtained, and the position transformation relationship of any pair of cameras can be obtained by analogy;

3.5b. Select the first image, and according to the relative position transformation relationship between the image and other images, select the image pair with the smallest rotation and translation scale as the matching image of the image;

3.5c. Use the matching image of the first image as the second image to be matched and execute 3.5a and 3.5b, and so on to determine the matching images of all images;

3.6. Assuming that the camera matrix of the first image is fixed and standard, use the position transformation relationship between the cameras obtained in step 3.5a to obtain the camera matrix of the other image in the matching image pair, and use this to obtain the cameras of all images matrix;

4. Feature point extraction that preserves the topological relationship of the contour texture;

4.1. Use the Canny edge extraction algorithm to extract all the contour texture features of the target features in each image. The detected contour texture does not establish a hierarchical relationship. The contour texture data of each image is stored in a two-dimensional container. Each contour Texture data is saved in point data format;

4.2. Use the Douglas-Puck algorithm to simplify the data of each contour texture point of each image:

4.2a. Retain the simplified contour texture points as feature points to be matched, and mark the image and contour number to which the point belongs, and the processing result of each image is stored separately in a two-dimensional container;

4.2b. Retain the contour texture points before simplification as the feature point library for feature point matching.

5. Feature point matching and filter mismatching;

5.1. Use the SIFT operator to describe the feature points to be matched;

5.2. Feature point matching:

5.2a, select the first image, and determine the matching image of the image according to the pair of images to be matched obtained in step 3.5;

5.2b. In the matching image pair, the feature points to be matched on one image and the feature point library on the matching image use the "epipolar constraint" relationship determined by the essential matrix obtained in step 3.4 to narrow the search range;

5.2c, using FLANN to achieve efficient matching;

5.3. Use RANSAC algorithm to filter false matches.

6. Use the matching feature point pairs obtained in step 5 to solve the 3D point cloud according to the triangle method;

6.1. Approximate a three-dimensional point by two two-dimensional points in a matching image pair, that is, use the constraint relationship between the camera matrix obtained in step 3.6 and the matching feature point pair obtained in step 5.3 to solve the spatial point three-dimensional coordinates;

6.2. Perform the step 6.1 operation on the matching point pairs through a loop to realize the complete triangle method, and obtain the 3D point cloud reconstructed from the two images, which is used as the initial structure of the 3D reconstruction of the sequence image;

6.3. Add the remaining images to this initial structure one by one, that is, find the image that matches the second image in the remaining images as the third reconstructed image, and repeat step 6.2 to obtain the three-dimensional sequence image point cloud.

7. Map the two-dimensional contour texture topological information contained in the feature points obtained in step 4 to a three-dimensional point cloud, and transform the unorganized three-dimensional point cloud into a classifiable three-dimensional point cloud with contour texture topological information.

7.1. During the calculation process of the triangle method in step 6.1, when calculating the coordinates of a 3D point through the two 2D matching points in the two images, the image and contour information to which the two feature points belong are retained in the calculated 3D point information, that is, the mapping from two-dimensional topological information to three-dimensional topological information is completed;

7.2. In the 3D point cloud obtained in step 7.1, each 3D point includes which two images the point comes from and its specific contour number on the image, and the point cloud can be classified accordingly:

7.2a. Perform primary classification on the obtained point cloud according to the image number of the 3D point;

7.2b. Perform secondary classification on the point cloud after the primary classification according to the contour number on the image corresponding to the three-dimensional point.

Beneficial effects: Compared with the traditional three-dimensional point cloud acquisition technology based on sequence images, this method supports surrounding and flat, orderly and disorderly photographing methods, and the acquired feature points are evenly distributed, and the matching efficiency of feature points is improved. The most important advantage is that this method retains 2D topological information and maps it to a 3D point cloud, which can be used to subsequently build a 3D model with topology constraints and improve model accuracy.

Description of drawings

Fig. 1 is a schematic flow sheet of the inventive method;

Fig. 2 is the three-dimensional point cloud of the reconstructed small house of conventional three-dimensional point cloud acquisition method;

Fig. 3 is the three-dimensional point cloud of the small house reconstructed by the three-dimensional point cloud acquisition method that retains topological information provided by the present invention;

Figure 4(a) is an unconstrained point cloud network;

Figure 4(b) is a schematic diagram of contour texture constraint information;

Figure 4(c) is a point cloud network with contour texture constraints.

Detailed ways

FIG. 1 shows the main flow of a method for acquiring a 3D point cloud that retains topological information in the present invention. Through the three-dimensional point cloud acquisition method that retains topological information provided by the present invention, the obtained three-dimensional point cloud data can be used for subsequent construction of a three-dimensional model with topology constraints to improve model accuracy. Taking the acquisition of 3D point cloud data of a house as an example, the following steps are described in detail in conjunction with Figure 1:

1. Camera calibration, obtain the internal parameters of the camera, and save them in the form of matrix K;

2. Obtain the image data of the target area through surrounding photography;

3. Perform image grayscale, Gaussian denoising, and photo alignment preprocessing on the acquired image data. The specific photo alignment steps are as follows;

3.1. Use the FAST operator to extract feature points (it is recommended to set the Fast operator threshold to 20, and set the maximum number of extraction points to 1000), and calculate the descriptor;

3.2. Use FLANN to achieve efficient feature point matching and bidirectional matching technology to reduce mismatching;

3.3. Use RANSAC to filter mismatches;

3.4. Use the feature point pair obtained in step 3.3 to solve the basic matrix F by using the 8-point method, and combine the existing camera calibration matrix K in step 1 to obtain the essential matrix E of the image pair;

3.5. Using the essential matrix E of the image pair obtained in step 3.4, estimate the matching images of all images:

3.5a. By decomposing the essential matrix E of the image pair into two parts of rotation R and translation t, the position transformation relationship between the two cameras can be obtained, and the position transformation relationship of any pair of cameras can be obtained by analogy;

3.5b. Select the first image, and according to the relative position transformation relationship between the image and other images, select the image pair with the smallest rotation and translation scale as the matching image of the image;

3.5c. Use the matching image of the first image as the second image to be matched and execute 3.5a and 3.5b, and so on to determine the matching images of all images;

3.6. Assuming that the camera matrix P ₀ of the first image is fixed and standard, use the position transformation relationship between the cameras obtained in step 3.5a to obtain the camera matrix P ₁ of the other image in the matching image pair, and obtain the camera matrix for all images;

4. Feature point extraction that preserves the topological relationship of the contour texture;

4.1. Use the Canny edge extraction algorithm to extract all the contour texture features of the target features in each image. The detected contour texture does not establish a hierarchical relationship. The contour texture data of each image is stored in a two-dimensional container. Each contour Texture data is saved in point data format;

4.2. Use the Douglas-Puke algorithm (recommended threshold is 5) to simplify the data of each contour texture point of each image:

4.2a. Retain the simplified contour texture points as feature points to be matched, and mark the image and contour number to which the point belongs, and the processing result of each image is stored separately in a two-dimensional container;

4.2b. Retain the contour texture points before simplification as the feature point library for feature point matching.

5. Feature point matching and filter mismatching;

5.1. Use the SIFT operator to describe the feature points to be matched;

5.2. Feature point matching:

5.2a, select the first image, and determine the matching image of the image according to the pair of images to be matched obtained in step 3.5;

5.2b. In the matching image pair, the feature points to be matched on one image and the feature point library on the matching image use the "epipolar constraint" relationship determined by the essential matrix obtained in step 3.4 to narrow the search range;

5.2c, using FLANN to achieve efficient matching;

5.3. Use RANSAC algorithm to filter false matches.

6. Use the matching feature point pairs obtained in step 5 to solve the 3D point cloud according to the triangle method;

6.1. Approximate a three-dimensional point by two two-dimensional points in a matching image pair, that is, use the constraint relationship between the camera matrix P obtained in step 3.6 and the matching feature point pair obtained in step 5.3 to solve the spatial point three-dimensional coordinates;

6.2. Perform the step 6.1 operation on the matching point pairs through a loop to realize the complete triangle method, and obtain the 3D point cloud reconstructed from the two images, which is used as the initial structure of the 3D reconstruction of the sequence image;

6.3. Add the remaining images to this initial structure one by one, that is, find the image that matches the second image in the remaining images as the third reconstructed image, and repeat step 6.2 to obtain the three-dimensional sequence image point cloud.

7. Map the two-dimensional contour texture topological information contained in the feature points obtained in step 4 to a three-dimensional point cloud, and transform the unorganized three-dimensional point cloud into a classifiable three-dimensional point cloud with contour texture topological information.

7.1. During the calculation process of the triangle method in step 6.1, when calculating the coordinates of a 3D point through the two 2D matching points in the two images, the image and contour information to which the two feature points belong are retained in the calculated 3D point information, that is, the mapping from two-dimensional topological information to three-dimensional topological information is completed;

7.2. In the 3D point cloud obtained in step 7.1, each 3D point includes which two images the point comes from and its specific contour number on the image, and the point cloud can be classified accordingly:

7.2a. Perform primary classification on the obtained point cloud according to the image number of the 3D point;

7.2b. Perform secondary classification on the point cloud after the primary classification according to the contour number on the image corresponding to the three-dimensional point.

The comparison between Figure 2 and Figure 3 shows that compared with the conventional method, the distribution of the obtained point cloud is more uniform, and the data of details such as the door frame is more abundant. Figure 4(a) is a point cloud network without constraints, 4(b) is a schematic diagram of contour texture constraint information, and 4(c) is a point cloud network with contour texture constraints, which embodies the preservation provided by the present invention. The point cloud data obtained by the 3D point cloud acquisition method of topological information has the advantage of constructing the surface mesh of the model, even if the model is closer to the real scene, and the model accuracy is improved.

Claims (7)

1. a kind of three-dimensional point cloud acquisition methods for retaining topology information, it is characterised in that comprise the following steps：

Step 1, camera calibration, obtain camera internal parameter, preserve with a matrix type；

Step 2, the view data for obtaining by way of taking photo by plane around formula or low latitude target area；

Step 3, the view data to acquisition pre-process；

Step 4, the feature point extraction for retaining profile texture topological relation；

Step 5, Feature Points Matching and filtering error hiding；

Step 6, using the matching characteristic point pair obtained in step 5, three-dimensional point cloud is resolved according to triangulation method；

Step 4, is obtained the two-dimensional silhouette texture topology information included in characteristic point and is mapped to three-dimensional point cloud by step 7, will be without group The three-dimensional point cloud knitted is changed into three-dimensional point cloud classifiable, with profile texture topology information.

2. the three-dimensional point cloud acquisition methods according to claim 1 for retaining topology information, it is characterised in that：The step 3 In data prediction include：Image gray processing, Gauss denoising and photo alignment.

3. the three-dimensional point cloud acquisition methods according to claim 2 for retaining topology information, it is characterised in that the photo pair Comprise the following steps together：

Step 3.1, using FAST operator extraction characteristic points, and calculate description son；

Step 3.2, realize the matching of efficient feature point and bi-directional matching technology reduction error hiding using FLANN；

Step 3.3, utilize RANSAC filtering error hidings；

Step 3.4, using the characteristic point obtained in step 3.3 to resolving basis matrixes using 8 methods, with reference to existing in step 1 Camera calibration matrix obtain image pair essential matrix；

Step 3.5, the essential matrix using the image pair obtained in step 3.4, estimate the matching image of all images：

3.5a, by the way that the essential matrix of image pair is decomposed into rotation and translation two parts, obtain the position between two cameras Transformation relation, and so on can obtain the evolution relation of any pair of camera；

3.5b, choose first image, according to the image and other image relative position transformation relations, is revolved between selection image pair Turn and translate the matching image as the image of scale minimum；

3.5c, using the matching image of first image as second image to be matched and perform 3.5a and 3.5b, and so on Determine the matching image of all images；

Step 3.6, assume that the camera matrix of first image is fixed and is standard type, using the camera obtained in step 3.5a it Between position transformation relation obtain the camera matrix of matching another image of image pair, and obtain with this camera square of all images Battle array.

4. the three-dimensional point cloud acquisition methods according to claim 1 for retaining topology information, it is characterised in that the step 4 Include the following steps：

Step 4.1, utilize all profile textural characteristics of Target scalar in the every piece image of Canny Boundary extracting algorithms extraction, inspection The profile texture of survey does not establish hierarchical relationship, and the profile data texturing per piece image is all stored in two-dimensional container, wherein often Bar profile data texturing saves as point data form；

Step 4.2, using Douglas-Pu Ke algorithms simplify each profile texture point data of every piece image：

4.2a, retain the profile Texture Points after simplifying, and as characteristic point to be matched, and marks the image and profile volume belonging to it Number, the handling result per piece image is individually stored in a two-dimensional container；

4.2b, retain the profile Texture Points before simplifying, the characteristic point storehouse as Feature Points Matching.

5. the three-dimensional point cloud acquisition methods according to claim 1 for retaining topology information, it is characterised in that the step 5 Include the following steps：

Step 5.1, utilize to be matched characteristic point progress feature description of the SIFT operators to every piece image；

Step 5.2, Feature Points Matching：

5.2a, choose first image, according to the image pair to be matched obtained in step 3.5, determines the matching image of the image；

5.2b, the characteristic point to be matched in matching image pair a, image are with matching the step of the characteristic point Cooley on image " epipolar-line constraint " relation that essential matrix determines is obtained in rapid 3.4 and reduces search range；

5.2c, using FLANN realize efficient matchings；

Step 5.3, utilize RANSAC algorithms filtering error hiding.

6. the three-dimensional point cloud acquisition methods according to claim 3 for retaining topology information, it is characterised in that the step 6 Specifically include：

Step 6.1, at one match image pair by two two-dimensional points come an approximate three-dimensional point, i.e., using in step 3.6 Restriction relation between the matching characteristic point pair obtained in the camera matrix and step 5.3 of acquisition, resolves the three-dimensional seat of spatial point Mark；

Step 6.2, by a circulation perform matching double points step 6.1 operation to realize complete triangulation method, obtains two The three-dimensional point cloud of width image reconstruction, in this, as the initial configuration of three-dimensional reconstruction of sequence image；

Step 6.3, add remaining image in this initial configuration one by one, i.e., is found in remaining image and second figure As image of the matched image as the 3rd reconstruction, step 6.2 is repeated, you can obtain the three-dimensional point cloud of sequence image.

7. the three-dimensional point cloud acquisition methods according to claim 6 for retaining topology information, it is characterised in that have in step 7 Body includes：

7.1st, in step 6.1 triangulation method solution process, one three is resolved by two two-dimentional match points in two images When tieing up point coordinates, the image belonging to two characteristic points and profile information are remained into the three-dimensional point information calculated, that is, completed Mapping of the two-dimensional topology information to three-dimensional topology information；

7.2nd, in the three-dimensional point cloud obtained in step 7.1, each three-dimensional point contains which two images the point comes from And its specific profile numbering on this image, it can classify accordingly to a cloud：

7.2a, according to the picture number of the three-dimensional point carry out first-level class to institute's invocation point cloud；

7.2b, according to the three-dimensional point correspond on the image profile numbering to after first-level class point cloud carry out secondary classification.