CN115760898A - World coordinate positioning method for road sprinklers in mixed Gaussian domain - Google Patents

Abstract

The invention provides a world coordinate positioning method of road sprinklers under a mixed Gaussian domain, which comprises the steps of confirming the foreground and background attribution of each pixel point in an acquired image by using a mixed Gaussian distribution model with a plurality of Gaussian distributions; continuously updating and extracting the required foreground to obtain a relatively complete foreground; analyzing whether the foreground or shadow contour by using a pixel averaging method; matching the target contour with the shadow contour; pre-storing the matched object shadow profiles, updating and screening; initializing and storing the suspected tossing shadow outline template and the posture of the matched result; outputting the coordinates of the projectile obtained by tracking and positioning; and collecting three-dimensional point cloud data of the throwing object by adopting a laser radar, establishing a three-dimensional KD tree assisted by Euclidean clustering, and sorting the collected disordered three-dimensional point cloud data to form an ordered three-dimensional point cloud number so as to complete the world coordinate positioning of the road throwing object. The method can effectively improve the problem of tracking drift of the projectile under the complex illumination environment, thereby improving the detection accuracy.

Description

A World Coordinate Positioning Method for Road Throwing Objects in Mixed Gaussian Domain

technical field

The invention belongs to the technical field of traffic safety management, and in particular relates to a world coordinate positioning method for road spills in a mixed Gaussian domain.

Background technique

With the rapid development of expressway construction, road traffic volume has increased significantly, and the number of accidents caused by spilled objects has increased sharply. As an important part of modern social traffic, highways have a huge traffic flow, and safety accidents have occurred frequently in recent years, and small road spills often cause huge accidents. Therefore, there is an urgent need for an efficient road information collection and detection system that can be placed on the vehicle body to sort and judge the information collected by the sensor, to quickly track the road parabola and to achieve precise positioning in world coordinates, as a It is an effective means of locating and throwing objects on the road, taking timely countermeasures, reducing the damage of throwing objects to subsequent vehicles, ensuring the safety of drivers and passengers, reducing the occurrence of traffic accidents, saving a lot of manpower and improving efficiency, and conforming to the road management The development trend from extensive to informatization.

As the two most widely used and convenient data acquisition sensors, cameras and lidar can collect various information on the road. Nowadays, the detection and tracking of road parabolas mostly use YOLO and other detection algorithms to detect and locate the parabolic objects in the frame of the spilled objects, and use traditional tracking algorithms such as kalman, mean-shift, pipeline, etc. or target tracking algorithms based on deep learning, etc. way to track. In the actual situation, the road traffic is huge and the road conditions are extremely complex. The mirrors and windows equipped with vehicles indirectly contribute to an extremely complex detection environment through light reflection and refraction. Traditional tracking methods are difficult to adapt to such complex changes. Therefore, it is necessary to use a deep learning-based target tracking algorithm to effectively track the target.

Contents of the invention

In view of the above defects, the present invention provides a world coordinate positioning method for road spills in a mixed Gaussian domain. The invention can effectively improve the detection accuracy of the spilled objects in the complex lighting environment, and effectively improve the tracking and drifting problem of the scattered objects in the complex lighting environment. Compared with traditional tracking algorithms, the present invention can effectively improve the tracking ability of small targets.

The present invention provides the following technical solution: a method for positioning world coordinates of road spills in a mixed Gaussian domain. The method includes the following steps:

S1. Collect images through the camera, use a mixed Gaussian distribution model with multiple Gaussian distributions to confirm the foreground and background attribution of each pixel in the collected image; continuously update and detect shadow pixels, extract the required foreground, and Perform morphological processing of corrosion and expansion closed operations to obtain a relatively complete prospect;

S2, use the pixel average method to analyze the contour category at the real-time image at time t collected, so as to analyze whether it is a foreground or a shadow contour;

S3. Introduce the Matchshape operator, use the translation, scaling and rotation invariance of the Hu matrix to match the target contour and shadow contour; pre-store the shadow contour for the successful object shadow matching, and update the screening; and the suspected matching result Throwing object shadow outline template and posture initialization and saving;

S4. After the S3 step, the suspected spilled object that has been positioned is obtained, and the method of the S2 step is used to remove the shadow of the new foreground to obtain the foreground target; use the template to perform shadow matching on the target in the foreground, and complete the tracking. And output the coordinates of the spilled objects obtained by tracking and positioning;

S5. Use lidar to collect 3D point cloud data of spilled objects, establish a 3D KD tree to organize the collected disordered 3D point cloud data into ordered 3D point cloud data, and then calculate the ordered 3D point cloud formed by Euclidean distance pairs The data is clustered, and then the ordered three-dimensional point cloud data after the clustering is converted into the coordinates of the spilled objects in the two-dimensional plane by matrix transformation, and the coordinates of the scattered objects in the two-dimensional plane after the conversion are combined with the output of the S4 step. The coordinates of the sprinkled objects are fused to complete the world coordinate positioning of the sprinkled objects on the road.

Further, the mixed Gaussian distribution model of multiple Gaussian distributions in the S1 step is as follows:

与

分别为t时刻像素点的均值与协方差矩阵，其中

为高斯分布方差估计值，I为单位矩阵。Wherein, M is the number of Gaussian distributions of the mixed Gaussian distribution model, χ _T is the collection sample set in the time window T from time t, χ _T = {x ^(t) , ..., x ^(tT) }, χ _T is used to indicate that new samples are continuously added to update the background model within the time window T to adapt to complex environmental changes; BG and FG represent the background component and foreground component respectively; π _m represents the weight of M mixed models; η is the Gaussian distribution Probability density function;

and

are the mean value and covariance matrix of pixels at time t, respectively, where

is the estimated value of Gaussian distribution variance, and I is the identity matrix.

Further, during the continuous update process of the mixed Gaussian distribution model in the S1 step, performing shadow pixel detection on the image includes the following steps:

S11. Construct the discriminant of the shadow pixel point S _t (x, y):

|I _t (x, y)HB _t (x, y)H|≤τ _H

|I _t (x, y)SB _t (x, y)S|≤τ _S

Wherein, I _t (x, y) is the t time real-time image collected by the S1 step; B _t (x, y) is the B component of the t time real-time image collected by the S1 step, H, S and V Respectively, the HSV components of the image RGB collected in the step S1 at time t converted to HSV space; α is the first adjustment parameter of brightness sensitivity for shadow detection, and β is the second adjustment parameter of brightness sensitivity for shadow detection , τ _H is the first adjustment parameter of noise sensitivity for shadow detection, and τ _S is the second adjustment parameter of noise sensitivity for shadow detection;

S12. Determine whether each pixel in the detected image conforms to the shaded pixel discriminant established in step S21, and if so, mark it as a shaded pixel.

Further, in the S1 step, the update equation of the mixed Gaussian distribution model of multiple Gaussian distributions is continuously updated as follows:

为中心向量；

为t时刻实时图像的像素

符合分布模型的归属因子，最符合的分布模型归属为1，其余模型为0；

为指数级下降的包络曲线，用以限制旧数据的影响；in,

is the center vector;

is the pixel of the real-time image at time t

The attribution factor conforming to the distribution model, the most conforming distribution model is attributed to 1, and the other models are assigned to 0;

is an exponentially decreasing envelope curve to limit the influence of old data;

的马氏距离

是否小于三倍的标准差用于判别该像素是否符合现有高斯分布，判断t时刻实时图像的像素

的马氏距离

的计算公式如下：In the mixed Gaussian distribution model that continuously updates multiple Gaussian distributions, determine the pixel of the real-time image at time t

Mahalanobis distance

Whether the standard deviation is less than three times is used to judge whether the pixel conforms to the existing Gaussian distribution, and to judge the pixel of the real-time image at time t

Mahalanobis distance

The calculation formula is as follows:

σ_m为高斯分布的方差。

σ _m is the variance of the Gaussian distribution.

Further, in the S2 step, the discriminant formula for analyzing the profile category at the real-time image at time t collected by using the pixel mean method is:

Wherein, I is the total number of pixels of the real-time image collected at time t, i is the i-th pixel of the real-time image collected at time t, and _b is the target pixel value;

When P>191, it indicates that the target foreground points in the collected real-time image at time t are more than shadow points, and it is judged that the contour target of the collected real-time image at time t is the foreground contour.

Further, in the step S5, establishing a three-dimensional KD tree and organizing the collected disordered three-dimensional point cloud data to form ordered three-dimensional point cloud data include the following steps:

进行初始化分割轴，对每个维度的点云位置数据进行方差和的计算，取最大方差的维度作为分割超平面，标记为r；S51. Collect the three-dimensional unordered point cloud data T={p ₁ , p ₂ , p ₃ } collected by the lidar, where

Initialize the segmentation axis, calculate the variance sum of the point cloud position data in each dimension, take the dimension with the largest variance as the segmentation hyperplane, and mark it as r;

S52. The current point cloud position data is retrieved according to the segmentation hyperplane dimension, the median data is found, and put into the current node, and the root node corresponds to the hyperplane area of the three-dimensional space containing T;

S53. Divide the current hyperplane dimension into two sub-hyperplane dimensions, divide all values smaller than the median into the left branch hyperplane dimension, and divide all values greater than or equal to the median into the right branch hyperplane dimension;

S54, update the segmented hyperplane r obtained in the S51 step, store the points that are included in the sub-hyperplane dimension on the current root node, update the formula for the segmented hyperplane r obtained in the S51 step: r=(r +1)%3, % is the remainder calculation;

S55: Repeat step S52 in the data of the left branch hyperplane dimension obtained in the step S53 until there is no data in the subdimension, and determine to obtain the left node; the right branch hyperplane dimension obtained in the step S53 The step S52 is repeated in the data until there is no data in the subdimension, and the right node is determined to be obtained;

S56: Outputting a KD tree corresponding to the point cloud, sorting the collected unordered 3D point cloud data to form ordered 3D point cloud data according to the KD tree corresponding to the point cloud.

Further, the Euclidean distance calculation formula for clustering the formed ordered three-dimensional point cloud data in the step S5 is as follows:

Among them, _xi is the abscissa of the i-th point cloud data in the ordered 3D point cloud data, y _i is the ordinate of the i-th point cloud data in the ordered 3D point cloud data, and n is the ordered 3D point The total number of point cloud data in the cloud data.

Further, in the step S5, converting the ordered three-dimensional point cloud data after clustering into two-dimensional in-plane sprinkler coordinates by matrix transformation includes the following steps:

1) Construct the transfer matrix M, and transfer the three-dimensional coordinates (X, Y, Z) in the point cloud space of the target spills collected by the lidar to the coordinates (u, v) in the two-dimensional plane of the target spills:

Wherein, m _ab is the transfer parameter of row a and column b in the transfer matrix M, a=1,2,3; b=1,2,3,4; Z _C is the z-axis coordinate of the camera coordinate system where the camera is located;

2), the formula for calculating the coordinates (u, v) in the two-dimensional plane of the target shed according to the transfer matrix is as follows:

3) After the calculation in the step 2), a series of linear equations are obtained, calibration parameters are solved, and then the coordinates (u, v) in the two-dimensional plane of the target spill are obtained.

Further, the three-dimensional coordinates (X, Y, Z) in the point cloud space in the step 1) are to divide the collected three-dimensional data into voxels according to the voxel grid, and create a three-dimensional space with a volume of 1 cubic centimeter. Voxel grid, each voxel uses the centroid of all points in the voxel to approximate other points in the voxel after being accommodated, and its centroid is calculated as follows:

Among them, m is the total number of voxel points contained in the voxel of the three-dimensional coordinates of the detected object, x _central , y _central and z _central are the three-dimensional x-axis coordinates, y-axis coordinates and z-axis coordinates of the centroid, then All points in the voxelized voxel of the detected target object are finally represented by the centroid; in the three-dimensional coordinates (X, Y, Z) in the point cloud space, X=x _central , Y=y _central , Z=z _central .

The present invention has the following beneficial effects:

1. This method can effectively improve the accuracy of detection of spilled objects in complex lighting environments.

2. This method can effectively improve the problem of tracking and drifting of spilled objects in complex lighting environments.

3. Compared with traditional tracking algorithms, the method provided by the present invention can effectively improve the tracking ability of small targets.

4. The present invention provides a method for positioning the world coordinates of road spills in the mixed Gaussian domain. It realizes real-time identification of spilled objects and trajectory tracking by using the camera's shadow matching and tracking algorithm. The real-time positioning and map construction computing technology of lidar combines voxel filtering algorithm and European clustering algorithm to collect and process road data in real time, and through Matrix transformation, through sensor fusion, realizes precise positioning of spilled objects from two-dimensional to three-dimensional world coordinate system, and provides early warning for high-speed spilled objects, prevents secondary accidents, and reduces unnecessary losses.

5. The world coordinate positioning method of mixed Gaussian domain system road spills provided by the present invention, after collecting images through the camera and performing target positioning, adopts the technical solution of collecting three-dimensional point cloud data by laser radar for data fusion, which avoids the need to be in the actual situation. Among them, the inevitable limitations of a single sensor avoid the situation that only a single camera sensor is prone to inaccuracy, no depth information, and a limited field of view. Therefore, after the camera collects images for target positioning, the laser radar is used to collect 3D point cloud data for data fusion to further improve the robustness of the system. A multi-sensor fusion scheme is adopted to integrate time synchronization and space synchronization of different sensors to improve road throwing. object positioning accuracy.

Description of drawings

Hereinafter, the present invention will be described in more detail based on the embodiments with reference to the accompanying drawings. in:

Fig. 1 is the flow chart of step S1-S4 in the method provided by the present invention carries out the coordinate positioning of throwing object;

Fig. 2 is a schematic flow diagram of setting up a three-dimensional KD tree for disordered three-dimensional point cloud data in S5 step in the method provided by the present invention;

Fig. 3 is a schematic flowchart of clustering using Euclidean distance-assisted three-dimensional KD tree provided by the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

The present invention provides a method for positioning world coordinates of road spills in a mixed Gaussian domain. The method includes the following steps:

S1. Collect images through the camera, use a mixed Gaussian distribution model with multiple Gaussian distributions to confirm the foreground and background attribution of each pixel in the collected image; continuously update and detect shadow pixels, extract the required foreground, and The morphological processing of erosion and dilation closed operation can effectively eliminate the noise generated in the complex environment, and highlight the edge sharpening of the image to obtain a more complete foreground;

S2, use the pixel average method to analyze the contour category at the real-time image at time t collected, so as to analyze whether it is a foreground or a shadow contour;

S3. Introduce the Matchshape operator, use the translation, scaling and rotation invariance of the Hu matrix (discrete image normalization center matrix) to match the target contour and the shadow contour; pre-store the shadow contour for the successful shadow matching, and update the screening; And initialize and save the silhouette template and posture of the suspected throwing shadow of the matching result;

S4. After the S3 step, the suspected spilled object that has been positioned is obtained, and the method of the S2 step is used to remove the shadow of the new foreground to obtain the foreground target; use the template to perform shadow matching on the target in the foreground, and complete the tracking. And output the coordinates of the spilled objects obtained by tracking and positioning;

S5. Use lidar to collect 3D point cloud data of spilled objects, establish a 3D KD tree to organize the collected disordered 3D point cloud data into ordered 3D point cloud data, and then calculate the ordered 3D point cloud formed by Euclidean distance pairs The data is clustered, and then the ordered three-dimensional point cloud data after the clustering is converted into the coordinates of the spilled objects in the two-dimensional plane by matrix transformation, and the coordinates of the scattered objects in the two-dimensional plane after the conversion are combined with the output of the S4 step. The coordinates of the sprinkled objects are fused to complete the world coordinate positioning of the sprinkled objects on the road.

As a preferred embodiment of the present invention, as shown in Figure 1, the mixed Gaussian distribution model of a plurality of Gaussian distributions in the S1 step is as follows:

与

分别为t时刻像素点的均值与协方差矩阵，其中

为高斯分布方差估计值，I为单位矩阵。Wherein, M is the number of Gaussian distributions of the mixed Gaussian distribution model, χ _T is the collection sample set in the time window T from time t, χ _T = {x ^(t) , ..., x ^(tT) }, χ _T is used to indicate that new samples are continuously added to update the background model within the time window T to adapt to complex environmental changes; BG and FG represent the background component and foreground component respectively; π _m represents the weight of M mixed models; η is the Gaussian distribution Probability density function;

and

are the mean value and covariance matrix of pixels at time t, respectively, where

is the estimated value of Gaussian distribution variance, and I is the identity matrix.

As another preferred embodiment of the present invention, in order to optimize the background update mode of the mixed Gaussian distribution model, during the continuous update process of the mixed Gaussian distribution model in the S1 step, a shadow detection module is introduced to perform shadow pixels on the image Point detection, including the following steps:

S11. Construct the discriminant of the shadow pixel point S _t (x, y):

|I _t (x, y)HB _t (x, y)H|≤τ _H

|I _t (x, y)SB _t (x, y)S|≤τ _S

Wherein, I _t (x, y) is the t time real-time image collected by the S1 step; B _t (x, y) is the B component of the t time real-time image collected by the S1 step, H, S and V Respectively, the HSV components of the image RGB collected in the step S1 at time t converted to HSV space; α is the first adjustment parameter of brightness sensitivity for shadow detection, and β is the second adjustment parameter of brightness sensitivity for shadow detection , τ _H is the first adjustment parameter of noise sensitivity for shadow detection, and τ _S is the second adjustment parameter of noise sensitivity for shadow detection;

S12. Determine whether each pixel in the detected image conforms to the shaded pixel discriminant established in step S21, and if so, mark it as a shaded pixel.

Shadow detection is performed on chromaticity analysis, which is suitable for shadow detection of moving objects and has a suppressive effect on background shadows.

Further preferably, in the S1 step, the update equation of the mixed Gaussian distribution model of multiple Gaussian distributions is continuously updated as follows:

为中心向量；

为t时刻实时图像的像素

符合分布模型的归属因子，最符合的分布模型归属为1，其余模型为0；

为指数级下降的包络曲线，用以限制旧数据的影响；in,

is the center vector;

is the pixel of the real-time image at time t

The attribution factor conforming to the distribution model, the most conforming distribution model is attributed to 1, and the other models are assigned to 0;

is an exponentially decreasing envelope curve to limit the influence of old data;

的马氏距离

是否小于三倍的标准差用于判别该像素是否符合现有高斯分布，判断t时刻实时图像的像素

的马氏距离

的计算公式如下：In the mixed Gaussian distribution model that continuously updates multiple Gaussian distributions, determine the pixel of the real-time image at time t

Mahalanobis distance

Whether the standard deviation is less than three times is used to judge whether the pixel conforms to the existing Gaussian distribution, and to judge the pixel of the real-time image at time t

Mahalanobis distance

The calculation formula is as follows:

σ_m为高斯分布的方差。

σ _m is the variance of the Gaussian distribution.

As another preferred embodiment of the present invention, in the S2 step, the discriminant formula for analyzing the profile category at the real-time image at time t collected by using the pixel mean method is:

Wherein, I is the total number of pixels of the real-time image collected at time t, i is the i-th pixel of the real-time image collected at time t, and _b is the target pixel value;

When P>191, it indicates that the target foreground points in the collected real-time image at time t are more than shadow points, and it is judged that the contour target of the collected real-time image at time t is the foreground contour.

As another preferred embodiment of the present invention, as shown in Figure 2, in the step S5, establishing a three-dimensional KD tree to organize the collected disordered three-dimensional point cloud data into ordered three-dimensional point cloud data includes the following steps:

进行初始化分割轴，对每个维度的点云位置数据进行方差和的计算，取最大方差的维度作为分割超平面，标记为r；S51. Collect the three-dimensional unordered point cloud data T={p ₁ , p ₂ , p ₃ } collected by the lidar, where

Initialize the segmentation axis, calculate the variance sum of the point cloud position data in each dimension, take the dimension with the largest variance as the segmentation hyperplane, and mark it as r;

S52. The current point cloud position data is retrieved according to the segmentation hyperplane dimension, the median data is found, and put into the current node, and the root node corresponds to the hyperplane area of the three-dimensional space containing T;

S53. Divide the current hyperplane dimension into two sub-hyperplane dimensions, divide all values smaller than the median into the left branch hyperplane dimension, and divide all values greater than or equal to the median into the right branch hyperplane dimension;

S54, update the segmented hyperplane r obtained in the S51 step, store the points that are included in the sub-hyperplane dimension on the current root node, update the formula for the segmented hyperplane r obtained in the S51 step: r=(r +1)%3, % is the remainder calculation;

S55: Repeat step S52 in the data of the left branch hyperplane dimension obtained in the step S53 until there is no data in the subdimension, and determine to obtain the left node; the right branch hyperplane dimension obtained in the step S53 The step S52 is repeated in the data until there is no data in the subdimension, and the right node is determined to be obtained;

S56: Outputting a KD tree corresponding to the point cloud, sorting the collected unordered 3D point cloud data to form ordered 3D point cloud data according to the KD tree corresponding to the point cloud.

Further preferably, as shown in FIG. 3, the Euclidean distance calculation formula for clustering the formed ordered three-dimensional point cloud data in the step S5 is as follows:

Among them, _xi is the abscissa of the i-th point cloud data in the ordered 3D point cloud data, y _i is the ordinate of the i-th point cloud data in the ordered 3D point cloud data, and n is the ordered 3D point The total number of point cloud data in the cloud data.

As another preferred embodiment of the present invention, in the step S5, converting the ordered three-dimensional point cloud data after clustering into two-dimensional in-plane sprinkler coordinates through matrix transformation includes the following steps:

1) Construct the transfer matrix M, and transfer the three-dimensional coordinates (X, Y, Z) in the point cloud space of the target spills collected by the lidar to the coordinates (u, v) in the two-dimensional plane of the target spills:

Wherein, m _ab is the transfer parameter of row a and column b in the transfer matrix M, a=1,2,3; b=1,2,3,4; Z _C is the z-axis coordinate of the camera coordinate system where the camera is located;

2), the formula for calculating the coordinates (u, v) in the two-dimensional plane of the target shed according to the transfer matrix is as follows:

3) After the calculation in the step 2), a series of linear equations are obtained, calibration parameters are solved, and then the coordinates (u, v) in the two-dimensional plane of the target spill are obtained.

Collect 3D point cloud data through lidar, read the 3D coordinates (X, Y, Z) in the point cloud space of the target sprinkler acquired by the lidar, and obtain the corresponding target sprinkler in the image plane through matrix change Coordinates (u, v) in a 2D plane. When it is judged that the calculated image pixel coordinates are inside the image read by the camera, the image pixels (R, G, B) are read out and assigned to the point cloud data to form a 3D coordinate color point cloud based on the RGB camera 3D coordinate system , to realize the integration of camera and lidar.

The laser radar has completed the ROI extraction, ground segmentation, European clustering and other operations on the point cloud data, and has generated the object matrix positioning frame. The image processing has basically determined the basic position of the scattered objects, and its three-dimensional position relative to the vehicle body can be located. , to complete the positioning, according to the topological relationship between the lane where the spilled object is located and the vehicle body, it can provide clues for the next driving route of the vehicle.

Although the collected lidar data can be converted into an ordered data group after steps S51-S56. However, due to the different viewing angles when the lidar collects obstacle points each time, the coordinates of some of the collected obstacle points change greatly. The point cloud is filtered to select the region of interest. In the present invention, the orderly point cloud data collected by the laser radar and sorted out by the KD tree through the KD tree, continues the clustering process, and then the point cloud data is regularly aggregated into a point set. Therefore, ROI extraction is used to retrieve the area containing roads and intersections, and at the same time, the data is segmented by building a top view based on planar grids to leave point cloud data of obstacles on the road. Then use the principal component analysis method to find the three main directions of the clustered point set, find the centroid, calculate the covariance, and obtain the covariance matrix, and use the Jacobian iterative method to obtain the eigenvalues and eigenvectors of the covariance matrix. The vector is the main direction, and the (X, Y, Z) coordinates of each point are projected onto the calculated coordinate axis. The position is obtained by accumulating all points and then calculating the average value, and the center point and half length are calculated to generate the smallest rotation. A rectangle that forms a bounding box along the direction of the principal components of the object.

Further preferably, the three-dimensional coordinates (X, Y, Z) in the point cloud space in the step 1) are to divide the collected three-dimensional data into voxels according to the voxel grid, and create a volume of 1 cubic centimeter Three-dimensional voxel grid, after containing each voxel, the centroid of all points in the voxel is used to approximate other points in the voxel, and its centroid is calculated as follows:

Among them, m is the total number of voxel points contained in the voxel of the three-dimensional coordinates of the detected object, x _central , y _central and z _central are the three-dimensional x-axis coordinates, y-axis coordinates and z-axis coordinates of the centroid, then All points in the voxelized voxel of the detected target object are finally represented by the centroid; in the three-dimensional coordinates (X, Y, Z) in the point cloud space, X=x _central , Y=y _central , Z=z _central .

Downsampling is achieved by using the voxel grid method, that is, without destroying the geometric structure of the point cloud, the number of points is reduced while the shape features of the point cloud are preserved, and a certain degree of noise points and outliers are removed. However, the point cloud data after voxel filtering is still unordered data, and the data volume is still huge, which is difficult to process. Therefore, it is necessary to use the KD tree to reduce a lot of time consumption, and at the same time ensure that the point cloud search and registration of associated points are in the real-time status.

It should be noted that the serial numbers of the above embodiments of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments. And herein the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, apparatus, article or method comprising a set of elements includes not only those elements, but also includes the elements not expressly included. other elements listed, or also include elements inherent in the process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional same elements in the process, apparatus, article or method comprising the element.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present invention can be embodied in the form of a software product in essence or in other words, the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM) as described above. , magnetic disk, optical disk), including several instructions to enable a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process conversion made by using the description of the present invention and the contents of the accompanying drawings, or directly or indirectly used in other related technical fields , are all included in the scope of patent protection of the present invention in the same way.

Claims (9)

1. A method for world coordinate positioning of a road projectile in a mixed gaussian domain, said method comprising the steps of:

s1, collecting an image through a camera, and confirming foreground and background attribution of each pixel point in the collected image by using a mixed Gaussian distribution model with a plurality of Gaussian distributions; continuously updating and carrying out shadow pixel point detection, extracting a required prospect, and carrying out morphological processing of corrosion and expansion closing operation on the prospect to obtain a relatively complete prospect;

s2, analyzing the contour type of the acquired real-time image at the time t by using a pixel mean value method, so as to be convenient for analyzing whether the contour is a foreground or shadow contour;

s3, introducing a Matchshape operator, and matching a target contour with a shadow contour by using the translation, scaling and rotation invariance of the Hu matrix; pre-storing the outline of the object shadow successfully matched with the appearing object shadow, and updating and screening; initializing and storing the suspected tossing shadow outline template and the posture of the matched result;

s4, obtaining a suspected throwing object which is positioned after the step S3, and performing shadow elimination on a new foreground by adopting the method of the step S2 to obtain a foreground target; performing object shadow matching on the target in the foreground by using the template to complete tracking, and outputting a coordinate of the tossing object obtained by tracking and positioning;

and S5, collecting three-dimensional point cloud data of the throwing object by adopting a laser radar, establishing a three-dimensional KD tree, arranging the collected disordered three-dimensional point cloud data to form ordered three-dimensional point cloud data, then calculating Euclidean distance to cluster the formed ordered three-dimensional point cloud data, converting the clustered ordered three-dimensional point cloud data into a throwing object coordinate in a two-dimensional plane in a matrix transformation mode, and fusing the converted two-dimensional plane throwing object coordinate with the throwing object coordinate output in the step S4 to complete the world coordinate positioning of the road throwing object.

2. The method according to claim 1, wherein the Gaussian mixture model of the Gaussian distribution in step S1 is as follows:

wherein M is the Gaussian distribution number, chi of the mixed Gaussian distribution model _T For the collection of sample sets within the time window T starting at time T χ _T ＝{x ^(t) ，...，x ^(t-T) }，χ _T The background model updating method is used for indicating that new samples are continuously added in a time window T to update the background model so as to adapt to the change of a complex environment; BG and FG represent background component and foreground component, respectively; pi _m Weights representing the M mixture models; eta is a probability density function of Gaussian distribution;

and

respectively, mean and covariance matrices of the pixels at time t, wherein

Is a Gaussian distribution variance estimation value, and I is an identity matrix.

3. The method according to claim 1, wherein shadow pixel point detection is performed on the image during the continuous updating process of the Gaussian mixture distribution model in the step S1, and the method comprises the following steps:

s11, constructing a shadow pixel point S _t The discriminant of (x, y):

|I _t (x，y)H-B _t (x，y)H|≤τ _H

|I _t (x，y)S-B _t (x，y)S|≤τ _S

wherein, I _t (x, y) is the step S1Collecting real-time images at the t moment; b is _t (x, y) is a component B of the real-time image at the time t acquired in the step S1, and H, S and V are HSV components obtained by converting the image at the time t acquired in the step S1 from RGB to HSV space respectively; α is a first adjustment parameter of the luminance sensitivity for shadow detection, β is a second adjustment parameter of the luminance sensitivity for shadow detection, τ _H First adjustment parameter for noise sensitivity for shadow detection, τ _S A second adjustment parameter for noise sensitivity for shadow detection;

and S12, judging whether each pixel point in the detected image accords with the shadow pixel point discriminant constructed in the step S21, and if so, marking the pixel point as a shadow pixel point.

4. The method according to claim 1, wherein the updating equation of the Gaussian mixture model for continuously updating the Gaussian distribution in the step S1 is as follows:

wherein,

is a central vector;

as pixels of the real-time image at time t

Attribution factors conforming to the distribution model, the attribution of the most conforming distribution model is 1, and the attribution of the rest models is 0;

an envelope curve which is exponentially descending and is used for limiting the influence of old data;

continuously updating multiple Gaussian distribution mixed Gaussian distribution models, and judging pixels of real-time images at t moment

Mahalanobis distance of

Judging whether the standard deviation is less than three times to judge whether the pixel accords with the existing Gaussian distribution or not and judging the pixel of the real-time image at the moment t

Mahalanobis distance of

The calculation formula of (a) is as follows:

σ _m is the variance of the gaussian distribution.

5. The method according to claim 1, wherein the discriminant of the contour type of the acquired real-time image at time t analyzed by a pixel mean value method in the step S2 is as follows:

wherein I isThe total number of pixel points of the collected real-time image at the time t, i is the ith pixel point of the collected real-time image at the time t, b _w Is a target pixel value;

and when P is larger than 191, indicating that the target foreground points in the acquired real-time image at the time t are more than the shadow points, and judging that the contour target of the acquired real-time image at the time t is a foreground contour.

6. The method according to claim 1, wherein the step of S5, establishing a three-dimensional KD tree and sorting the collected unordered three-dimensional point cloud data into ordered three-dimensional point cloud data, comprises the following steps:

s51, acquiring three-dimensional disordered point cloud data T = { p by using laser radar ₁ ，p ₂ ，p ₃ Therein of

Initializing a segmentation axis, calculating the sum of variances of point cloud position data of each dimension, taking the dimension with the maximum variance as a segmentation hyperplane, and marking as r;

s52, retrieving the current point cloud position data according to the segmented hyperplane dimension, finding median data, and putting the median data on a current node, wherein the root node corresponds to a hyperplane area of a three-dimensional space containing T;

s53, dividing the dimension of the current hyperplane into two sub hyperplane dimensions, dividing all values smaller than the median into the dimension of the left sub hyperplane, and dividing all values larger than or equal to the median into the dimension of the right sub hyperplane;

s54, updating the segmented hyperplane r obtained in the step S51, storing the point drawn into the dimension of the sub hyperplane on the current root node, and updating the formula of the segmented hyperplane r obtained in the step S51 as follows: r = (r + 1)% 3,% is calculated as the remainder;

s55: repeating the step S52 in the data of the left branch hyperplane dimension obtained in the step S53 until no data exists in the sub-dimension, and determining to obtain a left node; repeating the step S52 in the data of the right branch hyperplane dimension obtained in the step S53 until no data exists in the sub-dimension, and determining to obtain a right node;

s56: outputting a KD tree corresponding to the point cloud, and sorting the collected unordered three-dimensional point cloud data according to the KD tree corresponding to the point cloud to form ordered three-dimensional point cloud data.

7. The method according to claim 6, wherein the Euclidean distance calculation in step S5 is as follows:

wherein x is _i Is the abscissa, y, of the ith point cloud data in the ordered three-dimensional point cloud data _i The longitudinal coordinate of the ith point cloud data in the ordered three-dimensional point cloud data is shown, and n is the total number of point cloud data in the ordered three-dimensional point cloud data.

8. The method according to claim 1, wherein the step S5 of converting the clustered ordered three-dimensional point cloud data into coordinates of the projectile in the two-dimensional plane by means of matrix transformation comprises the following steps:

1) And constructing a transfer matrix M, and transferring the three-dimensional coordinates (X, Y, Z) in the point cloud space of the target throwing object acquired by the laser radar into the coordinates (u, v) in the two-dimensional plane of the target throwing object:

wherein m is _ab For the row a, column b transfer parameters in the transfer matrix M, a =1,2,3; b =1,2,3,4; z _c Is the position of a cameraZ-axis coordinates of a camera coordinate system;

2) And calculating the coordinates (u, v) in the two-dimensional plane of the target projectile according to the transfer matrix as follows:

3) And 2) obtaining a series of linear equations through the calculation of the step 2), solving the linear equations to obtain calibration parameters, and further obtaining coordinates (u, v) in a two-dimensional plane of the target throwing object.

9. The method according to claim 8, wherein the three-dimensional coordinates (X, Y, Z) in the point cloud space in step 1) are obtained by voxel-dividing the acquired three-dimensional data according to a voxel grid, creating a three-dimensional voxel grid with a volume of 1 cubic centimeter, and containing the center of mass of all the points in the voxel to approximately display other points in the voxel, wherein the center of mass is calculated as follows:

wherein m is the total number of voxel points contained in voxels voxelized by three-dimensional coordinates of the detected target object, and x _central 、y _central And z _central The three-dimensional x-axis coordinate, the y-axis coordinate and the z-axis coordinate of the centroid, and all points in voxels voxelized by the three-dimensional coordinates of the detection target object are finally represented by the centroid; x = X in three-dimensional coordinates (X, Y, Z) in a point cloud space _central ，Y＝y _central ，Z＝z _central 。

Images