WO2024199378A1

WO2024199378A1 - Obstacle feature recognition model training method and apparatus, device, and storage medium

Info

Publication number: WO2024199378A1
Application number: PCT/CN2024/084501
Authority: WO
Inventors: 伍俊杰; 王东伟
Original assignee: 北京罗克维尔斯科技有限公司
Priority date: 2023-03-30
Filing date: 2024-03-28
Publication date: 2024-10-03
Also published as: CN118736524A

Abstract

Provided are an obstacle feature recognition model training method and apparatus, a computer device, a computer-readable storage medium, a computer program product, and a computer program. The method comprises: acquiring a surround-view image and point cloud data that are corresponding to a same obstacle; on the basis of the point cloud data and voxels preset in a corresponding three-dimensional space, determining a real obstacle feature corresponding to an obstacle in each voxel; inputting the surround-view image into an initial model to be trained, to obtain a predicted obstacle feature corresponding to the obstacle, wherein the initial model comprises a feature extraction network, a feature conversion network, and an obstacle feature prediction network; and training the initial model according to the real obstacle features and the predicted obstacle feature that are corresponding to the obstacle, so as to obtain an obstacle feature recognition model.

Description

Training method, device, equipment and storage medium for obstacle feature recognition model

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chinese patent application No. 202310333989.7 filed in China on March 30, 2023, the entire contents of which are incorporated herein by reference.

Technical Field

The present disclosure relates to the technical field of obstacle detection, and in particular to a training method and apparatus for an obstacle feature recognition model, a computer device, a computer-readable storage medium, a computer program product, and a computer program.

Background Art

Self-driving cars, also known as driverless cars, are a technology that uses computer equipment to control vehicles to drive automatically on the road. The realization of self-driving relies on the collaboration of artificial intelligence, visual computing, radar, and positioning components. Due to the complex actual road conditions and the presence of a large number of obstacles such as pedestrians and vehicles, how to achieve obstacle recognition, clarify the characteristics of obstacles in three-dimensional space, and then plan a driving route to avoid obstacles has become the key to self-driving.

The related obstacle recognition technology mainly recognizes obstacles based on images captured by cameras due to the low cost of cameras. Although there is technology that can convert two-dimensional data collected by cameras into three-dimensional data, its accuracy is poor, resulting in the obstacle features obtained using images and recognition models being of low accuracy in three-dimensional space, affecting the subsequent decision-making of autonomous driving.

Therefore, how to obtain a recognition model that can output high-precision obstacle features becomes an urgent problem to be solved.

Summary of the invention

In order to solve the above technical problems or at least partially solve the above technical problems, at least one embodiment of the present disclosure provides a training method and apparatus for an obstacle feature recognition model, a computer device, a computer-readable storage medium, a computer program product, and a computer program.

In a first aspect, an embodiment of the present disclosure provides a method for training an obstacle feature recognition model, comprising:

Obtain surround view images and point cloud data corresponding to the same obstacle;

Based on the point cloud data and preset voxels in the corresponding three-dimensional space, determine the real obstacle features corresponding to the obstacle in each voxel;

Inputting the surround view image into an initial model to be trained to obtain obstacle prediction features corresponding to the obstacle, wherein the initial model includes a feature extraction network, a feature conversion network, and an obstacle feature prediction network;

The initial model is trained according to the real obstacle features and the predicted obstacle features corresponding to the obstacles to obtain an obstacle feature recognition model.

In a second aspect, an embodiment of the present disclosure provides a method for acquiring obstacle features, including:

Obtain a surround view of obstacles;

Inputting the surround view image into a pre-trained obstacle feature recognition model to obtain obstacle prediction features corresponding to the obstacle;

Among them, the obstacle feature recognition model includes a feature extraction network, a feature conversion network and an obstacle feature prediction network. The feature extraction network is used to extract the two-dimensional features corresponding to the surround view image, the feature conversion network is used to convert the two-dimensional features into three-dimensional features, and the obstacle feature prediction network is used to predict the obstacle prediction features corresponding to the obstacle based on the three-dimensional features.

In a third aspect, the present disclosure provides a training device for an obstacle feature recognition model, including:

The first acquisition module is used to acquire the surround view image and point cloud data corresponding to the same obstacle;

A determination module, configured to determine the real obstacle feature corresponding to each voxel of the obstacle based on the point cloud data and preset voxels in the corresponding three-dimensional space;

A second acquisition module is used to input the surround view image into an initial model to be trained to obtain obstacle prediction features corresponding to the obstacle, wherein the initial model includes a feature extraction network, a feature conversion network and an obstacle feature prediction network;

The training module is used to train the initial model according to the real obstacle features and the predicted obstacle features corresponding to the obstacles to obtain an obstacle feature recognition model.

In a fourth aspect, an embodiment of the present disclosure provides a device for acquiring obstacle characteristics, including:

An image acquisition module is used to acquire a surround view image of obstacles;

A feature prediction module, used to input the surround view image into a pre-trained obstacle feature recognition model to obtain obstacle prediction features corresponding to the obstacle;

In a fifth aspect, an embodiment of the present disclosure provides a computer device, including: a processor and a memory;

The processor is used to execute any of the obstacle feature recognition model training methods provided in the embodiments of the present disclosure, or to execute any of the obstacle feature acquisition methods provided in the embodiments of the present disclosure, by calling the program or instruction stored in the memory.

In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program or instruction, wherein the program or instruction enables a computer to execute any of the training methods for the obstacle feature recognition model provided in the embodiments of the present disclosure, or execute any of the methods for acquiring obstacle features provided in the embodiments of the present disclosure.

In the seventh aspect, an embodiment of the present disclosure provides a computer program product, which, when executed by a processor, implements the training method of the obstacle feature recognition model described in any embodiment of the first aspect of the present disclosure, or the method for obtaining obstacle features provided in any embodiment of the second aspect of the present disclosure.

In an eighth aspect, an embodiment of the present disclosure provides a computer program, wherein the computer program includes computer program code, and when the computer program code is run on a computer, the computer executes any one of the first aspects of the present disclosure. The training method of the obstacle feature recognition model described in the embodiment, or the method for obtaining obstacle features provided by any embodiment of the second aspect of the present disclosure.

Compared with the related art, the technical solution provided by the embodiments of the present disclosure has at least the following advantages:

In the disclosed embodiment, the surround view image and point cloud data corresponding to the same obstacle are obtained, and based on the point cloud data and the preset voxels in the corresponding three-dimensional space, the real obstacle features corresponding to the obstacle in each voxel are determined, and the surround view image is input into the initial model to be trained to obtain the obstacle prediction features corresponding to the obstacle, and then the initial model is trained according to the real obstacle features and the obstacle prediction features corresponding to the obstacle to obtain the obstacle feature recognition model. The above technical solution is adopted, the initial model is used to predict the obstacle features of the surround view image, and the real obstacle features determined based on the point cloud data are used as the training target, and the initial model is trained to obtain the obstacle feature recognition model, so that the trained model can learn accurate obstacle features from the surround view image, improve the recognition accuracy of the obstacle feature recognition model for the 3D features of the obstacle, and thus help improve the accuracy of obstacle recognition.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

FIG1 is a schematic diagram of a flow chart of a method for training an obstacle feature recognition model provided by an embodiment of the present disclosure;

FIG2 is a schematic flow chart of a method for training an obstacle feature recognition model provided by another embodiment of the present disclosure;

FIG3 is a flow chart of a method for training an obstacle feature recognition model provided by another embodiment of the present disclosure;

FIG4 is a flow chart of a method for training an obstacle feature recognition model provided in yet another embodiment of the present disclosure;

FIG5 is a schematic diagram showing a network structure of an obstacle feature recognition model according to a specific embodiment of the present disclosure;

FIG6 is a schematic flow chart of a method for acquiring obstacle features according to an embodiment of the present disclosure;

FIG7 is a schematic diagram of the structure of a training device for an obstacle feature recognition model provided by an embodiment of the present disclosure;

FIG8 is a schematic diagram of the structure of an obstacle feature acquisition device provided in an embodiment of the present disclosure.

DETAILED DESCRIPTION

In order to more clearly understand the above-mentioned purposes, features and advantages of the present disclosure, the present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It is understood that the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments, and the specific embodiments described herein are only used to explain the present disclosure, rather than to limit the present disclosure. In the absence of conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in the field belong to the scope of protection of the present disclosure.

In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only a part of the embodiments of the present disclosure. Rather than all embodiments.

Figure 1 is a flow chart of a training method for an obstacle feature recognition model provided in an embodiment of the present disclosure. The training method for an obstacle feature recognition model can be executed by a training device for an obstacle feature recognition model provided in an embodiment of the present disclosure. The training device for an obstacle feature recognition model can be implemented using software and/or hardware and can be integrated on a computer device, which can be an electronic device such as a computer or a server.

As shown in FIG. 1 , the training method of the obstacle feature recognition model provided in the embodiment of the present disclosure may include steps 101 to 104 .

Step 101, obtaining surround view images and point cloud data corresponding to the same obstacle.

The surround image includes multiple images, for example, the surround image may include images in six directions: front, rear, left front, right front, left rear and right rear. Obstacles may be, for example, buildings, trees, human bodies, vehicles and the like.

In the disclosed embodiment, for the same obstacle, a surround view image of the obstacle captured by a camera and point cloud data of the obstacle captured by various sensors may be obtained, for example, point cloud data of the obstacle may be obtained by a laser radar.

LiDAR is a system that integrates three technologies: laser, Global Navigation Satellite System (GNSS) and Inertial Navigation System (INS). Through the combination of these three technologies, it can not only actively and in real time perceive the environment and the dynamic spatial position relationship of objects, but also generate accurate three-dimensional spatial models under the condition of consistent absolute measurement points. It is used in surface remote sensing, such as ground elevation and landform, forestry survey and other data acquisition, as well as autonomous driving and high-precision map production.

LiDAR actively emits a laser beam and measures the time it takes for the light to hit an object or surface and then reflect back to calculate the distance from the target point to the laser radar. The above behavior will obtain millions of data points in a rapid and repeated process. Using these data points, the system will build a complex "map" of the geographic space surface it is measuring, called a "point cloud." Since the point cloud is the feedback value of the laser radar's received light beam on the object, each point in the point cloud contains three-dimensional coordinate values, which are the three elements we often call x, y, and z.

It is understood that the point cloud data can be collected manually or automatically. For example, the point cloud data within the obstacle range can be collected by manually controlling the laser radar. The point cloud data within the obstacle range can be collected autonomously by the laser radar on the autonomous mobile device during movement.

In the disclosed embodiment, in order to obtain rich training data, surround view images and point cloud data may be acquired for multiple obstacles, and corresponding surround view images and point cloud data may be acquired for each obstacle.

Step 102: Based on the point cloud data and preset voxels in the corresponding three-dimensional space, determine the real features of the obstacle corresponding to each voxel of the obstacle.

The preset voxels may be obtained by dividing the three-dimensional space into voxels according to a preset size.

In the disclosed embodiment, the acquired point cloud data can be converted into a three-dimensional space, and the three-dimensional space can be divided into voxels to obtain a plurality of voxels. The point cloud data in each voxel can then be processed to obtain the true features of the obstacle corresponding to the obstacle in each voxel.

The real features of the obstacle may include the real occupancy information of each voxel when the three-dimensional space corresponding to the obstacle is divided into multiple voxels. The occupancy information reflects whether the voxel contains an object. The occupied voxel contains an object, and the unoccupied voxel contains an object. Occupied voxels do not contain objects; they may also include the true position and size information of occupied voxels in obstacles.

In the disclosed embodiment, obstacles in the point cloud data are identified as real, and then the obstacles are used as a benchmark, and the determined real features of the obstacles are used as training targets to participate in subsequent supervised learning.

Step 103: input the surround view image into the initial model to be trained to obtain obstacle prediction features corresponding to the obstacle.

The initial model may include a feature extraction network, a feature conversion network, and an obstacle feature prediction network. It is understood that the network structure of the initial model is not limited to the above three networks and can be set according to actual needs, which is not limited in the embodiments of the present disclosure.

In some embodiments, the initial model may include a feature extraction network, a feature conversion network, and an obstacle feature prediction network, and each of the aforementioned networks may be one or more. Among them, the feature extraction network is used to extract different features. For example, the initial model may include the currently commonly used feature extraction network and feature conversion network. The feature extraction network is used to extract two-dimensional features based on the surround image, and the feature conversion network is used to convert the two-dimensional features into three-dimensional features, and then the obstacle feature prediction network predicts obstacle features based on the three-dimensional features. It can be understood that according to actual needs, the obstacle feature prediction network can be one or more. When there are multiple obstacle feature prediction networks, each network can be used to predict features of different categories, and the features predicted by each obstacle feature prediction network are used as the final obstacle features.

In the disclosed embodiment, the surround view image is input into the initial model to be trained, and the initial model performs feature extraction, feature conversion, obstacle feature prediction and other processing on the surround view image, and finally outputs the obstacle prediction features obtained based on the surround view image prediction.

The obstacle prediction features output by the initial model may include at least one of voxel occupancy prediction information, voxel position prediction information and voxel size prediction information.

Step 104: training the initial model according to the real obstacle features and the predicted obstacle features corresponding to the obstacles to obtain an obstacle feature recognition model.

In the disclosed embodiment, after obtaining the obstacle prediction features output by the initial model, the initial model can be trained based on the obstacle prediction features and the real features of the obstacle corresponding to the same obstacle. By continuously adjusting the network parameters of the initial model, a trained obstacle feature recognition model is obtained. The obstacle feature recognition model can more accurately identify the 3D features of obstacles in the surround view image, and provide data support for accurately identifying obstacles within obstacles.

In some embodiments, the network parameters of the initial model can be adjusted according to the difference between the predicted features of the obstacle and the real features of the obstacle. For example, if there is a convolutional neural network in the initial model, the number of convolution kernels, step size and other parameters can be adjusted. For another example, the parameters of the pooling layer can be adjusted.

In some embodiments, a preset loss function can be used to calculate the loss value based on the data of the obstacle prediction features and the obstacle real features. When the calculated loss value is different from the preset loss threshold or the difference exceeds the allowable error range, the network parameters of the initial model are adjusted; when there is no difference between the loss value and the loss threshold or the difference is within the allowable error range, the training is terminated to obtain a trained obstacle feature recognition model.

The obstacle feature recognition model training method of the disclosed embodiment obtains the surrounding view corresponding to the same obstacle The system uses the image and point cloud data, divides the point cloud data into voxels in three-dimensional space based on the point cloud data, determines the real obstacle features corresponding to the obstacle in each voxel, inputs the surround view image into the initial model to be trained, obtains the obstacle prediction features corresponding to the obstacle, and then trains the initial model according to the real obstacle features and the obstacle prediction features corresponding to the obstacle to obtain an obstacle feature recognition model. The above technical solution is adopted, the initial model is used to predict the obstacle features of the surround view image, and the real obstacle features determined based on the point cloud data are used as the training target to train the initial model to obtain the obstacle feature recognition model, so that the trained model can learn accurate obstacle features from the surround view image, improve the recognition accuracy of the obstacle feature recognition model for the 3D features of the obstacle, and thus help improve the accuracy of obstacle recognition.

In an optional implementation of the present disclosure, the real features of the obstacle may include real voxel occupancy information, and the obstacle feature prediction network may include a voxel occupancy prediction network. Thus, as shown in FIG. 2 , based on the aforementioned embodiment, step 103 may include steps 201 to 203.

Step 201: input the surround view image into a feature extraction network to obtain two-dimensional features corresponding to the surround view image.

The feature extraction network in the initial model may be a currently commonly used image feature extraction network, which is used to extract two-dimensional features from the surround image.

Step 202: input the two-dimensional features into the feature conversion network to obtain the three-dimensional features corresponding to the obstacles.

Among them, the feature conversion network in the initial model can be a currently commonly used feature conversion network, and the feature conversion network is used to perform feature conversion and restore the input two-dimensional features to three-dimensional features.

In the disclosed embodiment, after the acquired surround view image is input into the initial model, the surround view image is first subjected to feature extraction by the feature extraction network of the initial model to obtain two-dimensional features, and then the feature conversion network in the initial model performs feature conversion to restore the two-dimensional features to three-dimensional features to obtain the three-dimensional features of the obstacle.

Step 203: input the three-dimensional features into the voxel occupancy prediction network to obtain voxel occupancy prediction information output by the voxel occupancy prediction network.

Among them, the voxel occupancy prediction network is a network built according to actual needs, that is, the obstacle feature prediction network includes the voxel occupancy prediction network. The voxel occupancy prediction network can have a variety of structures, and it can predict the occupancy information of voxels based on three-dimensional features. The embodiments of the present disclosure do not limit the specific structure of the voxel occupancy prediction network.

In the disclosed embodiment, the three-dimensional features output by the feature conversion network of the initial network are input to the voxel occupancy prediction network, and the voxel occupancy prediction network outputs the voxel occupancy prediction information. It can be understood that in the disclosed embodiment, when the voxel occupancy prediction network makes a prediction based on the three-dimensional features, it can first divide the three-dimensional features into a plurality of voxels, and then predict the occupancy information corresponding to each voxel.

The voxel occupancy prediction information is used to reflect whether the voxel is occupied. If the voxel is occupied, it means that the voxel contains an object, and if the voxel is not occupied, it means that the voxel does not contain an object.

As an example, the voxel occupancy prediction information can be represented by a preset identifier. For example, "1" is preset to indicate that the voxel is occupied, and "0" is preset to indicate that the voxel is not occupied. When the voxel occupancy prediction network predicts the occupancy state of each voxel in the three-dimensional feature, for a certain voxel, if the prediction result is that the voxel is occupied, the voxel is marked as "1", and if the prediction result is that the voxel is not occupied, the voxel is marked as "0". The label information of the voxel represents the voxel occupancy prediction. Test information.

In the disclosed embodiment, by obtaining the real information of voxel occupancy as the learning target, and setting a feature extraction network, a feature conversion network and a voxel occupancy prediction network in the initial model, the initial model can learn the voxel occupancy of the input surround image. Since whether a voxel is occupied reflects whether there is an object in the voxel, this also provides conditions for further predicting the position of obstacles and narrowing the obstacle detection range.

In an optional implementation of the present disclosure, the real features of the obstacle may also include the real voxel position information and the real voxel size information corresponding to the voxel, and the obstacle feature prediction network may also include a voxel position prediction network and a voxel size prediction network, as shown in FIG. 2 , and step 103 may also include steps 204 to 205.

Step 204: input the three-dimensional features into the voxel position prediction network to obtain voxel position prediction information output by the voxel position prediction network.

Among them, the voxel position prediction network is a network built according to actual needs, that is, the obstacle feature prediction network also includes the voxel position prediction network. The voxel position prediction network can have a variety of structures, and it can predict the position information of voxels based on three-dimensional features. The embodiments of the present disclosure do not limit the specific structure of the voxel position prediction network.

In the disclosed embodiment, after the feature conversion network of the initial model extracts the three-dimensional features, the three-dimensional features can also be input into the voxel position prediction network, and the voxel position prediction network outputs voxel position prediction information based on the three-dimensional features.

Among them, the voxel position prediction information reflects the position of the object (ie, obstacle) within the voxel.

As an example, the position of the center of mass of an object within a voxel can be predicted as voxel position prediction information, and the voxel position prediction information can be expressed as (x, y, z), where x, y, and z represent the coordinate values of the predicted center of mass of the object on the x-axis, y-axis, and z-axis, respectively.

As an example, the coordinate offset of the center of mass of the object in the voxel relative to the center point of the voxel can be predicted as the voxel position prediction information, and the voxel position prediction information can be expressed as (dx, dy, dz), where dx represents the offset of the center point of the voxel relative to the center of mass of the object on the x-axis, dy represents the offset of the center point of the voxel relative to the center of mass of the object on the y-axis, and dz represents the offset of the center point of the voxel relative to the center of mass of the object on the z-axis. It can be understood that dx, dy and dz are predicted values.

Step 205: input the three-dimensional features into the voxel size prediction network to obtain voxel size prediction information output by the voxel size prediction network.

Among them, the voxel size prediction network is a network built according to actual needs, that is, the obstacle feature prediction network also includes a voxel size prediction network. The voxel size prediction network can have a variety of structures, and it is sufficient to predict the size information of voxels based on three-dimensional features. The embodiments of the present disclosure do not limit the specific structure of the voxel size prediction network.

In the disclosed embodiment, after the feature conversion network of the initial model extracts the three-dimensional features, the three-dimensional features can also be input into the voxel size prediction network, and the voxel size prediction network outputs voxel size prediction information based on the three-dimensional features.

Among them, the voxel size prediction information reflects the size of the object (ie, obstacle) within the voxel.

As an example, the length, width and height of the minimum circumscribed cuboid of the point cloud within the voxel can be predicted as voxel size prediction information. The voxel size prediction information can be expressed as (dl, dw, dh), where dl represents the length of the minimum circumscribed cuboid, dw represents the width of the minimum circumscribed cuboid, and dh represents the height of the minimum circumscribed cuboid.

In the implementation of the present disclosure, the initial model includes a voxel occupancy prediction network, a voxel position prediction network and a voxel size prediction network, so that voxel occupancy information, position information and size information can be predicted, and these information constitute obstacle prediction features.

In some embodiments, the obstacle prediction feature can be represented by the prediction features corresponding to multiple voxels, and the prediction features corresponding to each voxel can be expressed as (1, x, y, z, dx, dy, dz, dl, dw, dh), where 1 is the voxel occupancy prediction information, indicating that the voxel is occupied, (x, y, z) represents the center point coordinates of the voxel, (dx, dy, dz) represents the voxel position prediction information, and (dl, dw, dh) represents the voxel size prediction information. The obstacle prediction feature can also be expressed as (0), where 0 is the voxel occupancy prediction information, indicating that the voxel is unoccupied, and the unoccupied voxel has no voxel position prediction information and voxel size prediction information; or, the obstacle prediction feature can also be expressed as (0, x, y, z, 0, 0, 0, 0, 0), where the first 0 is the voxel occupancy prediction information, indicating that the voxel is unoccupied, (x, y, z) represents the center point coordinates of the unoccupied voxel, and the remaining 0s indicate that the unoccupied voxels have no voxel position prediction information and voxel size prediction information.

It should be noted that there is no order among the above steps 203, 204, and 205, and they can be executed together or separately.

The training method of the obstacle feature recognition model of the embodiment of the present disclosure obtains the real information of the voxel position and the real information of the voxel size as the learning target, and sets the voxel position prediction network and the voxel size prediction network in the initial model, so that the initial model can learn the voxel position and voxel size of the input surround image, so that the trained model can predict more accurate voxel positions and sizes based on the input surround image, and obtain accurate 3D features of obstacles.

In an optional embodiment of the present disclosure, when the initial model is trained according to the real obstacle features and the predicted obstacle features corresponding to the obstacles, a first loss value can be determined according to the voxel occupancy prediction information and the real voxel occupancy information, a second loss value can be determined according to the voxel position prediction information and the real voxel position information, and a third loss value can be determined according to the voxel size prediction information and the real voxel size information, and then the network parameters of the initial model can be adjusted based on the first loss value, the second loss value and the third loss value.

In the disclosed embodiment, when calculating the first loss value, the second loss value, and the third loss value, relevant technologies may be used to first determine at least one voxel pair from the real feature of the obstacle and the predicted feature of the obstacle. For example, for any voxel in the predicted feature of the obstacle, the target voxel with the same specific coordinates as the real feature of the obstacle may be found based on the coordinates of the voxel (i.e., the center point coordinates), and the voxel and the target voxel constitute a voxel pair. Then, the first loss value is calculated using the voxel occupancy prediction information and the real voxel occupancy information corresponding to the voxel pair, the second loss value is calculated using the voxel position prediction information and the real voxel position information corresponding to the voxel pair, and the third loss value is calculated using the voxel size prediction information and the real voxel size information corresponding to the voxel pair.

Among them, the loss function used to calculate each loss value can be pre-set according to actual needs, and the embodiment of the present disclosure does not limit the loss function used to calculate each loss value.

As an example, when adjusting the network parameters of the initial model based on the first loss value, the second loss value, and the third loss value, the network parameters of the initial model can be adjusted based on each loss value. For example, when the first loss value does not meet the corresponding loss threshold, the network parameters of the voxel occupancy prediction network in the initial model can be adjusted, and the voxel occupancy prediction network can also be adjusted. The network parameters of other networks before the prediction network can be adjusted; when the second loss value does not meet the corresponding loss threshold, the network parameters of the voxel position prediction network in the initial model can be adjusted, and the network parameters of other networks before the voxel position prediction network can also be adjusted; when the third loss value does not meet the corresponding loss threshold, the network parameters of the voxel size prediction network in the initial model can be adjusted, and the network parameters of other networks before the voxel size prediction network can also be adjusted.

As an example, when adjusting the network parameters of the initial model based on the first loss value, the second loss value, and the third loss value, the loss value of the initial model can be first calculated based on the first loss value, the second loss value, and the third loss value. For example, the average of the first loss value, the second loss value, and the third loss value can be calculated as the loss value of the initial model; for another example, the sum of the first loss value, the second loss value, and the third loss value can be calculated as the loss value of the initial model, and so on. Then, the network parameters of the initial model can be adjusted based on the loss value of the initial model. When the loss value of the initial model does not meet the preset loss threshold, at least one network parameter of the initial model is adjusted.

In the embodiment of the present disclosure, a first loss value is determined based on the voxel occupancy prediction information and the voxel occupancy real information, a second loss value is determined based on the voxel position prediction information and the voxel position real information, and a third loss value is determined based on the voxel size prediction information and the voxel size real information, and then based on the first loss value, the second loss value and the third loss value, the network parameters of the initial model are adjusted, thereby achieving iterative training of the initial model to obtain an obstacle feature recognition model that can accurately predict the 3D features of obstacles in the surround view image.

In an optional implementation of the present disclosure, as shown in FIG. 3 , based on the above-mentioned embodiment, step 102 may include steps 301 to 303 .

Step 301: convert the point cloud data into a three-dimensional coordinate system.

In the disclosed embodiment, point cloud data collected by a sensor (such as a laser radar) can be obtained, and the obtained point cloud data can be converted into a three-dimensional coordinate system. Since each point in the point cloud data contains a three-dimensional coordinate value, during the conversion, the obtained point cloud data can be converted into a three-dimensional coordinate system based on the three-dimensional coordinate value contained in each point.

Step 302 : voxelize the three-dimensional space in the three-dimensional coordinate system according to a preset size to obtain a plurality of voxels in the three-dimensional space.

Among them, the preset size can be pre-set according to actual needs. For example, the preset size can be set to 0.1m*0.1m*0.1m, 0.1m*0.2m*0.1m, 0.1m*0.1m*0.2m, 0.1m*0.2m*0.3m, and so on.

In the embodiment of the present disclosure, for a three-dimensional space in a three-dimensional coordinate system, the three-dimensional space may be divided into voxels according to a fixed preset size, and the three-dimensional space may be divided into a plurality of voxels of the same size.

In some embodiments, assuming that the spatial range of the three-dimensional space is 30m*30m*10m and the preset size is 0.1m*0.1m*0.1m, the three-dimensional space is voxelized according to the preset size, and the three-dimensional space can be divided into (300, 300, 100) voxels of equal size.

Step 303: Based on the point cloud data contained in each voxel of the multiple voxels, the real features of the obstacle corresponding to the obstacle in each voxel are generated by using the key points of the point cloud in the voxel and the external graphics of the point cloud. The real features of the obstacle include the real information of voxel occupancy corresponding to each voxel, the real information of voxel position corresponding to the non-empty voxel, and the real information of voxel size.

Among them, the key point of the point cloud within the voxel can be the centroid of the point cloud within the voxel, and the external graphics of the point cloud within the voxel can be is the minimum circumscribed cuboid of the point cloud within the voxel.

In the disclosed embodiment, for each divided voxel, the point cloud data contained in the voxel can be determined, and then the real voxel information of the voxel can be determined based on the point cloud data contained in the voxel, the key points of the point cloud in the voxel, and the external graphics of the point cloud, wherein the real voxel information includes the real voxel occupancy information of the voxel, the real voxel position information corresponding to the non-empty voxel (i.e., the voxel whose real voxel occupancy information is occupied voxel), and the real voxel size information. The real voxel occupancy information corresponding to each voxel, the real voxel position information corresponding to the occupied non-empty voxel, and the real voxel size information constitute the real obstacle feature corresponding to the obstacle to which the point cloud data belongs in each voxel.

The voxel occupancy real information can be determined based on whether the voxel contains a point, and can be represented by different marks. For example, for a voxel containing a point, "1" can be marked as the voxel occupancy real information of the voxel, indicating that an object exists in the voxel; for a voxel that does not contain a point, "0" can be marked as the voxel occupancy real information of the voxel, indicating that no object exists in the voxel.

In the disclosed embodiment, the real voxel position information and the real voxel size information of the voxel can be represented by different parameters, which are described below with examples.

In some embodiments, the true voxel position information of the voxel can be represented by the coordinates corresponding to the centroid (ie, key point) of the point cloud within the voxel, and the true voxel size information of the voxel can be represented by the coordinates of the vertices of the minimum circumscribed cuboid of the point cloud within the voxel.

In some embodiments, the true voxel position information of the voxel can be represented by the offset between the coordinates of the center point of the voxel and the coordinates corresponding to the center of mass of the point cloud within the voxel (i.e., the key point), and the true voxel size information of the voxel can be represented by the length, width, and height of the minimum circumscribed cuboid of the point cloud within the voxel.

The training method of the obstacle feature recognition model of the embodiment of the present disclosure converts point cloud data into a three-dimensional coordinate system, voxelizes the three-dimensional space in the three-dimensional coordinate system according to a preset size, and obtains multiple voxels in the three-dimensional space. Then, based on the point cloud data contained in each voxel of the multiple voxels, the key points of the point cloud in the voxel and the external graphics of the point cloud are used to generate the real obstacle features corresponding to the obstacle in each voxel. The real obstacle features include the real voxel occupancy information corresponding to each voxel, the real voxel position information corresponding to the non-empty voxel, and the real voxel size information. By adopting the above technical scheme, after dividing the three-dimensional space into multiple voxels of fixed size, the real voxel occupancy information, real voxel position information and real voxel size information corresponding to each voxel are further determined as the real voxel information of the voxel according to the point cloud data contained in each voxel. Thus, the real position and real size of the voxels belonging to the point cloud data are calculated according to the point cloud data, so that the sizes of different voxels can be different, and the accurate real position and real size of the voxels can be obtained, which is conducive to improving the representation accuracy of the obstacle position and size, and also provides data support for training a model that can accurately identify the characteristics of obstacles.

In an optional implementation of the present disclosure, as shown in FIG. 4 , based on the embodiment shown in FIG. 3 , step 303 may include steps 401 to 405 .

Step 401 : determining the real voxel occupancy information corresponding to each voxel according to the position of each point in the point cloud data, wherein the real voxel occupancy information includes occupied and unoccupied.

In the embodiment of the present disclosure, after the three-dimensional space is divided into a plurality of voxels, the position of each point in the point cloud data can be The coordinate value of each point is determined by the position, that is, the coordinate value of each point, to determine whether each voxel contains a point or a point cloud, and determine the voxel occupancy real information corresponding to each voxel according to the judgment result. Specifically, for a voxel containing a point, the corresponding voxel occupancy real information is determined to be occupied, and for a voxel not containing a point, the corresponding voxel occupancy real information is determined to be unoccupied.

In some embodiments, a non-empty voxel containing at least one point may be assigned a value of 1, indicating that the voxel is occupied and there is an object in the voxel; an empty voxel containing no point may be assigned a value of 0, indicating that the voxel is not occupied and there is no object in the voxel. "1" and "0" represent the real information of voxel occupancy.

Step 402 : for the non-empty voxels whose real voxel occupancy information is occupied, determine the centroid of the point cloud in the non-empty voxel as the key point based on a centroid calculation rule according to at least one point in the non-empty voxel.

Among them, the centroid calculation rule can be a preset rule for determining the centroid of the point cloud in a non-empty voxel. If there is only one point in the non-empty voxel, the point can be determined as the centroid, and the coordinates of the point are the coordinates of the centroid; if there are multiple points in the non-empty voxel, the centroid of the point cloud in the non-empty voxel and its corresponding coordinates can be calculated according to the coordinate values of each point in the non-empty voxel, the mass of each point and other information by the currently commonly used method of determining the centroid of the point cloud. The coordinates of the centroid can represent the position of the centroid, that is, the position coordinates.

In the embodiment of the present disclosure, for any non-empty voxel whose real voxel occupancy information is occupied, the center of mass of the point cloud in the non-empty voxel can be determined based on all points contained in the non-empty voxel (called a point cloud) and the center of mass calculation rule, and the center of mass can be used as the key point of the point cloud in the non-empty voxel.

Step 403: Determine the real voxel position information of the non-empty voxel according to the key point and based on a preset position determination rule.

In the embodiment of the present disclosure, for each non-empty voxel whose filtered voxel occupancy real information is occupied, the voxel position real information and voxel size real information corresponding to each non-empty voxel can be determined according to the key points of the point cloud in the non-empty voxels and based on the preset position determination rules.

In an optional implementation of the present disclosure, the position determination rule includes: determining the position coordinates corresponding to the key point as the real voxel position information of the non-empty voxel. Thus, in the embodiment of the present disclosure, for any non-empty voxel, when determining the real voxel position information of the non-empty voxel based on the key point and the preset position determination rule, the position coordinates of the determined key point can be directly determined as the real voxel position information corresponding to the non-empty voxel.

Since the key point of the point cloud in a non-empty voxel is the centroid of the point cloud, the centroid of the point cloud represents the average position of the mass distribution of the point cloud, and the point cloud describes obstacles in geographic space, in the embodiments of the present disclosure, the centroid of the point cloud can be used to characterize the true position of the obstacle. Since the general obstacle detection algorithm in three-dimensional space divides the three-dimensional space into voxels to identify obstacles, in the embodiments of the present disclosure, in order to determine the true position of the voxel, the coordinates of the centroid of the point cloud in the voxel (i.e., the key point) can be used as the true voxel position information of the voxel.

As a possible implementation, the position determination rule includes: determining the coordinate offset of the center point of the non-empty voxel relative to the key point as the position information of the non-empty voxel. Therefore, in the embodiment of the present disclosure, for any non-empty voxel, when determining the true voxel position information of the non-empty voxel based on the key point and the preset position determination rule, the first coordinate of the center point of the non-empty voxel can be obtained first, and then the coordinate of the center point of the non-empty voxel relative to the key point of the point cloud in the non-empty voxel can be determined based on the first coordinate and the second coordinate corresponding to the key point of the point cloud in the non-empty voxel. The offset, wherein the coordinate offset includes the offset corresponding to each coordinate axis in the three-dimensional coordinate system, and then the first coordinate of the center point of the non-empty voxel and the coordinate offset of the center point relative to the center of mass of the point cloud in the non-empty voxel are determined as the true voxel position information of the non-empty voxel.

It can be understood that after the spatial range of the three-dimensional space is divided into a plurality of voxels of the same size according to a fixed preset size, the coordinates of the center point of each voxel can be determined. For the convenience of description and to facilitate the distinction from the coordinates of the key points of the point cloud in the non-empty voxel, in the embodiment of the present disclosure, the coordinates corresponding to the center point of the non-empty voxel are referred to as the first coordinates, and the coordinates corresponding to the key points of the point cloud in the non-empty voxel are referred to as the second coordinates.

In the embodiment of the present disclosure, when determining the true voxel position information of each non-empty voxel, the coordinate offset of the center point of the non-empty voxel relative to the key point of the point cloud within the non-empty voxel can be calculated based on the first coordinate corresponding to the center point of the non-empty voxel and the second coordinate corresponding to the key point of the point cloud within the non-empty voxel, wherein the coordinate offset includes the offset of the center point relative to the key point on the x, y and z axes of the three-dimensional coordinate system, respectively, and the offset can be expressed by the coordinate difference between the center point and the key point on the same coordinate axis.

In some embodiments, assuming that the first coordinate of the center point of a non-empty voxel is (x ₁ , y ₁ , z ₁ ), and the second coordinate corresponding to the key point of the point cloud in the non-empty voxel is (x ₂ , y ₂ , z ₂ ), then the coordinate offset of the center point of the non-empty voxel relative to the key point of the point cloud can be recorded as (dx, dy, dz), where dx = x ₁ -x ₂ , dy = y ₁ -y ₂ , dz = z ₁ -z ₂ .

It can be understood that when a non-empty voxel includes only one point, the coordinates of the point can be regarded as the second coordinates of the key point, and the coordinate offset of the center point of the non-empty voxel relative to the point can be regarded as the coordinate offset of the center point of the non-empty voxel relative to the key point.

In some embodiments, assuming that the first coordinates of the center point of a non-empty voxel are ( _x1 , _y1 , _z1 ), and the non-empty voxel includes only one point, and the coordinates of the point are ( _x3 , _y3 , _z3 ), then the coordinate offset of the center point of the non-empty voxel relative to the key point of the point cloud (i.e., the point) can be recorded as (dx, dy, dz), where dx= _x1 - _x3 , dy= _y1 - _y3 , dz= _z1 - _z3 .

After that, after determining the coordinate offset of the center point of the non-empty voxel relative to the key point of the point cloud, the first coordinate corresponding to the center point and the determined coordinate offset can be determined as the real voxel position information of the non-empty voxel.

In some embodiments, the true voxel position information of a non-empty voxel can be expressed as (x ₁ , y ₁ , z ₁ , dx, dy, dz), where (x ₁ , y ₁ , z ₁ ) represents the coordinates of the center point of the non-empty voxel, and (dx, dy, dz) represents the coordinate offset of the center point of the non-empty voxel relative to the key point of the point cloud within the non-empty voxel.

In the embodiment of the present disclosure, the center of mass of the point cloud in the non-empty voxel is determined as the key point of the point cloud based on at least one point in the non-empty voxel, and the true voxel position information of the non-empty voxel is determined based on the center of mass. Thus, the true position of the voxel is determined according to the position of the center of mass of the point cloud in the non-empty voxel, which can improve the accuracy of the voxel position and make the voxel representation of the obstacle position more accurate.

Step 404: determine the real information of the voxel size of the non-empty voxel by using the circumscribed graph corresponding to at least one point in the non-empty voxel.

The circumscribed figure refers to the circumscribed figure of all points contained in the non-empty voxel, and the circumscribed figure may be, for example, a minimum circumscribed cuboid.

In an optional embodiment of the present disclosure, for any non-empty voxel, at least one of the non-empty voxels is used. When determining the size information of the non-empty voxel by using the circumscribed figure corresponding to each point, the minimum circumscribed figure of all the points in the non-empty voxel can be determined first, for example, the minimum circumscribed cuboid of all the points in the non-empty voxel can be determined, and then the coordinates of each vertex of the minimum circumscribed figure can be determined, and then the coordinates of each vertex can be used to represent the real voxel size information of the non-empty voxel.

In an optional embodiment of the present disclosure, when determining the true voxel size information of the non-empty voxel using the circumscribed figure corresponding to at least one point in the non-empty voxel, the minimum circumscribed cuboid of at least one point in the non-empty voxel can be first determined based on at least one point in the non-empty voxel, and then the true voxel size information of the non-empty voxel can be determined based on the length, width and height of the minimum circumscribed cuboid.

Among them, when determining the minimum circumscribed cuboid of at least one point in the non-empty voxel according to at least one point in the non-empty voxel, the currently commonly used method of determining the minimum circumscribed cuboid can be used for implementation, and the embodiments of the present disclosure do not limit the specific implementation method of determining the minimum circumscribed cuboid. It can be understood that the minimum circumscribed cuboid can be a cuboid or a cube. The determined minimum circumscribed rectangle can be represented by the coordinates of each vertex.

In the embodiment of the present disclosure, when determining the minimum circumscribed cuboid, different methods can be selected to determine the minimum circumscribed cuboid of at least one point in the non-empty voxel according to the different numbers and positions of points contained in the non-empty voxel. The following example is used for exemplary explanation.

As an example, assuming that a non-empty voxel contains at least two points, and at least two of the at least two points are not on the same plane, the currently commonly used method can be used to determine the minimum circumscribed cuboid of these points.

As an example, assuming that at least two points contained in a non-empty voxel are on the same plane, the currently commonly used method can be used to determine the minimum circumscribed rectangle of these points, and for another coordinate axis that is not involved in the plane where these points are located, the length on the coordinate axis can be determined based on the preset side length, thereby obtaining a cuboid, and the cuboid is used as the minimum circumscribed cuboid of these points. For example, assuming that multiple points contained in a non-empty voxel are all on the plane yoz, and the preset side length is 0.2, the minimum circumscribed rectangle of these points can be determined based on these points, and the coordinates of each vertex of the minimum circumscribed rectangle can be obtained, and for the x-axis, the coordinates on the x-axis can be determined based on the preset side length of 0.2 to obtain a cuboid. For example, the determined minimum circumscribed rectangle can be used as a face of the cuboid, and the coordinates of the vertex when the length is 0.2 in the positive or negative direction of the x-axis can be determined, and the coordinates of the other four vertices of the cuboid can be obtained, thereby obtaining the coordinates of each vertex on the minimum circumscribed cuboid. For another example, the determined minimum circumscribed rectangle can be used as the mid-section of the cuboid, and the x-coordinate values of the four vertices of the minimum circumscribed rectangle can be increased by 0.1 and decreased by 0.1 respectively, while the coordinates of the y-axis and the z-axis remain unchanged, to obtain eight new coordinate points. The cuboid enclosed by these eight new coordinate points is determined as the minimum circumscribed cuboid of at least two points in the non-empty voxel.

As an example, assuming that a non-empty voxel contains only one point, a cube containing the point can be determined as the minimum circumscribed cuboid of the point according to a preset side length. Alternatively, a cuboid containing the point can be determined as the minimum circumscribed cuboid of the point according to a preset length, a preset width, and a preset height. Alternatively, the non-empty voxel can also be used as the minimum circumscribed cuboid.

Next, after determining the minimum circumscribed cuboid of at least one point in the non-empty voxel, the length, width and height of the minimum circumscribed cuboid are also determined. Further, the real voxel size information of the non-empty voxel can be determined based on the length, width and height of the minimum circumscribed cuboid.

As a possible implementation, when determining the true voxel size information of the non-empty voxel according to the length, width and height of the minimum circumscribed cuboid, the length, width and height of the minimum circumscribed cuboid may be directly used as the true voxel size information of the non-empty voxel.

As a possible implementation method, when determining the true voxel size information of non-empty voxels based on the length, width and height of the minimum circumscribed cuboid, each side length of the minimum circumscribed cuboid can be compared with the preset length, wherein each side length includes the lengths corresponding to the length, width and height of the minimum circumscribed cuboid, and the preset length can be set according to actual needs, such as setting the preset length to 0.1; when there is a side length less than the preset length in each side length, the side length less than the preset length is updated to the preset length, and then based on each updated side length, the true voxel size information of the non-empty voxel is determined.

That is to say, in the embodiment of the present disclosure, for the determined minimum circumscribed cuboid, the length, width and height of the minimum circumscribed cuboid can be compared with the preset length respectively. If at least one side length of the length, width and height is less than the preset length, the side length less than the preset length is replaced with the preset length, and then the updated side length is used as the true voxel size information of the non-empty voxel. As a result, each side length in the true voxel size information of the non-empty voxel is not less than the preset length, so as to avoid the final non-empty voxel size being too small, resulting in a large number of holes, making the voxel discontinuous, thereby affecting the obstacle detection result.

In some embodiments, in the embodiments of the present disclosure, the true voxel size information of a non-empty voxel can be expressed as (dl, dw, dh), where dl represents the length of the minimum enclosing rectangle, dw represents the width of the minimum enclosing rectangle, and dh represents the height of the minimum enclosing rectangle.

For example, assuming that the length, width, and height of the minimum circumscribed cuboid determined according to the points in a non-empty voxel are 0.2, 0.1, and 0.08, respectively, then the true voxel size information of the non-empty voxel can be determined to be (0.2, 0.1, 0.08).

For another example, assuming the preset length is 0.1, the length, width, and height of the minimum circumscribed cuboid determined based on the points in a non-empty voxel are 0.2, 0.1, and 0.08, respectively. It can be determined that the height of the minimum circumscribed cuboid is less than the preset length, and the height of the minimum circumscribed cuboid is updated to 0.1. The final voxel size information of the non-empty voxel is (0.2, 0.1, 0.1).

In the embodiment of the present disclosure, the minimum circumscribed cuboid of at least one point in the non-empty voxel is determined based on at least one point in the non-empty voxel, and the real voxel size information of the non-empty voxel is determined based on the length, width and height of the minimum circumscribed cuboid. Thus, it is possible to determine the real size of the voxel based on the position of the point cloud contained in the non-empty voxel, which can improve the accuracy of the voxel size and make the voxel representation of the outline of the obstacle more precise.

In an optional embodiment of the present disclosure, when a non-empty voxel contains only one point, it can also be determined that the length, width and height of the minimum circumscribed cuboid of the point are all 0. In this case, the above-mentioned method of comparing the length, width and height of the minimum circumscribed cuboid with the preset length can be adopted. When there is a side length less than the preset length among the length, width and height, the side length less than the preset length is updated to the preset length, and then the true voxel size information of the non-empty voxel is determined based on each updated side length, so as to determine the true voxel size information of the non-empty voxel when the non-empty voxel contains only one point.

In the embodiment of the present disclosure, when a non-empty voxel contains only one point, the length, width and height of the smallest circumscribed cuboid of the point are all 0, and the preset length is a number greater than 0. The length, width and height of the smallest circumscribed cuboid are obviously smaller than the preset length. Therefore, the preset length can be determined as the true voxel size information of the non-empty voxel containing one point. Assuming the preset The length is 0.1. For a non-empty voxel containing only one point, (0.1, 0.1, 0.1) can be used as the actual voxel size information of the non-empty voxel.

It should be noted that in the embodiment of the present disclosure, there is no particular order for the execution of step 403 and step 404. The two can be executed simultaneously or one after the other. The embodiment of the present disclosure only takes the execution of step 404 after step 403 as an example, which cannot be used as a limitation of the present disclosure.

Step 405 : Generate a real obstacle feature corresponding to the obstacle in each voxel based on the real voxel occupancy information corresponding to each voxel, the real voxel position information corresponding to the non-empty voxel, and the real voxel size information.

In the disclosed embodiment, for each non-empty voxel, after determining the real voxel position information and the real voxel size information of the non-empty voxel, the real obstacle features of the obstacle belonging to the point cloud data can be generated based on the real voxel occupancy information, the real voxel position information and the real voxel size information of the same non-empty voxel, and the real voxel occupancy information corresponding to each unoccupied empty voxel.

It should be noted that, for an empty voxel whose voxel occupancy real information is unoccupied, the voxel position real information and the voxel size real information may not be provided, or the voxel position real information and the voxel size real information of the empty voxel may be set to 0.

In some embodiments, it is assumed that the real information representation method corresponding to each voxel in the real feature of the obstacle is represented by a unified format as (p, x, y, z, dl, dw, dh), where p represents the real information of voxel occupancy, (x, y, z) represents the coordinates corresponding to the centroid of the point cloud in the voxel, dl represents the length of the minimum bounding rectangle, dw represents the width of the minimum bounding rectangle, and dh represents the height of the minimum bounding rectangle. For unoccupied empty voxels, their real information can be expressed as (p ₁ ,,,,,,), where p ₁ indicates that the voxel is unoccupied, and the other elements are empty, indicating that there is no real information of the voxel position and the real information of the voxel size. For occupied non-empty voxels, their real information can be expressed as (p ₂ ,x ₀ ,y ₀ ,z ₀ ,dl,dw,dh), where p ₂ indicates that the voxel is occupied.

In some embodiments, the real information corresponding to the unoccupied empty voxels in the real feature of the obstacle may be represented by (p ₁ ), where p ₁ indicates that the voxel is unoccupied. Assuming that the true voxel position information of the occupied non-empty voxel is represented by the coordinates of the center point of the non-empty voxel and the coordinate offset of the center point relative to the center of mass of the point cloud in the non-empty voxel, and the true voxel size information of the non-empty voxel is represented by the length, width and height of the minimum circumscribed rectangle of all point clouds in the non-empty voxel, then the true information of the non-empty voxel can be expressed as (p ₂ ,x ₁ ,y ₁ ,z ₁ ,dx,dy,dz,dl,dw,dh), where p ₂ indicates that the voxel is occupied, (x ₁ ,y ₁ ,z ₁ ) represents the coordinates corresponding to the center point of the non-empty voxel, (dx,dy,dz) represents the coordinate offset of the center point of the non-empty voxel relative to the center of mass of the point cloud in the non-empty voxel, dl represents the length of the minimum circumscribed rectangle, dw represents the width of the minimum circumscribed rectangle, and dh represents the height of the minimum circumscribed rectangle.

The training method of the obstacle feature recognition model of the embodiment of the present disclosure determines the real information of voxel occupancy corresponding to each voxel according to the position of each point in the point cloud data, the real information of voxel occupancy includes occupied and unoccupied, and for non-empty voxels whose real information of voxel occupancy is occupied, the centroid of the point cloud in the non-empty voxel is determined as a key point based on a centroid calculation rule according to at least one point in the non-empty voxel, and the real information of the voxel position of the non-empty voxel is determined based on a preset position determination rule according to the determined key point, and the real information of the voxel size of the non-empty voxel is determined by using an external figure corresponding to at least one point in the non-empty voxel, and then based on the real information of voxel occupancy and the non-empty voxel corresponding to each voxel, the real information of the voxel size is determined. The real voxel position information and the real voxel size information corresponding to the voxel are obtained to generate the real features of the obstacle corresponding to each voxel. Therefore, the occupation status of the divided multiple voxels can be marked, and the real position and real size corresponding to each non-empty voxel can be determined. The real voxel information is dynamically generated according to the points in the voxel, which makes the position and contour of the obstacle more precise, which is conducive to improving the accuracy of obstacle feature recognition, thereby improving the accuracy of obstacle detection.

In an optional embodiment of the present disclosure, when converting the acquired point cloud data into a three-dimensional coordinate system, multiple frames of point cloud data collected by the laser radar for the same obstacle can be first obtained, and then the multiple frames of point cloud data can be converted into the same three-dimensional coordinate system. During the conversion, if the same three-dimensional coordinate position corresponds to multiple points in the multiple frames of point cloud data, the multiple points are deduplicated.

Usually, when a laser radar collects point cloud data of an obstacle, a frame of point cloud of the obstacle is obtained at one time. It is difficult for one frame of point cloud to fully describe all the information of the entire obstacle. Therefore, in the embodiment of the present disclosure, multiple frames of point cloud data collected by the laser radar for the same obstacle at different times can be obtained, and then the obtained multiple frames of point cloud data can be spliced. Splicing refers to converting multiple frames of point cloud data into the same three-dimensional coordinate system. It can be understood that for the same three-dimensional coordinate position in three-dimensional space, more than one frame of point cloud data may contain the point corresponding to the position. For this case, when splicing, the multiple points corresponding to the position can be deduplicated, and only one point is retained to be displayed in the three-dimensional coordinate system. Alternatively, for the case where a three-dimensional coordinate position corresponds to multiple points, the data of multiple points corresponding to the same three-dimensional coordinate position can also be averaged, and the average obtained is used as the information of the point corresponding to the three-dimensional coordinate position in the three-dimensional coordinate system.

In the embodiment of the present disclosure, by acquiring multi-frame point cloud data and converting the multi-frame point cloud data into the same coordinate system, a denser point cloud can be obtained, so that the point cloud data displayed in the three-dimensional coordinate system can better reflect all the information of the real obstacle.

FIG5 shows a schematic diagram of the network structure of an obstacle feature recognition model of a specific embodiment of the present disclosure. As shown in FIG5 , the obstacle feature recognition model includes a feature extraction network, a feature conversion network, a voxel occupancy prediction network, a voxel position prediction network and a voxel size prediction network, wherein the feature extraction network is used to extract features of the input surround view image to obtain two-dimensional features; the feature conversion network is used to convert the two-dimensional features extracted by the feature extraction network into three-dimensional features; the voxel occupancy prediction network is used to obtain voxel occupancy prediction information based on the three-dimensional features, the voxel position prediction network is used to obtain voxel position prediction information based on the three-dimensional features, and the voxel size prediction network is used to obtain voxel size prediction information based on the three-dimensional features. The obstacle feature recognition model shown in FIG5 can be obtained by training using the real features of obstacles obtained based on point cloud data as training targets, wherein the real features of obstacles include real voxel occupancy information, real voxel position information and real voxel size information. As shown in FIG5 , six surround view images are input into the obstacle feature recognition model, and two-dimensional features are first obtained through the feature extraction network, and the two-dimensional features are restored to three-dimensional features after passing through the feature conversion module. As shown in FIG5 , the obstacle feature recognition model may also include an upsampling network. After upsampling, the three-dimensional features are respectively input into the voxel occupancy prediction network, the voxel position prediction network and the voxel size prediction network to obtain voxel occupancy prediction information, voxel position prediction information and voxel size prediction information, and then based on the voxel occupancy prediction information, the voxel position prediction information and the voxel size prediction information, the obstacle prediction features are obtained. By using the obstacle feature recognition model shown in FIG5 , accurate obstacle 3D features can be obtained, which provides conditions for accurately identifying obstacles based on surround view images.

Corresponding to the above embodiment, the embodiment of the present disclosure further provides a method for acquiring obstacle features, which uses the obstacle feature recognition model trained by the above embodiment to obtain obstacle prediction features of the obstacle.

FIG6 is a flow chart of a method for acquiring obstacle features provided in an embodiment of the present disclosure. The method for acquiring obstacle features may be executed by an apparatus for acquiring obstacle features provided in an embodiment of the present disclosure. The apparatus for acquiring obstacle features may be implemented using software and/or hardware and may be integrated on a computer device, which may be an electronic device such as a computer or a server.

As shown in FIG. 6 , the method for acquiring obstacle features may include steps 501 and 502 .

Step 501: Acquire a surround view image of an obstacle.

The obstacles may be buildings, trees, human bodies, vehicles, etc.

In some embodiments, a camera may be used to capture a surround view image of an obstacle, the surround view image including multiple images, for example, the surround view image may include images in six directions: front, rear, left front, right front, left rear, and right rear.

Step 502: Input the surround view image into a pre-trained obstacle feature recognition model to obtain obstacle prediction features corresponding to the obstacle.

The obstacle feature recognition model is pre-trained by the obstacle feature recognition model training method provided in the above embodiment. The obstacle feature recognition model includes a feature extraction network, a feature conversion network and an obstacle feature prediction network. The feature extraction network is used to extract two-dimensional features corresponding to the surround view image, the feature conversion network is used to convert the two-dimensional features into three-dimensional features, and the obstacle feature prediction network is used to predict the obstacle prediction features corresponding to the obstacle based on the three-dimensional features.

In the disclosed embodiment, the acquired surround view image of the obstacle is input into the obstacle feature recognition model that has been trained in advance. The feature extraction network in the obstacle feature recognition model first extracts features from the surround view image to obtain two-dimensional features corresponding to the surround view image. Next, the two-dimensional features output by the feature extraction network are input into the feature conversion network, which performs feature conversion to convert the two-dimensional features into three-dimensional features corresponding to the obstacle. Finally, the three-dimensional features output by the feature conversion network are input into the obstacle feature prediction network, which performs obstacle feature prediction based on the input three-dimensional features and outputs obstacle prediction features corresponding to the obstacle.

Among them, the obstacle feature prediction network can be set according to actual needs. The obstacle feature prediction network may include but is not limited to at least one of a voxel occupancy prediction network, a voxel position prediction network, and a voxel size prediction network. Accordingly, at least one of the voxel occupancy prediction information, the voxel position prediction information, and the voxel size prediction information can be predicted as the obstacle prediction feature of the obstacle.

The obstacle feature acquisition method of the disclosed embodiment obtains a surround view image of the obstacle, inputs the surround view image into a pre-trained obstacle feature recognition model, and obtains obstacle prediction features corresponding to the obstacle. Thus, by using the pre-trained obstacle feature recognition model, it is possible to obtain high-precision obstacle features of the obstacle in three-dimensional space based on the surround view image of the obstacle, which is beneficial to improving the accuracy of obstacle recognition, thereby helping the vehicle to effectively avoid obstacles in autonomous driving.

In order to implement the above embodiment, the embodiment of the present disclosure also provides a training device for an obstacle feature recognition model.

FIG. 7 is a schematic diagram of the structure of a training device for an obstacle feature recognition model provided by an embodiment of the present disclosure. It can be implemented by software and/or hardware, and can be integrated on a computer device, which can be an electronic device such as a computer or a server.

As shown in FIG. 7 , the obstacle feature recognition model training device 60 provided in the embodiment of the present disclosure may include: a first acquisition module 601 , a determination module 602 , a second acquisition module 603 and a training module 604 .

The first acquisition module 601 is used to acquire surround view images and point cloud data corresponding to the same obstacle.

The determination module 602 is used to determine the real obstacle feature corresponding to each voxel of the obstacle based on the point cloud data and the preset voxels in the corresponding three-dimensional space.

The second acquisition module 603 is used to input the surround view image into the initial model to be trained to obtain obstacle prediction features corresponding to the obstacle, wherein the initial model includes a feature extraction network, a feature conversion network and an obstacle feature prediction network.

The training module 604 is used to train the initial model according to the real obstacle features and the predicted obstacle features corresponding to the obstacles to obtain an obstacle feature recognition model.

In some embodiments, the real feature of the obstacle includes real information of voxel occupancy, and the obstacle feature prediction network includes a voxel occupancy prediction network; the second acquisition module 603 is further used to:

Inputting the surround view image into a feature extraction network to obtain two-dimensional features corresponding to the surround view image;

Inputting the two-dimensional features into the feature conversion network to obtain three-dimensional features corresponding to the obstacles;

The three-dimensional features are input into the voxel occupancy prediction network to obtain voxel occupancy prediction information output by the voxel occupancy prediction network.

In some embodiments, the real feature of the obstacle further includes real information of the voxel position and real information of the voxel size corresponding to the voxel, and the obstacle feature prediction network further includes a voxel position prediction network and a voxel size prediction network; the second acquisition module 603 is further used to:

Inputting the three-dimensional features into the voxel position prediction network to obtain voxel position prediction information output by the voxel position prediction network;

The three-dimensional features are input into the voxel size prediction network to obtain voxel size prediction information output by the voxel size prediction network.

In some embodiments, the training module 604 is further configured to:

Determining a first loss value according to the voxel occupancy prediction information and the voxel occupancy actual information;

Determining a second loss value according to the voxel position prediction information and the voxel position real information;

Determining a third loss value according to the voxel size prediction information and the voxel size real information;

Based on the first loss value, the second loss value, and the third loss value, a network parameter of the initial model is adjusted.

In some embodiments, the determining module 602 includes:

A conversion submodule, used for converting the point cloud data into a three-dimensional coordinate system;

A segmentation submodule, used for voxelizing the three-dimensional space in the three-dimensional coordinate system according to a preset size to obtain a plurality of voxels in the three-dimensional space;

A generation submodule is used to generate, based on the point cloud data contained in each voxel of the multiple voxels, the real features of the obstacle corresponding to the obstacle in each voxel by using the key points of the point cloud in the voxel and the external graphics of the point cloud, wherein the real features of the obstacle include the real information of voxel occupancy corresponding to each voxel, the real information of voxel position corresponding to the non-empty voxel, and the real information of voxel size.

In some embodiments, the generating submodule comprises:

A first determining unit, configured to determine, according to the position of each point in the point cloud data, real voxel occupancy information corresponding to each voxel, wherein the real voxel occupancy information includes occupied and unoccupied;

A second determining unit is configured to determine, for the non-empty voxel whose real voxel occupancy information is occupied, the centroid of the point cloud in the non-empty voxel as the key point based on a centroid calculation rule according to at least one point in the non-empty voxel;

A third determination unit, configured to determine the real voxel position information of the non-empty voxel according to the key point and based on a preset position determination rule;

A fourth determining unit, configured to determine the real voxel size information of the non-empty voxel by using the circumscribed graph corresponding to at least one point in the non-empty voxel;

A generating unit is used to generate a real obstacle feature corresponding to the obstacle in each voxel based on the real voxel occupancy information corresponding to each voxel, the real voxel position information corresponding to the non-empty voxel, and the real voxel size information.

In some embodiments, the position determination rule includes: determining the position coordinates corresponding to the key point as the real voxel position information of the non-empty voxel.

In some embodiments, the position determination rule includes: determining the coordinate offset of the center point of the non-empty voxel relative to the key point as the position information of the non-empty voxel; the third determination unit is further used to:

Obtaining a first coordinate of a center point of the non-empty voxel;

Determine, according to the first coordinate and the second coordinate corresponding to the key point, a coordinate offset of the center point relative to the key point, wherein the coordinate offset includes an offset corresponding to each coordinate axis in the three-dimensional coordinate system;

The first coordinate and the coordinate offset are determined as the real voxel position information of the non-empty voxel.

In some embodiments, the fourth determining unit is further configured to:

Determine, according to at least one point in the non-empty voxel, a minimum circumscribed cuboid of the at least one point in the non-empty voxel;

According to the length, width and height of the minimum circumscribed cuboid, the real voxel size information of the non-empty voxel is determined.

In some embodiments, the fourth determining unit is further configured to:

When the non-empty voxel contains a point, it is determined that the length, width and height of the minimum circumscribed cuboid are all 0.

In some embodiments, the fourth determining unit is further configured to:

Compare each side length of the minimum circumscribed cuboid with a preset length, wherein each side length includes the lengths corresponding to the length, width, and height of the minimum circumscribed cuboid;

When there is a side length shorter than the preset length among each side length, updating the side length shorter than the preset length to the preset length;

Based on each updated edge length, the real voxel size information of the non-empty voxel is determined.

The training device for the obstacle feature recognition model that can be configured on a computer device provided in the embodiments of the present disclosure can execute any training method for the obstacle feature recognition model applied to a computer device provided in the embodiments of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method. For the contents not fully described in the embodiments of the device of the present disclosure, reference can be made to the description in any method embodiment of the present disclosure.

In order to implement the above embodiment, the embodiment of the present disclosure also provides a device for acquiring obstacle characteristics.

FIG8 is a schematic diagram of the structure of an obstacle feature acquisition device provided in an embodiment of the present disclosure. The device may be implemented using software and/or hardware and may be integrated on a computer device, which may be an electronic device such as a computer or a server.

As shown in FIG. 8 , the obstacle feature acquisition device 70 provided in the embodiment of the present disclosure may include: an image acquisition module 701 and a feature prediction module 702 .

The image acquisition module 701 is used to acquire a surround view image of an obstacle.

The feature prediction module 702 is used to input the surround view image into a pre-trained obstacle feature recognition model to obtain obstacle prediction features corresponding to the obstacle.

The obstacle feature acquisition device provided in the embodiments of the present disclosure and which can be configured on a computer device can execute any obstacle feature acquisition method provided in the embodiments of the present disclosure and which is applied to a computer device, and has the corresponding functional modules and beneficial effects of the execution method. For the contents not fully described in the embodiments of the present disclosure, reference can be made to the description in any method embodiment of the present disclosure.

The embodiments of the present disclosure further provide a computer device, including a processor and a memory; the processor is used to execute the steps of each embodiment of the training method of the obstacle feature recognition model as described in any of the aforementioned embodiments, or to execute the steps of each embodiment of the method for obtaining obstacle features as described in any of the aforementioned embodiments, by calling the program or instructions stored in the memory. To avoid repeated description, they are not repeated here.

The embodiments of the present disclosure further provide a computer-readable storage medium, which is non-transitory and stores programs or instructions. The programs or instructions enable a computer to execute the steps of each embodiment of the method for training an obstacle feature recognition model as described in any of the aforementioned embodiments, or to execute the steps of each embodiment of the method for acquiring obstacle features as described in any of the aforementioned embodiments. To avoid repeated description, they are not repeated here.

The embodiments of the present disclosure also provide a computer program product, which is used to execute the steps of each embodiment of the method for training an obstacle feature recognition model as described in any of the aforementioned embodiments, or to execute the steps of each embodiment of the method for acquiring obstacle features as described in any of the aforementioned embodiments.

The present disclosure also provides a computer program, wherein the computer program includes computer program code, and when the computer program code is run on a computer, the computer executes the obstacle control method as described in any of the above embodiments. A method for training a feature recognition model, or a method for acquiring obstacle features as described in any of the above embodiments.

It should be noted that the aforementioned explanations of the method and device embodiments are also applicable to the electronic device, computer-readable storage medium, computer program product and computer program of the above embodiments, and will not be repeated here.

It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

All embodiments of the present disclosure may be implemented individually or in combination with other embodiments, and are deemed to be within the protection scope required by the present disclosure.

Claims

A training method for an obstacle feature recognition model, characterized in that the method comprises:

Obtain surround view images and point cloud data corresponding to the same obstacle;

Based on the point cloud data and preset voxels in the corresponding three-dimensional space, determine the real obstacle features corresponding to the obstacle in each voxel;

Inputting the surround view image into an initial model to be trained to obtain obstacle prediction features corresponding to the obstacle, wherein the initial model includes a feature extraction network, a feature conversion network, and an obstacle feature prediction network;

The initial model is trained according to the real obstacle features and the predicted obstacle features corresponding to the obstacles to obtain an obstacle feature recognition model.
The method according to claim 1, characterized in that the real feature of the obstacle includes real information of voxel occupancy, and the obstacle feature prediction network includes a voxel occupancy prediction network;

The step of inputting the surround view image into the initial model to be trained to obtain obstacle prediction features corresponding to the obstacle includes:

Inputting the surround view image into the feature extraction network to obtain two-dimensional features corresponding to the surround view image;

Inputting the two-dimensional features into the feature conversion network to obtain three-dimensional features corresponding to the obstacles;

The three-dimensional features are input into the voxel occupancy prediction network to obtain voxel occupancy prediction information output by the voxel occupancy prediction network.
The method according to claim 2 is characterized in that the real feature of the obstacle also includes the real information of the voxel position and the real information of the voxel size corresponding to the voxel, and the obstacle feature prediction network also includes the voxel position prediction network and the voxel size prediction network;

The step of inputting the surround view image into the initial model to be trained to obtain obstacle prediction features corresponding to the obstacle further includes:

Inputting the three-dimensional features into the voxel position prediction network to obtain voxel position prediction information output by the voxel position prediction network;

The three-dimensional features are input into the voxel size prediction network to obtain voxel size prediction information output by the voxel size prediction network.
The method according to claim 3, characterized in that the training of the initial model according to the real obstacle features and the predicted obstacle features corresponding to the obstacles comprises:

Determining a first loss value according to the voxel occupancy prediction information and the voxel occupancy actual information;

Determining a second loss value according to the voxel position prediction information and the voxel position real information;

Determining a third loss value according to the voxel size prediction information and the voxel size real information;

Based on the first loss value, the second loss value, and the third loss value, a network parameter of the initial model is adjusted.
The method according to any one of claims 1 to 4, characterized in that based on the point cloud data, And corresponding to the preset voxels in the three-dimensional space, determining the real obstacle features corresponding to the obstacle in each voxel, including:

Converting the point cloud data into a three-dimensional coordinate system;

voxelize the three-dimensional space in the three-dimensional coordinate system according to a preset size to obtain a plurality of voxels in the three-dimensional space;

According to the point cloud data contained in each voxel of the multiple voxels, the real features of the obstacle corresponding to the obstacle in each voxel are generated by utilizing the key points of the point cloud in the voxel and the external graphics of the point cloud. The real features of the obstacle include the real information of voxel occupancy corresponding to each voxel, the real information of voxel position corresponding to the non-empty voxel, and the real information of voxel size.
The method according to claim 5, characterized in that the step of generating the real obstacle features corresponding to the obstacle in each voxel using the key points of the point cloud in the voxel and the circumscribed graphics of the point cloud according to the point cloud data contained in each voxel of the plurality of voxels comprises:

Determine, according to the position of each point in the point cloud data, the real voxel occupancy information corresponding to each voxel, wherein the real voxel occupancy information includes occupied and unoccupied;

For the non-empty voxel whose real voxel occupancy information is occupied, determining the centroid of the point cloud in the non-empty voxel as the key point based on a centroid calculation rule according to at least one point in the non-empty voxel;

According to the key points, based on a preset position determination rule, determining the real voxel position information of the non-empty voxel;

Determine the real voxel size information of the non-empty voxel by using the circumscribed figure corresponding to at least one point in the non-empty voxel;

Based on the voxel occupancy real information corresponding to each voxel, the voxel position real information corresponding to the non-empty voxel, and the voxel size real information, a real obstacle feature corresponding to the obstacle in each voxel is generated.
The method according to claim 6, characterized in that the location determination rule comprises:

The position coordinates corresponding to the key points are determined as the real voxel position information of the non-empty voxels.
The method according to claim 6 or 7, characterized in that the position determination rule comprises:

Determine the coordinate offset of the center point of the non-empty voxel relative to the key point as the position information of the non-empty voxel;

The step of determining the real voxel position information of the non-empty voxel based on the key point and a preset position determination rule includes:

Obtaining a first coordinate of a center point of the non-empty voxel;

Determine, according to the first coordinate and the second coordinate corresponding to the key point, a coordinate offset of the center point relative to the key point, wherein the coordinate offset includes an offset corresponding to each coordinate axis in the three-dimensional coordinate system;

The first coordinate and the coordinate offset are determined as the real voxel position information of the non-empty voxel.
The method according to any one of claims 6 to 8, characterized in that the determining the real voxel size information of the non-empty voxel by using the circumscribed graph corresponding to at least one point in the non-empty voxel comprises:

Determine the minimum circumscribed area of the at least one point in the non-empty voxel according to the at least one point in the non-empty voxel. cuboid;

According to the length, width and height of the minimum circumscribed cuboid, the real voxel size information of the non-empty voxel is determined.
The method according to claim 9, characterized in that the method further comprises:

When the non-empty voxel contains a point, it is determined that the length, width and height of the minimum circumscribed cuboid are all 0.
The method according to claim 9 or 10, characterized in that the determining the real voxel size information of the non-empty voxel according to the length, width and height of the minimum circumscribed cuboid comprises:

Compare each side length of the minimum circumscribed cuboid with a preset length, wherein each side length includes the lengths corresponding to the length, width, and height of the minimum circumscribed cuboid;

When there is a side length shorter than the preset length among each side length, updating the side length shorter than the preset length to the preset length;

Based on each updated edge length, the real voxel size information of the non-empty voxel is determined.
A method for acquiring obstacle features, characterized in that the method comprises:

Obtain a surround view of obstacles;

Inputting the surround view image into a pre-trained obstacle feature recognition model to obtain obstacle prediction features corresponding to the obstacle;

Among them, the obstacle feature recognition model includes a feature extraction network, a feature conversion network and an obstacle feature prediction network. The feature extraction network is used to extract the two-dimensional features corresponding to the surround view image, the feature conversion network is used to convert the two-dimensional features into three-dimensional features, and the obstacle feature prediction network is used to predict the obstacle prediction features corresponding to the obstacle based on the three-dimensional features.
A training device for an obstacle feature recognition model, characterized in that the device comprises:

The first acquisition module is used to acquire the surround view image and point cloud data corresponding to the same obstacle;

A determination module, used to determine the real features of the obstacle corresponding to the obstacle based on the point cloud data;

A second acquisition module is used to input the surround view image into the initial model to be trained to obtain obstacle prediction features corresponding to the obstacle;

The training module is used to train the initial model according to the real obstacle features and the predicted obstacle features corresponding to the obstacles to obtain an obstacle feature recognition model.
A device for acquiring obstacle characteristics, characterized in that the device comprises:

An image acquisition module is used to acquire a surround view image of obstacles;

A feature prediction module, used to input the surround view image into a pre-trained obstacle feature recognition model to obtain obstacle prediction features corresponding to the obstacle;

Among them, the obstacle feature recognition model includes a feature extraction network, a feature conversion network and an obstacle feature prediction network. The feature extraction network is used to extract the two-dimensional features corresponding to the surround view image, the feature conversion network is used to convert the two-dimensional features into three-dimensional features, and the obstacle feature prediction network is used to predict the obstacle prediction features corresponding to the obstacle based on the three-dimensional features.
A computer device, comprising: a processor and a memory;

The processor is used to execute the training method of the obstacle feature recognition model according to any one of claims 1 to 11, or to execute the method for acquiring obstacle features according to claim 12, by calling the program or instruction stored in the memory.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores a program or instruction, wherein the program or instruction enables a computer to execute the training method of the obstacle feature recognition model as described in any one of claims 1 to 11, or execute the method for obtaining obstacle features as described in claim 12.
A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method for training an obstacle feature recognition model as described in any one of claims 1 to 11, or the method for acquiring obstacle features as described in claim 12.
A computer program, characterized in that the computer program includes computer program code, and when the computer program code is run on a computer, the computer executes the training method of the obstacle feature recognition model as described in any one of claims 1 to 11, or the method for obtaining obstacle features as described in claim 12.