Nothing Special   »   [go: up one dir, main page]

CN112967296B - Point cloud dynamic region graph convolution method, classification method and segmentation method - Google Patents

Point cloud dynamic region graph convolution method, classification method and segmentation method Download PDF

Info

Publication number
CN112967296B
CN112967296B CN202110261653.5A CN202110261653A CN112967296B CN 112967296 B CN112967296 B CN 112967296B CN 202110261653 A CN202110261653 A CN 202110261653A CN 112967296 B CN112967296 B CN 112967296B
Authority
CN
China
Prior art keywords
point cloud
information
point
convolution
dynamic area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110261653.5A
Other languages
Chinese (zh)
Other versions
CN112967296A (en
Inventor
王勇
岳晨珂
汤鑫彤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sichuan Jiulai Technology Co ltd
Original Assignee
Chongqing University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing University of Technology filed Critical Chongqing University of Technology
Priority to CN202110261653.5A priority Critical patent/CN112967296B/en
Publication of CN112967296A publication Critical patent/CN112967296A/en
Application granted granted Critical
Publication of CN112967296B publication Critical patent/CN112967296B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/25Fusion techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10028Range image; Depth image; 3D point clouds
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20212Image combination
    • G06T2207/20221Image fusion; Image merging

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了一种点云动态区域图卷积方法,以及使用这种点云动态区域图卷积方法的一种点云动态区域图分类方法和一种点云动态区域图分割方法。本发明采用了一种针对点云的新的卷积运算形式,根据构建点云图结构,通过一种非线性方法聚合多个不同邻域的点特征信息,使得神经元能够自适应的选取区域大小。与PointNet等现有的在单个点上进行分析的技术方案相比,本发明构建了多个不同的局部邻域图结构,让每个神经元都能够自适应的选取适合的邻域感受野大小,然后利用每个点与邻域点之间的联系进行类似的卷积运算获取局部特征,能够更好的结合周围邻域信息,更加有效的提取到局部几何信息,最终提高对点云数据的分类或分割的准确率。

Figure 202110261653

The invention discloses a point cloud dynamic area map convolution method, a point cloud dynamic area map classification method and a point cloud dynamic area map segmentation method using the point cloud dynamic area map convolution method. The present invention adopts a new convolution operation form for point cloud, according to the construction of point cloud map structure, aggregates point feature information of multiple different neighborhoods through a nonlinear method, so that neurons can adaptively select the size of the area . Compared with existing technical solutions such as PointNet that analyze on a single point, the present invention constructs a plurality of different local neighborhood graph structures, so that each neuron can adaptively select a suitable neighborhood receptive field size , and then use the connection between each point and the neighborhood points to perform a similar convolution operation to obtain local features, which can better combine the surrounding neighborhood information, extract local geometric information more effectively, and finally improve the accuracy of point cloud data. Classification or segmentation accuracy.

Figure 202110261653

Description

一种点云动态区域图卷积方法、分类方法及分割方法A point cloud dynamic area map convolution method, classification method and segmentation method

技术领域technical field

本发明属于计算机视觉技术领域,具体涉及一种点云动态区域图卷积方法、分类方法及分割方法。The invention belongs to the technical field of computer vision, and in particular relates to a point cloud dynamic area map convolution method, a classification method and a segmentation method.

背景技术Background technique

点云数据包含了丰富的语义信息,具有高密度、高精度等特点,但是由于点云数据的不规则性和无序性,使得基于点云数据的语义分析仍是一项困难的挑战。早期的一些方法使用带有复杂规则的特征来进行手工提取解决此类问题。近年来随着深度学习和机器学习技术的火热增长,深度学习的方法也被引入用于点云数据的分析与处理。深度网络需要处理的数据为规则形状,点云数据从根本上说是不规则的,并且点云数据的空间分布不受点云的排列方式的影响,所以使用深度学习模型处理点云数据的常用方法是将原始点云数据转换成网格、体素、树形等数据结构形式。一些先进的深度学习网络如PointNet和PointNet++是专门为了处理点云的不规则性而设计,它可以直接处理原始点云数据,不需要把点云数据转换成规则形状再进行处理。但PointNet和PointNet++都不支持卷积操作,且不能有效的提取到局部几何信息。Point cloud data contains rich semantic information and has the characteristics of high density and high precision. However, due to the irregularity and disorder of point cloud data, semantic analysis based on point cloud data is still a difficult challenge. Some early methods used features with complex rules for manual extraction to solve such problems. In recent years, with the rapid growth of deep learning and machine learning technology, deep learning methods have also been introduced for the analysis and processing of point cloud data. The data that needs to be processed by the deep network is of regular shape, and the point cloud data is fundamentally irregular, and the spatial distribution of the point cloud data is not affected by the arrangement of the point cloud, so it is common to use the deep learning model to process point cloud data. The method is to convert the original point cloud data into grid, voxel, tree and other data structures. Some advanced deep learning networks such as PointNet and PointNet++ are specially designed to deal with the irregularities of point clouds. They can directly process raw point cloud data without converting point cloud data into regular shapes for further processing. However, neither PointNet nor PointNet++ supports convolution operations, and cannot effectively extract local geometric information.

目前很多工作都集中于利用卷积操作处理点云数据。2DCNN直接拓展到3D领域,将3D空间看作体积网格,使用3D卷积进行操作。虽然3D卷积在点云分类和分割任务上取得了不错的效果,但是它们对于存储性能的高要求以及所需的高计算成本使得它们在大规模数据集和大型场景上仍然存在精度不足的问题。Much work currently focuses on processing point cloud data using convolution operations. 2DCNN directly expands to the 3D field, treats the 3D space as a volume grid, and uses 3D convolution to operate. Although 3D convolutions have achieved good results in point cloud classification and segmentation tasks, their high requirements for storage performance and high computational costs make them still have insufficient accuracy on large-scale datasets and large scenes. .

综上所述,如何提高对点云数据的分类或分割的准确率成为了本领域技术人员急需解决的问题。To sum up, how to improve the accuracy of classification or segmentation of point cloud data has become an urgent problem to be solved by those skilled in the art.

发明内容Contents of the invention

针对现有技术中存在的上述不足,本发明实际解决的问题是:提高对点云数据的分类或分割的准确率。Aiming at the above-mentioned deficiencies in the prior art, the problem actually solved by the present invention is: to improve the accuracy rate of classification or segmentation of point cloud data.

为解决上述技术问题,本发明采用了如下的技术方案:In order to solve the problems of the technologies described above, the present invention adopts the following technical solutions:

一种点云动态区域图卷积方法,包括:A point cloud dynamic region map convolution method, comprising:

S1、获取三维点云数据X,X={α123,…,αi,…,αn},αi表示第i个点的数据,n表示所述三维点云数据中点的个数,αi={xi,yi,zi},xi、yi和zi表示αi的三维坐标;S1. Acquire 3D point cloud data X, X={α 1 , α 2 , α 3 ,…,α i ,…,α n }, α i represents the data of the i-th point, and n represents the 3D point cloud data The number of midpoints, α i ={ xi , y i , z i }, xi , y i and z i represent the three-dimensional coordinates of α i ;

S2、对三维点云数据X进行两次独立的k近邻运算,得到两个局部特征图y和z,所述两次独立的k近邻运算的k值不同;S2. Perform two independent k-nearest neighbor operations on the three-dimensional point cloud data X to obtain two local feature maps y and z, and the k values of the two independent k-nearest neighbor operations are different;

S3、将两个局部特征图y和z融合得到融合信息T,T=Sum(y,z);S3. Fusion of two local feature maps y and z to obtain fusion information T, T=Sum(y,z);

S4、对融合信息T进行池化操作得到特征通信信息s1,s1=MAX(T);S4. Perform pooling operation on the fusion information T to obtain characteristic communication information s 1 , s 1 =MAX(T);

S5、使用全连接层对特征通信信息s1进行紧凑降维得到紧凑特征s2,s2=FC(s1);S5. Use the fully connected layer to compact and reduce the dimensionality of the feature communication information s 1 to obtain a compact feature s 2 , s 2 =FC(s 1 );

S6、利用注意力机制从紧凑特征中自适应地选择不同区域的分支维数信息,并利用softmax对权重进行归一化,得到归一化信息a1和a2

Figure BDA0002970290580000021
Figure BDA0002970290580000022
FC1()和FC2()分别表示与y和z对应的全连接层操作;S6. Use the attention mechanism to adaptively select the branch dimension information of different regions from the compact features, and use softmax to normalize the weights to obtain normalized information a 1 and a 2 ,
Figure BDA0002970290580000021
Figure BDA0002970290580000022
FC 1 () and FC 2 () represent the fully connected layer operations corresponding to y and z, respectively;

S7、将归一化信息与局部特征图相乘后求和得到特征图谱U,U=Sum(a1*y,a2*z)。S7. The normalization information is multiplied by the local feature map and then summed to obtain the feature map U, U=Sum(a 1 *y, a 2 *z).

优选地,步骤S2中两次独立的k近邻运算的k值分别为15和25。Preferably, the k values of the two independent k-nearest neighbor operations in step S2 are 15 and 25 respectively.

一种点云动态区域图分类方法,采用上述的点云动态区域图卷积方法进行卷积操作,以所述特征图谱U作为每一次卷积操作得到的特征。A method for classifying point cloud dynamic area graphs, using the above-mentioned point cloud dynamic area graph convolution method for convolution operations, using the feature map U as a feature obtained by each convolution operation.

一种点云动态区域图分割方法,采用上述的点云动态区域图卷积方法进行卷积操作,以所述特征图谱U作为每一次卷积操作得到的特征。A method for segmenting a point cloud dynamic area map, using the above-mentioned point cloud dynamic area map convolution method for convolution operations, using the feature map U as a feature obtained by each convolution operation.

综上所述,本发明与现有技术相比,具有以下技术效果:In summary, compared with the prior art, the present invention has the following technical effects:

本发明采用了一种针对点云的新的卷积运算形式,根据构建点云图结构,通过一种非线性方法聚合多个不同邻域的点特征信息,使得神经元能够自适应的选取区域大小。与PointNet等现有的在单个点上进行分析的技术方案相比,本发明构建了多个不同的局部邻域图结构,让每个神经元都能够自适应的选取适合的邻域感受野大小,然后利用每个点与邻域点之间的联系进行类似的卷积运算获取局部特征,能够更好的结合周围邻域信息,更加有效的提取到局部几何信息,最终提高对点云数据的分类或分割的准确率。The present invention adopts a new convolution operation form for point cloud, according to the construction of point cloud map structure, through a nonlinear method to aggregate point feature information of multiple different neighborhoods, so that neurons can adaptively select the size of the area . Compared with existing technical solutions such as PointNet that analyze on a single point, the present invention constructs a plurality of different local neighborhood graph structures, so that each neuron can adaptively select a suitable neighborhood receptive field size , and then use the connection between each point and the neighborhood points to perform a similar convolution operation to obtain local features, which can better combine the surrounding neighborhood information, extract local geometric information more effectively, and finally improve the accuracy of point cloud data. Classification or segmentation accuracy.

附图说明Description of drawings

图1为本发明公开的一种点云动态区域图卷积方法的流程图;Fig. 1 is a flow chart of a method for convolution of point cloud dynamic region graphs disclosed by the present invention;

图2为局部点云空间的k近邻图。Figure 2 is the k-nearest neighbor map of the local point cloud space.

具体实施方式Detailed ways

下面结合附图对本发明作进一步的详细说明。The present invention will be further described in detail below in conjunction with the accompanying drawings.

如图1所示,本发明公开了一种点云动态区域图卷积方法,包括:As shown in Figure 1, the present invention discloses a method for convolution of a point cloud dynamic area graph, including:

S1、获取三维点云数据X,X={α123,…,αi,…,αn},αi表示第i个点的数据,n表示所述三维点云数据中点的个数,αi={xi,yi,zi},xi、yi和zi表示αi的三维坐标;S1. Acquire 3D point cloud data X, X={α 1 , α 2 , α 3 ,…,α i ,…,α n }, α i represents the data of the i-th point, and n represents the 3D point cloud data The number of midpoints, α i ={ xi , y i , z i }, xi , y i and z i represent the three-dimensional coordinates of α i ;

S2、对三维点云数据X进行两次独立的k近邻运算,得到两个局部特征图y和z,所述两次独立的k近邻运算的k值不同;S2. Perform two independent k-nearest neighbor operations on the three-dimensional point cloud data X to obtain two local feature maps y and z, and the k values of the two independent k-nearest neighbor operations are different;

如图2所示,为局部点云空间的k近邻图,定义αj1j2,…,αjk为αi的k个近邻点,eij为边缘特征,定义为eij=hθij),其中θ为训练参数,非线性函数hθij):RC×RC→RC,RC为聚合之后的特征。则图卷积中第i个点的输出可以表示为:

Figure BDA0002970290580000031
类似于2D图像中的卷积操作,将αi看作卷积区域的中心像素,αj则是αi周围的像素块。对图结构中αi和αj形成的有向边eij设计的边缘函数定义为:hθij)=hθiij)。这样的结构同时结合了全局形状信息以及局部邻域信息,且通过MLP实现。聚合函数选取max函数进行聚合操作。As shown in Figure 2, it is the k-nearest neighbor map of the local point cloud space, and α j1 , α j2 ,..., α jk are defined as the k neighbor points of α i , e ij is the edge feature, defined as e ij = h θ ( α ij ), where θ is the training parameter, the nonlinear function h θij ): R C × RCRC , and R C is the feature after aggregation. Then the output of the i-th point in the graph convolution can be expressed as:
Figure BDA0002970290580000031
Similar to the convolution operation in 2D images, α i is regarded as the central pixel of the convolution area, and α j is the pixel block around α i . The edge function designed for the directed edge e ij formed by α i and α j in the graph structure is defined as: h θij )=h θiij ). Such a structure combines global shape information and local neighborhood information at the same time, and is realized by MLP. The aggregation function selects the max function to perform the aggregation operation.

S3、将两个局部特征图y和z融合得到融合信息T,T=Sum(y,z);S3. Fusion of two local feature maps y and z to obtain fusion information T, T=Sum(y,z);

S4、对融合信息T进行池化操作得到特征通信信息s1,s1=MAX(T);S4. Perform pooling operation on the fusion information T to obtain characteristic communication information s 1 , s 1 =MAX(T);

S5、使用全连接层对特征通信信息s1进行紧凑降维得到紧凑特征s2,s2=FC(s1);S5. Use the fully connected layer to compact and reduce the dimensionality of the feature communication information s 1 to obtain a compact feature s 2 , s 2 =FC(s 1 );

步骤S3至S5是对从多个分支传过来的信息进行整合编码传送到下一步骤中,实现神经元自适应调整k邻域的大小。最后使用全连接网络对特征进行紧凑降维,既能实现精确自适应选择区域,还能降低尺寸,提高运算效率。Steps S3 to S5 are to integrate and encode the information transmitted from multiple branches and transmit them to the next step, so as to realize the adaptive adjustment of the size of the k-neighborhood of neurons. Finally, the fully connected network is used to perform compact dimensionality reduction on features, which can not only realize accurate adaptive selection of regions, but also reduce size and improve computing efficiency.

S6、利用注意力机制从紧凑特征中自适应地选择不同区域的分支维数信息,并利用softmax对权重进行归一化,得到归一化信息a1和a2

Figure BDA0002970290580000041
Figure BDA0002970290580000042
FC1()和FC2()分别表示与y和z对应的全连接层操作;S6. Use the attention mechanism to adaptively select the branch dimension information of different regions from the compact features, and use softmax to normalize the weights to obtain normalized information a 1 and a 2 ,
Figure BDA0002970290580000041
Figure BDA0002970290580000042
FC 1 () and FC 2 () represent the fully connected layer operations corresponding to y and z, respectively;

S7、将归一化信息与局部特征图相乘后求和得到特征图谱U,U=Sum(a1*y,a2*z)。S7. The normalization information is multiplied by the local feature map and then summed to obtain the feature map U, U=Sum(a 1 *y, a 2 *z).

a1∈1×C′,a2∈1×C′,y∈n×C′,z∈n×C′;C′表示特征通道数。a 1 ∈1×C′, a 2 ∈1×C′, y∈n×C′, z∈n×C′; C′ represents the number of feature channels.

具体实施时,步骤S2中两次独立的k近邻运算的k值分别为15和25。During specific implementation, the k values of the two independent k-nearest neighbor operations in step S2 are 15 and 25 respectively.

采用不同次数的k近邻运算以及k值时的平均类精度和总精度如表2所示,因此,本发明中,最佳的k近邻运算的次数为2,k值分别为15和25。The average class accuracy and total precision when using different times of k-nearest neighbor operations and k values are shown in Table 2. Therefore, in the present invention, the optimal number of k-nearest neighbor operations is 2, and the k-values are 15 and 25 respectively.

表2Table 2

Figure BDA0002970290580000043
Figure BDA0002970290580000043

Figure BDA0002970290580000051
Figure BDA0002970290580000051

本发明采用了一种针对点云的新的卷积运算形式,根据构建点云图结构,通过一种非线性方法聚合多个不同邻域的点特征信息,使得神经元能够自适应的选取区域大小。与PointNet等现有的在单个点上进行分析的技术方案相比,本发明构建了多个不同的局部邻域图结构,让每个神经元都能够自适应的选取适合的邻域感受野大小,然后利用每个点与邻域点之间的联系进行类似的卷积运算获取局部特征,能够更好的结合周围邻域信息,更加有效的提取到局部几何信息,最终提高对点云数据的分类或分割的准确率。The present invention adopts a new convolution operation form for point cloud, according to the construction of point cloud map structure, through a nonlinear method to aggregate point feature information of multiple different neighborhoods, so that neurons can adaptively select the size of the area . Compared with existing technical solutions such as PointNet that analyze on a single point, the present invention constructs a plurality of different local neighborhood graph structures, so that each neuron can adaptively select a suitable neighborhood receptive field size , and then use the connection between each point and the neighborhood points to perform a similar convolution operation to obtain local features, which can better combine the surrounding neighborhood information, extract local geometric information more effectively, and finally improve the accuracy of point cloud data. Classification or segmentation accuracy.

本发明还公开了一种点云动态区域图分类方法,采用上述的点云动态区域图卷积方法进行卷积操作,以所述特征图谱U作为每一次卷积操作得到的特征。The present invention also discloses a method for classifying point cloud dynamic area graphs, which uses the above-mentioned point cloud dynamic area graph convolution method to perform convolution operations, and uses the feature map U as a feature obtained by each convolution operation.

为了验证本发明公开的一种点云动态区域图分类方法的效果,在ModelNet40数据集上进行分类任务的评估。该数据集包含来自40个类别的12311个网格CAD模型,其中有9843个模型用于训练,2468个模型用于测试。本发明遵循DGCNN等模型的实验设置,对每个模型,从网格面均匀采样1024个点,仅使用采样点的三维坐标当作网络的输入数据。In order to verify the effect of a point cloud dynamic area map classification method disclosed in the present invention, the classification task is evaluated on the ModelNet40 data set. The dataset contains 12311 mesh CAD models from 40 categories, of which 9843 models are used for training and 2468 models are used for testing. The present invention follows the experimental settings of models such as DGCNN. For each model, 1024 points are evenly sampled from the grid surface, and only the three-dimensional coordinates of the sampling points are used as the input data of the network.

使用了四个DRG模块提取局部几何特征,每个DRG模块计算出来的特征用于下一个模块进行再计算。对于DRG模块本文选取了两个不同k邻域分支分别为15,25。之后将每个DRG模块得到的特征连接起来获得64+64+128+256=512维特征点云。然后分别使用全局最大池化和平均最大池化获取全局特征。最后使用两个全连接层(512,256)来进行特征分类。Four DRG modules are used to extract local geometric features, and the features calculated by each DRG module are used for recalculation in the next module. For the DRG module, two different k-neighborhood branches are selected as 15 and 25 respectively. Then connect the features obtained by each DRG module to obtain a 64+64+128+256=512-dimensional feature point cloud. Global features are then obtained using global max-pooling and average max-pooling, respectively. Finally, two fully connected layers (512, 256) are used for feature classification.

所有层都包含LeakyReLU和批处理正则化。实验还对不同k邻域数量进行对比,选取最优k邻域数量,并在测试数据集上对模型进行了评估。使用0.1学习率的SGD优化器,并对学习率进行衰减至0.001。训练数据选取批处理数量为24,测试数据选取为16。实验结果如表1所示。All layers incorporate LeakyReLU and batch regularization. The experiment also compares the number of different k-neighborhoods, selects the optimal number of k-neighborhoods, and evaluates the model on the test data set. Use the SGD optimizer with a learning rate of 0.1 and decay the learning rate to 0.001. The number of batches is selected as 24 for the training data and 16 for the test data. The experimental results are shown in Table 1.

表1Table 1

Figure BDA0002970290580000061
Figure BDA0002970290580000061

本发明还公开了一种点云动态区域图分割方法,采用上述的点云动态区域图卷积方法进行卷积操作,以所述特征图谱U作为每一次卷积操作得到的特征。The present invention also discloses a method for segmenting a point cloud dynamic area graph, which adopts the above-mentioned point cloud dynamic area graph convolution method for convolution operation, and uses the feature map U as a feature obtained by each convolution operation.

为了验证本发明公开的一种点云动态区域图分割方法的效果,在ShapeNet数据集上进行部分分割任务。该任务将点云中的每个点分类成物体的几个零件类别标签。该数据集包含来自16个对象类别的16881个3D形状,总共标注了50个零件,每个训练样本中采样了2048个点,同样遵循DGCNN等模型的实验方案。将三层DRGConv模块的输出连接起来,拼接到2048个点的特征中,然后通过MLP(256,256,128)进行特征变换。对于批处理数量、激活函数、学习率等选取与分类网络相同。In order to verify the effect of a point cloud dynamic region map segmentation method disclosed in the present invention, part of the segmentation task is performed on the ShapeNet data set. The task classifies each point in the point cloud into several part class labels of the object. The dataset contains 16881 3D shapes from 16 object categories, a total of 50 parts are labeled, and 2048 points are sampled in each training sample, which also follows the experimental scheme of models such as DGCNN. The outputs of the three-layer DRGConv modules are concatenated, concatenated into features of 2048 points, and then feature transformed by MLP (256, 256, 128). For the number of batches, activation function, learning rate, etc., the selection is the same as that of the classification network.

采用与PointNet一样得评估方法,通过平均在该形状中出现的不同部分的IOU来计算形状的IOU,并通过对属于该类别的所有形状的IOU进行平均计算来获得该类别的IOU。最后通过平均所有测试形状的IOU来计算平均IOU(mIOU)。通过与PointNet、PointNet++、PointCNN、DGCNN、Kd-Net进行比较。实验结果如表3所示。Using the same evaluation method as PointNet, the IOU of a shape is calculated by averaging the IOU of different parts that appear in the shape, and the IOU of the category is obtained by averaging the IOU of all shapes belonging to the category. Finally the average IOU (mIOU) is calculated by averaging the IOU of all test shapes. By comparing with PointNet, PointNet++, PointCNN, DGCNN, Kd-Net. The experimental results are shown in Table 3.

表3table 3

Figure BDA0002970290580000071
Figure BDA0002970290580000071

以上仅是本发明优选的实施方式,需指出是,对于本领域技术人员在不脱离本技术方案的前提下,还可以作出若干变形和改进,上述变形和改进的技术方案应同样视为落入本申请要求保护的范围。The above are only preferred embodiments of the present invention. It should be pointed out that those skilled in the art can also make some modifications and improvements without departing from the technical solution. The above-mentioned modifications and improved technical solutions should be regarded as falling into This application claims the protection scope.

Claims (2)

1.一种点云动态区域图卷积方法,其特征在于,包括:1. A method for convolution of point cloud dynamic area graphs, comprising: S1、获取三维点云数据X,X={α123,…,αi,…,αn},αi表示第i个点的数据,n表示所述三维点云数据中点的个数,αi={xi,yi,zi},xi、yi和zi表示αi的三维坐标;S1. Acquire 3D point cloud data X, X={α 1 , α 2 , α 3 ,…,α i ,…,α n }, α i represents the data of the i-th point, and n represents the 3D point cloud data The number of midpoints, α i ={ xi , y i , z i }, xi , y i and z i represent the three-dimensional coordinates of α i ; S2、对三维点云数据X进行两次独立的k近邻运算,得到两个局部特征图y和z,所述两次独立的k近邻运算的k值不同;S2. Perform two independent k-nearest neighbor operations on the three-dimensional point cloud data X to obtain two local feature maps y and z, and the k values of the two independent k-nearest neighbor operations are different; S3、将两个局部特征图y和z融合得到融合信息T,T=Sum(y,z);S3. Fusion of two local feature maps y and z to obtain fusion information T, T=Sum(y,z); S4、对融合信息T进行池化操作得到特征通信信息s1,s1=MAX(T);S4. Perform pooling operation on the fusion information T to obtain characteristic communication information s 1 , s 1 =MAX(T); S5、使用全连接层对特征通信信息s1进行紧凑降维得到紧凑特征s2,s2=FC(s1);S5. Use the fully connected layer to compact and reduce the dimensionality of the feature communication information s 1 to obtain a compact feature s 2 , s 2 =FC(s 1 ); S6、利用注意力机制从紧凑特征中自适应地选择不同区域的分支维数信息,并利用softmax对权重进行归一化,得到归一化信息a1和a2
Figure FDA0003735528030000011
Figure FDA0003735528030000012
FC1()和FC2()分别表示与y和z对应的全连接层操作;
S6. Use the attention mechanism to adaptively select the branch dimension information of different regions from the compact features, and use softmax to normalize the weights to obtain normalized information a 1 and a 2 ,
Figure FDA0003735528030000011
Figure FDA0003735528030000012
FC 1 () and FC 2 () represent the fully connected layer operations corresponding to y and z, respectively;
S7、将归一化信息与局部特征图相乘后求和得到特征图谱U,U=Sum(a1*y,a2*z)。S7. The normalization information is multiplied by the local feature map and then summed to obtain the feature map U, U=Sum(a 1 *y, a 2 *z).
2.如权利要求1所述的点云动态区域图卷积方法,其特征在于,步骤S2中两次独立的k近邻运算的k值分别为15和25。2. The point cloud dynamic area map convolution method as claimed in claim 1, wherein the k values of the two independent k nearest neighbor operations in step S2 are 15 and 25 respectively.
CN202110261653.5A 2021-03-10 2021-03-10 Point cloud dynamic region graph convolution method, classification method and segmentation method Active CN112967296B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110261653.5A CN112967296B (en) 2021-03-10 2021-03-10 Point cloud dynamic region graph convolution method, classification method and segmentation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110261653.5A CN112967296B (en) 2021-03-10 2021-03-10 Point cloud dynamic region graph convolution method, classification method and segmentation method

Publications (2)

Publication Number Publication Date
CN112967296A CN112967296A (en) 2021-06-15
CN112967296B true CN112967296B (en) 2022-11-15

Family

ID=76277614

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110261653.5A Active CN112967296B (en) 2021-03-10 2021-03-10 Point cloud dynamic region graph convolution method, classification method and segmentation method

Country Status (1)

Country Link
CN (1) CN112967296B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113516663B (en) * 2021-06-30 2022-09-27 同济大学 Point cloud semantic segmentation method and device, electronic equipment and storage medium
CN113628217A (en) * 2021-08-12 2021-11-09 江南大学 Three-dimensional point cloud segmentation method based on image convolution and integrating direction and distance

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109063753A (en) * 2018-07-18 2018-12-21 北方民族大学 A kind of three-dimensional point cloud model classification method based on convolutional neural networks
CN110188802A (en) * 2019-05-13 2019-08-30 南京邮电大学 SSD target detection algorithm based on multi-layer feature map fusion
CN110197223A (en) * 2019-05-29 2019-09-03 北方民族大学 Point cloud data classification method based on deep learning
CN111027559A (en) * 2019-10-31 2020-04-17 湖南大学 A Point Cloud Semantic Segmentation Method Based on Dilated Point Convolution Spatial Pyramid Pooling
CN111242208A (en) * 2020-01-08 2020-06-05 深圳大学 A point cloud classification method, segmentation method and related equipment
CN111476226A (en) * 2020-02-29 2020-07-31 新华三大数据技术有限公司 Text positioning method and device and model training method
CN111666836A (en) * 2020-05-22 2020-09-15 北京工业大学 High-resolution remote sensing image target detection method of M-F-Y type lightweight convolutional neural network
CN111723814A (en) * 2020-06-05 2020-09-29 中国科学院自动化研究所 Weakly supervised image semantic segmentation method, system and device based on cross-image association
CN111753698A (en) * 2020-06-17 2020-10-09 东南大学 A multimodal three-dimensional point cloud segmentation system and method
CN112036447A (en) * 2020-08-11 2020-12-04 复旦大学 Zero-sample target detection system and learnable semantic and fixed semantic fusion method
CN112184548A (en) * 2020-09-07 2021-01-05 中国科学院深圳先进技术研究院 Image super-resolution method, device, device and storage medium

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10013507B2 (en) * 2013-07-01 2018-07-03 Here Global B.V. Learning synthetic models for roof style classification using point clouds
CN106682233B (en) * 2017-01-16 2020-03-10 华侨大学 Hash image retrieval method based on deep learning and local feature fusion
CN107451616A (en) * 2017-08-01 2017-12-08 西安电子科技大学 Multi-spectral remote sensing image terrain classification method based on the semi-supervised transfer learning of depth
CN108198244B (en) * 2017-12-20 2020-11-10 中国农业大学 Apple leaf point cloud simplification method and device
CN109035329A (en) * 2018-08-03 2018-12-18 厦门大学 Camera Attitude estimation optimization method based on depth characteristic
CN109410321B (en) * 2018-10-17 2022-09-20 大连理工大学 Three-dimensional reconstruction method based on convolutional neural network
CN110081890B (en) * 2019-05-24 2023-02-03 长安大学 Dynamic k nearest neighbor map matching method combined with deep network
CN110443842B (en) * 2019-07-24 2022-02-15 大连理工大学 Depth map prediction method based on visual angle fusion
CN111583263B (en) * 2020-04-30 2022-09-23 北京工业大学 A point cloud segmentation method based on joint dynamic graph convolution
CN111915619A (en) * 2020-06-05 2020-11-10 华南理工大学 A fully convolutional network semantic segmentation method with dual feature extraction and fusion
CN112149725B (en) * 2020-09-18 2023-08-22 南京信息工程大学 Fourier transform-based spectrum domain map convolution 3D point cloud classification method
CN112329771B (en) * 2020-11-02 2024-05-14 元准智能科技(苏州)有限公司 Deep learning-based building material sample identification method

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109063753A (en) * 2018-07-18 2018-12-21 北方民族大学 A kind of three-dimensional point cloud model classification method based on convolutional neural networks
CN110188802A (en) * 2019-05-13 2019-08-30 南京邮电大学 SSD target detection algorithm based on multi-layer feature map fusion
CN110197223A (en) * 2019-05-29 2019-09-03 北方民族大学 Point cloud data classification method based on deep learning
CN111027559A (en) * 2019-10-31 2020-04-17 湖南大学 A Point Cloud Semantic Segmentation Method Based on Dilated Point Convolution Spatial Pyramid Pooling
CN111242208A (en) * 2020-01-08 2020-06-05 深圳大学 A point cloud classification method, segmentation method and related equipment
CN111476226A (en) * 2020-02-29 2020-07-31 新华三大数据技术有限公司 Text positioning method and device and model training method
CN111666836A (en) * 2020-05-22 2020-09-15 北京工业大学 High-resolution remote sensing image target detection method of M-F-Y type lightweight convolutional neural network
CN111723814A (en) * 2020-06-05 2020-09-29 中国科学院自动化研究所 Weakly supervised image semantic segmentation method, system and device based on cross-image association
CN111753698A (en) * 2020-06-17 2020-10-09 东南大学 A multimodal three-dimensional point cloud segmentation system and method
CN112036447A (en) * 2020-08-11 2020-12-04 复旦大学 Zero-sample target detection system and learnable semantic and fixed semantic fusion method
CN112184548A (en) * 2020-09-07 2021-01-05 中国科学院深圳先进技术研究院 Image super-resolution method, device, device and storage medium

Also Published As

Publication number Publication date
CN112967296A (en) 2021-06-15

Similar Documents

Publication Publication Date Title
WO2022160771A1 (en) Method for classifying hyperspectral images on basis of adaptive multi-scale feature extraction model
CN110322453B (en) 3D point cloud semantic segmentation method based on position attention and auxiliary network
CN111242208B (en) A point cloud classification method, segmentation method and related equipment
CN111861906B (en) Pavement crack image virtual augmentation model establishment and image virtual augmentation method
CN114119979A (en) A Fine-Grained Image Classification Method Based on Segmentation Mask and Self-Attention Neural Network
CN106228185A (en) A kind of general image classifying and identifying system based on neutral net and method
CN113705641A (en) Hyperspectral image classification method based on rich context network
CN104102919A (en) Image classification method capable of effectively preventing convolutional neural network from being overfit
CN112967296B (en) Point cloud dynamic region graph convolution method, classification method and segmentation method
CN107784305A (en) Facilities vegetable disease recognition method and device based on convolutional neural networks
CN107133640A (en) Image classification method based on topography's block description and Fei Sheer vectors
CN115294563A (en) 3D point cloud analysis method and device based on Transformer and capable of enhancing local semantic learning ability
CN108182316B (en) Electromagnetic simulation method based on artificial intelligence and electromagnetic brain thereof
CN110647977B (en) An optimization method of Tiny-YOLO network for on-board ship target detection
CN108846474A (en) The satellite cloud picture cloud amount calculation method of convolutional neural networks is intensively connected based on multidimensional
CN116434064A (en) A method for extracting soybean planting areas from remote sensing images based on SK-UNet deep learning network
Jin et al. Defect identification of adhesive structure based on DCGAN and YOLOv5
CN112487938A (en) Method for realizing garbage classification by utilizing deep learning algorithm
CN115512226A (en) LiDAR point cloud filtering method integrated with multi-scale CNN of attention mechanism
CN117671666A (en) A target recognition method based on adaptive graph convolutional neural network
CN117496260A (en) Pollen image classification method based on convolutional neural network and multi-scale cavity attention fusion
CN114742997B (en) Image segmentation-oriented full convolution neural network density peak pruning method
CN108537266A (en) A kind of cloth textured fault sorting technique of depth convolutional network
CN113592013B (en) A 3D point cloud classification method based on graph attention network
CN110544249A (en) A Convolutional Neural Network Quality Discrimination Method for Visual Inspection of Chassis Assembly at Any Angle

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20230628

Address after: No. 1811, 18th Floor, Building 19, Section 1201, Lushan Avenue, Wan'an Street, Tianfu New District, Chengdu, Sichuan, China (Sichuan) Pilot Free Trade Zone, 610213, China

Patentee after: Sichuan Jiulai Technology Co.,Ltd.

Address before: No. 69 lijiatuo Chongqing District of Banan City Road 400054 red

Patentee before: Chongqing University of Technology