CN109410307A - A kind of scene point cloud semantic segmentation method - Google Patents
A kind of scene point cloud semantic segmentation method Download PDFInfo
- Publication number
- CN109410307A CN109410307A CN201811204443.7A CN201811204443A CN109410307A CN 109410307 A CN109410307 A CN 109410307A CN 201811204443 A CN201811204443 A CN 201811204443A CN 109410307 A CN109410307 A CN 109410307A
- Authority
- CN
- China
- Prior art keywords
- point
- cloud
- convolution
- point cloud
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/10—Geometric effects
- G06T15/30—Clipping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Geometry (AREA)
- Computer Graphics (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to technical field of computer vision, provide a kind of scene point cloud semantic segmentation method, design the frame of the extensive intensive scene point cloud semantic segmentation model based on depth learning technology, for the extensive intensive scene point cloud of input, the two-dimensional signal that convolution can be handled directly can be converted by the three-dimensional information of cloud in the case where information is not lost, and complete a task for cloud semantic segmentation in conjunction with the technology that image, semantic is divided.Under this framework, it can effectively solve the semantic segmentation task of extensive intensive scene point cloud.The semantic segmentation result for the scene point cloud that method of the invention obtains can be utilized directly in tasks such as robot navigation, automatic Pilots.And this method effect in the natural scene of unartificial synthesis is especially significant.
Description
Technical field
The invention belongs to technical field of computer vision, more particularly to based on deep learning to extensive intensive point cloud field
The method of scape progress semantic segmentation.
Background technique
The development of modern computer vision is dominate using the method for convolutional neural networks processing two dimensional image.It is successfully
Key factor is convolution being effectively treated on the image.Convolution is defined on regular grid in the picture, the regular grid
Convolution operation is supported extremely efficiently to realize.This characteristic allows to using powerful deep layer architecture come to high-resolution
Large data collection handled.
When analyzing large-scale three-dimensional scenic, the direct extension of the above method is that three are carried out on voxel grid
Tie up convolution.However, this voxel-based method has significant limitation, cube growth and calculating effect including memory consumption
The problems such as rate.For this reason, voxel-based convolutional neural networks are mostly run on the voxel grid of low resolution, this limit
Their precision of prediction is made.Can be by alleviating these problems based on the technology of Octree, the technology is fixed on Octree
Adopted convolution and it is capable of handling slightly high-resolution data.However, these are still not sufficient to ensure that efficiently analysis large-scale three dimensional field
Scape.
The data of the 3D sensor such as RGB-D camera and Li-DAR capture typically represent the surface of object: i.e. one kind is embedded into
Two-dimensional structure in three-dimensional space.The three-dimensional data of this and real voxel form is contrasted, such as medical image.For dividing
Cloud is considered as a kind of potential surface texture of object by the classical feature for analysing such data, and this data is not regarded as body
Element.
The drawbacks of voxel-based three-dimensional data analysis method is obvious.Nearest some researchs are thought, are based on
The three-dimensional data structure of voxel is not the most natural form of Three dimensional convolution, and proposes based on unordered point set, graph structure and ball
The alternative of shape surface texture.Unfortunately, these methods have the defect of its own, such as have limit quick partial structurtes
Perception relies on restrictive topology hypothesis.
(1) three-dimensional point cloud semantic segmentation
The scene understanding of three-dimensional data, including cloud semantic segmentation, have a long history in computer vision.It starts
Property method be based on hand-made feature, it be suitable for aviation Li-DAR data.These methods can also be with advanced frame
Structure combines.The model of graphics, including condition random field is utilized in popular pre- flow gauge.Equally have in recent years
Method for interactive point cloud semantic segmentation is suggested.
(2) development of the deep learning in three-dimensional data
In recent years, the deep learning revolution of computer vision field has had spread over three-dimensional data analysis, some to be used for
The deep learning method of processing three-dimensional data is suggested.
The common expression of three-dimensional data for deep learning is voxel grid.But the time of cube rank and space are multiple
Miscellaneous degree, this run these methods can only with low resolution, and precision is limited.In order to overcome this limitation, people is studied
Member proposes the expression based on layering spatial data structure, and such as Octree and Kd-Tree, they have preferably storage and calculate
Efficiency, therefore can handle the data of higher resolution ratio.
The application of other some deep learning networks is using RGB-D image as input, later with full convolutional neural networks
Or it is handled based on the neural network of figure, but be generally unsuitable for unstructured unknown cloud of sensor visual angle.For
Solution this problem, Boulch et al. using the virtual camera randomly placed from a cloud rendering image, and with these pictures
Training convolutional neural networks.In the more controlled setting with fixed camera visual angle, multiple view method is used successfully to shape
Segmentation, shape recognition and shape synthesis.
Neat et al. to propose a kind of for analyzing the network of unordered cloud, which is independently handled and is made to a progress
With the information of maximum Chi Hualai polymerization context.But it is very weak due to putting communication between, when the network application in
When large scale scene with complex topology, this method can encounter many difficulties.
Summary of the invention
The present invention in order to solve conventional point cloud scene understanding vulnerable to data resolution limitation, the inadequate robust of local feature and
It is difficult to handle the technical problems such as extensive point off density cloud, devises the extensive intensive scene point based on deep learning technology
The frame of cloud semantic segmentation model can incite somebody to action the extensive intensive scene point cloud of input in the case where information is not lost
The three-dimensional information of point cloud is converted into the two-dimensional signal that convolution can be handled directly, and completes in conjunction with the technology that image, semantic is divided
The task of point cloud semantic segmentation.Under this framework, it can effectively solve the semantic segmentation task of extensive intensive scene point cloud.
Technical solution of the present invention:
A kind of scene point cloud semantic segmentation method, steps are as follows:
(1) building of local coordinate system planar convolution: in order to directly construct two-dimensional convolution on cloud, so that model
A local feature for cloud robust can be extracted in the lower situation of computation complexity, a cloud is projected to utilization by the present invention
PCA technology decomposes three coordinate planes generated to cloud, and constructs convolution module respectively in three coordinate planes and come to a cloud
Carry out the extraction of local feature.Local coordinate plane convolution module is described in detail below.
(1.1) local coordinate system plane is estimated:
For point p each in cloud, its local coordinate system plane is estimated by the analysis of covariance of part first；Tool
For body, for meeting | | p-q | | the point set q in a ball domain of < R, the estimation for tangent plane, the side of the tangent plane of point p
To being by covariance matrix ∑qrrTFeature vector determine, r=q-p；It is worth the smallest feature vector and determines tangent plane
Normal vector np, two feature vectors i and j in addition determine the direction of two reference axis of tangent plane；
(1.2) local coordinate system planar convolution:
Local message is extracted in order to carry out convolution operation on cloud, needs three coordinates by each cloud
Plane；The point in ball domain range that the radius from point p is R is indicated with point set q, and q is projected to three coordinates of p respectively
In plane；For each point p, the function that F (p) is point p is defined, for encoded colors, geometrical characteristic or is come in automatic network
The abstract characteristics of interbed；Building for convolution, the tangent plane π of point pp, defining S (u) is the continuous letter in tangent plane on the u of position
Number amount, c (u) is the convolution nuclear parameter on the u of position, wherein u ∈ R2；
Therefore the convolution operation at point p is defined as follows:
(1.3) signal difference:
For tangent plane, signal interpolation target is to estimate to participate in tangent plane with the semaphore F (q) of the neighborhood point set q of p
The semaphore S (u) of each position of convolution algorithm；Q is projected in the tangent plane of p first, generates a projection point set v=
(rTi,rTj)；Definition:
S (v)=F (q) (2)
In this way, point set v is scattered in the plane of delineation；Therefore these semaphores are subjected to interpolation to estimate that S (u) is participating in rolling up
The semaphore of each position of product operation:
∑v(w(u,v)·S(v)) (3)
Here, w (u, v) is the weight of convolution kernel, and meets ∑vW=1；The present invention is inserted using a kind of fairly simple
Value method: arest neighbors (NN) interpolation.In this interpolation strategies,
Finally again to the formula for carrying out tangent plane convolution operation at point p:
Note that the effect of tangent plane herein is more and more implicit: it provides range domain for u, and is convolution kernel w's
Deduction provides the foundation, but does not need clearly to safeguard.This enables the method to support in the point cloud with millions of a points
Upper building depth network.
(1.4) pond layer:
Convolutional network polymerize the signal on larger space region usually using pond layer.The present invention will be by that will put cloud signal
Pond is realized in amount hash to conventional 3D grid.For the point set being scattering into the same grid, it is polymerize by average Chi Hualai
Its semaphore.Consider point set P={ p } and corresponding semaphore { F (p) }.It enables g represent a voxel grid and enables VgIt represents in P
The point set being hashed into g.Assuming that VgNon-empty is then converged to the information of its all the points on one point by average pond:
(2) cloud semantic segmentation module is put:
(2.1) module inputs:
The input of the module is large-scale indoor and outdoor intensive scene point cloud, and putting the quantity of cloud, there is no limit put cloud
Input feature vector includes the information of RGBXYZ, needs to be converted into RGB, D (depth), H (height), N (normal vector) by pretreatment
As input feature vector；
(2.2) module architectures:
Point cloud semantic segmentation module is the convolutional neural networks from coding structure, and effect is realized to input point cloud
The prediction of semantic information, formula are as follows:
Iout=fseg(Iin；θf) (7)
In above formula, IoutIt is prediction of the network to cloud about n classification semantic information, IinIt is input comprising RGBDHN
The scene point cloud of information；fseg() indicates the convolutional neural networks from coding structure, θfIndicate the weight parameter of network model；
It wherein, include 2 pond layers from the encoder of the convolutional neural networks of coding structure, it is therefore an objective to polymerize volume by pond layer
The feature of volume module output and the Spatial Dimension for reducing feature；There can be 3 convolution modules to obtain a little before each pond layer
The local message of cloud；Restore the Spatial Dimension of feature, same packet before each up-sampling layer in decoder by up-sampling layer
Containing 3 convolution modules；Connection is jumped in increase by two between the respective layer of encoder and decoder makes network that mesh be better anticipated
Target details.
In each convolution module, due to using local coordinate system planar convolution that input feature vector can be projected to three planes
It causes the port number of feature to increase by 3 times of redundancies resulted in a feature that, therefore makes first after local coordinate system planar convolution
Feature port number is further expanded 2 times with 1 × 1 convolution, then separates convolution (n single pass volumes using depth
N channel of product core and input feature vector carries out one-to-one convolution operation) decoupling of the realization to redundancy feature, finally use one
1 × 1 convolution kernel comes fusion and compression to feature.
(3) training method
This patent is using the outdoor point cloud contextual data collection of the Semantic3D comprising 8 classifications and comprising 13 classifications
The outdoor scene data set of S3DIS；Model is trained using the method for data-driven, is lacked to solve 3-D data set
Weary problem, scene point cloud in Semantic3D data set and S3DIS data set is rotated horizontally 10 times respectively by this patent will
Sample size increases by 10 times.
Backpropagation and stochastic gradient descent are used from the convolutional neural networks of coding structure in point cloud semantic segmentation module
Method training.The scene point cloud inputted for one uses the cross entropy with class weight as loss function Lseg, benefit
Weight is calculated with formula (8), wherein the weight w of classification iiFor belong in sample classification i point quantity DiWith classes all in sample
The quantity D of other pointkRatio logarithm opposite number, this is prevented to alleviate the class imbalance phenomenon in data set
The training for the point cloud branch distribution network that quantity occupies the majority.
Network overall error is calculated using formula (9), wherein N indicates the number at scene point cloud midpoint, ylIndicate that the output of point l exists
Score corresponding to true classification, wlFor the weight of point l generic.
Obtain training error after, network will be updated the parameter of network along the opposite direction of gradient, iteration until
Convergence.
The present invention is had the significant advantage that compared with the method for same domain for a cloud semantic segmentation task, relatively more intuitive
Way be that entire point cloud scene is subjected to voxelization, then using three-dimensional convolution kernel and combine at full convolution technique
Reason, but since the problems such as dimension explosion and resolution limitations causes the computational efficiency of this method and accuracy to be unable to
To guarantee.Based on the neural network method of multi-layer perception (MLP) when solving the problems, such as some cloud semantic segmentations due to can not effectively mention
Get the local feature of a cloud so as to cause network cannot future position cloud scene well details.
And cloud is regarded a kind of table of object by the semantic segmentation method of extensive intensive scene point cloud proposed by the present invention
Face structure projects to local coordinate system plane by that will put cloud to directly build two-dimensional convolution on cloud, this makes the party
Method can effectively extract the local feature of a cloud under conditions of information lossless, jump over connection by building in a network and make
The textural characteristics and network high-rise semanteme abundant of Network Low-layer can fully be used when being predicted by obtaining model
Feature, so that network be helped preferably to realize to a prediction for cloud scene details.The semanteme for the scene point cloud that this method obtains point
Cutting result can directly utilize in tasks such as robot navigation, automatic Pilots.And this method is in the natural field of unartificial synthesis
Effect is especially significant in scape.
Detailed description of the invention
Fig. 1 (a) is the scene point cloud of a true meeting room, and Fig. 1 (b) is that the semantic segmentation of meeting room scene point cloud is true
Value.
Fig. 2 is a cloud semantic segmentation network structure.Using scene point cloud as input, by convolution, pondization and up-sampling
Deng operation, the semantic segmentation result of scene point cloud is finally entered.
Fig. 3 is the internal structure of each convolution block, is 1. the shape for converting feature vector to local coordinate system planar convolution
2. formula is that three n × 3 × 3 × d tensors are spliced into n × 3 × 3 × 3d tensor, 3. and is 5. 1 × 1 convolution, 4.
It is that depth separates convolution, is compressed to the dimension of tensor.
Specific embodiment
Invention is described in further detail With reference to embodiment, but the invention is not limited to specific implementations
Mode.
A method of semantic segmentation, including network model are carried out to extensive intensive scene point cloud based on deep learning
Training and model operating procedure part.
1. training network model
The semantic segmentation network of the training extensive intensive scene point cloud, it is necessary first to prepare sufficient point cloud data.Often
A scene point cloud sample should include semantic classes information belonging to RGBXYZ and each point.With S3DIS indoor scene data set
For, after data enhance, 2654 scene point cloud samples are shared as training set and 578 samples as verifying collection.
After obtaining enough data sets, it is necessary first to by the preprocessed information for being converted into RGBDHN of the feature of each point
Input as semantic segmentation network.Later by establish Kd-Tree come in Searching point cloud centered on each point, radius R
Ball domain in the information put, solve the local coordinate system of each point using PCA technology, and by the information put in ball domain by projecting
And etc. be converted into the form that can carry out local coordinate system planar convolution.
Training data, is transported to network to be trained by the semantic segmentation network that a cloud is then built according to attached drawing 2 in batches
In, the class weight of each point is calculated according to formula (8) and formula (9) respectively and puts the error of cloud semantic segmentation network, and according to
The iteration that the method for gradient backpropagation carries out parameter updates, and is accelerated using GPU, sets until the error of network is reduced to
Deconditioning within fixed threshold value or when the number of network iteration is met the requirements.
2. cloud semantic segmentation process
The scene point cloud indoor and outdoor for one, a cloud is converted to cloud feeding preprocessing module first can
The form for carrying out local coordinate system planar convolution, is then input to trained point Yun Yuyi for after pretreatment cloud
The semantic information of scene point cloud is obtained in parted pattern.The semantic information of scene point cloud can be then used for automatic Pilot,
In the tasks such as robot navigation.Process is as shown in Fig. 2.
Claims (1)
1. a kind of scene point cloud semantic segmentation method, which is characterized in that steps are as follows:
(1) cloud the building of local coordinate system planar convolution: is projected to three seats for being decomposed and being generated to cloud using PCA technology
Plane is marked, and constructs the extraction that convolution module to carry out a cloud local feature respectively in three coordinate planes；
(1.1) local coordinate system plane is estimated:
For point p each in cloud, its local coordinate system plane is estimated by the analysis of covariance of part first；It is specific next
It says, for meeting | | p-q | | the point set q in a ball domain of < R, the estimation for tangent plane, the direction of the tangent plane of point p is
By covariance matrix ∑qrrTFeature vector determine, r=q-p；It is worth the normal direction that the smallest feature vector determines tangent plane
Measure np, the direction of two reference axis of two feature vectors i and j decision tangent plane in addition；
(1.2) local coordinate system planar convolution:
The point in ball domain range that the radius from point p is R is indicated with point set q, and q is projected to three coordinates of p respectively
In plane；For each point p, the function that F (p) is point p is defined, for encoded colors, geometrical characteristic or is come in automatic network
The abstract characteristics of interbed；Building for convolution, the tangent plane π of point pp, defining S (u) is the continuous letter in tangent plane on the u of position
Number amount, c (u) is the convolution nuclear parameter on the u of position, wherein u ∈ R2；
Therefore the convolution operation at point p is defined as follows:
(1.3) signal difference:
For tangent plane, signal interpolation target is to estimate to participate in convolution in tangent plane with the semaphore F (q) of the neighborhood point set q of p
The semaphore S (u) of each position of operation；Q is projected in the tangent plane of p first, generates a projection point set v=
(rTi,rTj)；Definition:
S (v)=F (q) (2)
In this way, point set v is scattered in the plane of delineation；Therefore these semaphores are subjected to interpolation to estimate that S (u) is participating in convolution fortune
The semaphore for each position calculated:
∑v(w(u,v)·S(v)) (3)
Here, w (u, v) is the weight of convolution kernel, and meets ∑vW=1；With fairly simple interpolation method: arest neighbors (NN)
Interpolation；In this interpolation strategies,
Finally again to the formula for carrying out tangent plane convolution operation at point p:
(1.4) pond layer:
Pond is realized in cloud semaphore hash to conventional 3D grid by that will put；For the point set being scattering into the same grid,
It polymerize its semaphore by average Chi Hualai；Consider point set P={ p } and corresponding semaphore { F (p) }, g is enabled to represent a voxel
Grid simultaneously enables VgRepresent the point set being hashed into g in P；Assuming that VgThe information of its all the points is then passed through average pond Hua Hui by non-empty
Gather on a point:
(2) cloud semantic segmentation module is put:
(2.1) module inputs:
The input of the module is large-scale indoor and outdoor intensive scene point cloud, and putting the quantity of cloud, there is no limit put the input of cloud
Feature includes the information of RGBXYZ, and it is special as input to need to be converted into RGB, depth D, height H, normal vector N by pretreatment
Sign；
(2.2) module architectures:
Point cloud semantic segmentation module is the convolutional neural networks from coding structure, and effect is realized to input point cloud semanteme
The prediction of information, formula are as follows:
Iout=fseg(Iin；θf) (7)
In above formula, IoutIt is prediction of the network to cloud about n classification semantic information, IinIt is input comprising RGBDHN information
Scene point cloud；fseg() indicates the convolutional neural networks from coding structure, θfIndicate the weight parameter of network model；Wherein,
It include 2 pond layers from the encoder of the convolutional neural networks of coding structure, it is therefore an objective to polymerize convolution mould by pond layer
The feature of block output and the Spatial Dimension for reducing feature；There are 3 convolution modules before each pond layer to obtain the office of a cloud
Portion's information；Restore the Spatial Dimension of feature in decoder by up-sampling layer, is equally rolled up comprising 3 before each up-sampling layer
Volume module；Connection is jumped in increase by two between the respective layer of encoder and decoder makes network that the thin of target be better anticipated
Section；
In each convolution module, first using 1 × 1 convolution further by feature after local coordinate system planar convolution
Port number expands 2 times, then decoupling of the convolution realization to redundancy feature is separated using depth, finally using one 1 × 1
Convolution kernel comes fusion and compression to feature；
(3) training method
Using the room of outdoor point the cloud contextual data collection and the S3DIS comprising 13 classifications of the Semantic3D comprising 8 classifications
Outer scene data set；Model is trained using the method for data-driven, in order to solve the problems, such as that 3-D data set lacks,
Scene point cloud in the outdoor scene data set of the outdoor point cloud contextual data collection of Semantic3D and S3DIS is rotated horizontally respectively
10 times by sample size increase by 10 times；
The side of backpropagation and stochastic gradient descent is used in point cloud semantic segmentation module from the convolutional neural networks of coding structure
Method training；The scene point cloud inputted for one uses the cross entropy with class weight as loss function Lseg, utilize formula
(8) weight is calculated, wherein the weight w of classification iiFor belong in sample classification i point quantity DiWith all categories in sample
The quantity D of pointkRatio logarithm opposite number, this is to prevent quantity to alleviate the class imbalance phenomenon in data set
The training of the point cloud branch distribution network to occupy the majority:
Network overall error is calculated using formula (9), wherein N indicates the number at scene point cloud midpoint, ylIndicate the output of point l true
Score corresponding to classification, wlFor the weight of point l generic；
After obtaining training error, network will be updated the parameter of network along the opposite direction of gradient, and iteration is until convergence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811204443.7A CN109410307B (en) | 2018-10-16 | 2018-10-16 | Scene point cloud semantic segmentation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811204443.7A CN109410307B (en) | 2018-10-16 | 2018-10-16 | Scene point cloud semantic segmentation method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109410307A true CN109410307A (en) | 2019-03-01 |
| CN109410307B CN109410307B (en) | 2022-09-20 |
Family
ID=65467281
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811204443.7A Active CN109410307B (en) | 2018-10-16 | 2018-10-16 | Scene point cloud semantic segmentation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109410307B (en) |
Cited By (49)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110021072A (en) * | 2019-04-03 | 2019-07-16 | 武汉大学 | The multi-platform cloud intelligent processing method towards holography mapping |
| CN110020681A (en) * | 2019-03-27 | 2019-07-16 | 南开大学 | Point cloud feature extracting method based on spatial attention mechanism |
| CN110032962A (en) * | 2019-04-03 | 2019-07-19 | 腾讯科技（深圳）有限公司 | A kind of object detecting method, device, the network equipment and storage medium |
| CN110097556A (en) * | 2019-04-29 | 2019-08-06 | 东南大学 | Large-scale point cloud semantic segmentation algorithm based on PointNet |
| CN110163906A (en) * | 2019-05-22 | 2019-08-23 | 北京市商汤科技开发有限公司 | Processing Method of Point-clouds, device, electronic equipment and storage medium |
| CN110197215A (en) * | 2019-05-22 | 2019-09-03 | 深圳市牧月科技有限公司 | A kind of ground perception point cloud semantic segmentation method of autonomous driving |
| CN110222626A (en) * | 2019-06-03 | 2019-09-10 | 宁波智能装备研究院有限公司 | A kind of unmanned scene point cloud target mask method based on deep learning algorithm |
| CN110363776A (en) * | 2019-06-28 | 2019-10-22 | 联想(北京)有限公司 | Image processing method and electronic equipment |
| CN110458939A (en) * | 2019-07-24 | 2019-11-15 | 大连理工大学 | The indoor scene modeling method generated based on visual angle |
| CN110516751A (en) * | 2019-08-29 | 2019-11-29 | 上海交通大学 | Processing method, system and the equipment of three-dimensional data |
| CN110555847A (en) * | 2019-07-31 | 2019-12-10 | 瀚博半导体(上海)有限公司 | Image processing method and device based on convolutional neural network |
| CN110570429A (en) * | 2019-08-30 | 2019-12-13 | 华南理工大学 | Lightweight real-time semantic segmentation method based on three-dimensional point cloud |
| CN110706238A (en) * | 2019-09-12 | 2020-01-17 | 南京人工智能高等研究院有限公司 | Method and device for segmenting point cloud data, storage medium and electronic equipment |
| CN110827398A (en) * | 2019-11-04 | 2020-02-21 | 北京建筑大学 | Indoor three-dimensional point cloud automatic semantic segmentation algorithm based on deep neural network |
| CN110991373A (en) * | 2019-12-09 | 2020-04-10 | 北京字节跳动网络技术有限公司 | Image processing method, image processing apparatus, electronic device, and medium |
| CN111027559A (en) * | 2019-10-31 | 2020-04-17 | 湖南大学 | Point cloud semantic segmentation method based on expansion point convolution space pyramid pooling |
| CN111242952A (en) * | 2020-01-15 | 2020-06-05 | 腾讯科技（深圳）有限公司 | Image segmentation model training method, image segmentation device and computing equipment |
| CN111462137A (en) * | 2020-04-02 | 2020-07-28 | 中科人工智能创新技术研究院(青岛)有限公司 | Point cloud scene segmentation method based on knowledge distillation and semantic fusion |
| CN111507982A (en) * | 2019-06-28 | 2020-08-07 | 浙江大学 | Point cloud semantic segmentation method based on deep learning |
| CN111724478A (en) * | 2020-05-19 | 2020-09-29 | 华南理工大学 | Point cloud up-sampling method based on deep learning |
| CN111784699A (en) * | 2019-04-03 | 2020-10-16 | Tcl集团股份有限公司 | Method and device for carrying out target segmentation on three-dimensional point cloud data and terminal equipment |
| CN111833358A (en) * | 2020-06-26 | 2020-10-27 | 中国人民解放军32802部队 | Semantic segmentation method and system based on 3D-YOLO |
| CN111860138A (en) * | 2020-06-09 | 2020-10-30 | 中南民族大学 | Three-dimensional point cloud semantic segmentation method and system based on full-fusion network |
| CN111862101A (en) * | 2020-07-15 | 2020-10-30 | 西安交通大学 | 3D point cloud semantic segmentation method under aerial view coding visual angle |
| CN111968133A (en) * | 2020-07-31 | 2020-11-20 | 上海交通大学 | Three-dimensional point cloud data example segmentation method and system in automatic driving scene |
| CN112037138A (en) * | 2020-07-29 | 2020-12-04 | 大连理工大学 | Method for completing cloud scene semantics of single depth map point |
| CN112085066A (en) * | 2020-08-13 | 2020-12-15 | 南京邮电大学 | Voxelized three-dimensional point cloud scene classification method based on graph convolution neural network |
| WO2020253121A1 (en) * | 2019-06-17 | 2020-12-24 | 商汤集团有限公司 | Target detection method and apparatus, intelligent driving method and device, and storage medium |
| CN112149677A (en) * | 2020-09-14 | 2020-12-29 | 上海眼控科技股份有限公司 | Point cloud semantic segmentation method, device and equipment |
| CN112215231A (en) * | 2020-09-29 | 2021-01-12 | 浙江工业大学 | Large-scale point cloud semantic segmentation method combining space depth convolution and residual error structure |
| CN112446385A (en) * | 2021-01-29 | 2021-03-05 | 清华大学 | Scene semantic segmentation method and device and electronic equipment |
| CN112561950A (en) * | 2020-12-24 | 2021-03-26 | 福州大学 | Point cloud sampling method based on window function under PointTrack framework |
| CN112633330A (en) * | 2020-12-06 | 2021-04-09 | 西安电子科技大学 | Point cloud segmentation method, system, medium, computer device, terminal and application |
| CN112819833A (en) * | 2021-02-05 | 2021-05-18 | 四川大学 | Large scene point cloud semantic segmentation method |
| WO2021164469A1 (en) * | 2020-02-21 | 2021-08-26 | 北京市商汤科技开发有限公司 | Target object detection method and apparatus, device, and storage medium |
| CN113313161A (en) * | 2021-05-24 | 2021-08-27 | 北京大学 | Object shape classification method based on rotation invariant canonical invariant network model |
| CN113379898A (en) * | 2021-06-17 | 2021-09-10 | 西安理工大学 | Three-dimensional indoor scene reconstruction method based on semantic segmentation |
| CN113378756A (en) * | 2021-06-24 | 2021-09-10 | 深圳市赛维网络科技有限公司 | Three-dimensional human body semantic segmentation method, terminal device and storage medium |
| CN113392841A (en) * | 2021-06-03 | 2021-09-14 | 电子科技大学 | Three-dimensional point cloud semantic segmentation method based on multi-feature information enhanced coding |
| CN113421267A (en) * | 2021-05-07 | 2021-09-21 | 江苏大学 | Point cloud semantic and instance joint segmentation method and system based on improved PointConv |
| CN113449736A (en) * | 2021-01-14 | 2021-09-28 | 浙江工业大学 | Photogrammetry point cloud semantic segmentation method based on deep learning |
| CN113743417A (en) * | 2021-09-03 | 2021-12-03 | 北京航空航天大学 | Semantic segmentation method and semantic segmentation device |
| CN113762195A (en) * | 2021-09-16 | 2021-12-07 | 复旦大学 | Point cloud semantic segmentation and understanding method based on road side RSU |
| CN113837215A (en) * | 2021-04-27 | 2021-12-24 | 西北工业大学 | Point cloud semantic and instance segmentation method based on conditional random field |
| CN114035575A (en) * | 2021-11-04 | 2022-02-11 | 南京理工大学 | Unmanned vehicle motion planning method and system based on semantic segmentation |
| CN114341941A (en) * | 2019-08-29 | 2022-04-12 | 交互数字Vc控股法国公司 | Transmission format of encoding and decoding point cloud |
| CN114387289A (en) * | 2022-03-24 | 2022-04-22 | 南方电网数字电网研究院有限公司 | Semantic segmentation method and device for three-dimensional point cloud of power transmission and distribution overhead line |
| CN115131562A (en) * | 2022-07-08 | 2022-09-30 | 北京百度网讯科技有限公司 | Three-dimensional scene segmentation method, model training method and device and electronic equipment |
| CN115170585A (en) * | 2022-07-12 | 2022-10-11 | 上海人工智能创新中心 | Three-dimensional point cloud semantic segmentation method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105354591A (en) * | 2015-10-20 | 2016-02-24 | 南京大学 | High-order category-related prior knowledge based three-dimensional outdoor scene semantic segmentation system |
| CN106709481A (en) * | 2017-03-03 | 2017-05-24 | 深圳市唯特视科技有限公司 | Indoor scene understanding method based on 2D-3D semantic data set |
-
2018
- 2018-10-16 CN CN201811204443.7A patent/CN109410307B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105354591A (en) * | 2015-10-20 | 2016-02-24 | 南京大学 | High-order category-related prior knowledge based three-dimensional outdoor scene semantic segmentation system |
| CN106709481A (en) * | 2017-03-03 | 2017-05-24 | 深圳市唯特视科技有限公司 | Indoor scene understanding method based on 2D-3D semantic data set |
Non-Patent Citations (1)
| Title |
|---|
| 江文婷等: "基于增量计算的大规模场景致密语义地图构建", 《浙江大学学报(工学版)》 * |
Cited By (73)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110020681A (en) * | 2019-03-27 | 2019-07-16 | 南开大学 | Point cloud feature extracting method based on spatial attention mechanism |
| CN110032962A (en) * | 2019-04-03 | 2019-07-19 | 腾讯科技（深圳）有限公司 | A kind of object detecting method, device, the network equipment and storage medium |
| CN111784699A (en) * | 2019-04-03 | 2020-10-16 | Tcl集团股份有限公司 | Method and device for carrying out target segmentation on three-dimensional point cloud data and terminal equipment |
| CN110021072A (en) * | 2019-04-03 | 2019-07-16 | 武汉大学 | The multi-platform cloud intelligent processing method towards holography mapping |
| CN110097556A (en) * | 2019-04-29 | 2019-08-06 | 东南大学 | Large-scale point cloud semantic segmentation algorithm based on PointNet |
| CN110163906A (en) * | 2019-05-22 | 2019-08-23 | 北京市商汤科技开发有限公司 | Processing Method of Point-clouds, device, electronic equipment and storage medium |
| CN110197215A (en) * | 2019-05-22 | 2019-09-03 | 深圳市牧月科技有限公司 | A kind of ground perception point cloud semantic segmentation method of autonomous driving |
| CN110163906B (en) * | 2019-05-22 | 2021-10-29 | 北京市商汤科技开发有限公司 | Point cloud data processing method and device, electronic equipment and storage medium |
| CN110222626B (en) * | 2019-06-03 | 2021-05-28 | 宁波智能装备研究院有限公司 | Unmanned scene point cloud target labeling method based on deep learning algorithm |
| CN110222626A (en) * | 2019-06-03 | 2019-09-10 | 宁波智能装备研究院有限公司 | A kind of unmanned scene point cloud target mask method based on deep learning algorithm |
| WO2020253121A1 (en) * | 2019-06-17 | 2020-12-24 | 商汤集团有限公司 | Target detection method and apparatus, intelligent driving method and device, and storage medium |
| CN111507982A (en) * | 2019-06-28 | 2020-08-07 | 浙江大学 | Point cloud semantic segmentation method based on deep learning |
| CN111507982B (en) * | 2019-06-28 | 2022-04-26 | 浙江大学 | Point cloud semantic segmentation method based on deep learning |
| CN110363776A (en) * | 2019-06-28 | 2019-10-22 | 联想(北京)有限公司 | Image processing method and electronic equipment |
| CN110458939A (en) * | 2019-07-24 | 2019-11-15 | 大连理工大学 | The indoor scene modeling method generated based on visual angle |
| CN110458939B (en) * | 2019-07-24 | 2022-11-18 | 大连理工大学 | Indoor scene modeling method based on visual angle generation |
| CN110555847A (en) * | 2019-07-31 | 2019-12-10 | 瀚博半导体(上海)有限公司 | Image processing method and device based on convolutional neural network |
| CN114341941A (en) * | 2019-08-29 | 2022-04-12 | 交互数字Vc控股法国公司 | Transmission format of encoding and decoding point cloud |
| CN110516751A (en) * | 2019-08-29 | 2019-11-29 | 上海交通大学 | Processing method, system and the equipment of three-dimensional data |
| CN110570429B (en) * | 2019-08-30 | 2021-12-17 | 华南理工大学 | Lightweight real-time semantic segmentation method based on three-dimensional point cloud |
| CN110570429A (en) * | 2019-08-30 | 2019-12-13 | 华南理工大学 | Lightweight real-time semantic segmentation method based on three-dimensional point cloud |
| CN110706238A (en) * | 2019-09-12 | 2020-01-17 | 南京人工智能高等研究院有限公司 | Method and device for segmenting point cloud data, storage medium and electronic equipment |
| CN110706238B (en) * | 2019-09-12 | 2022-06-17 | 南京人工智能高等研究院有限公司 | Method and device for segmenting point cloud data, storage medium and electronic equipment |
| CN111027559A (en) * | 2019-10-31 | 2020-04-17 | 湖南大学 | Point cloud semantic segmentation method based on expansion point convolution space pyramid pooling |
| CN110827398B (en) * | 2019-11-04 | 2023-12-26 | 北京建筑大学 | Automatic semantic segmentation method for indoor three-dimensional point cloud based on deep neural network |
| CN110827398A (en) * | 2019-11-04 | 2020-02-21 | 北京建筑大学 | Indoor three-dimensional point cloud automatic semantic segmentation algorithm based on deep neural network |
| CN110991373A (en) * | 2019-12-09 | 2020-04-10 | 北京字节跳动网络技术有限公司 | Image processing method, image processing apparatus, electronic device, and medium |
| CN111242952B (en) * | 2020-01-15 | 2023-06-30 | 腾讯科技（深圳）有限公司 | Image segmentation model training method, image segmentation device and computing equipment |
| CN111242952A (en) * | 2020-01-15 | 2020-06-05 | 腾讯科技（深圳）有限公司 | Image segmentation model training method, image segmentation device and computing equipment |
| WO2021164469A1 (en) * | 2020-02-21 | 2021-08-26 | 北京市商汤科技开发有限公司 | Target object detection method and apparatus, device, and storage medium |
| CN111462137A (en) * | 2020-04-02 | 2020-07-28 | 中科人工智能创新技术研究院(青岛)有限公司 | Point cloud scene segmentation method based on knowledge distillation and semantic fusion |
| CN111462137B (en) * | 2020-04-02 | 2023-08-08 | 中科人工智能创新技术研究院(青岛)有限公司 | Point cloud scene segmentation method based on knowledge distillation and semantic fusion |
| CN111724478A (en) * | 2020-05-19 | 2020-09-29 | 华南理工大学 | Point cloud up-sampling method based on deep learning |
| CN111724478B (en) * | 2020-05-19 | 2021-05-18 | 华南理工大学 | Point cloud up-sampling method based on deep learning |
| CN111860138B (en) * | 2020-06-09 | 2024-03-01 | 中南民族大学 | Three-dimensional point cloud semantic segmentation method and system based on full fusion network |
| CN111860138A (en) * | 2020-06-09 | 2020-10-30 | 中南民族大学 | Three-dimensional point cloud semantic segmentation method and system based on full-fusion network |
| CN111833358A (en) * | 2020-06-26 | 2020-10-27 | 中国人民解放军32802部队 | Semantic segmentation method and system based on 3D-YOLO |
| CN111862101A (en) * | 2020-07-15 | 2020-10-30 | 西安交通大学 | 3D point cloud semantic segmentation method under aerial view coding visual angle |
| CN112037138A (en) * | 2020-07-29 | 2020-12-04 | 大连理工大学 | Method for completing cloud scene semantics of single depth map point |
| CN111968133A (en) * | 2020-07-31 | 2020-11-20 | 上海交通大学 | Three-dimensional point cloud data example segmentation method and system in automatic driving scene |
| CN112085066A (en) * | 2020-08-13 | 2020-12-15 | 南京邮电大学 | Voxelized three-dimensional point cloud scene classification method based on graph convolution neural network |
| CN112085066B (en) * | 2020-08-13 | 2022-08-26 | 南京邮电大学 | Voxelized three-dimensional point cloud scene classification method based on graph convolution neural network |
| CN112149677A (en) * | 2020-09-14 | 2020-12-29 | 上海眼控科技股份有限公司 | Point cloud semantic segmentation method, device and equipment |
| CN112215231B (en) * | 2020-09-29 | 2024-03-08 | 浙江工业大学 | Large-scale point cloud semantic segmentation method combining spatial depth convolution and residual error structure |
| CN112215231A (en) * | 2020-09-29 | 2021-01-12 | 浙江工业大学 | Large-scale point cloud semantic segmentation method combining space depth convolution and residual error structure |
| CN112633330A (en) * | 2020-12-06 | 2021-04-09 | 西安电子科技大学 | Point cloud segmentation method, system, medium, computer device, terminal and application |
| CN112633330B (en) * | 2020-12-06 | 2024-02-02 | 西安电子科技大学 | Point cloud segmentation method, system, medium, computer equipment, terminal and application |
| CN112561950A (en) * | 2020-12-24 | 2021-03-26 | 福州大学 | Point cloud sampling method based on window function under PointTrack framework |
| CN113449736A (en) * | 2021-01-14 | 2021-09-28 | 浙江工业大学 | Photogrammetry point cloud semantic segmentation method based on deep learning |
| CN113449736B (en) * | 2021-01-14 | 2022-09-23 | 浙江工业大学 | Photogrammetry point cloud semantic segmentation method based on deep learning |
| CN112446385A (en) * | 2021-01-29 | 2021-03-05 | 清华大学 | Scene semantic segmentation method and device and electronic equipment |
| CN112819833B (en) * | 2021-02-05 | 2022-07-12 | 四川大学 | Large scene point cloud semantic segmentation method |
| CN112819833A (en) * | 2021-02-05 | 2021-05-18 | 四川大学 | Large scene point cloud semantic segmentation method |
| CN113837215A (en) * | 2021-04-27 | 2021-12-24 | 西北工业大学 | Point cloud semantic and instance segmentation method based on conditional random field |
| CN113837215B (en) * | 2021-04-27 | 2024-01-12 | 西北工业大学 | Point cloud semantic and instance segmentation method based on conditional random field |
| CN113421267B (en) * | 2021-05-07 | 2024-04-12 | 江苏大学 | Point cloud semantic and instance joint segmentation method and system based on improved PointConv |
| CN113421267A (en) * | 2021-05-07 | 2021-09-21 | 江苏大学 | Point cloud semantic and instance joint segmentation method and system based on improved PointConv |
| CN113313161A (en) * | 2021-05-24 | 2021-08-27 | 北京大学 | Object shape classification method based on rotation invariant canonical invariant network model |
| CN113313161B (en) * | 2021-05-24 | 2023-09-26 | 北京大学 | Object shape classification method based on rotation-invariant standard isomorphism network model |
| CN113392841A (en) * | 2021-06-03 | 2021-09-14 | 电子科技大学 | Three-dimensional point cloud semantic segmentation method based on multi-feature information enhanced coding |
| CN113392841B (en) * | 2021-06-03 | 2022-11-18 | 电子科技大学 | Three-dimensional point cloud semantic segmentation method based on multi-feature information enhanced coding |
| CN113379898B (en) * | 2021-06-17 | 2022-11-11 | 西安理工大学 | Three-dimensional indoor scene reconstruction method based on semantic segmentation |
| CN113379898A (en) * | 2021-06-17 | 2021-09-10 | 西安理工大学 | Three-dimensional indoor scene reconstruction method based on semantic segmentation |
| CN113378756A (en) * | 2021-06-24 | 2021-09-10 | 深圳市赛维网络科技有限公司 | Three-dimensional human body semantic segmentation method, terminal device and storage medium |
| CN113378756B (en) * | 2021-06-24 | 2022-06-14 | 深圳市赛维网络科技有限公司 | Three-dimensional human body semantic segmentation method, terminal device and storage medium |
| CN113743417B (en) * | 2021-09-03 | 2024-02-23 | 北京航空航天大学 | Semantic segmentation method and semantic segmentation device |
| CN113743417A (en) * | 2021-09-03 | 2021-12-03 | 北京航空航天大学 | Semantic segmentation method and semantic segmentation device |
| CN113762195A (en) * | 2021-09-16 | 2021-12-07 | 复旦大学 | Point cloud semantic segmentation and understanding method based on road side RSU |
| CN114035575B (en) * | 2021-11-04 | 2023-03-31 | 南京理工大学 | Unmanned vehicle motion planning method and system based on semantic segmentation |
| CN114035575A (en) * | 2021-11-04 | 2022-02-11 | 南京理工大学 | Unmanned vehicle motion planning method and system based on semantic segmentation |
| CN114387289A (en) * | 2022-03-24 | 2022-04-22 | 南方电网数字电网研究院有限公司 | Semantic segmentation method and device for three-dimensional point cloud of power transmission and distribution overhead line |
| CN115131562A (en) * | 2022-07-08 | 2022-09-30 | 北京百度网讯科技有限公司 | Three-dimensional scene segmentation method, model training method and device and electronic equipment |
| CN115170585A (en) * | 2022-07-12 | 2022-10-11 | 上海人工智能创新中心 | Three-dimensional point cloud semantic segmentation method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109410307B (en) | 2022-09-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109410307A (en) | A kind of scene point cloud semantic segmentation method | |
| CN110458939B (en) | Indoor scene modeling method based on visual angle generation | |
| CN111832655B (en) | Multi-scale three-dimensional target detection method based on characteristic pyramid network | |
| Wei et al. | Aa-rmvsnet: Adaptive aggregation recurrent multi-view stereo network | |
| CN111753698B (en) | Multi-mode three-dimensional point cloud segmentation system and method | |
| Ma et al. | Binary volumetric convolutional neural networks for 3-D object recognition | |
| CN111862101A (en) | 3D point cloud semantic segmentation method under aerial view coding visual angle | |
| Guo et al. | JointPruning: Pruning networks along multiple dimensions for efficient point cloud processing | |
| CN113569979B (en) | Three-dimensional object point cloud classification method based on attention mechanism | |
| EP4365841A1 (en) | Object pose detection method and apparatus, computer device, and storage medium | |
| CN108764019A (en) | A kind of Video Events detection method based on multi-source deep learning | |
| CN109063549A (en) | High-resolution based on deep neural network is taken photo by plane video moving object detection method | |
| CN111028335B (en) | Point cloud data block surface patch reconstruction method based on deep learning | |
| CN113554653B (en) | Semantic segmentation method based on mutual information calibration point cloud data long tail distribution | |
| CN114693744A (en) | Optical flow unsupervised estimation method based on improved cycle generation countermeasure network | |
| Wang et al. | DepthNet Nano: A highly compact self-normalizing neural network for monocular depth estimation | |
| Gu et al. | Ue4-nerf: Neural radiance field for real-time rendering of large-scale scene | |
| CN111597367B (en) | Three-dimensional model retrieval method based on view and hash algorithm | |
| CN112989952A (en) | Crowd density estimation method and device based on mask guidance | |
| Maisano et al. | Reducing complexity of 3D indoor object detection | |
| Gao et al. | Semantic Segmentation of Substation Site Cloud Based on Seg-PointNet | |
| CN116452750A (en) | Object three-dimensional reconstruction method based on mobile terminal | |
| CN116912486A (en) | Target segmentation method based on edge convolution and multidimensional feature fusion and electronic device | |
| CN116228986A (en) | Indoor scene illumination estimation method based on local-global completion strategy | |
| Bao et al. | Pose ResNet: a 3D human pose estimation network model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |