CN113850748A - Point cloud quality evaluation system and method - Google Patents
Point cloud quality evaluation system and method Download PDFInfo
- Publication number
- CN113850748A CN113850748A CN202010517761.XA CN202010517761A CN113850748A CN 113850748 A CN113850748 A CN 113850748A CN 202010517761 A CN202010517761 A CN 202010517761A CN 113850748 A CN113850748 A CN 113850748A
- Authority
- CN
- China
- Prior art keywords
- signal gradient
- point cloud
- signal
- attribute information
- key
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000013441 quality evaluation Methods 0.000 title claims abstract description 22
- 238000011156 evaluation Methods 0.000 claims abstract description 3
- 238000011176 pooling Methods 0.000 claims description 20
- 230000002776 aggregation Effects 0.000 claims description 18
- 238000004220 aggregation Methods 0.000 claims description 18
- 238000005070 sampling Methods 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 17
- 238000004364 calculation method Methods 0.000 claims description 14
- 238000012935 Averaging Methods 0.000 claims description 9
- 238000010606 normalization Methods 0.000 claims description 9
- 238000010276 construction Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Quality & Reliability (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a point cloud quality evaluation method and an evaluation system, which are used for evaluating the quality of point clouds.
Description
Technical Field
The invention relates to the application field of quality evaluation of multiple distortion types existing in point cloud, in particular to a point cloud quality evaluation system and method.
Background
With the widespread use of Virtual Reality (VR), Augmented Reality (AR), and mixed Reality, point clouds have become one of the most interesting immersive media types. A point cloud is a series of point sets containing characteristic attribute information, including coordinates, colors, normal vectors, and the like. In order to make the point cloud be well applied to various application occasions such as compression, transmission, data recovery and the like, the quality of the point cloud must be accurately evaluated. However, compared with traditional media types such as pictures, the points of the point cloud are scattered and distributed in the space, and there is no definite connection relation, so that the quality is difficult to measure. The existing objective evaluation algorithm has large limitation, so that an implementation scheme for designing accurate and efficient objective quality evaluation aiming at point cloud is urgently needed.
Disclosure of Invention
The invention provides a point cloud quality evaluation system and method, which are used for evaluating the point cloud quality by adopting a method for extracting a spatial signal gradient, thereby solving the problem that the distortion degree of the point cloud is difficult to evaluate.
The invention provides a point cloud quality evaluation method, which comprises the following steps:
step S1, sampling the reference point cloud, and extracting key points;
step S2, constructing space neighborhoods, namely a first space neighborhood and a second space neighborhood, by respectively adopting the reference point cloud and the distortion point cloud; respectively selecting neighborhoods with the key point as the center in the first space neighborhood and the second space neighborhood, namely a first neighborhood and a second neighborhood, and respectively establishing the connection relation between the points in the first neighborhood and the second neighborhood;
step S3, generating the signal gradient similarity of the key point corresponding to the attribute information according to the connection relation and the attribute information between the points in the first neighborhood and the second neighborhood of the key point;
step S4, combining the signal gradient similarity of the attribute information corresponding to the key points to generate the signal gradient similarity of the attribute characteristics corresponding to the key points, wherein the attribute characteristics are represented by at least one attribute information;
and step S5, calculating to obtain a quality evaluation result of the corresponding attribute features of the point cloud by combining the signal gradient similarity of the corresponding attribute features of each key point.
Preferably, in step S1, the reference point cloud is sampled, and high-pass filtering or random down-sampling is performed.
Preferably, in step S2, the first and second spatial neighborhoods are constructed based on geometric or spatial composition.
Preferably, in step S3, each attribute information corresponding to the key point has at least one type of signal gradient feature that can be used to calculate a signal gradient similarity.
Preferably, the step S3 includes the steps of:
step S31, calculating the signal gradient of the key point corresponding to the attribute information under the signal gradient characteristic;
step S32, normalizing the signal gradient of the attribute information under the signal gradient characteristic corresponding to the key point to obtain the gradient similarity of the attribute information under the signal gradient characteristic corresponding to the key point;
and step S32, pooling the signal gradient similarity of the attribute information corresponding to the key point under the signal gradient characteristic to obtain the signal gradient similarity of the key point corresponding to the attribute information.
Preferably, each attribute information corresponding to the key point has the following signal gradient characteristics that can be used to calculate the signal gradient similarity: a signal gradient aggregation feature, a signal gradient averaging feature, and a signal gradient covariance feature.
Preferably, the signal gradient of the key point corresponding to the attribute information under the signal gradient aggregation characteristic is: the sum of the signal gradients of the keypoints in the first neighbourhood and all points in the neighbourhood which are in operative connection with the keypoints, and the sum of the signal gradients of the keypoints in the second neighbourhood and all points in the neighbourhood which are in operative connection with the keypoints.
Preferably, the signal gradient of the key point corresponding to the attribute information under the signal gradient average characteristic is: the average value of the signal gradients of the key points in the first neighborhood and all points in the neighborhood which are in effective connection with the key points, and the average value of the signal gradients of the key points in the second neighborhood and all points in the neighborhood which are in effective connection with the key points.
Preferably, the signal gradient of the attribute information corresponding to the key point under the signal gradient covariance characteristic is: the covariance of the sequence of weighted signal gradients for points in the first neighborhood that are connected to the keypoint and the sequence of weighted signal gradients for points in the second neighborhood that are connected to the keypoint.
Preferably, in step S4, the method for generating the signal gradient similarity of the corresponding attribute feature of the key point includes:
when the attribute features of the key points are represented by only one attribute information, the signal gradient similarity of the key points corresponding to the attribute information is taken as the signal gradient similarity of the corresponding attribute features of the key points;
when the attribute features of the key points are represented by adopting various attribute information, weighting, multiplying, adding or averaging the signal gradient similarity of the various attribute information which is used for representing the same attribute feature and corresponds to the key points, and taking the signal gradient similarity of the corresponding attribute feature of the key points.
Optionally, in step S5, performing average calculation or weighted average calculation on the signal gradient similarity of the attribute feature corresponding to each key point to obtain a quality evaluation result of the attribute feature corresponding to the point cloud.
The invention provides a point cloud quality evaluation system, which is used for evaluating the quality of distorted point clouds and comprises a point cloud sampling module, a point cloud space construction module, a point cloud space gradient extraction module and a characteristic pooling module, and is characterized in that:
the point cloud sampling module is used for sampling the reference point cloud and extracting key points;
the point cloud space construction module is used for constructing two space neighborhoods, namely a first space neighborhood and a second space neighborhood, by respectively adopting the reference point cloud and the distortion point cloud, respectively selecting neighborhoods with the key point as the center in the two space neighborhoods, namely the first neighborhood and the second neighborhood, and respectively establishing the connection relation between the points in the two neighborhoods;
the point cloud spatial gradient extraction module generates signal gradients of the attribute information corresponding to the key points under the signal gradient characteristics according to the connection relation, the attribute information and the signal gradient characteristics between the points in two adjacent regions of the key points;
the feature pooling module is used for performing normalization processing and pooling processing on the signal gradient of the attribute information under the signal gradient feature corresponding to the key point to generate signal gradient similarity of the key point corresponding to the attribute information and further generating the signal gradient similarity of the key point corresponding to the attribute feature and a quality evaluation result of the point cloud corresponding to the attribute feature;
the attribute features are represented by at least one attribute information, and each attribute information has at least more than one type of signal gradient features which can be used for calculating the signal gradient similarity.
Preferably, the feature pooling module performs normalization processing on the signal gradient of the attribute information under the signal gradient feature corresponding to the key point to obtain the gradient similarity of the attribute information under the signal gradient feature corresponding to the key point; and performing pooling treatment to obtain the signal gradient similarity of the key point to the attribute information.
Preferably, the feature pooling module generates the signal gradient similarity of the corresponding attribute feature of the keypoint by:
when the attribute features of the key points are represented by only one attribute information, the signal gradient similarity of the key points corresponding to the attribute information is taken as the signal gradient similarity of the corresponding attribute features of the key points;
when the attribute features of the key points are represented by adopting various attribute information, weighting, multiplying, adding or averaging the signal gradient similarity of the various attribute information which is used for representing the same attribute feature and corresponds to the key points, and taking the signal gradient similarity of the corresponding attribute feature of the key points.
Preferably, the method for generating the quality evaluation result of the corresponding attribute feature of the point cloud by the feature pooling module is as follows:
and carrying out average calculation or weighted average calculation on the signal gradient similarity of the corresponding attribute features of each key point in the point cloud.
Compared with the prior art, the invention has the following beneficial effects:
according to the method, the key points are screened out and the signal gradients of the key points are extracted for evaluating the distortion degree by adopting a method for evaluating the quality of the distortion degree of the point cloud quality based on the spatial signal gradients, so that the problem that the distortion degree of the point cloud is difficult to evaluate is solved, and a basis is provided for the correct processing of subsequent point cloud data.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
fig. 1 is a schematic flow chart of the method for evaluating the point cloud quality according to the present invention.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.
Fig. 1 is a flow chart of a method for evaluating the quality of a point cloud according to the present invention, which mainly includes the following parts:
firstly, sampling a reference point cloud by using a point cloud sampling method so as to obtain a series of key points. The point cloud sampling module is used for executing the processing procedure of the steps. The sampling can be performed by various sampling methods such as high-pass filtering or random down-sampling.
Recording the reference point cloud as P, wherein P belongs to Rm×nM represents the number of points, n represents the attribute characteristics of each point, and the attribute characteristics comprise geometry, color, normal vector, reflectivity and the like.
In a preferred embodiment, the reference point cloud P is recorded for the human visual system, taking into account the geometric (G) and color (C) featuresG,PC]∈Rm×6，PG,PC∈Rm×3(ii) a Sampling the reference point cloud to obtain a key point set PS，PS∈Rk×6K is less than m, and k is the number of key points.
In a preferred embodiment, the point cloud sampling module comprises a high-pass filter, and the high-pass filter can filter out points in the point cloud which meet a certain characteristic, or filter out points in the point cloud which are larger than a certain value.
The attribute features of each point in the reference point cloud and the distortion point cloud may be represented by one kind of attribute information or multiple kinds of attribute information according to actual needs. In a preferred embodiment, the attribute characteristics of the color are represented by three attribute information of R (red ), G (green ) and B (blue ), and red, green and blue are three primary colors.
In the step, the distorted point cloud does not need to be sampled, and the same key points are adopted for the subsequent data processing of the distorted point cloud and the reference point cloud, namely the key points obtained by sampling the reference point cloud.
And secondly, constructing expression modes of the reference graph and the distortion graph, wherein the point cloud space construction module is used for executing the processing process of the step.
(1) Constructing expression modes of the reference picture: and constructing a spatial neighborhood of the reference point cloud, namely a first spatial neighborhood, by using the reference point cloud. The first spatial neighborhood may be constructed centered on the keypoint. And selecting a first neighborhood taking the key point as a center in the first spatial neighborhood, wherein the first neighborhood of each key point comprises the key point and points adjacent to the key point in the reference point cloud. And establishing a connection relation between points in the first neighborhood, namely a first connection relation.
(2) Constructing a distortion map expression mode: and constructing a spatial neighborhood of the distorted point cloud, namely a second spatial neighborhood, by using the distorted point cloud. The second spatial neighborhood may also be constructed centered around the keypoint. And selecting a second neighborhood taking the key point as the center in a second space neighborhood, wherein the second neighborhood of each key point comprises the key point and the points adjacent to the key point in the distortion point cloud. And establishing a connection relationship between points in the second neighborhood, namely a second connection relationship.
The first spatial neighborhood and the second spatial neighborhood may be established based on geometric construction, spatial construction, and the like. In a preferred embodiment, the first spatial neighborhood and the second spatial neighborhood are established based on euclidean distances.
The first neighborhood and the second neighborhood are the same in size and are selected based on requirements. In a preferred embodiment, a gaussian function is used to establish the connection relationship between each point in the first neighborhood and each point in the second neighborhood, and the specific calculation method of the connection relationship between each point in the first neighborhood and each point in the second neighborhood is as follows:
for two points a, b,
wherein, tau represents the Euclidean distance,
the position information of points a and b in the neighborhood is respectively shown, G represents the geometrical attribute characteristic, and sigma represents the variance of a Gaussian function. The adjacent points are adjacent points where two points a and b different from 0 are opposite to each other.
If Wa,bNot equal to 0, the representative points a and b have a spatial dependence relationship, and the dependence degree thereof has a positive correlation with the Euclidean distance. Based on this, for any neighborhood Φ centered on a key point, an adjacency matrix W describing the connection relationship of each point included in the neighborhood Φ can be obtained.
And thirdly, extracting the spatial signal gradient, wherein the point cloud spatial gradient extraction module is used for executing the processing procedure of the step.
One attribute feature of each point in the reference point cloud and the distortion point cloud corresponds to one signal; an attribute feature may be represented by one or more types of attribute information, and one or more channels may exist for a corresponding signal, one channel being matched with one attribute information.
And respectively generating the signal gradient similarity of the key point corresponding to the attribute information in the first neighborhood and the second neighborhood of the key point according to the connection relation between the points and the available signal gradient characteristics of the attribute information.
In a preferred embodiment, the signal f is characterized by a selected attribute of color (C, color), f ∈ PCThe attribute features of the color are represented by three types of attribute information, namely R (red), G (green ), and B (blue). In one neighborhood (first neighborhood or second neighborhood) of the keypoint, signal f may select attribute information (R, G or B) of points within the neighborhood.
For each attribute information, there are at least three types of signal gradient features that can be used to calculate signal gradients and/or signal gradient similarities, respectively: a signal gradient aggregation feature, a signal gradient averaging feature, and a signal gradient covariance feature.
First-class signal gradient characteristics: signal gradient aggregation characteristics. Defining the signal gradient z (a) of the attribute information corresponding to any point a in the neighborhood under the signal gradient aggregation characteristic as follows:
wherein, Wa,bValues representing the direct adjacency of points a and b, f (a), f (b) values representing the attribute information corresponding to signal f at points a and b, phiaRepresenting the set of points in the neighborhood that have a connection relationship with a.
The signal gradient z (i) of the key point i under the signal gradient aggregation characteristic can be calculated by adopting the formula.
Second-class signal gradient characteristics: signal gradient average characteristics. Based on the signal gradient z (a) of the point a corresponding to certain attribute information under the signal gradient aggregation characteristic, the signal gradient k (a) of the point a corresponding to the attribute information in the neighborhood under the signal gradient average characteristic can be further calculated,
k(a)＝z(a)/N,
n represents the number of points in the geometric neighborhood which have a definite connection relation with the point a.
The signal gradient k (i) of the key point i under the signal gradient average characteristic can be calculated by adopting the formula.
The third type of signal gradient feature: signal gradient covariance characterization. Aiming at a first neighborhood and a second neighborhood of a certain key point, the following processing is carried out: firstly, comparing the number of adjacent points of the key points in the first neighborhood and the second neighborhood (namely, the points with the adjacent relation with the key points not being zero), searching a neighborhood with less adjacent points of the key points, and recording the neighborhood as a reference neighborhood of the key points, and recording the other neighborhood as a candidate neighborhood of the key points; respectively extracting points (namely adjacent points) with definite connection relation with the key points from the reference neighborhood and the candidate neighborhood, respectively randomly arranging the points into a queue, and marking the queue as QbAnd QcThe dimensions of the two may not be the same, i.e. the number of extracted neighboring points may not be the same. For QbEvery point within (every point is the adjacent point of the key point), from QcIf the geometric nearest neighbor point is searched, two sequences L with the same dimensionality and with the point at each position being the Euclidean nearest neighbor point can be generatedbAnd Lc. Calculating weighted signal gradients separately for points in both sequences, i.e.
G(j)＝Wj,i(f(j)-f(i))。
Wherein G (j) represents the sequence Lb、LcF (j) represents the sequence Lb、LcSignal intensity of the j-th point in (1), Wj,iRepresenting the adjacency matrix coefficient of the j-th point and the key point i. Based on the above formula, can respectively aim at the sequence LbAnd LcRespectively calculating the sequence G of weighted signal gradientsbAnd Gc。
Then using GbAnd GcCalculating the signal gradient cov (G) of the key points in the neighborhood corresponding to the attribute information under the signal gradient covariance characteristicb，Gc)，
Wherein cov () represents covariance calculation, cov (a, b) ═ E (ab) -E (a) E (b), and E () represents average value calculation.
And fourthly, pooling characteristics, wherein the characteristic pooling module is used for executing the processing procedure of the step and comprises two substeps.
Firstly, each attribute information corresponding to a key point has at least three types of signal gradient features which can be used for calculating signal gradient and signal gradient similarity, and the signal gradient features are respectively as follows: a signal gradient aggregation feature, a signal gradient averaging feature, and a signal gradient covariance feature. The key point corresponds to certain attribute information and has two signal gradients respectively under the signal gradient aggregation characteristic and the signal gradient average characteristic, namely a first signal gradient in a first neighborhood and a second signal gradient in a second neighborhood; the key point corresponds to a certain attribute information, and only one signal gradient exists under the signal gradient covariance characteristic.
And respectively carrying out normalization processing on the signal gradients of a certain attribute information corresponding to the key point under the signal gradient aggregation characteristic, the signal gradient average characteristic and the signal gradient covariance characteristic to generate the signal gradient similarity of the attribute information corresponding to the key point under the corresponding signal gradient characteristic.
In a preferred embodiment, the normalization process is performed as follows:
for the key point i, the signal gradients corresponding to certain signal information (namely attribute information) on the reference point cloud and the distorted point cloud (namely the signal gradients corresponding to certain signal information in the first neighborhood and the second neighborhood) are respectively z (i)r、z(i)dThen the similarity between the two can be calculated by the following formula:
wherein T is a small positive real number, such as 0.001, which prevents the denominator from being zero and ensures the stability of the calculation result. s (i) belongs to (0,1), when s → 1, it shows that the Quality of the distorted point cloud and the reference point cloud is close to each other at the point, and the Quality of Experience (Quality of Experience) can be obtained at the time of human eye consumption. And normalizing the signal gradient of certain attribute information corresponding to the key point under the signal gradient aggregation characteristic and the signal gradient average characteristic, and respectively obtaining the signal gradient similarity of the attribute information corresponding to the key point under the signal gradient aggregation characteristic and the signal gradient average characteristic by adopting the normalization processing mode.
In a preferred embodiment, the normalization process is performed as follows:
sequence G Using Signal gradientsbAnd GcCalculating the gradient similarity o (i) of a certain attribute information corresponding to the key point i under the characteristic of signal gradient covariance,
wherein cov () represents covariance calculation, cov (a, b) ═ E (ab) -E (a) E (b), E () represents average value calculation,
each represents a group Gb、GcT is a small constant that prevents the denominator from being zero.
And then, performing pooling processing on the signal gradient similarity of a certain attribute information corresponding to the key point under the characteristics of a plurality of signal gradients, and finally obtaining the signal gradient similarity of the attribute information corresponding to the key point.
In a preferred embodiment, pooling is performed on the signal gradient similarity of a key point to certain attribute information under the signal gradient aggregation characteristic, the signal gradient average characteristic and the signal gradient covariance characteristic, respectively, to obtain the signal gradient similarity (i.e., the signal gradient similarity under a certain channel of the signal f) of the key point to the attribute information:
Q'＝sim(z(i)r,z(i)d)αsim(k(i)r,k(i)d)βo(i)γ,
wherein Q' represents the signal gradient similarity of the key point i corresponding to the attribute information; z (i)r、z(i)dRepresenting the signal gradient of the attribute information (namely signal information) corresponding to the key point i on the reference point cloud and the distortion point cloud under the signal gradient aggregation characteristic (namely the key point i in the first neighborhood and the second neighborhood corresponds to the signal gradient of certain signal information under the signal gradient aggregation characteristic); k (i)r、k(i)dRepresenting reference point clouds and distortion pointsThe key point i on the cloud corresponds to the signal gradient of the attribute information (i.e. the signal information) under the signal gradient average characteristic (i.e. the key point i in the first neighborhood and the key point i in the second neighborhood correspond to the signal gradient of certain signal information under the signal gradient average characteristic); o (i) representing the gradient similarity of the key points i on the reference point cloud and the distorted point cloud corresponding to the attribute information under the signal gradient covariance characteristic; alpha, beta and gamma respectively represent the enhancement coefficients of the similarity of the three signal gradients under the signal gradient aggregation characteristic, the signal gradient average characteristic and the signal gradient covariance characteristic.
In a preferred embodiment, the signal gradient similarity of a certain attribute information under a plurality of signal gradient characteristics is corresponding to the key point, and the signal gradient similarity corresponding to the attribute information is obtained as the key point by adding or averaging the signal gradient similarities.
Finally, when only one channel exists in the signal f, directly taking the signal gradient similarity of the key point corresponding to the attribute information as the signal gradient similarity of the corresponding attribute characteristic of the key point; and when the signal f has a plurality of channels, multiplying the signal gradient similarity weights of a plurality of types of attribute information which are used for representing the same attribute characteristics and correspond to the key points (namely weighting the signal gradients of the key points in the plurality of channels) to obtain the signal gradient similarity of the corresponding attribute characteristics of the key points.
The method takes the key points extracted in the first step as an object for calculating the signal gradient similarity, namely calculating the similarity of k key points, then adding the similarity, and carrying out average calculation or reinforced average calculation to obtain the quality evaluation result of the corresponding attribute characteristics of the point cloud.
In a preferred embodiment, the attribute features of the color (C, color) of the signal f include three channels (corresponding to three attribute information) of R (red ), G (green ), and B (blue ), and for the extracted k key points, the signal gradient similarity of the color (C, color) of the signal f is a weighted sum of R, G, B channels, that is:
wherein, Q'R,iRepresenting the signal gradient similarity, Q 'of the key point i in the R channel'G,iRepresenting the signal gradient similarity, Q 'of the key point i in the G channel'B,iSignal gradient, s, representing the critical point i in the B channel1，s2，s3Enhancement coefficients representing the similarity of the signal gradients of R, G, B channels, S ═ S1+s2+s3And | represents absolute value operation, and Q is the final objective quality evaluation score (quality evaluation result of corresponding attribute features of the point cloud).
According to the technical scheme, the method for evaluating the quality of the distortion degree of the point cloud quality based on the spatial signal gradient is adopted, the key points are screened out, the signal gradients of the key points are extracted, the signal gradient similarity of the key points in the reference point cloud and the distortion point cloud is compared, and therefore the distortion degree of the point cloud is evaluated, the problem that the distortion degree of the point cloud is difficult to evaluate is solved, a basis is provided for correction compensation in the processes of transmission, compression, encoding and the like of point cloud data, and the accuracy of information is guaranteed.
Those skilled in the art will appreciate that, in addition to implementing the systems, apparatus, and individual modules thereof provided by the present invention in purely computer readable program code, the same procedures can be implemented entirely by logically programming method steps into logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system, the device and the modules thereof provided by the present invention can be considered as a hardware component, and the modules included in the system, the device and the modules thereof for implementing various programs can also be considered as structures in the hardware component; modules for performing various functions may also be considered to be both software programs for performing the methods and structures within hardware components.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.
Claims (15)
1. A point cloud quality evaluation method is characterized by comprising the following steps:
step S1: sampling the reference point cloud, and extracting key points;
step S2: respectively adopting the reference point cloud and the distortion point cloud to construct space neighborhoods, namely a first space neighborhood and a second space neighborhood; respectively selecting neighborhoods with the key point as the center in the first space neighborhood and the second space neighborhood, namely a first neighborhood and a second neighborhood, and respectively establishing the connection relation between the points in the first neighborhood and the second neighborhood;
step S3: generating signal gradient similarity of the key point corresponding to the attribute information according to the connection relation and the attribute information between the points in the first neighborhood and the second neighborhood of the key point;
step S4: combining the signal gradient similarity of the attribute information corresponding to the key points to generate the signal gradient similarity of the attribute characteristics corresponding to the key points, wherein the attribute characteristics are represented by at least one attribute information;
step S5: and calculating to obtain a quality evaluation result of the corresponding attribute features of the point cloud by combining the signal gradient similarity of the corresponding attribute features of each key point.
2. The method of claim 1, wherein the point cloud quality is evaluated,
in step S1, the reference point cloud is sampled, which may be high-pass filtered or randomly down-sampled.
3. The method of claim 1, wherein the point cloud quality is evaluated,
in step S2, the first and second spatial neighborhoods are constructed based on geometric or spatial composition.
4. The method of claim 1, wherein the point cloud quality is evaluated,
in step S3, each attribute information corresponding to the key point has at least one type of signal gradient feature that can be used to calculate a signal gradient similarity.
5. The method of claim 4, wherein the point cloud quality is evaluated,
the step S3 includes the following steps:
step S31, calculating the signal gradient of the key point corresponding to the attribute information under the signal gradient characteristic;
step S32, aiming at the signal gradient of the attribute information corresponding to the key point under the signal gradient characteristic, carrying out normalization processing to obtain the gradient similarity of the attribute information corresponding to the key point under the signal gradient characteristic;
and step S32, pooling the signal gradient similarity of the attribute information corresponding to the key point under the signal gradient characteristic to obtain the signal gradient similarity of the key point corresponding to the attribute information.
6. The method of claim 5, wherein the point cloud quality is evaluated,
each attribute information corresponding to the key point has the following signal gradient characteristics which can be used for calculating the signal gradient similarity: a signal gradient aggregation feature, a signal gradient averaging feature, and a signal gradient covariance feature.
7. The method of claim 6, wherein the point cloud quality is evaluated,
the key point corresponds to the signal gradient of the attribute information under the signal gradient aggregation characteristic: the sum of the signal gradients of the keypoints in the first neighbourhood and all points in the neighbourhood which are in operative connection with the keypoints, and the sum of the signal gradients of the keypoints in the second neighbourhood and all points in the neighbourhood which are in operative connection with the keypoints.
8. The method of claim 6, wherein the point cloud quality is evaluated,
the key points correspond to the signal gradients of the attribute information under the signal gradient average characteristic: the average value of the signal gradients of the key points in the first neighborhood and all points in the neighborhood which are in effective connection with the key points, and the average value of the signal gradients of the key points in the second neighborhood and all points in the neighborhood which are in effective connection with the key points.
9. The method of claim 6, wherein the point cloud quality is evaluated,
the key points correspond to the signal gradients of the attribute information under the signal gradient covariance characteristic: the covariance of the sequence of weighted signal gradients for points in the first neighborhood that are connected to the keypoint and the sequence of weighted signal gradients for points in the second neighborhood that are connected to the keypoint.
10. The method of claim 1, wherein the point cloud quality is evaluated,
in step S4, the method for generating the signal gradient similarity of the corresponding attribute feature of the keypoint comprises:
when the attribute features of the key points are represented by only one attribute information, the signal gradient similarity of the key points corresponding to the attribute information is taken as the signal gradient similarity of the corresponding attribute features of the key points;
when the attribute features of the key points are represented by adopting various attribute information, weighting, multiplying, adding or averaging the signal gradient similarity of the various attribute information which is used for representing the same attribute feature and corresponds to the key points, and taking the signal gradient similarity of the corresponding attribute feature of the key points.
11. The method of claim 1, wherein the point cloud quality is evaluated,
in step S5, the signal gradient similarity of the attribute features corresponding to each key point is averaged or weighted average calculated to obtain a quality evaluation result of the corresponding attribute features of the point cloud.
12. The utility model provides an evaluation system of point cloud quality for carry out the quality evaluation to the distortion point cloud, including point cloud sampling module, point cloud space construction module, point cloud space gradient extraction module and characteristic pooling module, its characterized in that:
the point cloud sampling module is used for sampling the reference point cloud and extracting key points;
the point cloud space construction module is used for constructing two space neighborhoods, namely a first space neighborhood and a second space neighborhood, by respectively adopting the reference point cloud and the distortion point cloud, respectively selecting neighborhoods with the key point as the center in the two space neighborhoods, namely the first neighborhood and the second neighborhood, and respectively establishing the connection relation between the points in the two neighborhoods;
the point cloud spatial gradient extraction module generates signal gradients of the attribute information corresponding to the key points under the signal gradient characteristics according to the connection relation, the attribute information and the signal gradient characteristics between the points in two adjacent regions of the key points;
the feature pooling module is used for performing normalization processing and pooling processing on the signal gradient of the attribute information under the signal gradient feature corresponding to the key point to generate signal gradient similarity of the key point corresponding to the attribute information and further generating the signal gradient similarity of the key point corresponding to the attribute feature and a quality evaluation result of the point cloud corresponding to the attribute feature;
the attribute features are represented by at least one type of attribute information, and each type of attribute information has at least more than one type of signal gradient features which can be used for calculating the similarity of signal gradients.
13. The system for evaluating the quality of a point cloud of claim 12,
the feature pooling module is used for carrying out normalization processing on the signal gradient of the attribute information under the signal gradient feature corresponding to the key point to obtain the gradient similarity of the attribute information under the signal gradient feature corresponding to the key point; and performing pooling treatment to obtain the signal gradient similarity of the key point corresponding to the attribute information.
14. The system for evaluating the quality of a point cloud of claim 12,
the method for generating the signal gradient similarity of the corresponding attribute features of the key points by the feature pooling module comprises the following steps:
when the attribute features of the key points are represented by only one attribute information, the signal gradient similarity of the key points corresponding to the attribute information is taken as the signal gradient similarity of the corresponding attribute features of the key points;
when the attribute features of the key points are represented by adopting various attribute information, weighting, multiplying, adding or averaging the signal gradient similarity of the various attribute information which is used for representing the same attribute feature and corresponds to the key points, and taking the signal gradient similarity of the corresponding attribute feature of the key points.
15. The system for evaluating the quality of a point cloud of claim 12,
the method for generating the quality evaluation result of the corresponding attribute features of the point cloud by the feature pooling module comprises the following steps:
and carrying out average calculation or weighted average calculation on the signal gradient similarity of the corresponding attribute features of each key point in the point cloud.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010517761.XA CN113850748B (en) | 2020-06-09 | 2020-06-09 | Evaluation system and method for point cloud quality |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010517761.XA CN113850748B (en) | 2020-06-09 | 2020-06-09 | Evaluation system and method for point cloud quality |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113850748A true CN113850748A (en) | 2021-12-28 |
| CN113850748B CN113850748B (en) | 2024-07-26 |
Family
ID=78971994
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010517761.XA Active CN113850748B (en) | 2020-06-09 | 2020-06-09 | Evaluation system and method for point cloud quality |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113850748B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114782332A (en) * | 2022-04-02 | 2022-07-22 | 宁波大学 | Color point cloud quality evaluation method based on geometric and color joint perception |
| CN116542981A (en) * | 2023-07-06 | 2023-08-04 | 无锡晨智物联科技有限公司 | Quality assessment method for reference-point-free cloud |
| CN117011299A (en) * | 2023-10-07 | 2023-11-07 | 华侨大学 | Reference point cloud quality assessment method and system integrating graph resampling and gradient characteristics |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190087979A1 (en) * | 2017-09-18 | 2019-03-21 | Apple Inc. | Point cloud compression |
| US20190156518A1 (en) * | 2017-11-22 | 2019-05-23 | Apple Inc. | Point cloud compression with closed-loop color conversion |
| CN111080684A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨工程大学 | Point cloud registration method for point neighborhood scale difference description |
-
2020
- 2020-06-09 CN CN202010517761.XA patent/CN113850748B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190087979A1 (en) * | 2017-09-18 | 2019-03-21 | Apple Inc. | Point cloud compression |
| US20190156518A1 (en) * | 2017-11-22 | 2019-05-23 | Apple Inc. | Point cloud compression with closed-loop color conversion |
| CN111080684A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨工程大学 | Point cloud registration method for point neighborhood scale difference description |
Non-Patent Citations (2)
| Title |
|---|
| QI YANG等: "Inferring Point Cloud Quality via Graph Similarity", 《 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 》, 24 December 2020 (2020-12-24) * |
| 王新;刘永山;朱代春;刘新;张小潮;: "三维点云配准算法的研究", 燕山大学学报, no. 06, 30 November 2016 (2016-11-30) * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114782332A (en) * | 2022-04-02 | 2022-07-22 | 宁波大学 | Color point cloud quality evaluation method based on geometric and color joint perception |
| CN116542981A (en) * | 2023-07-06 | 2023-08-04 | 无锡晨智物联科技有限公司 | Quality assessment method for reference-point-free cloud |
| CN116542981B (en) * | 2023-07-06 | 2023-08-29 | 无锡晨智物联科技有限公司 | Quality assessment method for reference-point-free cloud |
| CN117011299A (en) * | 2023-10-07 | 2023-11-07 | 华侨大学 | Reference point cloud quality assessment method and system integrating graph resampling and gradient characteristics |
| CN117011299B (en) * | 2023-10-07 | 2024-02-20 | 华侨大学 | Reference point cloud quality assessment method and system integrating graph resampling and gradient characteristics |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113850748B (en) | 2024-07-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108492343B (en) | Image synthesis method for training data for expanding target recognition | |
| CN102542058B (en) | Hierarchical landmark identification method integrating global visual characteristics and local visual characteristics | |
| CN107123088B (en) | A kind of method of automatic replacement photo background color | |
| Li et al. | Example-based image colorization using locality consistent sparse representation | |
| US6674915B1 (en) | Descriptors adjustment when using steerable pyramid to extract features for content based search | |
| Krig et al. | Image pre-processing | |
| CN107967482A (en) | Icon-based programming method and device | |
| CN113850748A (en) | Point cloud quality evaluation system and method | |
| WO2017181892A1 (en) | Foreground segmentation method and device | |
| CN111046789A (en) | Pedestrian re-identification method | |
| CN107944459A (en) | A kind of RGB D object identification methods | |
| CN110188217A (en) | Image duplicate checking method, apparatus, equipment and computer-readable storage media | |
| CN107622280B (en) | Modularized processing mode image saliency detection method based on scene classification | |
| CN104732534B (en) | Well-marked target takes method and system in a kind of image | |
| CN107392211B (en) | Salient target detection method based on visual sparse cognition | |
| CN113066030A (en) | Multispectral image panchromatic sharpening method and system based on space-spectrum fusion network | |
| CN109165551B (en) | Expression recognition method for adaptively weighting and fusing significance structure tensor and LBP characteristics | |
| CN111784658A (en) | Quality analysis method and system for face image | |
| CN113223098A (en) | Preprocessing optimization method for image color classification | |
| CN112884884A (en) | Candidate region generation method and system | |
| CN109377524B (en) | Method and system for recovering depth of single image | |
| CN116071625B (en) | Training method of deep learning model, target detection method and device | |
| CN108230409B (en) | Image similarity quantitative analysis method based on multi-factor synthesis of color and content | |
| Viacheslav et al. | Low-level features for inpainting quality assessment | |
| Tang et al. | A GMS-guided approach for 2D feature correspondence selection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |