JP2024152169A - Point group decoding device, point group decoding method and program - Google Patents

Abstract

[Problem] To improve the compression performance of encoding. [Solution] The point cloud decoding device 200 according to the present invention includes a tree synthesis unit 2020 that, when selecting an inter-predictor based on the laser ID of the parent node of a processing target node and an azimuth obtained by subtracting an offset value from the azimuth, selects a second candidate from among first candidates of predefined offset values using a predetermined method and specifies the offset value from among the second candidates. [Selected Figure] Figure 2

Description

The present invention relates to a point cloud decoding device, a point cloud decoding method, and a program.

Non-Patent Document 1 discloses a technique for performing intra-prediction in predictive coding.

Non-Patent Document 2 discloses a technology for performing inter-prediction using an inter-predictor selected from a reference frame in predictive coding.

G-PCC codec description, ISO/IEC JTC1/SC29/WG7 N00271 G-PCC 2nd Edition codec description, ISO/IEC JTC1/SC29/WG7 N00314

However, the method in Non-Patent Document 1 only performs intra-prediction, which causes problems in that prediction residuals become large in non-uniform scenes, which can impair compression performance.

In addition, in the method of Non-Patent Document 2, in addition to intra prediction, inter prediction is performed. However, since the inter predictor is selected from the point of the reference frame based on the laser ID and azimuth angle of the point decoded immediately before, if there is a large positional deviation between the frame to be decoded and the reference frame, an appropriate predictor cannot be selected, which can result in a loss of compression performance.

The present invention has been made in consideration of the above-mentioned problems, and aims to provide a point cloud decoding device, a point cloud decoding method, and a program that can improve the compression performance of encoding.

A first feature of the present invention is a point cloud decoding device comprising a tree synthesis unit that, in predictive coding, when selecting an inter-predictor based on an azimuth obtained by subtracting an offset value from the laser ID and azimuth of a parent node of a node to be processed, selects a second candidate from among first candidates of predefined offset values by a predetermined method, and identifies the offset value from the second candidate.A second feature of the present invention is a point cloud decoding method that, in predictive coding, when selecting an inter-predictor based on an azimuth obtained by subtracting an offset value from the laser ID and azimuth of a parent node of a node to be processed, comprises a step of selecting a second candidate from among first candidates of predefined offset values by a predetermined method, and identifying the offset value from the second candidate.

The third feature of the present invention is a program for causing a computer to function as a point cloud decoding device, the point cloud decoding device including a tree synthesis unit that, when selecting an inter-predictor based on the laser ID of the parent node of a processing target node and an azimuth obtained by subtracting an offset value from the azimuth in predictive coding, selects a second candidate from among first candidates of predefined offset values using a predetermined method, and identifies the offset value from among the second candidates.

The present invention provides a point cloud decoding device, a point cloud decoding method, and a program that can improve the compression performance of encoding.

FIG. 1 is a diagram illustrating an example of the configuration of a point cloud processing system 10 according to an embodiment. FIG. 2 is a diagram illustrating an example of functional blocks of a point group decoding device 200 according to an embodiment. FIG. 3 is a diagram showing an example of the configuration of encoded data (bit stream) received by the geometric information decoding unit 2010 of the point cloud decoding device 200 according to an embodiment. FIG. 4 is a diagram showing an example of the syntax configuration of GPS2011. FIG. 5 is a flowchart showing an example of processing in the tree synthesis unit 2020 of the point group decoding device 200 according to an embodiment. FIG. 6 is a flowchart showing an example of the slice data decoding process in step S505. FIG. 7 is a flowchart showing an example of the coordinate prediction process in step S604. FIG. 8 is a diagram showing an example of the process of selecting a predictor from a reference frame in step S704. FIG. 9 is a diagram showing an example of the process of selecting a predictor from a reference frame in step S704. FIG. 10 is a diagram showing an example of functional blocks of the point group encoding device 100 according to this embodiment.

The following describes embodiments of the present invention with reference to the drawings. Note that the components in the following embodiments can be replaced with existing components as appropriate, and various variations, including combinations with other existing components, are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

First Embodiment
A point cloud processing system 10 according to a first embodiment of the present invention will be described below with reference to Figures 1 to 10. Figure 1 is a diagram showing a point cloud processing system 10 according to the present embodiment.

As shown in FIG. 1, the point cloud processing system 10 has a point cloud encoding device 100 and a point cloud decoding device 200.

The point cloud encoding device 100 is configured to generate encoded data (bit stream) by encoding an input point cloud signal. The point cloud decoding device 200 is configured to generate an output point cloud signal by decoding the bit stream.

The input point cloud signal and the output point cloud signal are composed of position information and attribute information of each point in the point cloud. The attribute information is, for example, color information and reflectance of each point.

Here, such a bit stream may be transmitted from the point cloud encoding device 100 to the point cloud decoding device 200 via a transmission path. Also, the bit stream may be stored in a storage medium and then provided from the point cloud encoding device 100 to the point cloud decoding device 200.

(Point Cloud Decoding Device 200)
Hereinafter, the point group decoding device 200 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of functional blocks of the point group decoding device 200 according to this embodiment.

As shown in FIG. 2, the point cloud decoding device 200 includes a geometric information decoding unit 2010, a tree synthesis unit 2020, an approximate surface synthesis unit 2030, a geometric information reconstruction unit 2040, an inverse coordinate transformation unit 2050, an attribute information decoding unit 2060, an inverse quantization unit 2070, a RAHT unit 2080, an LoD calculation unit 2090, an inverse lifting unit 2100, an inverse color transformation unit 2110, and a frame buffer 2120.

The geometric information decoding unit 2010 is configured to receive as input a bit stream related to geometric information (geometric information bit stream) from the bit streams output from the point cloud encoding device 100, and to decode the syntax.

The decoding process is, for example, a context-adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the position information.

The tree synthesis unit 2020 is configured to receive as input the control data decoded by the geometric information decoding unit 2010 and an occurrence code indicating at which node in the tree (described later) the point group exists, and generate tree information indicating in which area in the decoding target space the point exists.

The decoding process of the occasion code may be configured to be performed within the tree synthesis unit 2020.

This process divides the space to be decoded into rectangular parallelepipeds, refers to the occupancy code to determine whether a point exists in each rectangular parallelepiped, divides the rectangular parallelepiped in which the point exists into multiple rectangular parallelepipeds, and then refers to the occupancy code. This process is repeated recursively to generate tree information.

Here, when decoding such an occasion code, inter prediction, which will be described later, may be used.

In this embodiment, a method called "Octree" can be used, which recursively performs octree division on the above-mentioned rectangular parallelepiped, always treating it as a cube, and a method called "QtBt" can be used, which performs quadtree division and binary tree division in addition to octree division. Whether or not to use "QtBt" is transmitted as control data from the point cloud encoding device 100.

Alternatively, when the control data specifies that predictive geometry coding is to be used, the tree synthesis unit 2020 is configured to decode the coordinates of each point based on an arbitrary tree configuration determined by the point cloud encoding device 100.

The approximate surface synthesis unit 2030 is configured to generate approximate surface information using the tree information generated by the tree synthesis unit 2020, and to decode the point cloud based on the approximate surface information.

When decoding three-dimensional point cloud data of an object, for example, if the points are densely distributed on the object's surface, approximate surface information is used to represent the area in which the points exist by approximating the area using a small plane, rather than decoding each point individually.

Specifically, the approximate surface synthesis unit 2030 can generate approximate surface information and decode the point cloud using, for example, a method called "Trisoup." A specific processing example of "Trisoup" will be described later. Also, when decoding a sparse point cloud acquired by Lidar or the like, this processing can be omitted.

The geometric information reconstruction unit 2040 is configured to reconstruct the geometric information (position information in the coordinate system assumed by the decoding process) of each point of the point cloud data to be decoded based on the tree information generated by the tree synthesis unit 2020 and the approximate surface information generated by the approximate surface synthesis unit 2030.

The inverse coordinate transformation unit 2050 is configured to receive the geometric information reconstructed by the geometric information reconstruction unit 2040 as input, transform it from the coordinate system assumed by the decoding process to the coordinate system of the output point cloud signal, and output position information.

The frame buffer 2120 is configured to receive the geometric information reconstructed by the geometric information reconstruction unit 2040 as an input and store it as a reference frame. The stored reference frame is read from the frame buffer 2130 and used as a reference frame when the tree synthesis unit 2020 performs inter-prediction of temporally different frames.

Here, which reference frame to use for each frame may be determined based on, for example, control data transmitted as a bit stream from the point cloud encoding device 100.

The attribute information decoding unit 2060 is configured to receive as input a bit stream relating to attribute information (attribute information bit stream) from the bit streams output from the point cloud encoding device 100, and to decode the syntax.

The decoding process is, for example, a context-adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the attribute information.

The attribute information decoding unit 2060 is also configured to decode the quantized residual information from the decoded syntax.

The inverse quantization unit 2070 is configured to perform an inverse quantization process based on the quantized residual information decoded by the attribute information decoding unit 2060 and the quantization parameter, which is one of the control data decoded by the attribute information decoding unit 2060, to generate inverse quantized residual information.

The dequantized residual information is output to either the RAHT unit 2080 or the LoD calculation unit 2090 depending on the characteristics of the point group to be decoded. The control data decoded by the attribute information decoding unit 2060 specifies which unit the information is output to.

The RAHT unit 2080 is configured to receive the inverse quantized residual information generated by the inverse quantization unit 2070 and the geometric information generated by the geometric information reconstruction unit 2040 as input, and to decode the attribute information of each point using a type of Haar transform (inverse Haar transform in the decoding process) called RAHT (Region Adaptive Hierarchical Transform). As a specific example of the RAHT process, the method described in Non-Patent Document 1 can be used.

The LoD calculation unit 2090 is configured to receive the geometric information generated by the geometric information reconstruction unit 2040 as input and generate the LoD (Level of Detail).

LoD is information for defining a reference relationship (a referencing point and a referenced point) to realize predictive coding, such as predicting attribute information of a certain point from attribute information of another point and encoding or decoding the prediction residual.

In other words, LoD is information that defines a hierarchical structure in which each point contained in the geometric information is classified into multiple levels, and the attributes of points belonging to lower levels are encoded or decoded using attribute information of points belonging to higher levels.

As a specific method for determining LoD, for example, the method described in the above-mentioned non-patent document 1 may be used.

The inverse lifting unit 2100 is configured to decode attribute information of each point based on the hierarchical structure defined by the LoD, using the LoD generated by the LoD calculation unit 2090 and the inverse quantized residual information generated by the inverse quantization unit 2070. As a specific example of the inverse lifting process, the method described in the above-mentioned non-patent document 1 can be used.

The inverse color conversion unit 2110 is configured to perform inverse color conversion processing on the attribute information output from the RAHT unit 2080 or the inverse lifting unit 2100 when the attribute information to be decoded is color information and color conversion has been performed on the point cloud encoding device 100 side. Whether or not such inverse color conversion processing is performed is determined by the control data decoded by the attribute information decoding unit 2060.

The point cloud decoding device 200 is configured to decode and output attribute information for each point in the point cloud through the above processing.

(Geometric information decoding unit 2010)
The control data decoded by the geometric information decoding unit 2010 will be described below with reference to FIGS.

Figure 3 shows an example of the structure of the encoded data (bit stream) received by the geometric information decoding unit 2010.

First, the bit stream may include a GPS2011. A GPS2011 is also called a geometry parameter set, and is a collection of control data related to decoding of geometric information. A specific example will be described later. Each GPS2011 includes at least GPS id information for identifying each GPS2011 when multiple GPS2011 exist.

Secondly, the bit stream may include GSH2012A/2012B. GSH2012A/2012B is also called a geometry slice header or geometry data unit header, and is a collection of control data corresponding to a slice, which will be described later. In the following description, the term "slice" will be used, but slice can also be read as data unit. Specific examples will be described later. GSH2012A/2012B includes at least GPS id information for specifying the GPS2011 corresponding to each GSH2012A/2012B.

Thirdly, the bit stream may include slice data 2013A/2013B following GSH 2012A/2012B. The slice data 2013A/2013B includes data in which geometric information is encoded. An example of the slice data 2013A/2013B is data encoded using an occupancy code or predictive coding, which will be described later.

As described above, the bit stream is structured so that each slice data 2013A/2013B corresponds to one GSH 2012A/2012B and one GPS 2011.

As described above, the GPS ID information is used to specify which GPS 2011 to refer to in GSH 2012A/2012B, so a common GPS 2011 can be used for multiple slice data 2013A/2013B.

In other words, GPS2011 does not necessarily need to be transmitted for each slice. For example, as shown in FIG. 3, the bit stream can be configured so that GPS2011 is not encoded immediately before GSH2012B and slice data 2013B.

Note that the configuration in FIG. 3 is merely an example. As long as GSH 2012A/2012B and GPS 2011 correspond to each slice data 2013A/2013B, elements other than those described above may be added as components of the bit stream.

For example, as shown in FIG. 3, the bitstream may include a sequence parameter set (SPS) 2001. Similarly, when transmitted, the bitstream may be shaped into a configuration different from that shown in FIG. 3. Furthermore, the bitstream may be combined with a bitstream decoded by an attribute information decoding unit 2060 (described later) and transmitted as a single bitstream.

Figure 4 is an example of the syntax configuration of GPS2011.

Note that the syntax names explained below are merely examples. If the syntax functions explained below are similar, the syntax names may be different.

GPS2011 may include GPS ID information (gsps_geom_parameter_set_id) for identifying each GPS2011.

The Descriptor column in Figure 4 indicates how each syntax is coded. ue(v) indicates an unsigned zeroth-order exponential Golomb code, and u(1) indicates a 1-bit flag.

GPS2011 may include a flag (interprediction_enabled_flag) that controls whether or not inter prediction is performed in the tree synthesis unit 2020.

For example, when the value of interprediction_enabled_flag is "0", it may be defined that inter prediction is not performed, and when the value of interprediction_enabled_flag is "1", it may be defined that inter prediction is performed.

Note that interpretation_enabled_flag may be included in SPS2001 instead of GPS2011.

GPS2011 may include a flag (geom_tree_type) for controlling the tree type in the tree synthesis unit 2020. For example, when the value of geom_tree_type is "1", it may be defined that predictive coding is used, and when the value of geom_tree_type is "0", it may be defined that predictive coding is not used.

Note that geom_tree_type may be included in SPS2001 instead of GPS2011.

GPS2011 may include a flag (geom_angular_enabled) to control whether processing is performed in angular mode in the tree synthesis unit 2020.

For example, when the value of geom_angular_enabled is "1", it may be defined that predictive coding is performed in angular mode, and when the value of geom_angular_enabled is "0", it may be defined that predictive coding is not performed in angular mode.

Note that geom_angular_enabled may be included in SPS2001 instead of GPS2011.

GPS2011 may include a flag (global_motion_enabled_flag) that controls whether or not global motion compensation is performed for inter prediction in the tree synthesis unit 2020.

For example, when the value of global_motion_enabled_flag is "0", it may be defined that global motion compensation is not performed, and when the value of global_motion_enabled_flag is "1", it may be defined that global motion compensation is performed.

When global motion compensation is performed, each slice data may include a global motion vector.

Note that global_motion_enabled_flag may be included in SPS2001 instead of GPS2011.

(Tree synthesis unit 2020)
The processing of the tree merging unit 2020 will be described below with reference to Fig. 5 to Fig. 9. Fig. 5 is a flowchart showing an example of the processing in the tree merging unit 2020. Note that the following describes an example in which trees are merged using "Predictive geometry coding".

Note that terms such as "Predictive geometry," "Predictive geometry coding," and "Predictive tree" may be used instead of "Predictive coding."

As shown in FIG. 5, in step S501, the tree synthesis unit 2020 determines whether to use inter prediction based on the value of interprediction_enabled_flag.

If the tree synthesis unit 2020 determines that inter prediction is to be used, the process proceeds to step S502; if it determines that inter prediction is not to be used, the process proceeds to step S505.

In step S502, the tree synthesis unit 2020 obtains a reference frame from the frame buffer 2120. The frame buffer 2120 may store one previously decoded frame, and a decoded frame may be added to the frame buffer 2020 each time the decoding of one or a specified number of frames is completed. After obtaining the reference frame, the tree synthesis unit 2020 proceeds to step S503.

In step S503, the tree synthesis unit 2020 determines whether to perform global motion compensation based on global_motion_enabled_flag.

If the tree synthesis unit 2020 determines that global motion compensation is to be performed, the process proceeds to step S504; if the tree synthesis unit 2020 determines that global motion compensation is not to be performed, the process proceeds to step S505.

In step S504, the tree synthesis unit 2020 performs global motion compensation on the reference frame obtained in step S502.

Here, global motion compensation is a process that corrects the global position shift for each frame, and applies rotation and translation based on the global motion vector decoded by the geometric information decoding unit 2010 to the entire reference frame or to the point group within a specified range.

After performing global motion compensation, the tree synthesis unit 2020 proceeds to step S505.

In step S505, the tree synthesis unit 2020 decodes the slice data. The specific processing of step S505 will be described later. After decoding the slice data, the tree synthesis unit 2020 proceeds to step S506.

In step S506, the tree synthesis unit 2020 ends the process. Note that the processes in steps S503 and S504, i.e., the determination and execution of global motion compensation, may be performed during the slice data decoding process in step S505.

Figure 6 is a flowchart showing an example of the slice data decoding process in step S505.

As shown in FIG. 6, in step S601, the tree synthesis unit 2020 constructs a prediction tree corresponding to the slice data.

The slice data may include a depth-first list of the number of children for each node in the prediction tree. The prediction tree may be constructed by starting with the root node and adding the number of children to each node in depth-first order, as specified in the list above.

After completing construction of the prediction tree, the tree synthesis unit 2020 proceeds to step S602.

In step S602, the tree synthesis unit 2020 determines whether processing of all nodes in the prediction tree has been completed.

If the tree synthesis unit 2020 determines that processing of all nodes of the prediction tree has been completed, it proceeds to step S607, and if it determines that processing of all nodes of the prediction tree has not been completed, it proceeds to step S603.

In step S603, the tree synthesis unit 2020 selects a node to be processed from the prediction tree.

The tree synthesis unit 2020 may select the node next to the previous node to be processed in depth-first order as the node to be processed.

After completing the selection of the node to be processed, the tree synthesis unit 2020 proceeds to step S604.

In step S604, the tree synthesis unit 2020 predicts the coordinates of the point corresponding to the node to be processed. A specific method for predicting such coordinates will be described later.

After completing this prediction, the tree synthesis unit 2020 proceeds to step S605.

In step S605, the tree synthesis unit 2020 decodes the prediction residual of the coordinates of the point corresponding to the node to be processed. The slice data may include the prediction residual of the coordinates of the point corresponding to each node.

After completing decoding of the prediction residual for the node being processed, the tree synthesis unit 2020 proceeds to step S606.

In step S606, the tree synthesis unit 2020 reconstructs the coordinates of the point corresponding to the node to be processed. The tree synthesis unit 2020 may obtain the coordinates of the point by adding the coordinates predicted in step S604 and the residual decoded in step S605.

When the angular mode is used, the tree synthesis unit 2020 may reconstruct the coordinates using the methods described in Non-Patent Documents 1 and 2, taking into account that the prediction residuals and predicted coordinates are values based on a spherical coordinate system.

When the angular mode is used, the tree synthesis unit 2020 may convert the reconstructed coordinates from a spherical coordinate system to a Cartesian coordinate system using the methods described in Non-Patent Documents 1 and 2.

After the tree synthesis unit 2020 completes the coordinate reconstruction, it returns to step 602.

In step S607, the tree synthesis unit 2020 ends the processing of step S505.

Here, the order of steps S604 and S605 may be reversed.

Figure 7 is a flowchart showing an example of the coordinate prediction process in step S604.

As shown in FIG. 7, in step S701, the tree synthesis unit 2020 decodes the predictor flag.

Here, the slice data may include a flag indicating the predictor to be used for each node. For example, the slice data may include flags similar to those described in Non-Patent Documents 1 and 2, such as a flag indicating whether the predictor is an inter predictor or an intra predictor, and an index of the inter predictor. The slice data may also include other flags, which will be described later.

After decoding the predictor flag, the tree synthesis unit 2020 proceeds to step S702.

In step S702, the tree synthesis unit 2020 determines whether to use an inter-predictor based on the predictor flag decoded in step S701.

If the tree synthesis unit 2020 determines that an inter-predictor is to be used, the process proceeds to step S704; if the tree synthesis unit 2020 determines that an inter-predictor is not to be used, the process proceeds to step S703.

In step S703, the tree synthesis unit 2020 performs intra prediction on the coordinates of the node to be processed.

When performing intra prediction, the tree synthesis unit 2020 constructs a predictor based on the coordinates of the parent node or ancestor node (e.g., the parent node of the parent node) of the node to be processed, and predicts the coordinates of the node to be processed.

The tree synthesis unit 2020 may use the methods described in Non-Patent Documents 1 and 2 to configure an intra predictor, and may use the predictor indicated by the predictor flag decoded in step S701 from among multiple intra predictors.

After intra prediction is completed, the tree synthesis unit 2020 proceeds to step S705.

In step S704, the tree synthesis unit 2020 performs inter prediction on the coordinates of the node to be processed.

When performing inter prediction, the tree synthesis unit 2020 selects a node corresponding to the node to be processed from the reference frame as a predictor, and sets the coordinates of the selected predictor as the predicted value of the coordinates of the node to be processed. Here, the method of selecting a predictor from the reference frame will be described later.

After inter prediction is completed, the tree synthesis unit 2020 proceeds to step S705.

In step S705, the tree synthesis unit 2020 ends the processing of step S605.

Figures 8 and 9 show an example of the process of selecting a predictor from a reference frame in step S704 of the tree synthesis unit 2020.

However, it is assumed that angular mode is used. In angular mode, the point of the parent node of the node being processed can be considered to have been decoded immediately before or earlier.

In the example of FIG. 8, the tree synthesis unit 2020 searches the reference frame for nodes that have the same laser ID (L) and a large azimuth angle (φ) as the parent node of the node to be processed, and selects the two with the smallest azimuth angles (φ) as predictor 1 and predictor 2, respectively.

In the example of Figure 9, the tree synthesis unit 2020 searches the reference frame for nodes that have the same laser ID (L) and an azimuth angle (φ) greater than the angle obtained by subtracting the offset value (φ _offset ) from the parent node of the node to be processed, and selects the two with the smallest azimuth angles (φ) as predictor 1 and predictor 2, respectively.

The tree synthesis unit 2020 may take any value from negative to positive as this offset value.

Based on the flag, the tree synthesis unit 2020 may determine which predictor to use from among predictor 1 and predictor 2 shown in FIG. 8 and predictor 1 and predictor 2 shown in FIG. 9. The slice data may include such a flag for each node. Furthermore, such a flag may be decoded in step S701.

As described above, the tree synthesis unit 2020 selects a predictor based on the laser ID, azimuth angle, and azimuth angle offset value of the parent node of the node to be processed, making it possible to robustly select an appropriate predictor even when there is a misalignment or difference in the frame to be processed or the reference frame.

In addition, such offset values may be included in the slice data as values for each node or for multiple nodes.

Furthermore, such offset values may be included in the header (GSH, SPS, GPS or other header) as per laser ID, per slice, per frame or per sequence values.

Such offset values may be derived from predefined offset value candidates.

For example, such offset value candidates may be defined as values obtained by multiplying all or some of the integers from α1 to α2 by a fixed value s.

Here, α1 and α2 are arbitrary integers set in advance, and α1<=α2. The fixed value s may be set based on the LiDAR laser speed, etc.

As some of the integers from α1 to α2, for example, if α1 is -8 and α2 is 8, then integers at regular intervals such as {-8, -4, 0, 4, 8} or integers at exponential intervals such as {-8, -4, -2, -1, 0, 1, 2, 4, 8} are possible.

The tree synthesis unit 2020 may set the above-mentioned offset value candidate as a first candidate for the offset value, and derive the offset value from a second candidate that is obtained by narrowing down the first candidate based on a predetermined condition.

The predetermined condition may be based on the offset value of the node that was processed immediately before.

For example, the specified condition may be that, if the offset value of the most recently processed node is i x s, then the first candidate is greater than or equal to (i + β1) x s and less than or equal to (i + β2) x s. Here, β1 and β2 are arbitrary integers set in advance, and β1 <= β2.

The tree synthesis unit 2020 may set a value that satisfies such a condition as the second candidate. For example, if i+β1>α1 or i+β2<α2, the tree synthesis unit 2020 can narrow down the second candidate from the first candidate.

The predetermined condition may also be, for example, a condition based on the magnitude of the global motion vector.

For example, the predetermined condition may be that if the rotation amount or translation amount of the global motion vector is equal to or less than a threshold, then the first candidate is equal to or greater than γ1 and equal to or less than γ2. However, γ1 and γ2 are arbitrary integers that are set in advance, and γ1<=γ2.

The tree synthesis unit 2020 may set a value that satisfies such a condition as the second candidate. For example, if γ1>α1 or γ2<α2, the tree synthesis unit 2020 can narrow down the second candidate from the first candidate.

The global motion vector may be included in the slice data or in the header (GSH, SPS, GPS or other header).

A flag may be included in the slice data or header (GSH, SPS, GPS or other header) indicating whether the rotation or translation of the global motion vector is less than or equal to a threshold.

In either case, the data may have been decrypted prior to such processing (e.g., step S502 or step S504).

If the number of second candidates is narrowed down to one, the tree synthesis unit 2020 may skip decoding the data indicating the offset. In addition, the tree synthesis unit 2020 may derive the offset value from the one narrowed down candidate.

For example, in an example where the specified condition is a condition based on the offset value of the most recently processed node, if β1 and β2 are both 0, the tree synthesis unit 2020 determines that the first candidate is {-4×s, -3×s, -2×s, -1×s, 0, 1×s, 2×s, 3×s, 4×s}, and if the offset value of the most recently processed node is -1×s, the second candidate is the only one, {-1×s}, and the offset value may be set to -1×s.

For example, in an example where the specified condition is based on the magnitude of the global motion vector, when γ1 and γ2 are both 0, the tree synthesis unit 2020 determines that the first candidate is {-4×s, -3×s, -2×s, -1×s, 0×s, 1×s, 2×s, 3×s, 4×s}, and if the magnitude of the global motion vector of the frame to be processed is equal to or less than a threshold, the second candidate is the only one, {0×s}, and the offset value may be set to 0×s.

By narrowing down the candidates for the offset value or skipping the decoding, the tree synthesis unit 2020 can reduce the amount of coding data indicating the offset value and efficiently set the offset during coding, thereby shortening the coding time.

As described above, in predictive coding, when an inter-predictor is selected based on the laser ID of the parent node of the node to be processed and the azimuth obtained by subtracting an offset value from the azimuth, the tree synthesis unit 2020 is configured to select a second candidate from among first candidates for predefined offset values using a predetermined method, and to identify an offset value from among the second candidates.

The tree synthesis unit 2020 may also be configured to select the second candidate based on the offset value of the node processed immediately before, as the above-mentioned predetermined method.

The tree synthesis unit 2020 may also be configured to select the second candidate based on the magnitude of the global motion vector of the frame to be processed, as the above-mentioned predetermined method.

Furthermore, the tree synthesis unit 2020 may be configured to skip decoding the data indicating the offset value when the number of second candidates is narrowed down to one.

(Point cloud encoding device 100)
Hereinafter, the point cloud encoding device 100 according to this embodiment will be described with reference to Fig. 10. Fig. 10 is a diagram showing an example of functional blocks of the point cloud encoding device 100 according to this embodiment.

As shown in FIG. 10, the point cloud encoding device 100 includes a coordinate transformation unit 1010, a geometric information quantization unit 1020, a tree analysis unit 1030, an approximate surface analysis unit 1040, a geometric information encoding unit 1050, a geometric information reconstruction unit 1060, a color conversion unit 1070, an attribute transfer unit 1080, a RAHT unit 1090, an LoD calculation unit 1100, a lifting unit 1110, an attribute information quantization unit 1120, an attribute information encoding unit 1130, and a frame buffer 1140.

The coordinate conversion unit 1010 is configured to perform a conversion process from the three-dimensional coordinate system of the input point cloud to any different coordinate system. The coordinate conversion may, for example, convert the x, y, and z coordinates of the input point cloud into any s, t, and u coordinates by rotating the input point cloud. In addition, as one variation of the conversion, the coordinate system of the input point cloud may be used as is.

The geometric information quantization unit 1020 is configured to quantize the position information of the input point group after coordinate transformation and remove points with overlapping coordinates. Note that when the quantization step size is 1, the position information of the input point group and the position information after quantization match. In other words, when the quantization step size is 1, it is equivalent to not performing quantization.

The tree analysis unit 1030 is configured to receive position information of the quantized point group as input, and generate an occurrence code that indicates at which node in the encoding target space the point exists, based on the tree structure described below.

In this process, the tree analysis unit 1030 is configured to generate a tree structure by recursively dividing the encoding target space into rectangular parallelepipeds.

If a point exists within a certain rectangular parallelepiped, a tree structure can be generated by recursively dividing the rectangular parallelepiped into multiple rectangular parallelepipeds until the rectangular parallelepiped reaches a specified size. Each such rectangular parallelepiped is called a node. Each rectangular parallelepiped generated by dividing a node is called a child node, and the occurrence code is expressed as 0 or 1 to indicate whether or not a point is contained within a child node.

As described above, the tree analysis unit 1030 is configured to generate an occupancy code while recursively splitting the node until it reaches a predetermined size.

In this embodiment, a method called "Octree" can be used, which recursively performs octree division on the above-mentioned rectangular parallelepiped, always treating it as a cube, and a method called "QtBt" can be used, which performs quadtree division and binary tree division in addition to octree division.

Here, whether or not to use "QtBt" is transmitted to the point cloud decoding device 200 as control data.

Alternatively, predictive coding using an arbitrary tree structure may be specified. In such a case, the tree analysis unit 1030 determines the tree structure, and the determined tree structure is transmitted to the point cloud decoding device 200 as control data.

For example, the tree-structured control data may be configured so that it can be decoded using the procedures described in Figures 5 to 9.

The approximate surface analysis unit 1040 is configured to generate approximate surface information using the tree information generated by the tree analysis unit 1030.

When decoding three-dimensional point cloud data of an object, for example, if the points are densely distributed on the object's surface, approximate surface information is used to represent the area in which the points exist by approximating the area using a small plane, rather than decoding each point individually.

Specifically, the approximate surface analysis unit 1040 may be configured to generate approximate surface information using, for example, a method called "Trisoup." In addition, when decoding a sparse point cloud acquired by Lidar or the like, this process can be omitted.

The geometric information encoding unit 1050 is configured to generate a bit stream (geometric information bit stream) by encoding syntax such as the occupancy code generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040. Here, the bit stream may include, for example, the syntax described in FIG. 4.

The encoding process is, for example, a context-adaptive binary arithmetic encoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the position information.

The geometric information reconstruction unit 1060 is configured to reconstruct the geometric information of each point of the point cloud data to be encoded (the coordinate system assumed by the encoding process, i.e., the position information after the coordinate transformation in the coordinate transformation unit 1010) based on the tree information generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040.

The frame buffer 1140 is configured to receive the geometric information reconstructed by the geometric information reconstruction unit 1060 and store it as a reference frame.

The stored reference frame is read from the frame buffer 1140 and used as a reference frame when inter prediction is performed in the tree analysis unit 1030.

The color conversion unit 1070 is configured to perform color conversion when the input attribute information is color information. Color conversion does not necessarily have to be performed, and whether or not the color conversion process is performed is coded as part of the control data and transmitted to the point cloud decoding device 200.

The attribute transfer unit 1080 is configured to correct the attribute values so as to minimize distortion of the attribute information, based on the position information of the input point cloud, the position information of the point cloud after reconstruction in the geometric information reconstruction unit 1060, and the attribute information after color change in the color conversion unit 1070. As a specific correction method, for example, the method described in Non-Patent Document 2 can be applied.

The RAHT unit 1090 is configured to receive the attribute information transferred by the attribute transfer unit 1080 and the geometric information generated by the geometric information reconstruction unit 1060 as input, and to generate residual information for each point using a type of Haar transform called RAHT (Region Adaptive Hierarchical Transform). As a specific example of the RAHT process, the method described in the above-mentioned non-patent document 2 can be used.

The LoD calculation unit 1100 is configured to receive the geometric information generated by the geometric information reconstruction unit 1060 as input and generate the LoD (Level of Detail).

LoD is information for defining a reference relationship (a referencing point and a referenced point) to realize predictive coding, such as predicting attribute information of a certain point from attribute information of another point and encoding or decoding the prediction residual.

In other words, LoD is information that defines a hierarchical structure in which each point contained in the geometric information is classified into multiple levels, and the attributes of points belonging to lower levels are encoded or decoded using attribute information of points belonging to higher levels.

As a specific method for determining LoD, for example, the method described in the above-mentioned non-patent document 2 may be used.

The lifting unit 1110 is configured to generate residual information by a lifting process using the LoD generated by the LoD calculation unit 1100 and the attribute information after attribute transfer in the attribute transfer unit 1080.

Specific lifting processing may be performed, for example, using the method described in the above-mentioned non-patent document 2.

The attribute information quantization unit 1120 is configured to quantize the residual information output from the RAHT unit 1090 or the lifting unit 1110. Here, a quantization step size of 1 is equivalent to no quantization being performed.

The attribute information encoding unit 1130 is configured to perform encoding processing using the quantized residual information, etc. output from the attribute information quantization unit 1120 as syntax, and generate a bit stream related to the attribute information (attribute information bit stream).

The encoding process is, for example, a context-adaptive binary arithmetic encoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the attribute information.

The point cloud encoding device 100 is configured to perform encoding processing using the position information and attribute information of each point in the point cloud as input, and to output a geometric information bit stream and an attribute information bit stream through the above processing.

Furthermore, the above-mentioned point cloud encoding device 100 and point cloud decoding device 200 may be realized as a program that causes a computer to execute each function (each process).

In each of the above embodiments, the present invention has been described using the point cloud encoding device 100 and the point cloud decoding device 200 as examples, but the present invention is not limited to such examples and can be similarly applied to a point cloud encoding/decoding system having the functions of the point cloud encoding device 100 and the point cloud decoding device 200.

In addition, according to this embodiment, for example, it is possible to realize an improvement in the overall service quality in video communication, which makes it possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, which is to "build resilient infrastructure, promote sustainable industrialization and foster innovation."

10... Point cloud processing system 100... Point cloud encoding device 1010... Coordinate conversion unit 1020... Geometric information quantization unit 1030... Tree analysis unit 1040... Approximate surface analysis unit 1050... Geometric information encoding unit 1060... Geometric information reconstruction unit 1070... Color conversion unit 1080... Attribute transfer unit 1090... RAHT unit 1100... LoD calculation unit 1110... Lifting unit 1120... Attribute information quantization unit 1130... Attribute Information encoding unit 1140...frame buffer 200...point cloud decoding device 2010...geometric information decoding unit 2020...tree synthesis unit 2030...approximate surface synthesis unit 2040...geometric information reconstruction unit 2050...inverse coordinate transformation unit 2060...attribute information decoding unit 2070...inverse quantization unit 2080...RAHT unit 2090...LoD calculation unit 2100...inverse lifting unit 2110...inverse color transformation unit 2120...frame buffer

Claims (6)

A point cloud decoding device, comprising:
A point cloud decoding device comprising: a tree synthesis unit that, in predictive coding, when selecting an inter-predictor based on the laser ID of a parent node of a node to be processed and an azimuth angle obtained by subtracting an offset value from the azimuth angle, selects a second candidate from among first candidates of predefined offset values using a predetermined method, and identifies the offset value from among the second candidates. The point cloud decoding device according to claim 1, characterized in that the tree synthesis unit selects the second candidate based on the offset value of the node processed immediately before as the predetermined method. The point cloud decoding device according to claim 1, characterized in that the tree synthesis unit selects the second candidate based on the magnitude of the global motion vector of the frame to be processed as the predetermined method. The point cloud decoding device according to any one of claims 1 to 3, characterized in that the tree synthesis unit skips decoding of the data indicating the offset value when the second candidates are narrowed down to one. 1. A point cloud decoding method, comprising:
A point cloud decoding method comprising the steps of: in predictive coding, when selecting an inter-predictor based on the laser ID of a parent node of a node to be processed and an azimuth angle obtained by subtracting an offset value from the azimuth angle, selecting a second candidate from among first candidates of offset values defined in advance by a predetermined method, and specifying the offset value from among the second candidates. A program for causing a computer to function as a point group decoding device,
The point group decoding device comprises:
a tree synthesis unit that, in predictive coding, when selecting an inter predictor based on the laser ID of a parent node of a node to be processed and an azimuth angle obtained by subtracting an offset value from the azimuth angle, selects a second candidate from among first candidates of predefined offset values using a predetermined method, and identifies the offset value from among the second candidates.

Images