WO2025009868A1 - Mesh data transmission device, mesh data transmission method, mesh data reception device, and mesh data reception method - Google Patents

Abstract

A mesh data decoding method according to embodiments may comprise the steps of: receiving a file including mesh data; decapsulating the file; and decoding the mesh data, wherein the file includes a bitstream including the mesh data and parameter information about the mesh data. A mesh data encoding method according to embodiments may comprise the steps of: encoding mesh data; encapsulating the mesh data into a file; and transmitting the file, wherein a bitstream including the mesh data and parameter information about the mesh data is encapsulated into a single track of the file.

Description

Mesh data transmission device, mesh data transmission method, mesh data reception device and mesh data reception method

The embodiments provide a method for providing Point Cloud content to provide various services to users, such as Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), and autonomous driving services.

A point cloud is a collection of points in 3D space. There is a problem in that it is difficult to create point cloud data because there are a large number of points in 3D space.

There is a problem that a lot of processing is required to transmit and receive point cloud data.

The technical problem according to the embodiments is to provide a point cloud data transmission device, a transmission method, a point cloud data reception device, and a reception method for efficiently transmitting and receiving point clouds in order to solve the problems described above.

Technical problems according to embodiments are directed to providing a point cloud data transmission device, a transmission method, a point cloud data reception device, and a reception method for resolving latency and encoding/decoding complexity.

However, the scope of the embodiments is not limited to the technical tasks described above, and the scope of the embodiments may be expanded to other technical tasks that can be inferred by a person skilled in the art based on the entire contents of this document.

In order to achieve the above-described object and other advantages, a mesh data decoding method according to embodiments includes the steps of: receiving a file including mesh data; decapsulating the file; and decoding the mesh data; wherein the file may include a bitstream including the mesh data and parameter information about the mesh data. A mesh data encoding method according to embodiments includes the steps of: encoding the mesh data; encapsulating the mesh data into a file; and transmitting the file; wherein the bitstream including the mesh data and the parameter information about the mesh data may be encapsulated into a single track of the file.

The point cloud data transmission method, transmission device, point cloud data reception method, and reception device according to the embodiments can provide a quality point cloud service.

The point cloud data transmission method, transmission device, point cloud data reception method, and reception device according to the embodiments can achieve various video codec methods.

The point cloud data transmission method, transmission device, point cloud data reception method, and reception device according to the embodiments can provide general-purpose point cloud content such as autonomous driving services.

The drawings are included to further understand the embodiments, and the drawings illustrate the embodiments together with the description related to the embodiments. For a better understanding of the various embodiments described below, reference should be made to the following description of the embodiments in conjunction with the following drawings, in which like reference numerals correspond to corresponding parts throughout the drawings.

Figure 1 illustrates a system for providing dynamic mesh content according to embodiments.

Figure 2 illustrates a V-MESH compression method according to embodiments.

Figure 3 illustrates pre-processing of V-MESH compression according to embodiments.

Figure 4 illustrates a mid-edge subdivision method according to embodiments.

Figure 5 shows a displacement generation process according to embodiments.

Figure 6 illustrates an intra-frame encoding process of a V-MESH compression method according to embodiments.

Figure 7 illustrates an inter-frame encoding process of a V-MESH compression method according to embodiments.

Figure 8 shows a lifting conversion process for displacement according to embodiments.

Figure 9 shows a process of packing transformation coefficients according to embodiments into a 2D image.

Figure 10 illustrates an attribute transfer process of a V-MESH compression method according to embodiments.

Figure 11 illustrates an intra frame decoding process of a V-MESH compression method according to embodiments.

Fig. 12 shows a V-MES and Fig. 13 shows a point cloud data transmission device according to embodiments.

Fig. 13 shows a point cloud data transmission device according to embodiments.

Fig. 14 shows a point cloud data receiving device according to embodiments.

Figure 15 shows a mesh system structure according to embodiments.

Figure 16 illustrates a video-based dynamic mesh coding bitstream structure according to embodiments.

Fig. 17 shows a mesh data receiving device according to embodiments.

Figure 18 shows a mesh data encoding method according to embodiments.

Figure 19 shows a mesh data decryption method according to embodiments.

The following detailed description of preferred embodiments of the embodiments is specifically described, examples of which are illustrated in the accompanying drawings. The following detailed description with reference to the accompanying drawings is intended to explain preferred embodiments of the embodiments rather than to illustrate only embodiments that can be implemented according to the embodiments of the embodiments. The following detailed description includes details to provide a thorough understanding of the embodiments. However, it will be apparent to one skilled in the art that the embodiments may be practiced without these details.

Most of the terms used in the examples are selected from those commonly used in the field, but some terms are arbitrarily selected by the applicant and their meanings are described in detail in the following description as needed. Therefore, the examples should be understood based on the intended meaning of the terms and not on the simple names or meanings of the terms.

Figure 1 illustrates a system for providing dynamic mesh content according to embodiments.

The system of FIG. 1 includes a point cloud data transmission device (100) and a point cloud data reception device (110) according to embodiments. The point cloud data transmission device may include a dynamic mesh video acquisition unit (101), a dynamic mesh video encoder (102), a file/segment encapsulator (103), and a transmitter (104). The point cloud data reception device (110) may include a reception unit (111), a file/segment decapsulator (112), a dynamic mesh video decoder (113), and a renderer (114). Each component of FIG. 1 may correspond to hardware, software, a processor, and/or a combination thereof. Hereinafter, the point cloud data transmission device according to embodiments may be interpreted as a term referring to the transmission device (100) or a dynamic mesh video encoder (hereinafter, referred to as an encoder) (102). The point cloud data receiving device according to the embodiments may be interpreted as a term referring to a receiving device (110) or a dynamic mesh video decoder (hereinafter, decoder) (113).

The system of Fig. 1 can perform video-based dynamic mesh compression and decompression.

With the advancement of 3D capture, modeling, and rendering, users can use various forms of 3D content, such as AR, XR, metaverse, and holograms, on various platforms and devices. 3D contents express objects more precisely and realistically so that users can enjoy immersive experiences, and for this purpose, a large amount of data is required to create and use 3D models. Among various types of 3D content, 3D mesh is widely used for efficient data utilization and realistic object expression. Embodiments include a series of processing processes in a system that uses such mesh content.

First, the method of compressing dynamic mesh data starts from the V-PCC (Video-based point cloud compression) standard technology. Point cloud data is data that has color information in the vertex coordinates (X, Y, Z). Mesh data refers to the vertex information to which connectivity information between vertices is added. When creating content, it can be created in the form of mesh data from the beginning. Connectivity information can be added to point cloud data and converted into mesh data for use.

Currently, the MPEG standards body defines the data types of dynamic mesh data as the following two types. Category 1: Mesh data with texture maps as color information. Category 2: Mesh data with vertex colors as color information.

Mesh coding standards for Category 1 data are in progress, and work on Category 2 data standards will also be carried out in the future. The entire process for providing a Mesh content service may include an acquisition process, an encoding process, a transmission process, a decoding process, a rendering process, and/or a feedback process, as shown in Fig. 1.

In order to provide a mesh content service, 3D data acquired through multiple cameras or special cameras can be processed into a mesh data type through a series of processes and then generated as a video. The generated mesh video is transmitted through a series of processes, and the receiving end can process the received data again into a mesh video and render it. Through this, a mesh video is provided to the user, and the user can use the mesh content according to his/her intention through interaction.

A mesh compression system may include a transmitting device and a receiving device. The transmitting device may encode mesh video to output a bitstream, which may be transmitted to a receiving device via a digital storage medium or a network in the form of a file or streaming (streaming segment). The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.

The transmitting device may roughly include a mesh video acquisition unit, a mesh video encoder, and a transmitting unit. The receiving device may roughly include a receiving unit, a mesh video decoder, and a renderer. The encoder may be called a mesh video/video/picture/frame encoding device, and the decoder may be called a mesh video/video/picture/frame decoding device. The transmitter may be included in the mesh video encoder. The receiver may be included in the mesh video decoder. The renderer may include a display unit, and the renderer and/or the display unit may be configured as separate devices or external components. The transmitting device and the receiving device may further include separate internal or external modules/units/components for a feedback process.

Mesh data represents the surface of an object as a number of polygons. Each polygon is defined by vertices in 3D space and connection information that describes how the vertices are connected. It can also include vertex properties such as vertex color, normal, etc. Mapping information that allows the surface of the mesh to be mapped to a 2D planar area can also be included as a mesh property. The mapping can be described as a set of parameter coordinates, commonly called UV coordinates or texture coordinates, associated with the mesh vertices. Meshes contain 2D attribute maps, which can be used to store high-resolution attribute information such as textures, normals, displacement, etc.

The mesh video acquisition unit may include processing 3D object data acquired through a camera, etc. into a mesh data type having the properties described above through a series of processes and generating a video composed of such mesh data. The mesh video may have properties of the mesh, such as vertices, polygons, connection information between vertices, colors, normals, etc., that may change over time. A mesh video having properties and connection information that change over time in this way may be expressed as a dynamic mesh video.

A mesh video encoder can encode an input mesh video into one or more video streams. A single video can include multiple frames, and a single frame can correspond to a still image/picture. In this document, a mesh video can include a mesh image/frame/picture, and the mesh video can be used interchangeably with the mesh image/frame/picture. The mesh video encoder can perform a Video-based Dynamic Mesh (V-Mesh) Compression procedure. The mesh video encoder can perform a series of procedures such as prediction, transformation, quantization, and entropy coding for compression and coding efficiency. The encoded data (encoded video/image information) can be output in the form of a bitstream.

An encapsulation processing unit (file/segment encapsulation module) can encapsulate encoded mesh video data and/or mesh video related metadata in the form of a file, etc. Here, the mesh video related metadata may be received from a metadata processing unit, etc. The metadata processing unit may be included in the mesh video encoder, or may be configured as a separate component/module. The encapsulation processing unit may encapsulate the corresponding data in a file format such as ISOBMFF, or process it in the form of other DASH segments, etc. The encapsulation processing unit may include mesh video related metadata in the file format according to an embodiment. The mesh video metadata may be included in boxes at various levels in the ISOBMFF file format, for example, or may be included as data in a separate track within a file. According to an embodiment, the encapsulation processing unit may encapsulate mesh video related metadata itself in a file.

The transmission processing unit can process encapsulated mesh video data for transmission according to a file format. The transmission processing unit can be included in the transmission unit, or can be configured as a separate component/module. The transmission processing unit can process mesh video data according to any transmission protocol. The processing for transmission can include processing for transmission through a broadcast network and processing for transmission through broadband. According to an embodiment, the transmission processing unit can receive not only mesh video data, but also mesh video-related metadata from the metadata processing unit, and process this for transmission.

The transmission unit can transmit encoded video/image information or data output in the form of a bitstream to the reception unit of the receiving device through a digital storage medium or network in the form of a file or streaming. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit can include an element for generating a media file through a predetermined file format and can include an element for transmission through a broadcasting/communication network. The reception unit can extract the bitstream and transmit it to a decoding device.

The receiver can receive mesh video data transmitted by the mesh video transmission device. Depending on the channel through which it is transmitted, the receiver can receive mesh video data through a broadcast network, through a broadband, or through a digital storage medium.

The receiving processing unit can perform processing according to a transmission protocol on the received mesh video data. The receiving processing unit can be included in the receiving unit, or can be configured as a separate component/module. In order to correspond to the processing performed for transmission on the transmitting side, the receiving processing unit can perform the reverse process of the aforementioned transmitting processing unit. The receiving processing unit can transfer the acquired mesh video data to the decapsulation processing unit, and transfer the acquired mesh video-related metadata to the metadata parser. The mesh video-related metadata acquired by the receiving processing unit can be in the form of a signaling table.

A decapsulation processing unit (file/segment decapsulation module) can decapsulate mesh video data in a file format received from a receiving processing unit. The decapsulation processing unit can decapsulate files according to ISOBMFF, etc., to obtain a mesh video bitstream or mesh video related metadata (metadata bitstream). The obtained mesh video bitstream can be transmitted to a mesh video decoder, and the obtained mesh video related metadata (metadata bitstream) can be transmitted to a metadata processing unit. The mesh video bitstream may include metadata (metadata bitstream). The metadata processing unit may be included in the mesh video decoder, or may be configured as a separate component/module. The mesh video related metadata obtained by the decapsulation processing unit may be in the form of a box or track within a file format. If necessary, the decapsulation processing unit may receive metadata required for decapsulation from the metadata processing unit. Mesh video related metadata can be passed to a Mesh video decoder for use in the Mesh video decoding process, or can be passed to a renderer for use in the Mesh video rendering process.

The Mesh Video Decoder can receive a bitstream as input and perform operations corresponding to the operations of the Mesh Video Encoder to decode the video/image. The decoded Mesh Video can be displayed through the display unit. The user can view all or part of the rendered result through a VR/AR display or a general display.

The feedback process may include a process of transmitting various feedback information that may be acquired during the rendering/display process to the transmitter or to the decoder of the receiver. Interactivity may be provided in mesh video consumption through the feedback process. According to an embodiment, head orientation information, viewport information indicating an area that the user is currently viewing, etc. may be transmitted during the feedback process. According to an embodiment, the user may interact with things implemented in the VR/AR/MR/autonomous driving environment, in which case information related to the interaction may be transmitted to the transmitter or the service provider during the feedback process. Depending on the embodiment, the feedback process may not be performed.

Head orientation information can mean information about the user's head position, angle, movement, etc. Based on this information, information about the area the user is currently viewing within the mesh video, i.e. viewport information, can be calculated.

Viewport information can be information about the area that the current user is viewing in the Mesh video. This can be used to perform gaze analysis to determine how the user consumes the Mesh video, which area of the Mesh video they gaze at and for how long. The gaze analysis can be performed on the receiving side and transmitted to the transmitting side through a feedback channel. Devices such as VR/AR/MR displays can extract the viewport area based on the user's head position/orientation, the vertical or horizontal FOV supported by the device, etc.

Depending on the embodiment, the aforementioned feedback information may be consumed by the receiver as well as transmitted to the transmitter. That is, the decoding and rendering processes of the receiver may be performed using the aforementioned feedback information. For example, only the mesh video for the area currently being viewed by the user may be preferentially decoded and rendered using head orientation information and/or viewport information.

This document relates to dynamic mesh video compression as described above. The method/embodiment disclosed in this document can be applied to the Video-based Dynamic Mesh compression method (V-Mesh) standard of MPEG (Moving Picture Experts Group) or the next-generation video/image coding standard. Dynamic mesh video compression is a method for processing mesh connection information and properties that change over time, and it can perform lossy and lossless compression for various applications such as real-time communication, storage, free-viewpoint video, and AR/VR.

The dynamic mesh video compression method described below is based on MPEG's V-Mesh method.

In this document, picture/frame can generally mean a unit representing one video image of a specific time period.

A pixel or pel can mean the smallest unit that constitutes a picture (or image). In addition, a 'sample' can be used as a term corresponding to a pixel. A sample can generally represent a pixel or a pixel value, and can represent only a pixel/pixel value of a luma component, only a pixel/pixel value of a chroma component, or only a pixel/pixel value of a depth component.

A unit may represent a basic unit of image processing. A unit may include at least one of a specific region of a picture and information related to the region. A unit may be used interchangeably with terms such as block or area, depending on the case. In general, an MxN block may include a set (or array) of samples (or sample array) or transform coefficients consisting of M columns and N rows.

The encoding process of Fig. 1 is as follows.

Video-based dynamic mesh compression (V-Mesh) compression method can provide a method of compressing dynamic mesh video data based on 2D video codecs such as HEVC and VVC. In the V-Mesh compression process, the following data is received as input and compression is performed.

Input mesh: Contains the 3D coordinates (geometry) of the vertices that make up the mesh, normal information for each vertex, mapping information that maps the mesh surface to a 2D plane, connection information between the vertices that make up the surface, etc. The mesh surface can be expressed as triangles or more polygons, and connection information between the vertices that make up each surface is stored according to a set shape. The input mesh can be saved in the OBJ file format.

Attribute map: (Hereinafter, texture map is also used with the same meaning): Contains information on the properties (color, normal, displacement, etc.) of the mesh, and stores data in the form of mapping the surface of the mesh onto a 2D image. Mapping which part (surface or vertex) of the mesh each data of this attribute map corresponds to is based on the mapping information included in the input mesh. Since the attribute map has data for each frame of the mesh video, it can also be expressed as an attribute map video (or attribute for short). The attribute map in the V-Mesh compression method mainly contains the color information of the mesh, and is stored in an image file format (PNG, BMP, etc.).

Material Library File: Contains information about the material properties used in the mesh, and in particular, information that links the input mesh to its corresponding attribute map. It is saved in the Wavefront Material Template Library (MTL) file format.

In the V-Mesh compression method, the following data and information can be generated through the compression process.

Base mesh: The input mesh is simplified (decimated) through a preprocessing process to express the objects of the input mesh using the minimum number of vertices determined by the user's criteria.

Displacement: This is displacement information used to express the input mesh as similarly as possible to the base mesh, and is expressed in the form of three-dimensional coordinates.

Atlas information: This is metadata required to reconstruct the mesh using the base mesh, displacement, and attribute map information. This can be created and utilized as a sub-unit (sub-mesh, patch, etc.) that constitutes the mesh.

Referring to FIGS. 2 to 7, a method for encoding mesh location information (vertex) is described, and referring to FIGS. 6-10, etc., a method for encoding attribute information (attribute map) by restoring mesh location information is described.

Figure 2 illustrates a V-MESH compression method according to embodiments.

Fig. 2 shows the encoding process of Fig. 1, and the encoding process may include a pre-processing and an encoding process. The encoder of Fig. 1 may include a pre-processor (200) and an encoder (201) as in Fig. 2. The transmitting device of Fig. 1 may be broadly referred to as an encoder, and the dynamic mesh video encoder of Fig. 1 may be referred to as an encoder. The V-Mesh compression method may include a pre-processing (200) and an encoding (201) process as in Fig. 2. The pre-processor of Fig. 2 may be located in front of the encoder of Fig. 2. The pre-processor and the encoder of Fig. 2 may be referred to as one encoder.

The pre-processor can receive a static dynamic mesh and/or an attribute map. The pre-processor can generate a base mesh and/or a displacement through preprocessing. The pre-processor can receive feedback information from an encoder and generate the base mesh and/or the displacement based on the feedback information.

An encoder can receive a base mesh, a displacement mesh, a static dynamic mesh, and/or an attribute map. The encoder can encode mesh-related data to generate a compressed bitstream.

Figure 3 illustrates pre-processing of V-MESH compression according to embodiments.

Figure 3 shows the configuration and operation of the pre-processor of Figure 2.

Fig. 3 shows a process of performing preprocessing on an input mesh. The preprocessing process (200) can largely include four steps: 1) GoF (Group of Frame) generation, 2) Mesh Decimation, 3) UV parameterization, and 4) Fitting subdivision surface (300). The preprocessor (200) can receive an input mesh, generate a displacement and/or base mesh, and transmit the generated mesh to the encoder (201). The preprocessor (200) can transmit GoF information related to GoF generation to the encoder (201).

Below, each step of Fig. 3 is described.

GoF Generation: This is the process of generating a reference structure for mesh data. If the number of vertices, the number of texture coordinates, the vertex connection information, and the texture coordinate connection information of the mesh of the previous frame and the current mesh are all the same, the previous frame can be set as the reference frame. In other words, if only the vertex coordinate values are different between the current input mesh and the reference input mesh, inter frame encoding can be performed. Otherwise, the frame performs intra frame encoding.

Mesh Decimation: This is the process of simplifying the input mesh to create a simplified mesh, or base mesh. After selecting vertices to be removed from the original mesh based on criteria defined by the user, the selected vertices and triangles connected to the selected vertices can be removed.

In the process of performing mesh simplification (Mesh decimation), the input mesh (voxelized), target triangle ratio (TTR), and minimum triangle component (CCCount) information are passed as input, and the simplified mesh (decimated mesh) can be obtained as output. In this process, connected triangle components smaller than the set minimum triangle component (CCCount) can be removed.

UV parameterization: This is the process of mapping a 3D surface to a texture domain for a decimated mesh. Parameterization can be performed using the UVAtlas tool. Through this process, mapping information is generated that shows where each vertex of the decimated mesh can be mapped to on a 2D image. The mapping information is expressed and saved as texture coordinates, and the final base mesh is generated through this process.

Fitting subdivision surface: This is the process of performing subdivision on a simplified mesh. A user-defined method, such as the mid-edge method, can be applied as the subdivision method. The fitting process is performed so that the input mesh and the mesh on which subdivision is performed are similar to each other.

Figure 4 illustrates a mid-edge subdivision method according to embodiments.

Figure 4 shows the mid-edge method of the fitting subdivision surface described in Figure 3. Referring to Figure 4, an original mesh including four vertices is subdivided to create a sub-mesh. A sub-mesh can be created by creating a new vertex in the middle of an edge between vertices.

When a fitted subdivided mesh (hereinafter referred to as the fitted subdivided mesh) is generated, displacement is calculated using this result and a pre-compressed and decoded base mesh (hereinafter referred to as the reconstructed base mesh). That is, the reconstructed base mesh is subdivided in the same way as the fitting subdivision surface. The difference in position of each vertex between this result and the fitted subdivided mesh is the displacement for each vertex. Since the displacement represents the position difference in three-dimensional space, it is also expressed as a value in the (x, y, z) space of the Cartesian coordinate system. Depending on the user input parameters, the (x, y, z) coordinate values can be converted to (normal, tangential, bi-tangential) coordinate values in the local coordinate system.

Figure 5 shows a displacement generation process according to embodiments.

Fig. 5 illustrates in detail the displacement calculation method of the fitting subdivision surface (300) as described in Fig. 4.

An encoder and/or pre-processor according to embodiments may include 1) a subdivision unit, 2) a local coordinate system calculation unit, and 3) a displacement calculation unit. The subdivision unit may receive a reconstructed base mesh and generate a subdivided reconstructed base mesh. The local coordinate system calculation unit may receive a fitted subdivision mesh and a subdivided reconstructed base mesh, and transform a coordinate system of the meshes into a local coordinate system. The local coordinate system calculation operation may be optional. The displacement calculation unit may calculate a positional difference between the fitted subdivision mesh and the subdivided reconstructed base mesh. For example, a positional difference value between vertices of two input meshes may be generated. The vertex positional difference value becomes a displacement.

The method and device for transmitting point cloud data according to the embodiments can encode the point cloud as follows. The point cloud data (which may be referred to as point cloud for short) according to the embodiments can refer to data including vertex coordinates and color information. The point cloud is a term including mesh data, and in this document, the point cloud and mesh data can be used interchangeably.

The V-Mesh compression (decompression) method according to the embodiments may include intra frame encoding (Fig. 6) and inter frame encoding (Fig. 7).

Based on the result of the GoF generation described above, intra frame encoding or inter frame encoding is performed. In the case of intra encoding, the data to be compressed can be a base mesh, displacement, attribute map, etc. In the case of inter encoding, the data to be compressed can be a displacement, an attribute map, and a motion field between a reference base mesh and a current base mesh.

Figure 6 illustrates an intra-frame encoding process of a V-MESH compression method according to embodiments.

The encoding process of Fig. 6 details the encoding of Fig. 1. That is, it shows the configuration of an encoder when the encoding of Fig. 1 is an intra-frame method. The encoder of Fig. 6 may include a preprocessor (200) and/or an encoder (201).

The preprocessor can receive an input mesh and perform the preprocessing described above. The preprocessing can generate a base mesh and/or a fitted subdivided mesh. The quantizer can quantize the base mesh and/or the fitted subdivided mesh. The static mesh encoder can encode the static mesh. The static mesh encoder can generate a bitstream including the encoded base mesh. The static mesh decoder can decode the encoded static mesh. The inverse quantizer can inversely quantize the quantized static mesh. The displacement calculation unit can receive the restored static mesh and generate displacement, which is a position difference, based on the fitted subdivided mesh. The forward linear lifting unit can receive the displacement and generate lifting coefficients. The quantizer can quantize the lifting coefficients. The image packing unit can pack an image based on the quantized lifting coefficients. A video encoder can encode a packed image. A video decoder decodes an encoded video. An image unpacker can unpack a packed image. An inverse quantizer can inversely quantize an image. An inverse linear lifting unit applies inverse lifting to an image to generate a reconstructed displacement. A mesh restoration unit restores a warped mesh using the reconstructed displacement and the reconstructed base mesh. An attribute transfer unit receives an input mesh and/or an input attribute map, and generates an attribute map based on the reconstructed warped mesh. A push-pull padding unit can pad data in an attribute map based on a push-pull method. A color space transformation unit can transform a space of a color component, which is an attribute. A video encoder can encode an attribute. A multiplexer can generate a bitstream by multiplexing a compressed base mesh, a compressed displacement, and a compressed attribute.

Figure 7 illustrates an inter-frame encoding process of a V-MESH compression method according to embodiments.

The encoding process of Fig. 7 illustrates the encoding of Fig. 1 in detail. That is, it illustrates the configuration of an encoder when the encoding of Fig. 1 is in an inter-frame manner. The encoder of Fig. 7 may include a preprocessor (200) and/or an encoder (201).

Among the encoding operations of Fig. 7, the configuration corresponding to the encoding operation of Fig. 6 refers to the description of Fig. 7. For the inter-frame-based encoding of Fig. 7, the motion encoder can encode motion based on the restored quantized reference base mesh. The base mesh restoration unit can restore the base mesh based on the restored quantized reference base mesh.

The encoder of Fig. 6 generates a bitstream by compressing the base mesh, displacement, and attributes within a frame, and the encoder of Fig. 7 generates a bitstream by compressing the motion, displacement, and attributes between the current frame and the reference frame.

The encoding method according to the embodiments includes base mesh encoding (intra encoding). When performing intra frame encoding on a current input mesh frame, a base mesh generated in a preprocessing process can be encoded using a static mesh compression technique after going through a quantization process. In the V-Mesh compression method, for example, Draco technology is applied, and vertex position information, mapping information (texture coordinates), vertex connection information, etc. of the base mesh become compression targets.

The encoding method according to the embodiments may include motion field encoding (inter encoding). Inter frame encoding may be performed when a reference mesh and a current input mesh have a one-to-one correspondence of vertices and only the position information of the vertices is different. When performing inter frame encoding, instead of compressing the base mesh, the difference between the vertices of the reference base mesh and the current base mesh, i.e., the motion field, may be calculated and this information may be encoded. The reference base mesh is the result of quantizing the already decoded base mesh data and is determined according to the reference frame index determined in the GoF generation. The motion field may be encoded as a value. Alternatively, the motion fields of the restored vertices among the vertices connected to the current vertex may be averaged to calculate the predicted motion field, and this predicted motion field The residual motion field, which is the difference between the value and the motion field value of the current vertex, can be encoded. This value can be encoded using entropy coding. The process of encoding the displacement and attribute map, excluding the motion field encoding process of the inter frame encoding, is the same as the remaining structure of the intra frame encoding method, excluding the base mesh encoding.

Figure 8 shows a lifting conversion process for displacement according to embodiments.

Figure 9 shows a process of packing transformation coefficients according to embodiments into a 2D image.

Figures 8-9 illustrate the process of transforming the displacement of the encoding process of Figure 6-7 and the process of packing the transform coefficients, respectively.

The encoding method according to the embodiments includes displacement encoding.

After encoding the base mesh through base mesh encoding and/or motion field encoding, a reconstructed base mesh is generated through restoration and dequantization, and the displacement between the result of performing subdivision on the reconstructed base mesh and the fitted subdivided mesh generated through the fitting subdivision surface can be calculated. For effective encoding, a data transform process such as wavelet transform can be applied to the displacement information.

Fig. 8 shows a process of transforming displacement information using a lifting transform in V-Mesh. The transform coefficients generated through the transform process are quantized and then packed into a 2D image as shown in Fig. 9. The transform coefficients are organized into one block for every 256 (=16×16) units, and each block can be packed in z-scan order. The horizontal number of blocks is fixed to 16, but the vertical number of blocks can be determined according to the number of vertices of the subdivided base mesh. The transform coefficients can be packed by sorting them with a Morton code within a block. The packed images generate a displacement video for every GoF unit, and the displacement video can be encoded using an existing video compression codec.

Referring to FIG. 8, the base mesh (original) may include vertices and edges for LoD0. The first sub-division mesh generated by dividing the base mesh includes vertices generated by further dividing edges of the base mesh. The first sub-division mesh includes vertices for LoD0 and vertices for LoD1. LoD1 includes the sub-divided vertices and the vertices (LoD0) of the base mesh. The first sub-division mesh may be generated by dividing the first sub-division mesh. The second sub-division mesh includes LoD2. LoD2 includes the base mesh vertices (LoD0), LoD1 including vertices additionally generated from LoD0, and vertices additionally divided from LoD1. LoD is a level indicating the degree of detail (Level of Detail), and as the index of the level increases, the distance between vertices becomes closer and the level of detail increases. LoD N includes the vertices included in the previous LoDN-1 as they are. When a vertex is further divided through subdivision, considering the previous vertices v1, v2 and the subdivided vertex v, the mesh can be encoded based on a prediction and/or update method. Instead of still encoding information about the current LoD N, a residual value between the previous LoD N-1 can be generated, and the mesh can be encoded using the residual value to reduce the size of the bitstream. The prediction process means the operation of predicting the current vertex v through the previous vertices v1, v2. Since adjacent subdivision meshes have similar data, efficient encoding can be achieved by utilizing this property. The current vertex position information is predicted as the residual for the previous vertex position information, and the previous vertex position information is updated through the residual.

Referring to Figure 9, the vertices have coefficients generated through lifting transformation. The coefficients of the vertices related to lifting transformation can be packed into an image and encoded.

Figure 10 illustrates an attribute transfer process of a V-MESH compression method according to embodiments.

Figure 10 shows the detailed operation of attribute transfer of encoding such as Figure 6-7.

Encoding according to the embodiments includes attribute map encoding.

Information about the input mesh is compressed through base mesh encoding, motion field encoding, and displacement encoding. In the encoding process, the compressed input mesh is restored through base mesh decoding (intra frame), motion field decoding (inter frame), and displacement video decoding, and the restored result, the reconstructed deformed mesh (hereinafter referred to as Recon. deformed mesh), is used to compress the input attribute map as shown in FIGS. 6 and 7. The reconstructed deformed mesh (Recon. deformed mesh) has position information of vertices, texture coordinates, and corresponding connection information, but does not have color information corresponding to the texture coordinates. Therefore, as shown in Fig. 10, in the V-Mesh compression method, a new attribute map having color information corresponding to the texture coordinates of the reconstructed deformed mesh is created through the attribute transfer process.

Attribute transfer first checks whether each point P(u, v) in the 2D texture domain belongs to the texture triangle of the reconstructed deformed mesh, and if it exists in the texture triangle T, calculates the barycentric coordinate (α, β γ) of P(u, v) according to the triangle T. Then, using the 3D vertex position and (α, β γ) of triangle T, calculate the 3D coordinate M(x, y, z) of P(u, v). Find the vertex coordinate M'(x', y', z') that corresponds to the position most similar to the calculated M(x, y, z) in the input mesh domain and the triangle T' that contains this point. Then, the center of mass coordinates (α', β', γ') of M'(x', y', z') are calculated in this triangle T'. Using the texture coordinates and (α', β', γ') corresponding to the three vertices of triangle T', the texture coordinates (u', v') are calculated, and the color information corresponding to these coordinates is searched for in the input attribute map. The color information found in this way is immediately assigned to the (u, v) pixel location in the new attribute map. If P(u, v) does not belong to any triangle, the pixel at that location in the new attribute map can be filled with a color value using a padding algorithm such as the push-pull algorithm.

The new attribute map generated through attribute transfer is grouped into GoF units to form an attribute map video, which is then compressed using a video codec.

Referring to Figure 10, the reference relationship between the input mesh, input attribute map, restored mesh, and generated attribute map can be seen.

The decoding process of Fig. 1 can perform the reverse process of the corresponding process of the encoding process of Fig. 1. The specific decoding process is as follows.

Figure 11 illustrates an intra frame decoding process of a V-MESH compression method according to embodiments.

Fig. 11 shows the configuration and operation of a decoder of a receiving device such as Fig. 1.

Fig. 11 shows the intra decoding process of V-Mesh technology. First, the input bitstream can be separated into a mesh sub-stream, a displacement sub-stream, an attribute map sub-stream, and a sub-stream containing patch information of the mesh such as V3C/V-PCC.

The mesh sub-stream is decoded by a decoder of a static mesh codec used in encoding, such as Google Draco, so that the connection information, vertex geometry information, and vertex texture coordinates of the base mesh can be restored. The displacement sub-stream is decoded into a displacement video by a decoder of a video compression codec used in encoding, and goes through image unpacking, inverse quantization, and inverse transform processes to restore displacement information for each vertex. Inverse quantization is applied to the restored base mesh, and this result is combined with the restored displacement information to generate the final decoded mesh.

The attribute map sub-stream is decoded through the decoder of the video compression codec used in encoding, and then restored to the final attribute map through processes such as color format conversion.

The restored decoded mesh and decoded attribute map can be utilized by the receiver as final mesh data that can be utilized by the user.

Referring to FIG. 11, the bitstream includes patch information, a mesh substream, a displacement substream, and an attribute map substream. The substream is interpreted as a term referring to some bitstreams included in the bitstream. The bitstream includes patch information (data), mesh information (data), displacement information (data), and attribute cap information (data).

The decoder performs the following decoding operations within a frame. The static mesh decoder decodes the mesh to generate a reconstructed quantized base mesh, and the inverse quantizer inversely applies the quantization parameters of the quantizer to generate the reconstructed base mesh. The video decoder decodes the displacement, the unpacker unpacks the decoded video image, and the inverse quantizer inversely quantizes the quantized image. The linear lifting inverse transform applies a lifting transform in the reverse process of the encoder to generate the reconstructed displacement. The mesh restoration unit generates a warped mesh based on the base mesh and the displacement. The video decoder decodes an attribute map, and the color transformation unit transforms a color format and/or space to generate a decoded attribute map.

Figure 12 shows the inter-frame decoding process of the V-MESH compression method.

Fig. 12 shows the configuration and operation of a decoder of a receiving device such as Fig. 1.

Fig. 12 shows the inter decoding process of V-Mesh technology. First, the input bitstream can be separated into a motion sub-stream, a displacement sub-stream, an attribute sub-stream, and a sub-stream containing patch information of the mesh such as V3C/V-PCC.

The motion sub-stream is decoded through the entropy decoding and inverse prediction processes, and the reconstructed motion information is combined with the already reconstructed reference base mesh to generate a reconstructed quantized base mesh for the current frame. The result of applying inverse quantization to this is combined with the displacement information reconstructed in the same way as the intra decoding described above to generate the final decoded mesh. The attribute map sub-stream is decoded in the same way as the intra decoding. The reconstructed decoded mesh and the decoded attribute map can be utilized by the receiver as the final mesh data that can be utilized by the user.

Referring to Fig. 12, the bitstream includes motion, displacement, and attribute maps. Since inter-frame decoding is performed, a process of decoding inter-frame motion information is further included. The motion is decoded, and a restored quantized base mesh for the motion is generated based on the reference base mesh to generate the restored base mesh. For a description of the operation of Fig. 12, which is the same as Fig. 11, refer to the description of Fig. 11.

Fig. 13 shows a point cloud data transmission device according to embodiments.

Fig. 13 corresponds to the transmitting device (100), the dynamic mesh video encoder (102), the Fig. 2 encoder (pre-processor and encoder), and/or the transmitting encoding device corresponding thereto of Fig. 1. Each component of Fig. 13 corresponds to hardware, software, a processor, and/or a combination thereof.

The operation process of a transmitter for compressing and transmitting dynamic mesh data using V-Mesh compression technology can be as shown in Fig. 13.

The mesh preprocessor receives the original mesh as input and generates a simplified mesh (decimated mesh). Simplification can be performed based on the target number of vertices or the target number of polygons that constitute the mesh. Parameterization can be performed to generate texture coordinates and texture connection information per vertex for the simplified mesh. In addition, a task of quantizing floating-point type mesh information into fixed-point type can be performed. This result can be encoded as a base mesh through a static mesh encoding unit. The mesh preprocessor can perform mesh subdivision on the base mesh to generate additional vertices. Depending on the subdivision method, vertex connection information, texture coordinates, and texture coordinate connection information including the added vertices can be generated. The subdivided mesh can be fitted by adjusting the vertex positions to be similar to the original mesh, thereby generating a fitted subdivided mesh.

When performing intra encoding for the corresponding mesh frame, the base mesh generated through the mesh preprocessing unit can be compressed through the static mesh encoding unit. In this case, encoding can be performed on the connection information, vertex geometry information, vertex texture information, normal information, etc. of the base mesh. The base mesh bitstream generated through encoding is transmitted to the multiplexing unit.

When performing inter encoding for the corresponding mesh frame, a motion vector encoding unit is performed, which can calculate a motion vector between the two meshes using a base mesh and a reference restoration base mesh as inputs, and encode the value. The motion vector encoding unit can perform prediction based on connection information using a previously encoded/decoded motion vector as a predictor, and encode a residual motion vector obtained by subtracting the predicted motion vector from the current motion vector. The motion vector bitstream generated through encoding is transmitted to the multiplexing unit.

The encoded base mesh and motion vectors can be used to generate a restored base mesh through the base mesh restoration unit.

The displacement vector calculator can perform mesh refinement on the restored base mesh. The displacement vector can be calculated as the difference value of the vertex position between the refined restored base mesh and the fitted subdivision mesh generated in the preprocessing unit. As a result, the displacement vector can be calculated as many as the number of vertices of the refined mesh. The displacement vector calculation unit can convert the displacement vector calculated in the 3D Cartesian coordinate system into the local coordinate system based on the normal vector of each vertex.

The displacement vector video generator can transform the displacement vector for effective encoding. The transform can be performed by a lifting transform, a wavelet transform, etc., depending on the embodiment. In addition, quantization can be performed on the transformed displacement vector value, that is, the transform coefficient. Different quantization parameters can be applied to each axis of the transform coefficient, and the quantization parameters can be derived by the promise of the encoder/decoder. The transformed and quantized displacement vector information can be packed into a 2D image. The packed 2D images can be bundled for each frame to generate a displacement vector video, and the displacement vector video can be generated for each GoF (Group of Frame) unit of the input mesh.

A displacement vector video encoder can encode the generated displacement vector video using a video compression codec. The generated displacement vector video bitstream is transmitted to a multiplexer.

The displacement vector restored through the displacement vector restorer and the base mesh restored through the base mesh restorer and refined are restored through the mesh restorer, and the restored mesh has restored vertices, connection information between vertices, texture coordinates, and connection information between texture coordinates.

The texture map of the original mesh can be regenerated as a texture map for the restored mesh through the texture map video generation unit. The color information per vertex of the texture map of the original mesh can be assigned to the texture coordinates of the restored mesh. The regenerated texture maps for each frame can be grouped by GoF unit to generate a texture map video.

The generated texture map video can be encoded using a video compression codec through a texture map video encoding unit. The texture map video bitstream generated through encoding is transmitted to a multiplexing unit.

The generated motion vector bitstream, base mesh bitstream, displacement vector bitstream and texture map bitstream may be multiplexed into a single bitstream and transmitted to a receiver through a transmitter. Alternatively, the generated motion vector bitstream, base mesh bitstream, displacement vector bitstream and texture map bitstream may be generated into a file with one or more track data or encapsulated into segments and transmitted to a receiver through a transmitter.

Referring to FIG. 13, a transmitting device (encoder) can encode a mesh in an intra-frame or inter-frame manner. A transmitting device according to intra-encoding can generate a base mesh, a displacement vector (displacement), and a texture map (attribute map). A transmitting device according to inter-encoding can generate a motion vector (motion), a base mesh, a displacement vector (displacement), and a texture map (attribute map). A texture map obtained from a data input unit is generated and encoded based on a restored mesh. Displacement is generated and encoded through a difference in vertex positions between the base mesh and the divided mesh. The base mesh is generated by preprocessing, simplifying, and encoding an original mesh. Motion is generated as a motion vector for a mesh of a current frame based on a reference base mesh of a previous frame.

Fig. 14 shows a point cloud data receiving device according to embodiments.

Fig. 14 corresponds to the receiving device (110) of Fig. 1, the dynamic mesh video decoder (113), the decoder of Figs. 11-12, and/or the receiving decoding device corresponding thereto. Each component of Fig. 14 corresponds to hardware, software, a processor, and/or a combination thereof. The receiving (decoding) operation of Fig. 14 can follow the reverse process of the corresponding process of the transmitting (encoding) operation of Fig. 13.

The received Mesh bitstream is demultiplexed into a compressed motion vector bitstream or base mesh bitstream, displacement vector bitstream, and texture map bitstream after file/segment decapsulation.

If the current mesh has inter-screen encoding applied according to the frame header information, the motion vector decoding unit can perform decoding on the motion vector bitstream. The final motion vector can be reconstructed by adding the previously decoded motion vector to the residual motion vector decoded from the bitstream using the previously decoded motion vector as a predictor.

If the current mesh has in-screen encoding applied according to the frame header information, the base mesh bitstream can restore the connection information, vertex geometry information, texture coordinates, normal information, etc. of the base mesh through the static mesh encoding unit.

In the base mesh restoration unit, if the current mesh has inter-screen encoding applied, the current base mesh can be restored by adding the decoded motion vector to the reference base mesh and then performing inverse quantization. If the current mesh has intra-screen encoding applied, the decoded mesh can be dequantized through the static mesh decoding unit to generate a restored base mesh.

The displacement vector bitstream can be decoded as a video bitstream using a video codec in a displacement vector video decoder.

In the displacement vector restoration unit, displacement vector transformation coefficients are extracted from the decoded displacement vector video, and the displacement vector is restored through the inverse quantization and inverse transformation processes. If the restored displacement vector is a value in the local coordinate system, a process of inverse transformation to the Cartesian coordinate system can be performed.

In the mesh restoration section, additional vertices can be generated by performing subdivision on the restored base mesh. Through subdivision, vertex connection information including the added vertices, texture coordinates, and texture coordinate connection information can be generated. The subdivided restored base mesh can be combined with the restored displacement vector to generate the final restored mesh.

The texture map bitstream can be decoded as a video bitstream using a video codec in a texture map video decoding unit. The restored texture map has color information for each vertex contained in the restored mesh, and the color value of each vertex can be obtained from the texture map using the texture coordinates of each vertex.

The restored mesh and texture map are shown to the user through a rendering process using a mesh data renderer, etc.

Referring to FIG. 14, a receiving device (decoder) can decode a mesh in an intra-frame or inter-frame manner. A receiving device according to intra-decoding can receive a base mesh, a displacement vector (displacement), and a texture map (attribute map), and can decode a restoration mesh and a restoration texture map to render mesh data. A receiving device according to inter-decoding can receive a motion vector (motion), a base mesh, a displacement vector (displacement), and a texture map (attribute map), and can decode a restoration mesh and a restoration texture map to render mesh data.

The point cloud data transmission device and method according to the embodiments can encode mesh data and transmit a bitstream including the encoded mesh data. The point cloud data reception device and method according to the embodiments can receive a bitstream including mesh data and decode the mesh data. The point cloud data transmission and reception method/device according to the embodiments may be referred to as the method/device according to the embodiments. The point cloud data transmission and reception method/device according to the embodiments may also be referred to as the mesh data transmission and reception method/device according to the embodiments.

Figure 15 shows a mesh system structure according to embodiments.

Embodiments include a method of storing and transmitting a dynamic mesh coding bitstream in a file (Carriage of Dynamic Mesh Coding Bitstream in Single Track).

Embodiments include methods for storing and/or signaling a Video-based Dynamic Mesh Coding (VDMC) bitstream as a single track within a file.

Embodiments include a transmission method/device and/or a reception method/device for providing a mesh content service that efficiently stores a V-DMC (video-based dynamic mesh coding) or mesh bitstream in a single track within a file and provides signaling therefor.

Embodiments include a transmitting method/device and/or a receiving method/device for providing mesh content services that handle file storage techniques to enable efficient access to stored V-DMC bitstreams.

Although the embodiments describe the embodiments using V-DMC (video-based dynamic mesh coding), the embodiments can be applied not only to V-DMC but also to bitstreams using other mesh codings configured in the same manner or in a similar bitstream format.

Embodiments can encapsulate mesh content encoded with V-DMC (Video-based Dynamic Mesh Coding) into a single track based on an ISOBMFF file to provide a service such as streaming.

The embodiments generate and transmit information contained in a sample entry within a single track of a file.

The embodiments generate and transmit decoder configuration information as signaling information included in a sample entry within a single track of a file.

The embodiments generate and transmit information contained in a sample format within a single track of a file.

The decoder receives a single track of the file, parses the information within the single track, and efficiently decodes the mesh data.

Fig. 15 is a structure of a mesh system. Scenes and/or objects acquired using multiple cameras, sensors and/or virtual cameras in the real world can be encoded into V-DMC bitstreams based on mesh data pre-processing and/or mesh data encoding. The mesh bitstreams can be generated in a file format that can be efficiently stored and transmitted through file encapsulation.

The dynamic mesh player can recover the acquired and received files through this process into a mesh bitstream through file decapsulation. The recovered mesh bitstream can be provided to a display device and displayed based on mesh data decoding and/or mesh data processing/rendering.

Examples include mesh encoding, file/segment encapsulation, file/segment decapsulation, mesh decoding, etc.

Figure 16 illustrates a video-based dynamic mesh coding bitstream structure according to embodiments.

Figure 16 shows an example of a bitstream structure including mesh data encoded in Figure 15, etc.

The sequence header of the bitstream may include decoder configuration information. The encoded base mesh of the bitstream is generated by encoding the input mesh file. The encoding method may include Edgebreaker or Draco, etc. The encoded displacement (which may be referred to as displacement information or geometry, etc.) of the bitstream means the difference information between vertices between the reconstructed subdivision meshes after fitting and subdividing the base mesh. It is generated by packing the difference information between vertices into a 2D image and encoding the packed image based on a video method. The video method may include HEVC, etc. Or, the displacement may be generated by encoding the packed image based on an arithmetic coding method. The encoded attribute (or texture) of the bitstream is generated by generating and encoding a new 2D attribute (color information, PNG image) that matches the coordinates of the attribute (texture, 2D) of the reconstructed subdivision mesh using the input attribute map (e.g., PNG image). Encoding methods may include HEVC, etc.

The bitstream according to the embodiments may be a bitstream resulting from video-based dynamic mesh encoding. As described above, the bitstream may include base mesh data, displacement data, texture or attribute data, each encoded.

A bitstream according to the embodiments may be generated as shown in Fig. 16b. The bitstream may include a sample stream header and a sample stream unit. The unit may be a DMC unit. The header and the unit may be referred to as a sample stream DMC header, a sample stream DMC unit, etc., respectively.

The sample stream DMC unit according to the embodiments contains the following information:

VPS (V-DMC Parameter Set): May include parameter set information such as decoder configuration information related to mesh encoding/decoding and sequence number.

AD (Atlas Data): May contain information related to 2D mapping or texture mapping for 3D objects.

BMD (Base Mesh Data): Encoded base mesh information for mesh coding/decoding.

DD(Displacements Data): Displacement information encoded using arithmetic coding. However, in case of lossless encoding, displacement data may not be included.

GVD(Displacements Video Data): Displacement information encoded with a video codec.

AVD (Attribute Video Data): Attribute or texture information encoded with a video codec.

The format of a V-DMC bitstream can be defined by the syntax and semantics of the sample stream DMC header and the sample stream DMC unit as follows.

The sample stream data syntax and semantics can be the sample stream format defined in the V3C codec standard, i.e., ISO/IEC 23090-5, which is referenced in the V-DMC standard. In addition, it can be flexibly implemented based on a bitstream format newly defined in the V-DMC or V3C codec.

The syntax of the sample stream DMC header is as follows:

sample_stream_dmc_header() { Descriptor sdmh_unit_size_precision_bytes_minus1 u(3) sdmh_reserved_zero_5bits u(5) }

Sample Stream Header Precision (sdmh_unit_size_precision_bytes_minus1): This value plus 1 represents the precision (in bytes) of sdmu_dmc_unit_size elements in every sample stream DMC unit. The range for sdmh_unit_size_precision_bytes_minus1 is 0 to 7.

The syntax of a sample stream DMC unit is as follows. Each sample stream DMC unit contains one type of DMC unit among VPS, AD, BMD, DD, GVD, and AVD. The contents of each sample stream DMC unit are associated with the same access unit as the DMC unit contained in the sample stream DMC unit.

sample_stream_dmc_unit() { Descriptor sdmu_dmc_unit_size u(v) dmc_unit(sdmu_dmc_unit_size ) }

The sample stream unit size (sdmu_dmc_unit_size) represents the size (in bytes) of the subsequent dmc_unit. The number of bits used to express sdmu_dmc_unit_size is equal to (sdmh_unit_size_precision_bytes_minus1 + 1) * 8.

A DMC unit (dmc_unit) may contain a unit header and a unit payload. The unit header may contain type information about the unit payload. The unit payload may contain data of that type.

DMC unit (dmc_unit) data syntax and semantics can be in the format of V3C unit (v3c_unit) used in the V3C codec specification, i.e. ISO/IEC 23090-5, which is referenced in the V-DMC standard. It can also be flexibly implemented based on a bitstream format newly defined in V-DMC or V3C codec.

Common data structures contained in the Sample Stream DMC Header and/or Sample Stream DMC Unit are as follows:

DMC decoder configuration record:

Decoder configuration information contains decoder configuration information for mesh-based point cloud content. This decoder configuration information contains a version field. The decoder configuration information can define version 1. Incompatible changes to the decoder configuration information are indicated by a change in the version number. Compatible extensions to this record extend it and do not change the configuration version code.

The syntax of the DMC decoder configuration information is as follows.

aligned(8) class DMCDecoderConfigurationRecord {

unsigned int(8) configurationVersion = 1;

unsigned int(8) num_of_setup_units;

for (i=0; I < num_of_setup_units; i++) {

unsigned int(1) array_completeness;

unsigned int(7) setup_unit_type; / VPS, SPS, FPS, GPS, APS, etc.

SetupUnit setup_unit; }

/ additional fields

}

Setup unit ( setup_unit ): Contains a set of V3C parameters. A setup unit contains NAL units according to their associated nal unit type ( nal_unit_type ). If present in the setup_unit , it contains SEI messages of type NAL_PREFIX_ESEI , NAL_PREFIX_NSEI , NAL_SUFFIX_ESEI , or NAL_SUFFIX_NSEI , i.e. SEI messages that provide information about the stream as a whole.

The setup_unit array contains a set of atlas sub-bitstream parameters and atlas sub-bitstream SEI messages that are constants for the CVS referenced by the sample entry for which a decoder configuration record exists.

The semantics of the DMC decoder configuration information are as follows.

ConfigurationVersion is a version field. Incompatible changes to a record are indicated by a change in the version number.

The number of setup units (num_of_setup_units) indicates the number of DMC configuration units in the decoder configuration record.

An array completeness of 1 indicates that all the constituent units of the indicated type are in the following array and none are in the stream. A value of 0 indicates that additional constituent units of the indicated type may be in the stream. The default and allowed values are restricted by the sample item name.

The setup unit type (setup_unit_type) indicates the DMC-related parameter set type. This parameter set type can be information based on the parameter set type defined in the V-DMC specification.

A SetupUnit contains a VPS (V-DMC Parameter Set), a SPS (Sequence Parameter Set), a FPS (Frame Parameter Set), a DPS (n Parameter Set), a GPS (Geometry Parameter Set), a BPS (Base Mesh Parameter Set), and/or an APS (Attribute Parameter Set), and the parameter sets may be information based on parameter sets defined in the V-DMC standard.

A BPS (BaseMesh Parameter Set) can contain the following elements:

The sequence parameter set ID (bmsps_sequence_parameter_set_id) represents an identifier for a BaseMesh sequence parameter set so that it can be referenced from other syntax elements.

The intra-mesh codec ID (bmsps_intra_mesh_codec_id) indicates the identifier of the codec used to compress the static mesh. bmsps_intra_mesh_codec_id is in the range 0 to 255. The codec may be identified by a profile defined in ISO/IEC 23090-29, a component codec mapping SEI message, or means external to this document. It may also be associated with a particular mesh or motion mesh codec via a profile specified in that specification, or it may be explicitly indicated in an SEI message, as is done in the V3C specification for video sub-bitstreams.

bmsps_inter_mesh_codec_id indicates the identifier of the codec used to compress the motion data. bmsps_intr_mesh_codec_id is in the range 0 to 255. The codec may be identified by a profile defined in ISO/IEC 23090-29, a component codec mapping SEI message, or means external to this document. It may also be associated with a particular mesh or motion mesh codec via a profile specified in that specification, or it may be explicitly indicated in an SEI message, as is done in the V3C specification for video sub-bitstreams.

Inter-mesh motion group size (bmsps_inter_mesh_motion_group_size_minus1): Adding 1 to this value indicates the size of vertex grouping in motion vector coding. The range of bmsps_inter_mesh_motion_group_size_minus1 is 0 to 255.

Inter-mesh maximum number of neighbors (bmsps_inter_mesh_max_num_neighbours_minus1): Adding 1 to this value indicates the maximum number of vertex neighbors to use for calculating the motion vector estimator. The range of bmsps_inter_mesh_max_num_neighbours_minus1 is 0 to 255.

Geometry length (bmsps_geometry_3d_bit_length_minus1): Adding 1 to this value indicates the bit depth of the geometry coordinates of the reconstructed mesh.

The facegroup segmentation method (bmsps_facegroup_segmentation_method) indicates the identifier of the method for deriving facegroup IDs from a mesh.

Mesh attribute count (bmsps_mesh_attribute_count) indicates the number of attributes associated with the mesh.

Mesh attribute type ID (bmsps_mesh_attribute_type_id[ i ]) indicates the attribute type of the attribute with index i for the mesh. Types include texture, materialID, reflectance, normal, facegroupID, etc.

Attribute bit length (bmsps_attribute_bit_length_minus1[ i ]): Adding 1 to this value indicates the bit depth of the attribute with index i for the mesh.

bmsps_log2_max_mesh_frame_order_cnt_lsb_minus4 + 4 represents the values of the Log2MaxMeshFrmOrderCntLsb and MaxMeshFrmOrderCntLsb variables used in the decoding process for the atlas frame order number:

Log2MaxMeshFrmOrderCntLsb =

bmsps_log2_max_mesh_frame_order_cnt_lsb_minus4 + 4

MaxMeshFrmOrderCntLsb = 2Log2MaxMeshFrmOrderCntLsb

bmsps_max_dec_mesh_frame_buffering_minus1 + 1 represents the maximum required size of the decoded atlas frame buffer for CAS in atlas frame storage buffer units.

If bmsps_long_term_ref_mesh_frames_flag is 0, it indicates that long-term reference atlas frames are not used for inter prediction of coded atlas frames in CAS. If bmsps_long_term_ref_mesh_frames_flag is 1, it indicates that long-term reference atlas frames may be used for inter prediction of one or more coded atlas frames in CAS.

bmsps_num_ref_mesh_frame_lists_in_bmsps indicates the number of bmesh_ref_list_struct( rlsIdx ) syntax structures included in the atlas sequence parameter set.

The decoder can have one bmesh_ref_list_struct(rlsIdx) syntax structure that directly signals the atlas tile header of the current atlas tile, so it allocates memory for a total number of bmesh_ref_list_struct( rlsIdx ) syntax structures equal to (bmsps_num_ref_mesh_frame_lists_in_bmsps + 1).

BPS further contains bmesh_profile_tier_level(　). bmesh_profile_tier_level(　) further contains the following elements:

bmptl_tier_flag indicates the hierarchy context for interpretation of bmptl_level_idc as specified in ISO/IEC 23090-29.

bmptl_profile_codec_group_idc indicates a codec group profile component that the CVS complies with as specified in ISO/IEC 23090-29.

bmptl_profile_toolset_idc represents a toolset combination profile component that CVS complies with as specified in ISO/IEC 23090-29.

bmptl_profile_reconstruction_idc represents the reconstruction profile component that CVS is recommended to comply with as specified in ISO/IEC 23090-29.

bmptl_level_idc indicates the level to which the CVS complies as specified in ISO/IEC 23090-29.

bmptl_num_sub_profiles represents the number of bmptl_sub_profile_idc[ i ] syntax elements.

If bmptl_extended_sub_profile_flag is 1, it indicates that the bmptl_sub_profile_idc[ i ] syntax element, if present, should be represented using 64 bits. If bmptl_extended_sub_profile_flag is 0, it indicates that the bmptl_sub_profile_idc[ i ] syntax element, if present, should be represented using 32 bits.

bmptl_sub_profile_idc[ i ] represents the ith interoperability metadata registered as specified in Rec.

If bmptl_toolset_constraints_present_flag is 1, it indicates that the additional structure profile_toolset_constraints_information() is present in the bitstream. If bmptl_toolset_constraints_present_flag is 0, it indicates that the profile_toolset_constraints_information() structure is not present.

The bmesh_profile_tier_level(　) of BPS further contains bmesh_profile_toolset_constraints_information. bmesh_profile_toolset_constraints_information further contains the following elements:

When bmptc_one_mesh_frame_only_flag is present, it has the semantics specified in ISO/IEC 23090-29. A bmptc_intra_frames_only_flag of 1 indicates that the bitstream contains only sdu_intra_sub_mesh_unit().

The setup unit may further include a base frame parameter. The base frame parameter includes the following elements:

bfps_mesh_sequence_parameter_set_id indicates the bmsps_sequence_parameter_set_id value for the active basemesh sequence parameter set.

bfps_mesh_parameter_set_id indicates the basemesh frame parameters set for reference in other syntax elements.

If bfps_output_flag_present_flag is 1, it indicates that the smh_output_flag syntax element is present in the associated submesh header. If bfps_output_flag_present_flag is 0, it indicates that the smh_output_flag syntax element is not present in the associated submesh header.

bfps_num_ref_idx_default_active_minus1 + 1 represents the inferred value of variable NumRefIdxActive for tiles with smh_num_ref_idx_active_override_flag equal to 0.

bfps_additional_lt_mfoc_lsb_len indicates the value of the MaxLtMeshFrmOrderCntLsb variable used in the decoding process for the reference atlas frame list.

MaxLtMeshFrmOrderCntLsb =

2 * (Log2MaxMeshFrmOrderCntLsb + bfps_additional_lt_mfoc_lsb_len)

If bfps_extension_present_flag is 1, it indicates that the syntax element bfps_extension_8bits is present in the basemesh frame parameter set. If bfps_extension_present_flag is 0, it indicates that the syntax element bfps_extension_8bits is not present.

If bfps_extension_8bits is 0, it indicates that there is no bfps_extension_data_flag syntax element in the AFPS RBSP syntax structure.

In addition, the bitstream may further include parameters relating to arithmetic-encoded displacement information. The parameters relating to the displacement information may be included in the bitstream as raw units. The parameters relating to the displacement information may further include a parameter set relating to the displacement information, a profile of the displacement information, a frame parameter relating to the displacement information, etc.

DMC Decoder Configuration Box:

The DMC Decoder Configuration box contains a DMCDecoderConfigurationRecord.

The syntax of the DMC decoder configuration box is as follows:

class DMCConfigurationBox extends FullBox('dmcC', version = 0, 0) {

DMCDecoderConfigurationRecord();

}

The semantics of the DMC decoder configuration box are the same as the DMCDecoderConfigurationRecord described above.

Encoded bitstreams according to the embodiments can be transmitted and received encapsulated in a single track.

Single Track Encapsulation:

A V-DMC bitstream sample entry for a single track is as follows:

Sample Entry Type: 'dme1', 'dmeg'

Container: SampleDescriptionBox

Mandatory: A 'dme1' or 'dmeg' sample entry is mandatory

Quantity: One or more

A transmitting device according to embodiments may encapsulate a V-DMC bitstream into a single track. A sample entry of a V-DMC bitstream track may include DMCConfigurationBox information. If a sample entry type is 'dme1', all related parameter sets may be included in a setup unit (setup_unit) of the DMCConfigurationBox. If a sample entry type is 'dmeg', all related parameter sets may be included in the setup unit (setup_unit) of the DMCConfigurationBox, and/or may be included in a sample of a DMC bitstream track.

The syntax of a V-DMC bitstream sample entry for a single track is as follows:

aligned(8) class DMCBitstreamSampleEntry() extends VolumetricVisualSampleEntry (type) {

/ type is 'dme1' or 'dmeg'

DMCConfigurationBox config;

}

The V-DMC bitstream track sample format for a single track is as follows:

definition:

A V-DMC bitstream sample must contain one or more DMC units belonging to the same presentation time, i.e., a V-DMC constituent unit. A sample may be self-contained (e.g., a synchronization sample) or may be decoded dependent on other samples in the V-DMC bitstream track.

A V-DMC bitstream track sample according to the embodiments is described to include V-DMC bitstream data in a sample stream format referenced in the V-DMC standard document, but may also include V-DMC bitstream data in a format defined other than the sample stream format.

The V-DMC bitstream sample syntax for a single track is as follows:

aligned(8) class DMCBitstreamSample {

/ sample_size size of sample from SampleSizeBox

for (int i = 0; i < sample_size; ) {

sample_stream_dmc_unit ss_dmc_unit;

i += ss_dmc_unit.sdmu_dmc_unit_size;

}

}

The V-DMC bitstream sample semantics of a single track are as follows:

The DMC unit (ss_dmc_unit) contains a single DMC unit in the DMC unit sample stream format defined in the V-DMC specification.

DMC unit size (sdmu_dmc_unit_size) indicates the size (in bytes) of the sample stream DMC unit.

Fig. 17 illustrates a mesh data receiving device according to embodiments.

A flow diagram from the receiver perspective for decoder configuration and track parsing might look like this:

File Receiver: The receiver can receive the V-DMC bitstream encapsulated in a single track file format, either in its entirety or in segments, via a file delivery method such as streaming.

File Parser: It can find the track(s) of the received file and parse the sample entry of each track. At this time, the sample entry type can be distinguished as 'dme1' or 'dmeg' to find out what type of track it is. In addition, by parsing the receiver initialization information included in the sample entry, i.e., decoder configuration information, the receiver can prepare for decoding using parameter sets related to decoding.

Additionally, data contained in a sample can be collected by parsing information about the sample contained in the sample entry, such as sample size, sample offset, etc.

Bitstream Packager: As described above, the collected data can be packaged in a form similar to the V-DMC bitstream structure embodiment as shown in FIG. 16.

Decoder (V-DMC Decoder): Can decode bitstream files packaged in the V-DMC bitstream structure and format.

Figure 18 shows a mesh data encoding method according to embodiments.

A mesh data encoding method according to embodiments may include a step of encoding mesh data (S1800); a step of encapsulating mesh data into a file (S1810); and a step of transmitting the file (S1820).

A bitstream containing mesh data and parameter information about the mesh data can be encapsulated into a single track of a file.

A bitstream contains a sample stream header and one or more sample stream units,

The sample stream header contains information about one or more sample stream units,

One or more sample stream units may include at least one of atlas data about the mesh data, information about a base mesh, displacement information encoded based on arithmetic coding, displacement information encoded based on a video codec, and attribute data encoded based on a video codec.

The file comprises a bitstream based on a single track including mesh data and parameter information about the mesh data, a sample entry of the single track including decoder configuration information, the decoder configuration information including information indicating a number of setup units, information indicating a type of the setup units, or at least one of the setup units, and the setup unit may include a parameter set about the mesh data.

A parameter set of a setup unit can include at least one of a VPS (V-DMC parameter set), a SPS (sequence parameter set), a FPS (frame parameter set), a DPS (displacement parameter set), a GPS (geometry parameter set), a BPS (basemesh parameter set), or an APS (attribute parameter set).

The file comprises a bitstream based on a single track, which includes mesh data and parameter information about the mesh data, and samples of the single track can include parameter information about the mesh data.

A sample of a single track may contain one or more Dynamic Mesh Coding (DMC) units belonging to composition time.

A mesh data encoding device includes an encoder for encoding mesh data; a file encapsulator for encapsulating mesh data into a file; and a transmission unit for transmitting the file; wherein a bitstream including mesh data and parameter information about the mesh data can be encapsulated into a single track of the file.

Figure 19 shows a mesh data decryption method according to embodiments.

A method for decrypting mesh data according to embodiments may include a step of receiving a file including mesh data (S1900); a step of decapsulating the file (S1910); and a step of decrypting the mesh data (S1920).

Referring to FIGS. 1, 15, and 19, a method for decoding mesh data includes the steps of: receiving a file including mesh data; decapsulating the file; and decoding the mesh data; and the file may include a bitstream including mesh data and parameter information about the mesh data.

Referring to FIG. 16b, a bitstream includes a sample stream header and one or more sample stream units, the sample stream header includes information about the one or more sample stream units, and the one or more sample stream units may include at least one of atlas data about mesh data, information about a base mesh, displacement information encoded based on arithmetic coding, displacement information encoded based on a video codec, and attribute data encoded based on a video codec.

The file can be generated as a single track, and when a sample entry type of the single track is a specific value (dme1), the file includes a bitstream including the mesh data and parameter information about the mesh data based on the single track, the sample entry of the single track includes decoder configuration information, and the decoder configuration information includes at least one of information indicating a number of setup units, information indicating a type of the setup units, or a setup unit, and the setup unit can include a parameter set about the mesh data.

With respect to a V-DMC setup unit, a parameter set of the setup unit may include at least one of a VPS (V-DMC parameter set), an SPS (sequence parameter set), an FPS (frame parameter set), a DPS (displacement parameter set), a GPS (geometry parameter set), a BPS (basemesh parameter set), or an APS (attribute parameter set).

A file can be generated as a single track, and if a sample entry type of the single track is a specific value (dmeg), the file includes a bitstream including the mesh data and parameter information about the mesh data based on the single track, and a sample of the single track can include parameter information about the mesh data.

With respect to the V-DMC bitstream track sample format, a sample of a single track may contain one or more DMC (Dynamic Mesh Coding) units belonging to a composition time.

A sample entry may have at least one type of information among a first type indicating whether parameter information about mesh data is included in a setup unit or a second type indicating whether parameter information about the mesh data is included in a sample.

Referring to FIG. 17, a mesh data decoding device includes a receiving unit that receives a file including mesh data; a file decapsulator that decapsulates the file; and a decoder that decodes the mesh data; and the file may include a bitstream including mesh data and parameter information about the mesh data.

The examples provide the following effects:

A transmitter or receiver for providing mesh content services can construct a V-DMC bit stream and store and transmit a file. The transmitter or receiver can efficiently store and transmit a file of a V-DMC bit stream through a storage technique and signaling of the V-DMC bit stream as one track in a file.

The embodiments have been described in terms of methods and/or devices, and the descriptions of methods and devices may be applied complementarily.

For the convenience of explanation, each drawing has been described separately, but it is also possible to design a new embodiment by combining the embodiments described in each drawing. In addition, designing a computer-readable recording medium in which a program for executing the previously described embodiments is recorded according to the needs of a person skilled in the art also falls within the scope of the embodiments. The devices and methods according to the embodiments are not limited to the configurations and methods of the embodiments described above, but the embodiments may be configured by selectively combining all or part of the embodiments so that various modifications can be made. Although the preferred embodiments of the embodiments have been illustrated and described, the embodiments are not limited to the specific embodiments described above, and various modifications can be made by a person skilled in the art without departing from the gist of the embodiments claimed in the claims, and such modifications should not be individually understood from the technical idea or prospect of the embodiments.

The various components of the device of the embodiments may be performed by hardware, software, firmware, or a combination thereof. The various components of the embodiments may be implemented as one chip, for example, one hardware circuit. According to embodiments, the components according to the embodiments may be implemented as separate chips, respectively. According to embodiments, at least one of the components of the device of the embodiments may be configured with one or more processors capable of executing one or more programs, and the one or more programs may perform, or include instructions for performing, one or more of the operations/methods according to the embodiments. The executable instructions for performing the methods/operations of the device of the embodiments may be stored in non-transitory CRMs or other computer program products configured to be executed by one or more processors, or may be stored in temporary CRMs or other computer program products configured to be executed by one or more processors. In addition, the memory according to the embodiments may be used as a concept including not only volatile memory (e.g., RAM, etc.), but also non-volatile memory, flash memory, PROM, etc. Additionally, it may include implementations in the form of carrier waves, such as transmission over the Internet. Additionally, the processor-readable recording medium may be distributed over network-connected computer systems, so that the processor-readable code may be stored and executed in a distributed manner.

In this document, “/” and “,” are interpreted as “and/or”. For example, “A/B” is interpreted as “A and/or B”, and “A, B” is interpreted as “A and/or B”. Additionally, “A/B/C” means “at least one of A, B, and/or C”. Also, “A, B, C” means “at least one of A, B, and/or C”. Additionally, “or” in this document is interpreted as “and/or”. For example, “A or B” can mean 1) “A” only, 2) “B” only, or 3) “A and B”. In other words, “or” in this document can mean “additionally or alternatively”.

Terms such as first, second, etc. may be used to describe various components of the embodiments. However, the various components according to the embodiments should not be limited in their interpretation by the above terms. These terms are merely used to distinguish one component from another. For example, a first user input signal may be referred to as a second user input signal. Similarly, a second user input signal may be referred to as a first user input signal. The use of these terms should be construed as not departing from the scope of the various embodiments. Although the first user input signal and the second user input signal are both user input signals, they do not mean the same user input signals unless the context clearly indicates otherwise.

The terminology used to describe the embodiments is used for the purpose of describing particular embodiments and is not intended to be limiting of the embodiments. As used in the description of the embodiments and in the claims, the singular is intended to include the plural unless the context clearly dictates otherwise. The expressions “and” and “or” are used to mean including all possible combinations of the terms. The expression “includes” describes the presence of features, numbers, steps, elements, and/or components, and does not mean that additional features, numbers, steps, elements, and/or components are not included. Conditional expressions such as “if,” “when,” etc., used to describe the embodiments are not intended to be limited to only optional cases. When a particular condition is satisfied, a related action is performed in response to a particular condition, or a related definition is intended to be interpreted.

In addition, the operations according to the embodiments described in this document may be performed by a transceiver device including a memory and/or a processor according to the embodiments. The memory may store programs for processing/controlling the operations according to the embodiments, and the processor may control various operations described in this document. The processor may be referred to as a controller, etc. The operations according to the embodiments may be performed by firmware, software, and/or a combination thereof, and the firmware, software, and/or a combination thereof may be stored in the processor or in the memory.

Meanwhile, the operations according to the embodiments described above may be performed by a transmitting device and/or a receiving device according to the embodiments. The transmitting/receiving device may include a transmitting/receiving unit for transmitting and receiving media data, a memory for storing instructions (program codes, algorithms, flowcharts, and/or data) for a process according to the embodiments, and a processor for controlling operations of the transmitting/receiving device.

The processor may be referred to as a controller, etc., and may correspond to, for example, hardware, software, and/or a combination thereof. The operations according to the embodiments described above may be performed by the processor. In addition, the processor may be implemented as an encoder/decoder, etc. for the operations of the embodiments described above.

As described above, the relevant contents have been described in the best form for carrying out the embodiments.

As described above, the embodiments can be applied in whole or in part to a point cloud data transmission and reception device and system.

Those skilled in the art may make various changes or modifications to the embodiments within the scope of the embodiments.

Embodiments may include modifications/changes, which do not depart from the scope of the claims and their equivalents.

Claims (15)

A step of receiving a file containing mesh data; a step of decapsulating the above file; and A step of decrypting the above mesh data; comprising: The above file includes a bitstream including the mesh data and parameter information about the mesh data. How to decode mesh data. In the first paragraph, The above bitstream includes a sample stream header and one or more sample stream units, The above sample stream header contains information about one or more sample stream units, The one or more sample stream units include at least one of atlas data about the mesh data, information about a base mesh, displacement information encoded based on arithmetic coding, displacement information encoded based on a video codec, and attribute data encoded based on a video codec. How to decode mesh data. In the first paragraph, The above file comprises a bitstream including the mesh data and parameter information about the mesh data based on a single track, The sample entry of the above single track contains decoder configuration information, The above decoder configuration information includes information indicating the number of setup units, information indicating the type of the setup units, or at least one of the setup units. The above setup unit comprises a set of parameters relating to the mesh data. How to decode mesh data In the third paragraph, The parameter set of the above setup unit includes at least one of VPS (V-DMC parameter set), SPS (sequence parameter set), FPS (frame parameter set), DPS (displacement parameter set), GPS (geometry parameter set), or APS (attribute parameter set). How to decode mesh data. In the first paragraph, The above file comprises a bitstream including the mesh data and parameter information about the mesh data based on a single track, The sample of the above single track contains parameter information about the mesh data. How to decode mesh data In the third paragraph, The sample of the above single track includes one or more DMC (Dynamic Mesh Coding) units belonging to the composition time. How to decode mesh data. In clause 3 or clause 5, The above sample entry has at least one type of information among a first type indicating whether parameter information about the mesh data is included in the setup unit or a second type indicating whether parameter information about the mesh data is included in the sample. How to decode mesh data. A receiving unit for receiving a file containing mesh data; A file decapsulator for decapsulating the above file; and A decoder for decoding the above mesh data; The above file includes a bitstream including the mesh data and parameter information about the mesh data. Mesh data decoding device. Step of encoding mesh data; A step of encapsulating the above mesh data into a file; and comprising the steps of transmitting the above file; wherein the bitstream including the mesh data and parameter information about the mesh data is encapsulated into a single track of the file. How to encode mesh data. In Article 9, The above bitstream includes a sample stream header and one or more sample stream units, The above sample stream header contains information about one or more sample stream units, The one or more sample stream units include at least one of atlas data about the mesh data, information about a base mesh, displacement information encoded based on arithmetic coding, displacement information encoded based on a video codec, and attribute data encoded based on a video codec. How to encode mesh data. In Article 9, The above file comprises a bitstream including the mesh data and parameter information about the mesh data based on a single track, The sample entry of the above single track contains decoder configuration information, The above decoder configuration information includes information indicating the number of setup units, information indicating the type of the setup units, or at least one of the setup units. The above setup unit comprises a set of parameters relating to the mesh data. How to encode mesh data In Article 11, The parameter set of the above setup unit includes at least one of VPS (V-DMC parameter set), SPS (sequence parameter set), FPS (frame parameter set), DPS (displacement parameter set), GPS (geometry parameter set), or APS (attribute parameter set). How to encode mesh data. In Article 9, The above file comprises a bitstream including the mesh data and parameter information about the mesh data based on a single track, The sample of the above single track contains parameter information about the mesh data. How to encode mesh data In Article 11, The sample of the above single track contains one or more DMC (Dynamic Mesh Coding) units belonging to the composition time. How to encode mesh data. An encoder that encodes mesh data; A file encapsulator that encapsulates the above mesh data into a file; and A transmission unit for transmitting the above file; including; wherein the bitstream including the mesh data and parameter information about the mesh data is encapsulated into a single track of the file. Mesh data encoding device.

Images