RU2808631C2 - Maintaining tables for storing candidates for prediction of motion vector based on history - Google Patents

Abstract

FIELD: video encoding/decoding.

SUBSTANCE: video processing method is proposed, comprising: determining, during conversion between a first block of visual media data and a corresponding encoded representation of the visual media data, a first block encoded in a geometric partitioning mode; determining, based on at least one table storing history-based motion vector prediction (HMVP) candidates that include motion information based on previously encoded blocks, motion information of at least one sub-part of the first block; transforming the first block using certain motion information.

EFFECT: increase in the efficiency of encoding/decoding.

16 cl, 42 dwg, 4 tbl

Description

Field of technology to which the invention relates

This document relates to video and image encoding and decoding technologies.

State of the art

Digital video accounts for the largest share of bandwidth usage on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, the demand for bandwidth for digital video use is expected to continue to increase.

Disclosure of the invention

The disclosed technologies may be used in embodiments of a video or image decoder or encoder that utilize geometric partitions using history-based motion vector prediction (HMVP).

In one exemplary aspect, a method for processing video is disclosed. The method includes determining by the processor that the first video block is internally encoded or externally encoded without merging; determining by the processor a first subpart and a second subpart for the first video block based on determining that the first video block is internally encoded or externally encoded without merging, wherein one or both of the first subpart or the second subpart is non-rectangular and non-square portions of the first video block; and further processing the first video block using the first sub-part and the second sub-part.

In another exemplary aspect, a method of processing video includes determining by a processor that a first video block is internally encoded or externally encoded without merging; determining by the processor a first subpart and a second subpart for the first video block, wherein one or both of the first subpart or the second subpart are non-rectangular and non-square portions of the first video block; and further processing the first video block using the first sub-part and the second sub-part, wherein the at least one sub-part is not externally encoded with or without merging and uses the current image as a reference image.

In another exemplary aspect, a method of processing video includes determining by a processor that a first video block is internally encoded or externally encoded without merging; determining by the processor a first subpart and a second subpart for the first video block, wherein one or both of the first subpart or the second subpart are non-rectangular and non-square portions of the first video block; and further processing the first video block using the first sub-part and the second sub-part, wherein the additional processing of the first video block using the first sub-part and the second sub-part is based on inter- or intra-encoded information about non-contiguous spatial video blocks with respect to the first video block.

In another exemplary aspect, a video processing method includes determining that a first video block is encoded in a triangular portion mode (TPM) using the triangular prediction portion of the first video block, and that a second video block is encoded in a non-TPM using the non-triangular prediction portion of the second video block; further processing the first video block and the second video block using the stored HMVP candidates; and storing the HMVP candidates associated with the first video block and the second video block.

In another exemplary aspect, a method of processing video includes determining by a processor that a first video block includes prediction portions that are non-rectangular and non-square portions of the first video block; identification of the HMVP candidate; adding one or more motion candidates obtained from the HMVP candidate to a merge list associated with video blocks that include prediction portions that are non-rectangular and non-square; and further processing the first video block using the merge list.

In another exemplary aspect, a method of processing video includes: determining, during conversion between a current block of visual media data and a corresponding encoded representation of visual media data, a plurality of subparts for the current block; determining inter prediction information of the plurality of sub-parts; transforming the current block using inter prediction information of the plurality of sub-parts; and where the current block is intra-coded and at least one of the plurality of sub-parts is a non-rectangular and non-square sub-part.

In another exemplary aspect, a method of processing video includes: determining, during conversion between a current block of visual media data and a corresponding encoded representation of visual media data, a plurality of subparts for the current block; determining information about the movement of multiple sub-parts; transforming the current block using motion information of the plurality of sub-parts; and where the current block is not externally merge encoded and at least one of the plurality of subparts is a non-rectangular and non-square subpart.

In another exemplary aspect, a method of processing video includes: converting between a current block of visual media data and a corresponding encoded representation of the visual media data, wherein the current block is divided into multiple subparts according to a partitioning pattern, wherein the first subpart has a non-rectangular, non-square shape; processing the first subpart in internal coding mode; and processing the second sub-part in an externally encoding mode.

In another exemplary aspect, a method of processing video includes: converting between a current block of visual media data and a corresponding encoded representation of the visual media data, wherein the current block is divided into multiple subparts according to a partitioning pattern, wherein the first subpart has a non-rectangular, non-square shape; wherein at least one of the plurality of sub-parts is externally encoded with or without merging and uses the current image as a reference image.

In another exemplary aspect, a method for processing video includes: converting between a current block of visual media data and a corresponding encoded representation of the visual media data, wherein the current block is partitioned into multiple subparts according to a partitioning pattern, wherein the first prediction partition has a non-rectangular, non-square shape; and transforming using inter- or intra-encoded information about one or more spatial blocks.

In another exemplary aspect, a method of processing video includes: determining, during conversion between a first block of visual media data and a corresponding encoded representation of the visual media data, a first block encoded in a geometric partitioning mode; determining, based on at least one table storing history-based motion vector prediction (HMVP) candidates that include motion information based on previously encoded blocks, motion information of at least one sub-part of the first block; transforming the first block using certain motion information.

In another exemplary aspect, a method of processing video includes: determining, during conversion between a first block of visual media data and a corresponding encoded representation of the visual media data, a first block encoded in a geometric partitioning mode; determining motion information of at least one sub-part of the first block; transforming the first block using motion information of the at least one sub-part; wherein determining the motion information of the at least one sub-part comprises using at least one history-based motion vector prediction (HMVP) candidate that includes the motion information based on the previously encoded block to create a list of motion candidates and determine the motion information from list of movement candidates.

In another exemplary aspect, the above-described method may be implemented by a video encoder device that includes a processor.

In another exemplary aspect, the above-described method may be implemented by a video decoder device that includes a processor.

In yet another exemplary aspect, these methods may be embodied in instructions executable by a processor and stored on a computer-readable program medium.

These and other aspects are further described herein.

Brief description of drawings

Figure 1 is an example of a retrieval process for creating a list of merge candidates.

Figure 2 is an example of the positions of spatial merger candidates.

Figure 3 is an example of candidate pairs to be tested for redundancy of a spatial merge candidate.

Figure 4 is an example of the positions of the second PU partitions N×2N and 2N×N.

Figure 5 is an example of motion vector scaling for a temporary merge candidate.

Fig. 6 is an example of candidate positions for temporary merge candidates C0 and C1.

Figure 7 is an example of a merged bi-prediction fusion candidate.

FIG. 8 is an example of a process for obtaining motion vector predictions for candidates.

Figure 9 is an example of motion vector scaling for a spatial motion vector candidate.

Fig. 10 is an example of ATMVP motion prediction for a CU.

Fig. 11 is an example of one CU with four sub-blocks (AD) and its neighboring blocks (a-d).

Fig. 12 is an example of a block diagram of coding with different MV precision.

FIG. 13 shows examples of partitioning a CU into two triangular prediction units (two types of partitioning).

Fig. 14 is an example of the positions of neighboring blocks.

Fig. 15 is an example of a CU applying the 1st group of weighting coefficients.

Fig. 16 is an example of storing a motion vector.

FIG. 17 is an example of adjacent blocks (A and L) used for context selection when encoding a TPM flag.

FIG. 18 is an example of subblocks where OBMC is applied.

FIG. 19 is an example of adjacent samples used to obtain IC parameters.

Fig. 20 is an example of a simplified model of affine motion.

Fig. 21 is an example of an affine MVF for each subblock.

Fig. 22 is an example of (a) a 4-parameter affine model and (b) a 6-parameter affine model.

Fig. 23 is an example of MVP for AF_INTER.

Fig. 24 is an example of candidates for AF_MERGE.

Fig. 25 is an example of the candidate position for the affine merge mode.

Fig. 26 is an example of an optical flow trajectory.

Fig. 27 is an example of a BIO extension of a block w/o: a) access positions are outside the block; b) padding is used to avoid unnecessary memory accesses and computations.

FIG. 28 is an example of DMVR based on two-way template matching.

FIG. 29 is a block diagram of an example of a video processing apparatus.

FIG. 30 is a block diagram of an exemplary implementation of a video encoder.

Fig. 31 is a flowchart for an example of a video processing method.

Fig. 32 is a flowchart for an example of a video processing method.

Fig. 33 is a flowchart for an example of a video processing method.

Fig. 34 is a flowchart for an example of a video processing method.

Fig. 35 is a flowchart for an example of a video processing method.

Fig. 36 is a flowchart for an example of a video processing method.

Fig. 37 is a flowchart for an example of a video processing method.

Fig. 38 is a flowchart for an example of a video processing method.

Fig. 39 is a flowchart for an example of a video processing method.

Fig. 40 is a flowchart for an example of a video processing method.

Fig. 41 is a flowchart for an example of a video processing method.

Fig. 42 is a flowchart for an example of a video processing method.

Carrying out the invention

This document presents various technologies that can be used in a video or image bitstream decoder to improve the quality of reconstructed or decoded digital video or digital images. For brevity, the term "video" is used herein to refer to both a sequence of images (traditionally called a video) and individual images. In addition, the video encoder also allows these technologies to be implemented during the encoding process to recover decoded frames used for further encoding.

Section headings are used herein for ease of understanding and are not intended to limit the embodiments and technologies to their respective sections. Thus, embodiments from one section may be combined with embodiments from other sections.

1. Summary of the invention

This patent document relates to video encoding technologies. In particular, it relates to motion vector encoding under geometric partitioning conditions in video encoding. It can be applied to an existing video encoding standard, such as HEVC, or to a standard (Universal Video Coding) that is yet to be finalized. It may also be applicable to future video coding standards or video codecs.

2. Prior art

Video encoding standards have evolved primarily through the development of the well-known ITU-T and ISO/IEC standards. ITU-T released the H.261 and H.263 standards, ISO/IEC released the MPEG-1 and MPEG-4 Visual standards, and the two organizations jointly released the H.262/MPEG-2 Video, H.264/MPEG-4 standards advanced video coding (AVC) and H.265/HEVC. Starting with the H.262 standard, video coding standards are based on a hybrid video coding structure that uses temporal prediction plus transform coding. To explore future video encoding technologies beyond HEVC, the Joint Video Experts Team (JVET) was founded in 2015 with VCEG and MPEG. Since then, JVET has adopted many new techniques that are included in reference software called the Collaborative Exploration Model (JEM). In April 2018, the Joint Video Expert Team (JVET) was established between VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11 (MPEG) to work on a VVC standard aimed at reducing data rates by 50% compared to with HEVC.

FIG. 30 is a block diagram of an exemplary implementation of a video encoder. 30 shows that the encoder implementation has a built-in feedback path in which the video encoder also performs video decoding functionality (reconstructing a compressed representation of the video data for use in encoding the next video data).

2.1 Inter-prediction in HEVC/H.265

Each externally predicted PU has motion parameters for one or two reference image lists. Motion parameters include motion vector and reference image index. The use of one of the two reference picture lists may also be signaled using inter_pred_idc. Motion vectors can be explicitly coded as deltas with respect to the predictors.

When CUs are encoded in skip mode, one PU is associated with the CU, and there are no significant residual coefficients, uncoded motion vector delta, or reference frame index. The fusion mode is precisely determined, whereby the motion parameters for the current PU are obtained from neighboring PUs, including spatial and temporal candidates. Merge mode can be applied to any externally predicted PU, not just skip mode. An alternative to the fusion mode is the explicit transmission of motion parameters, where the motion vector (more precisely, the difference of motion vectors compared to the motion vector predictor), the corresponding reference frame index for each reference picture list, and the use of the reference picture list are signaled explicitly for each PU . In the present disclosure, such a mode is called enhanced motion vector prediction (AMVP).

When signaling indicates that one of two reference picture lists should be used, a PU is created from one block of samples. This is called "uni-prediction". Uni-prediction is available for both P-slices and B-slices.

When signaling indicates that both reference picture lists should be used, a PU is created from two sample blocks. This is referred to as "bi-prediction". Bi-prediction is only available for B-slices.

The following text provides details of the inter-prediction modes defined in HEVC. The description begins with the merge mode.

2.1.1. List of reference images

In HEVC, the term "external prediction" is used to refer to a prediction derived from data elements (eg, sample values or motion vectors) of reference pictures that are different from the currently decoded picture. As in H.264/AVC, a picture can be predicted from multiple reference pictures. The reference pictures that are used for inter-prediction are combined into one or more reference picture lists. The reference index indicates which of the reference pictures in the list should be used to create the prediction signal.

One reference picture list, list 0, is used for the P-slice, and two reference picture lists, list 0 and list 1, are used for the B-slice. It should be noted that the reference pictures included in the 0/1 list may be from past and future pictures in terms of capture/display order.

2.1.2 Merge mode

2.1.2.1 Obtaining candidates for merge mode

When a PU is predicted using the merge mode, an index pointing to an entry in the merge candidate list is parsed from the bitstream and used to extract motion information. The construction of this list is precisely defined in the HEVC standard and can be summarized according to the following sequence of steps:

Step 1: Obtaining Initial Candidates

Step 1.1: Obtaining spatial candidates

Step 1.2: Redundancy check for spatial candidates

Stage 1.3: Obtaining temporary candidates

Step 2: Adding Additional Candidates

Step 2.1: Generating bi-prediction candidates

Step 2.2: Inserting Zero Motion Candidates

These steps are also schematically depicted in FIG. 1. To obtain a spatial fusion candidate, a maximum of four fusion candidates are selected from candidates that are located at five different positions. To obtain a temporary merge candidate, a maximum of one merge candidate is selected from two candidates. Since the decoder assumes a constant number of candidates for each PU, additional candidates are produced when the number of candidates does not reach the maximum number of merge candidates (MaxNumMergeCand), which is signaled in the slice header. Since the number of candidates is constant, the index of the best merge candidate is encoded using truncated unary binarization (TU). If the CU size is 8, all PUs of the current CU share one merge candidate list, which is identical to the 2N×2N prediction block merge candidate list.

The following describes in detail the operations associated with the above steps.

Figure 1 shows an example of a retrieval process for constructing a list of merge candidates.

2.1.2.2. Retrieving spatial candidates

When obtaining spatial fusion candidates, a maximum of four fusion candidates are selected from the candidates located at the positions shown in FIG. 2. The order of receipt is as follows: A ₁ , B ₁ , B ₀ , A ₀ and B ₂ . Position B ₂ is considered only when any PU from position A ₁ , B ₁ , B ₀ , A ₀ is not available (for example, because it belongs to another slice or tile) or is internally encoded. Once a candidate at position A ₁ is added, the addition of the remaining candidates is subject to a redundancy check, which ensures that candidates with the same motion information are eliminated from the list, so coding efficiency is improved. To reduce computational complexity, not all possible candidate pairs are considered in the redundancy check mentioned. Instead, only pairs connected by the arrow shown in Fig. 3 are considered, and a candidate is added to the list only if the corresponding candidate used for redundancy testing does not have the same motion information. Another source of duplicate motion information is the "second PU" associated with partitions other than 2Nx2N. As an example, FIG. 4 shows the second PU for the N×2N and 2N×N cases, respectively. When the current PU is partitioned as N×2N, the candidate at position A ₁ is not considered for list construction. In some embodiments, adding this candidate may result in two prediction blocks having the same motion information, which is redundant to having only one PU in a coding block. Similarly, position B ₁ is not taken into account when the current PU is divided as 2N×N. In fact, adding this candidate will result in two prediction units having the same motion information, which is redundant to have only one PU in a coding unit. Likewise, position B ₁ is not considered when the current PU is divided as 2N×N.

Figure 2 shows an example of the positions of spatial merge candidates.

Figure 3 shows an example of candidate pairs to be tested for redundancy of a spatial merge candidate.

Figure 4 shows an example of positions for the second PU from N×2N and 2N×N partitions.

2.1.2.3. Getting temporary candidates

At this stage, only one candidate is added to the list. Specifically, when obtaining this temporary fusion candidate, a scaled motion vector is obtained based on the fused PU belonging to the image that has the smallest POC difference with the current image in the given reference image list. The list of reference pictures that should be used to obtain the combined PU is explicitly signaled in the slice header. A scaled motion vector for the temporary fusion candidate is obtained, which is illustrated by the dashed line in FIG. 5, and which is scaled relative to the motion vector of the fused PU using the POC distances, tb and td, where tb is defined as the difference in POC between the reference image of the current image and the current image, and td is defined as the POC difference between the reference image of the fusion image and the fusion image. The reference image index of the temporary merge candidate is set to zero. The practical implementation of the scaling process is described in the HEVC specification. For a B-slice, two motion vectors, one for reference picture list 0 and the other for reference picture list 1, are obtained and combined to generate a bi-prediction fusion candidate.

Figure 5 shows an example of motion vector scaling for a temporal fusion candidate.

In the combined PU (Y) belonging to the reference frame, positions for the temporary candidate are selected between candidates C ₀ and C ₁ , as shown in FIG. 6. If the PU at position _C0 is not available, is internally encoded, or is outside the current CTU, position _C1 is used. In other cases, position C ₀ is used when obtaining a temporary merge candidate.

FIG. 6 shows an example of candidate positions for temporary merge candidates _C0 and _C1 .

2.1.2.4. Inserting Additional Candidates

In addition to spatiotemporal fusion candidates, there are two additional types of fusion candidates: a bi-predictive fusion candidate and a null fusion candidate. Bi-predictive fusion candidates are generated using spatio-temporal fusion candidates. The merged bidirectional prediction merge candidate is used for B-slice only. Merged bi-prediction candidates are generated by combining motion parameters from a first reference picture list of one source candidate with motion parameters from a second reference picture list of another source candidate. If the two tuples provide different motion hypotheses, they will form a new bi-prediction candidate. As an example, FIG. 7 shows a case where two candidates in the original list (left) that have mvL0 and refIdxL0 or mvL1 and refIdxL1 are used to create a combined bi-prediction fusion candidate added to the final list (right). There are many rules regarding the combinations that are considered to generate these additional fusion candidates.

Figure 7 shows an example of a merged bi-prediction fusion candidate.

Null motion candidates are inserted to fill the remaining entries in the merge candidate list and hence reach the maximum capacity of MaxNumMergeCand. These candidates have a spatial offset of zero and a reference image index that starts at zero and is incremented each time a new zero motion candidate is added to the list. The number of reference frames used by these candidates is one and two for unidirectional prediction (uni-prediction) and bidirectional prediction (bi-prediction), respectively. Finally, no redundancy check is performed on these candidates.

2.1.2.5. Motion estimation areas for parallel processing

To speed up the encoding process, motion estimation can be performed in parallel, resulting in motion vectors for all prediction units within a given region being obtained simultaneously. Receiving fusion candidates from a spatial neighborhood can interfere with parallel processing since one prediction unit cannot obtain motion parameters from a neighboring PU until its associated motion estimation is completed. To mitigate the trade-off between encoding efficiency and processing latency, HEVC defines a motion estimation region (MER), the size of which is signaled in the image parameter set using the "log2_parallel_merge_level_minus2" syntax element. Once the MER is determined, merge candidates that fall in the same region are marked as unavailable and hence are not considered when constructing the list.

2.1.3. AMVP

AMVP uses spatio-temporal correlation of the motion vector with neighboring PUs, which is used to communicate motion parameters explicitly. For each reference image list, a motion vector candidate list is created by first checking the availability of the left, top temporal neighboring PU positions, removing redundant candidates, and adding a null vector to make a constant-length candidate list. The encoder can then select the best predictor from the list of candidates and pass the corresponding index indicating the selected candidate. Similar to merge index signaling, the index of the best motion vector candidate is encoded using a truncated unary code. The maximum value that must be encoded in this case is 2 (see FIG. 8). The following sections provide details about the process of obtaining a motion vector prediction candidate.

2.1.3.1 Receiving AMVP candidates

FIG. 8 summarizes the process of obtaining a motion vector prediction candidate.

In motion vector prediction, two types of motion vector candidates are considered: a spatial motion vector candidate and a temporal motion vector candidate. To obtain motion vector candidates, two motion vector candidates are ultimately obtained based on the motion vectors of each PU located at five different positions, as previously shown in Fig. 2.

To obtain temporary motion vector candidates, one motion vector candidate is selected from two candidates that are obtained based on two different aligned positions. After the first list of spatiotemporal candidates is compiled, duplicate motion vector candidates in the list are removed. If the number of potential candidates is greater than two, motion vector candidates whose reference picture index in the associated reference picture list is greater than 1 are removed from the list. If the number of spatiotemporal motion vector candidates is less than two, additional zero motion vector candidates are added to the list.

2.1.3.2. Spatial candidates of motion vectors

When obtaining spatial motion vector candidates, a maximum of two candidates are considered out of five potential candidates, which are obtained from PUs located at positions as previously shown in FIG. 2, these positions being the same as the motion fusion positions. The acquisition order for the left side of the current PU is defined as A0, A1, scaled A0, scaled A1. The acquisition order for the above side of the current PU is defined as B0, B1, B2, scaled B0, scaled B1, scaled B2. Thus, for each side, there are four cases that can be used as motion vector candidates: two cases where spatial scaling is not required and two cases where spatial scaling is used. The four different cases are summarized as follows:

In the absence of spatial scaling

(1) Same reference picture list and same reference picture index (same POC)

(2) Different list of reference pictures, but the same reference picture (same POC)

In the presence of spatial scaling

(3) Same reference picture list, but different reference picture (different POC)

(4) Another reference picture list, and another reference picture (different POC)

Cases that lack spatial scaling are tested first, followed by cases that allow spatial scaling. Spatial scaling is considered when the POC differs between the reference image of a neighboring PU and the reference image of the current PU, regardless of the list of reference images. If all left candidate PUs are unavailable or internally encoded, scaling is allowed for the above motion vector to facilitate parallel acquisition of left and top candidate MVs. In other cases, spatial scaling for the above motion vector is not allowed.

FIG. 9 shows an example of motion vector scaling for a spatial motion vector candidate.

In the spatial scaling process, the motion vector of the neighboring PU is scaled in the same way as in temporal scaling, as shown in Fig. 9 . One difference is that the list of reference images and the index of the current PU are given as input, and the actual scaling process is the same as in temporal scaling.

2.1.3.3. Temporary Motion Vector Candidates

Apart from the acquisition index reference image, all processes for obtaining temporal fusion candidates are the same as for obtaining spatial motion vector candidates (see FIG. 6). The reference picture index is signaled to the decoder.

2.2. New ways of external prediction in JEM

2.2.1. Sub-CU Based Motion Vector Prediction

In JEM, when using QTBT, each CU can have at most one set of motion parameters for each prediction direction. Two methods of motion vector prediction at the sub-CU level are considered in the encoder by dividing the large CU into sub-CUs and obtaining motion information for all sub-CUs of the large CU. The Alternate Temporal Motion Vector Prediction (ATMVP) method allows each CU to extract multiple sets of motion information from multiple blocks smaller than the current CU in a co-located reference picture. In the spatio-temporal motion vector prediction (STMVP) method, sub-CU motion vectors are obtained recursively using a temporal motion vector predictor and a neighboring motion vector.

To maintain a more accurate motion field for sub-CU motion prediction, you can disable motion compression for reference frames.

FIG. 10 shows an example of ATMVP motion prediction for a CU.

2.2.1.1. Alternative prediction of temporal motion vector

In the alternative temporal motion vector prediction (ATMVP) method, the temporal motion vector prediction (TMVP) of motion vectors is modified by extracting multiple sets of motion information (including motion vectors and reference indices) from blocks smaller in size than the current CU. Sub-CUs are N×N square blocks (N is set to 4 by default).

ATMVP predicts sub-CU motion vectors in CUs in two steps. In the first step, the corresponding block in the reference frame is identified using a so-called time vector. The reference image is called the motion source image. The second step divides the current CU into sub-CUs and obtains the motion vectors as well as the reference indices of each sub-CU from the block corresponding to each sub-CU.

In the first stage, the reference image and the corresponding block are determined using the motion information of the spatial neighboring blocks of the current CU. To avoid the process of rescanning neighboring blocks, the first merge candidate in the current CU's merge candidate list is used. The first available motion vector, as well as its associated reference index, is set to the time vector, and the index is set to the motion source image. Thus, in ATMVP, the corresponding block can be more accurately identified compared to TMVP, where the corresponding block (sometimes called a co-located block) is always in the lower right or center position with respect to the current CU.

In the second step, the corresponding sub-CU is identified with a time vector in the motion source image by adding the time vector to the coordinate of the current CU. For each sub-CU, the motion information of its corresponding block (the smallest motion grid that spans the center sample) is used to obtain the motion information for the sub-CU. Once the motion information of the corresponding N×N block is identified, it is converted into motion vectors and reference indices of the current sub-CU in the same way as TMVP in HEVC, and motion scaling and other procedures are applied. For example, the decoder checks whether a low latency condition is satisfied (that is, whether the POC values of all reference pictures of the current picture are less than the POC values of the current picture) and optionally uses a motion vector MV _x (a motion vector corresponding to a list of X reference pictures) to make a prediction motion vector MV _y (with X equal to 0 or 1 and Y equal to 1-X) for each sub-CU.

2.2.1.2. Spatio-temporal motion vector prediction (STMVP)

In this method, the sub-CU motion vectors are obtained recursively by following the raster scanning order. Figure 11 illustrates this concept. Consider a CU of size 8x8 that contains four sub-CUs A, B, C and D of size 4x4. Adjacent 4x4 blocks in the current frame are labeled a, b, c, and d.

Retrieving motion for sub-CU A begins with identifying its two spatial neighbors. The first neighbor is an N×N block above the sub-CU A (block c). If this block c is not available or internally encoded, other N×N blocks above sub-CU A are checked (from left to right, starting from block c). The second neighbor is the block to the left of sub-CU A (block b). If block b is not available or internally encoded, other blocks to the left of sub-CU A are checked (from top to bottom, starting with block b). The motion information obtained from neighboring blocks for each list is scaled according to the first reference frame for that list. The temporal motion vector predictor (TMVP) of subblock A is then obtained by following the same TMVP derivation procedure as defined in HEVC. The motion information of the aligned block at location D is extracted and scaled accordingly. Finally, after motion information is extracted and scaled, all available motion vectors (up to 3) are averaged separately for each reference image list. The averaged motion vector is assigned as the motion vector of the current sub-CU.

2.2.1.3. Sub-CU motion prediction mode signaling

The sub-CU modes are activated as additional merge candidates, and no additional syntax element is required to signal the modes. Two additional merge candidates are added to each CU's merge candidate list to represent the ATMVP mode and the STMVP mode. If the sequence parameter set specifies that ATMVP and STMVP are enabled, then up to seven merge candidates are used. The logic for encoding additional merge candidates is the same as for merge candidates in HM, which means that for each CU in a P or B slice, two more RD checks are required for two additional merge candidates.

In JEM, all merge index bins represent CABAC encoded context. In HEVC, only the first bin is context-encoded, and the remaining bins are context-bypassed.

2.2.2. Adaptive Motion Vector Difference Resolution

In HEVC, motion vector differences (MVD) (between the motion vector and the PU's predicted motion vector) are transmitted in units of one-quarter luminance sample when use_integer_mv_flag is 0 in the slice header. JEM introduces locally adaptive motion vector resolution (LAMVR). In JEM, MVD can be encoded in units of one-quarter luminance samples, integer luminance samples, or four luminance samples. MVD resolution control is performed at the coding unit (CU) level, and MVD resolution flags are conditionally signaled for each CU that has at least one non-zero MVD component.

For a CU that has at least one non-zero MVD component, the first flag is signaled to indicate whether the MV precision of one-quarter luminance sampling is used in the CU. When the first flag (equal to 1) indicates that the MV precision of one quarter luminance sample is not used, another flag is signaled to indicate whether the MV precision of integer luminance samples or the MV precision of four luminance samples is used.

When the first MVD resolution flag of a CU is zero or not encoded for the CU (meaning all MVDs in the CU are zero), the MV resolution of one quarter brightness is used for the CU. When the CU uses the MV precision of an integer luma sample or the MV precision of four luma samples, the MVPs in the AMVP candidate list for the CU are rounded to the corresponding precision.

In the encoder, CU-level RD checks are used to determine which MVD resolution should be used for the CU. That is, the CU-level RD check is performed three times for each MVD resolution. To increase encoder speed, JEM uses the following encoding schemes.

During the RD CU test with normal MVD resolution of one quarter luminance, motion information of the current CU (integer luminance sampling precision) is stored. The stored motion information (after rounding) is used as a starting point for further low-range motion vector refinement during RD inspection for the same CU with integer luminance sampling and MVD resolution with 4 luminance samples, so the time-consuming motion estimation process is not duplicated three times.

The RD check of a CU with MVD resolution with 4 brightness samples is called conditionally. For a CU, when the MVD resolution of an integer luminance sample with RD overhead is much larger than the MVD resolution of one-quarter luminance sample, the RD check of the MVD resolution of 4 luminance samples for the CU is skipped.

The encoding process is shown in Fig. 12. First, the 1/4 pel MV is tested, the RD cost is calculated and denoted as RDCost0, then the integer MV is tested, and the RD cost is denoted as RDCost1. If RDCost1 < th * RDCost0 (where th is a positive value), then MV 4-pel is checked; otherwise MV 4-pel is skipped. Basically, traffic information, RD costs, etc. are already known for MV 1/4 pel when checking integer MV or MV 4-pel, which can be reused to speed up the encoding process of integer MV or MV 4 pel.

2.2.3. Triangle Prediction Mode

The concept of Triangular Prediction Mode (TPM) is to introduce a new triangular partition for motion compensated prediction. As shown in FIG. 13, it splits the CU into two triangular prediction units, in the diagonal or reverse diagonal direction. Each triangular shape prediction module in the CU is interpredicted using its own uni-prediction motion vector and reference frame index, which are obtained from the uni-prediction candidate list. The adaptive weighting process is performed with respect to the diagonal edge after predicting the triangular prediction units. The conversion and quantization process is then applied to the entire CU. It should be noted that this mode only applies to skip and merge modes.

2.2.3.1 List of uni-prediction candidates for TPM

The uni-prediction candidate list consists of five uni-prediction motion vector candidates. It is derived from seven adjacent blocks, including five spatial adjacent blocks (1-5) and two temporal adjacent blocks (6-7), as shown in FIG. 14. The motion vectors of seven neighboring blocks are collected and placed into a uni-prediction candidate list according to the order of the uni-prediction motion vectors, the L0 motion vector of the bi-prediction motion vectors, the L1 motion vector of the bi-prediction motion vectors, and the average motion vector of the L0 motion vectors. and L1 bi-prediction motion vectors. If the number of candidates is less than five, a zero motion vector is added to the list.

More specifically, the following are the steps:

Obtain motion candidates from A1, B1, B0, A0, B2, Col and Col2 (corresponding to block 1-7 in Fig. 14) without any truncation operations.

Set variable numCurrMergeCand = 0

For each motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2, and numCurrMergeCand is less than 5, if the motion candidate is a uni-prediction (either from list 0 or list 1), it is added to the list merging with numCurrMergeCand increased by 1. Such added motion candidates are called "originally uni-predicted candidates".

Full truncation is applied.

For each motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2, and numCurrMergeCand is less than 5, if the motion candidate is a bi-prediction, the motion information from list 0 is added to the merge list (i.e., modified by uni-prediction from list 0), and numCurrMergeCand is incremented by 1. Such added motion candidates are called "predicted candidate from truncated list 0".

Full truncation is applied.

For each motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2, and numCurrMergeCand is less than 5, if the motion candidate is a bi-prediction, the motion information from list 1 is added to the merge list (i.e., modified by uni-prediction from list 1), and numCurrMergeCand is incremented by 1. Such added motion candidates are called "predicted candidate from truncated list 1".

Full truncation is applied.

For each motion candidate derived from A1, B1, B0, A0, B2, Col and Col2, and numCurrMergeCand is less than 5, if the motion candidate is a bi-prediction,

If the QP slice of the reference picture from list 0 is smaller than the QP slice of the reference picture from list 1, the motion information from list 1 is first scaled to the reference picture list 0, and the average of the two MVs (one is obtained from the original list 0 and the other is the scaled MV from list 1) is added to the merge list, which is the average of the uni-prediction from the motion candidate from list 0 and numCurrMergeCand increased by 1.

Otherwise, the motion information from list 0 is first scaled to the reference image from list 1, and the average of the two MVs (one is obtained from the original list 1, and the other is the scaled MV from list 0) is added to the merge list, which is the averaged uni -prediction from a motion candidate from list 1, and numCurrMergeCand increased by 1.

Full truncation is applied.

If numCurrMergeCand is less than 5, zero motion vector candidates are added.

2.2.3.2. Adaptive Weighing Process

After predicting each triangular prediction unit, an adaptive weighting process is applied to the diagonal edge between two triangular prediction units to obtain the final prediction for the entire CU. The two groups of weighting coefficients are defined as follows:

1st group of weighting coefficients: {7/8, 6/8, 4/8, 2/8, 1/8} and {7/8, 4/8, 1/8} are used for luma and chrominance samples, respectively ;

2nd group of weighting coefficients: {7/8, 6/8, 5/8, 4/8, 3/8, 2/8, 1/8} and {6/8, 4/8, 2/8} used for luma and chrominance samples, respectively.

A group of weighting coefficients is selected based on a comparison of the motion vectors of two triangular prediction blocks. The 2nd group of weight coefficients is used when the reference images of two triangular prediction blocks are different from each other, or their motion vector difference exceeds 16 pixels. Otherwise, the first group of weighting coefficients is used. An example is shown in Fig. 15.

2.2.3.3. Motion vector storage

The motion vectors (Mv1 and Mv2 in FIG. 16) of the triangular prediction blocks are stored in 4x4 grids. For each 4x4 grid, a motion vector is stored with either uni-prediction or bi-prediction, depending on the position of the 4x4 grid in the CU. As shown in FIG. 16, the uni-prediction motion vector, Mv1 or Mv2, is stored for a 4x4 grid located in the unweighted region (ie, not on the diagonal edge). On the other hand, the bi-predictive motion vector is stored for a 4x4 grid located in the weighted region. The bidirectional predicted motion vector is obtained from Mv1 and Mv2 according to the following rules:

In the case where Mv1 and Mv2 have a motion vector from different directions (L0 or L1), Mv1 and Mv2 are simply combined to form a bi-predictive motion vector.

In the case where both Mv1 and Mv2 originate from the same direction L0 (or L1),

If the reference picture Mv2 is the same as the picture in the reference picture list L1 (or L0), Mv2 is scaled to the picture Mv1. Mv1 and scaled Mv2 are combined to form a bi-predictive motion vector.

If the reference picture Mv1 is the same as the picture in the reference picture list L1 (or L0), Mv1 is scaled to the picture Mv2. The scaled Mv1 and Mv2 are combined to form a bi-predictive motion vector.

Otherwise, only Mv1 is stored for the weighted region.

2.2.3.4. Triangle Prediction Mode (TPM) signaling

First, a one-bit flag may be signaled to indicate whether the TPM is in use. Thereafter, the indications of the two partition patterns (as shown in FIG. 13) and the selected merge indices for each of the two partitions are further signaled.

2.2.3.4.1 TPM flag signaling

Let's denote the width and height of one brightness block by W and H, respectively. If W*H < 64, triangular prediction mode is disabled.

When one block is encoded in affine mode, the triangular prediction mode is also disabled.

When one block is encoded in merge mode, a one-bit flag may be signaled to indicate whether triangular prediction mode is enabled or disabled for the block.

The flag is encoded with 3 contexts based on the following equation:

Index Ctx = ((available left block L && L encoded with TPM?) 1:0)

+((available above block A && A is encoded with TPM?) 1:0);

2.2.3.4.2. Signaling the indication of two partition patterns (as shown in FIG. 13) and selected merge indices for each of the two partitions

It should be noted that the partition patterns, merging indices of two partitions are encoded together. The limitation is that two partitions cannot use the same reference index. So there are 2 (partition patterns) * N (maximum number of merge candidates) * (N-1) possible options, where N is set to 5. One hint is encoded, a mapping is performed between partition types, and two merge indices are obtained from the array. defined below:

const uint8_t g_TriangleCombination [TRIANGLE_MAX_NUM_CANDS] [3] = {

{0, 1, 0}, {1, 0, 1}, {1, 0, 2}, {0, 0, 1}, {0, 2, 0},

{1, 0, 3}, {1, 0, 4}, {1, 1, 0}, {0, 3, 0}, {0, 4, 0},

{0, 0, 2}, {0, 1, 2}, {1, 1, 2}, {0, 0, 4}, {0, 0, 3},

{0, 1, 3}, {0, 1, 4}, {1, 1, 4}, {1, 1, 3}, {1, 2, 1},

{1, 2, 0}, {0, 2, 1}, {0, 4, 3}, {1, 3, 0}, {1, 3, 2},

{1, 3, 4}, {1, 4, 0}, {1, 3, 1}, {1, 2, 3}, {1, 4, 1},

{0, 4, 1}, {0, 2, 3}, {1, 4, 2}, {0, 3, 2}, {1, 4, 3},

{0, 3, 1}, {0, 2, 4}, {1, 2, 4}, {0, 4, 2}, {0, 3, 4}};

Split type (45 degrees or 135 degrees) = g_TriangleCombination[signaled hint][0];

Candidate A merge index = g_TriangleCombination[signaled hint][1];

Candidate B merge index = g_TriangleCombination[signaled hint][2].

Once two motion candidates A and B are received, the motion information of the two partitions (PU1 and PU2) can be established from either A or B. Whether PU1 uses the motion information of merge candidate A or B depends on the prediction directions of the two candidates of the movement. Table 1 presents the relationships between the two obtained motion candidates A and B for two partitions.

Table 1. Obtaining motion information during partitioning from the resulting two merge candidates (A, B)

Predicted direction A Predicted direction B PU1 traffic information Traffic information PU2 L0 L0 A (L0) B (L0) L1 L1 B (L1) A (L1) L0 L1 A (L0) B (L1) L1 L0 B (L0) A (L1)

2.2.3.4.3. Entropy encoding of an indication (denoted merge_triangle_idx)

merge_triangle_idx is in the range [0, 39] inclusive. A K-th order exponential Golomb (EG) code is used to binarize merge_triangle_idx when K is set to 1.

EG kth order

To encode larger numbers with fewer bits (at the expense of using more bits to encode smaller numbers), this can be generalized by using a non-negative integer parameter k. To encode a non-negative integer x in the kth order exponential Golomb code:

encode using the 0th order exponential Golomb code described above, then

encode x mod 2 ^k in binary

Table 2: Examples of coding using k-th order exponential Golomb code

x k=0 k=1 k=2 x k=0 k=1 k=2 0 1 10 100 10 0001011 001100 01110 1 010 eleven 101 eleven 0001100 001101 01111 2 011 0100 110 12 0001101 001110 0010000 3 00100 0101 111 13 0001110 001111 0010001 4 00101 0110 01000 14 0001111 00010000 0010010 5 00110 0111 01001 15 000010000 00010001 0010011 6 00111 001000 01010 16 000010001 00010010 0010100 7 0001000 001001 01011 17 000010010 00010011 0010101 8 0001001 001010 01100 18 000010011 00010100 0010110 9 0001010 001011 01101 19 000010100 00010101 0010111

2.2.4. Compensating for motion of overlapping blocks

Overlapping block motion compensation (OBMC) was previously used in H.263. In JEM, unlike H.263, OBMC can be turned on and off using CU-level syntax. When OBMC is used in JEM, OBMC is performed on all motion compensation (MC) block boundaries except the right and bottom CU boundaries. Moreover, it applies to both luma and chrominance components. In JEM, the MC block corresponds to the encoding block. When a CU is encoded in sub-CU mode (including sub-CU merging, affine mode, and FRUC mode), each CU sub-block is an MC block. To handle CU boundaries in a uniform manner, OBMC is performed at the sub-block level for all MC block boundaries, where the sub-block size is set to 4x4, as shown in FIG. 18.

When OBMC is applied to the current sub-block, in addition to the current motion vectors, the motion vectors of the four associated neighboring sub-blocks, if available and not identical to the current motion vector, are also used to obtain the prediction block for the current sub-block. These multiple prediction blocks based on multiple motion vectors are combined to produce a final prediction signal of the current subblock.

A prediction block based on the motion vectors of a neighboring sub-block is denoted as P _N , where N indicates an index for the adjacent top, bottom, left and right sub-blocks, and a prediction block based on the motion vectors of the current sub-block is denoted as P _C . When P _N is based on the motion information of a neighboring subblock that contains the same motion information as the current subblock, OBMC is not performed from P _N . Otherwise, each sample P _N is added to the same sample in P _C , that is, four rows/columns of P _N are added to P _C . Weights {1/4, 1/8, 1/16, 1/32} are used for P _N , and weights {3/4, 7/8, 15/16, 31/32} are used for P _C . The exception is small MC blocks (that is, when the encoding block height or width is 4, or the CU is encoded in sub-CU mode), for which only two rows/columns P _N are added to P _C . In this case, weights {1/4, 1/8} are used for P _N and weights {3/4, 7/8} are used for P _C . For P _N generated based on the motion vectors of a vertically (horizontally) adjacent subblock, samples in the same row (column) of P _N are added to P _C with the same weighting factor.

In JEM, for a CU with a size less than or equal to 256 luma samples, a CU level flag is signaled to indicate whether or not OBMC is applied for the current CU. For CUs larger than 256 luma samples or not AMVP encoded, OBMC is used by default. In the encoder, when OBMC is applied to a CU, its influence is taken into account in the motion estimation stage. The prediction signal generated by the OBMC using the motion information of the top neighbor block and the left neighbor block is used to compensate the top and left boundaries of the original signal of the current CU, and then the normal motion estimation process is applied.

2.2.5. Local illumination compensation

Local illuminance compensation (LIC) is based on a linear model for illuminance variations using a scaling factor a and an offset b, and is turned on or off adaptively for each coding unit (CU) encoded in externally encoded mode.

When LIC is applied to a CU, the least squares error method is used to obtain the parameters a and b using the neighboring samples of the current CU and their corresponding reference samples. More specifically, as shown in FIG. 19, the reference picture uses subsampled (2:1 downsampled) neighboring CU samples and corresponding samples (identified by motion information of the current CU or sub-CU). The IC parameters are obtained and applied separately for each prediction direction.

When a CU is encoded in merge mode, the LIC flag is copied from neighboring blocks in a similar manner to copying motion information in merge mode; otherwise, the LIC flag is signaled to the CU to indicate whether LIC is applied or not.

When LIC is enabled for an image, an additional RD check at the CU level is required to determine whether or not LIC is applied to the CU. When LIC is enabled for CU, mean-remote sum of absolute difference (MR-SAD) and mean-remote sum of absolute difference (MR) are used instead of SAD and SATD for integer pixel motion search and fractional pixel motion search, respectively. -SATD).

To reduce coding complexity, JEM uses the following coding scheme.

LIC is disabled for the entire image when there is no obvious change in illumination between the current image and its reference images. To determine this situation, the encoder calculates histograms of the current image and each reference image of the current image. If the histogram difference between the current image and each reference image of the current image is less than a specified threshold, LIC is disabled for the current image; otherwise, LIC is enabled for the current image.

2.2.6. Affine motion compensation prediction

In HEVC, only the translational motion model is used for motion compensation prediction (MCP). In the real world, there are many types of motion, such as zoom in/out, rotation, perspective motion, and other types of uneven motion. JEM uses simplified motion compensation prediction with an affine transformation. As shown in Fig. 20, the affine motion field of a block is described by two motion vectors of the control point.

The vector field (MVF) of block motion is described by the following equation:

(1-a)

For a 6-parameter affine candidate

(1-b)

where (v _0x , v _0y ) is the movement vector of the control point in the upper left corner, (v _1x , v _1y ) is the movement vector of the control point in the upper right corner, and (v _2x , v _2y ) is the movement vector of the control point in the lower left corner, (x, y) represents the coordinate of the representative point relative to the top left sample in the current block. In VTM, a representative point is defined as the center position of a subblock, for example, when the coordinate of the upper left corner of a subblock relative to the upper left sample in the current block is (xs, ys), the coordinate of the representative point is defined as (xs+2, ys+2).

To further simplify motion compensation prediction, subblock-based affine transformation prediction is applied. The sub-block size is obtained as in equation (2), where MvPre is the motion vector fraction precision (1/16 in JEM), (v _2x , v _2y ) is the motion vector of the bottom left control point calculated according to equation (1).

(2)

After obtaining M and N from equation (2), they must be adjusted downward if necessary to make them divisors of w and h, respectively.

To obtain the motion vector of each M×N sub-block, the motion vector of the center sample of each sub-block, as shown in FIG. 21, is calculated according to Equation (1) and rounded to a precision of 1/16. Then, the motion compensation interpolation filters mentioned herein are applied to generate a prediction of each sub-block with the resulting motion vector.

After MCP, the high-precision motion vector of each sub-block is rounded and stored with the same precision as the normal motion vector.

2.2.6.1. AF_INTER mode

There are two affine motion modes in JEM: AF_INTER mode and AF_MERGE mode. For CUs with width and height greater than 8, the AF_INTER mode can be used. An affine flag is signaled in the bitstream at the CU level to indicate whether the AF_INTER mode is used. In this mode, a list of candidates with a pair of motion vectors created using adjacent blocks. As shown in Fig. 23, is selected from the motion vectors of block A, B, or C. The motion vector from the adjacent block is scaled according to the list of reference pictures and the relationship between the POC of the reference picture for the neighboring block, the POC of the reference picture for the current CU, and the POC of the current CU. Selection approach from the adjacent block D and E is similar. If the number of candidate lists is less than 2, the list is supplemented with a pair of motion vectors constructed by duplicating each of the AMVP candidates. When the candidate list is larger than 2, the candidates are first sorted according to the compatibility of adjacent motion vectors (e.g., based on the similarity of two motion vectors in a candidate pair), and only the first two candidates are retained. The RD overhead test is used to determine which candidate of a pair of motion vectors is selected as the reference motion vector prediction (CPMVP) of the current CU. An index indicating the position of the CPMVP in the candidate list is signaled in the bit stream. Once the CPMVP of the current affine CU is determined, the affine motion estimate is applied and the control point motion vector (CPMV) is determined. The difference between CPMV and CPMVP is then signaled in the bitstream.

In AF_INTER mode, when 4-/6-parameter affine mode is used, 2/3 control points are required, and therefore 2/3 MVDs must be encoded for these control points, as shown in FIG. 22. It is proposed to obtain MV as follows, that is, mvd ₁ and mvd ₂ are predicted from mvd0.

Where , mvd _i and mv ₁ represent the predicted motion vector, the difference of motion vectors, and the motion vector of the top left pixel (i = 0), top right pixel (i = 1), or bottom left pixel (i = 2.), respectively, as shown in Fig. 22(b). It should be noted that the addition of two motion vectors (e.g. mvA(xA, yA) and mvB(xB, yB)) is equal to the addition of the two components separately, that is, newMV = mvA + mvB, and the two newMV components are set to (xA + xB ) and (yA + yB), respectively.

2.2.6.2. AF_MERGE mode

When the CU is applied in AF_MERGE mode, it obtains the first affine-mode encoded block from the valid neighboring reconstructed blocks. As shown in FIG. 24a, the selection order for a candidate block is left, top, top right, bottom left, top left. If the adjacent lower left block A is encoded in affine mode, as shown in Fig. 24b, the motion vectors are calculated , And in the upper left corner, upper right corner and lower left corner of the CU block that contains block A. And the motion vector in the upper left corner of the current CU is calculated according to , And . Secondly, the vector is calculated movement in the upper right corner of the current CU.

Once the CPMV of the current CU is received And , according to Equation (1) of the simplified affine motion model, the MVF of the current CU is generated. To determine whether the current CU is AF_MERGE encoded, an affine flag is signaled in the bitstream when there is at least one adjacent block encoded in affine mode.

The list of affine fusion candidates is created in the following steps:

Insert legacy affine candidates

A legacy affine candidate means that the candidate is derived from the affine motion model of its actual neighboring affine coded block. In general, as shown in FIG. 25, the scanning order of candidate positions is A1, B1, B0, A0 and B2.

Once a candidate is received, a full truncation process is performed to check whether the same candidate has been inserted into the list. If an identical candidate exists, the resulting candidate is discarded.

Insert constructed affine candidates

If the number of candidates in the affine fusion candidate list is less than MaxNumAffineCand (set to 5 in this message), the constructed affine candidates are inserted into the candidate list. The phrase “constructed affine candidate” means that the candidate is created by combining the motion information of the neighbors of each reference point.

Motion information for reference points is first obtained from the specified spatial neighbors and temporal neighbors shown in FIG. 25. CPk (k = 1, 2, 3, 4) represents the kth control point. A0, A1, A2, B0, B1, B2 and B3 – spatial positions for CPk prediction (k = 1, 2, 3); T is the time position for CP4 prediction.

The coordinates of CP1, CP2, CP3 and CP4 are (0, 0), (W, 0), (H, 0) and (W, H), respectively, where W and H are the width and height of the current block.

Information about the movement of each control point is obtained in accordance with the following order of priority:

For CP1, the check priority is B2->B3->A2. B2 is used if available. Otherwise, if B2 is not available, B3 is used. If B2 and B3 are both unavailable, A2 is used. If all three candidates are unavailable, CP1 traffic information cannot be obtained.

For CP2 the test priority is B1->B0.

For CP3, the test priority is A1->A0.

For CP4, T is used.

Secondly, combinations of checkpoints are used to construct an affine merge candidate.

Information about the motion of the three control points is necessary to construct a 6-parameter affine candidate. The three control points can be selected from one of the following four combinations ({CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}). The combinations {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4} will be converted into a 6-parameter motion model represented by the upper left, upper right and lower left control points.

Information about the motion of two control points is necessary to construct a 4-parameter affine candidate. The two control points can be selected from one of the following six combinations ({CP1, CP4}, {CP2, CP3}, {CP1, CP2}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4}). The combinations {CP1, CP4}, {CP2, CP3}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4} will be converted into a 4-parameter motion model represented by the top left and top right control points.

Combinations of constructed affine candidates are inserted into the candidate list in the following order:

{CP1, CP2, CP3}, {CP1, CP2, CP4}, {CP1, CP3, CP4}, {CP2, CP3, CP4}, {CP1, CP2}, {CP1, CP3}, {CP2, CP3}, {CP1, CP4}, {CP2, CP4}, {CP3, CP4}

For checklist X (X is 0 or 1) combination, the check index with the highest utilization rate in check points is selected as the check index of list X, and the motion vectors that point to the difference reference image will be scaled.

Once a candidate is received, a full truncation process is performed to check whether the same candidate has been inserted into the list. If an identical candidate exists, the resulting candidate is discarded.

Filling with zero motion vectors

If the number of candidates in the affine fusion candidate list is less than 5, zero motion vectors with zero support indices are inserted into the candidate list until the list is full.

2.2.7. Bidirectional optical flow

Bidirectional optical flow (BIO) is a sample-based motion refinement that is performed on top of block-based motion compensation for bi-prediction. Sample-level motion refinement does not use signaling.

Let I ^(k) be the brightness value from reference image k (k=0, 1) after block motion compensation, and denote And as horizontal and vertical components of the gradient I ^(k) , respectively. Assuming that the optical flow is real, the vector field motion is given by the equation

(3)

Combining this optical flow equation with Hermitian interpolation for each sample path results in a unique third-order polynomial that corresponds to both the values of the I ^(k) function and the derivatives And at the ends. The value of this polynomial at t=0 is the BIO prediction:

(3)

In this document And indicate the distances to the reference frames, as shown in Fig. 26. Distances And are calculated based on POC for Ref0 and Ref1: τ ₀ =POC(current) − POC(Ref0), τ ₁ = POC(Ref1) − POC(current). If both predictions come from the same time direction (either the past or the future), then the signs are different (i.e. ). In this case, BIO only applies in cases where the prediction does not refer to the same point in time (i.e. ), both mentioned areas have non-zero motion ( ), and the movement vectors of the blocks are proportional to the time distance ( ).

Vector field motion is determined by minimizing the difference between the values at points A and B (at the intersection points of the motion trajectories and the planes of the reference frames in Fig.9). The model uses only the first linear term of the local Taylor expansion for :

(5)

All values in equation (5) are location dependent samples, which has been omitted in the notation so far. Assuming motion is consistent in the local surrounding region, we minimize the square window size (2M+1)×(2M+1) centered on the current predicted point , where M is equal to 2:

(6)

For this optimization problem, JEM uses a simplified approach by first performing minimization in the vertical direction and then in the horizontal direction. This leads to the equation:

(7)

(8)

Where,

(9)

To avoid division by zero or a very small value, regularization parameters r and m are introduced into equations (7) and (8).

(10)

(eleven)

Here – bit depth of video samples.

To keep memory access for BIO the same as for regular bi-prediction motion compensation, all prediction and gradient values are calculated only for positions within the current block. In equation (9), the square window of size (2M+1)×(2M+1) centered at the current predicted point on the boundary of the predicted block is necessary to access positions outside the block (as shown in Fig. 27(a)). In JEM values outside the block are set equal to the nearest available value inside the block. For example, it can be implemented as a padding, as shown in Fig. 27(b).

When using BIO, it is possible that the motion field can be refined for each sample. To reduce computational complexity, JEM uses a block-based BIO design. Motion refinement is calculated based on a 4x4 block. In block BIO, the s _n values in equation (9) for all samples in a 4x4 block are aggregated, and then the aggregated s _n values are used to offset the resulting BIO motion vectors for the 4x4 block. More specifically, the following formula is used to obtain block-based BIO:

(12)

where b _k denotes the set of samples belonging to the kth 4x4 block of the predicted block. s _n in equations (7) and (8) are replaced by ((s _n,bk ) >> 4) to obtain the associated motion vector displacements.

In some scenarios, multiple MV BIOs may be unreliable due to noise or irregular movement. Thus, in BIO, the magnitude of numerous MVs is limited by the thBIO threshold. The threshold value is determined based on whether all reference images of the current image are coming from the same direction. If all reference images of the current image come from the same direction, the threshold value is set to ; in other cases the value is set .

Gradients for BIO are calculated simultaneously with motion compensation interpolation using operations corresponding to the HEVC (2D shared FIR) motion compensation process. The input signal to this 2D shared FIR is the same reference frame sample as for the motion compensation process and the fractional position (fracX, fracY) according to the fractional part of the block motion vector. In case of signal horizontal gradient, it is first interpolated vertically using a BIOfilterS corresponding to the fractional position of fracY with a descale offset of d−8, then the gradient filter BIOfilterG is applied in the horizontal direction corresponding to the fractional position of fracX with a descale offset of 18−d. In the case of a vertical gradient , first applies a gradient filter in the vertical direction using BIOfilterG corresponding to the fractional position of fracY with a descale offset of d−8, then shifts the signal using BIOfilterS in the horizontal direction corresponding to the fractional position of fracX with a descale offset of 18−d. The length of the interpolation filter for calculating the gradients BIOfilterG and the offset BIOfilterF of the signal is shorter (has 6 taps) in order to keep the complexity at an acceptable level. Table 3 lists the filters used to compute gradients for various fractional block motion vector positions in BIO. Table 4 lists the interpolation filters used to generate the prediction signal in BIO.

Table 3. Filters for calculating gradients in BIO

Fractional pixel position Interpolation filter for gradient (BIOfilterG) 0 {8,−39,−3.46,−17.5} 1/16 {8,−32,−13.50,−18.5} 1/8 {7,−27,−20.54,−19.5} 3/16 {6,−21,−29.57,−18.5} 1/4 {4,−17,−36.60,−15.4} 5/16 {3,−9,−44.61,−15.4} 3/8 {1,−4,−48.61,−13.3} 7/16 {0.1,−54.60,−9.2} 1/2 {−1.4,−57.57,−4.1}

Table 4. Interpolation filters for generating a prediction signal in BIO

Fractional pixel position Interpolation filter for prediction signal (BIOfilterS) 0 {0,0,64,0,0,0} 1/16 {1,−3.64,4,−2.0} 1/8 {1,−6,62,9,−3,1} 3/16 {2,−8,60,14,−5,1} 1/4 {2,−9.57,19,−7.2} 5/16 {3,−10,53,24,−8,2} 3/8 {3,−11,50,29,−9,2} 7/16 {3,−11,44,35,−10,3} 1/2 {3,−10,35,44,−11,3}

In JEM, BIO is applied to all bi-prediction blocks when two predictions are obtained from different reference images. When LIC is enabled for a CU, BIO is disabled.

In JEM, OBMC is applied to a block after the normal MC process. To reduce computational complexity, BIO is not applied during the OBMC process. This means that BIO is only applied in the MC process for a block when its own MV is used, and is not applied in the MC process when the MV of an adjacent block is used during the OBMC process.

2.2.8. Motion vector refinement on the decoder side

In a bi-prediction operation for predicting one block region, two prediction blocks generated using a motion vector (MV) from list 0 (list0) and MV from list 1 (list1), respectively, are combined to form a single prediction signal. In the decoder-side motion vector refinement (DMVR) method, two bi-prediction motion vectors are further refined through a two-way template matching process. Two-way template matching is used in the decoder to perform a distortion-based search between the two-way template and reconstructed samples in the reference images in order to obtain a refined MV without transmitting additional motion information.

In DMVR, the two-way template is generated as a weighted combination (ie, average) of two prediction blocks, the initial MV0 of list0 and MV1 of list1, respectively, as shown in FIG. 28. The template matching operation consists of calculating a measure of the overhead between the generated template and the sample region (around the initial prediction block) in the reference image. For each of the two reference images, the MV that has the minimum template overhead is considered to be an updated MV from this list to replace the original one. In JEM, nine MV candidates are found for each list. The nine candidate MVs include the original MV and 8 surrounding MVs with one luminance sample offset from the original MV in either the horizontal or vertical direction or both. Finally, two new MVs, i.e., MV0' and MV1', as shown in Fig. 28, are used to generate the final bi-prediction results. Sum of absolute differences (SAD) is used as a measure of cost. It should be noted that when calculating the overhead of a prediction block produced by one surrounding MV, the rounded MV (to an integer pel) is actually used to obtain the prediction block instead of the actual MV.

DMVR is used for bi-prediction fusion mode with one MV from a reference picture in the past and another MV from a reference picture in the future without passing additional syntax elements. In JEM, when LIC, affine motion, FRUC, or a sub-CU fusion candidate is enabled for a CU, DMVR does not apply.

LUT-based motion vector prediction based on the disclosed technology, which improves both current and future video coding standards, is illustrated through the following examples described for various implementations. Since LUTs allow the encoding/decoding process to be performed based on historical data (eg, blocks that have been processed), LUT-based motion vector prediction can also be called a history-based motion vector prediction (HMVP) method. In the LUT-based motion vector prediction method, one or more tables of motion information from previously encoded blocks are maintained during the encoding/decoding process. These motion candidates stored in the LUT are called HMVP candidates. During encoding/decoding of one block, associated motion information in the LUT may be added to motion candidate lists (eg, merge/AMVP candidate lists) and after encoding/decoding of one block, the LUT may be updated. The updated LUTs are then used to encode subsequent blocks. Thus, the update of motion candidates in the LUT is based on the encoding/decoding order of the blocks. The examples below should be considered examples to explain general concepts. These examples should not be interpreted in a narrow sense. Moreover, these examples can be combined in any way.

3. Examples of Problems Solved by Embodiments

In a triangular partition configuration, one block can be divided into two partitions. To save memory bandwidth due to motion compensation, the two partitions need to be uni-predictable. In the motion compensation process, bi-prediction is used for the diagonal edge, and uni-prediction is used for all other parts. If bi-prediction is allowed for each partition, then samples located on the diagonal edge will have four MVs: two from one partition and two from another partition. This configuration has the following problems:

During the merge list creation process, only spatial neighbors and temporal blocks are checked.

History-based motion vector prediction technology is not allowed for triangular partitioning.

It is unknown how to handle triangular split mode in intrablock copy mode.

4. Examples of embodiments

The proposed technologies can be applied to any non-square/non-rectangular partitions, for example, to geometric partitions. In the following description, we use "triangular partitioning mode" to represent it as an example of the non-square/non-rectangular partitioning mode (TPM). It should be noted that other types of partitions can be used.

The technologies detailed below should be considered as examples to explain general concepts. These technologies should not be interpreted in a narrow sense. Moreover, these technologies according to the invention can be combined in any way. Let's denote the block size as W×H.

1. Instead of always using triangular prediction mode with merge mode, it is proposed to enable triangular prediction mode for intra-coded blocks or outer-coded blocks without merging.

a) In one example, two partitions are predicted in different inter-prediction modes.

b) Alternatively, the prediction values of the samples along the edge boundaries may be further filtered.

c) In one example, the motion information (such as MV) of two triangular partitions cannot predict each other;

i) Alternatively, the motion information (eg, MV) of one triangular partition can be used to predict another partition.

2. It is proposed that when one block is divided into two geometric partitions, one partition can be encoded in an inter prediction mode, and the other partition can be encoded in an inter prediction mode.

a) In one example, bi-prediction may also be used for externally encoded partitioning.

b) In one example, such a method may be signaled when the current block is encoded in merge mode, that is, for an externally encoded partition, motion information is signaled using the merge index.

c) Additionally, as an alternative, the prediction values of the samples along the edge boundaries can be further filtered.

d) In one example, only a subset of inter prediction modes may be used.

i) The subsets for the two partitions may be different.

ii) Subsets may depend on the partition position.

iii) Subsets may depend on the block size and/or block shape.

3. As described in claims 1 and 2, the externally coded block with or without merging may use the current picture as a reference picture.

4. It is proposed that when encoding one block in the triangular partition mode, the externally/internally encoded information of non-adjacent spatial blocks could also be considered as predictors for encoding the current block.

a) One example may use information about the motion of non-adjacent spatial blocks.

b) In one example, a non-contiguous spatial block inter prediction mode may be used.

c) Alternatively, the encoded time block information may be further used to encode one block with a TPM.

5. It is proposed to add motion candidates derived from HMVP candidates in the process of creating a merge list for TPM-encoded blocks, where the HMVP (History-Based Motion Vector Prediction) candidate is motion information inherited or derived from previously encoded blocks.

Maintaining tables to store HMVP candidates

a) Maintaining HMVP candidates and/or updating storage tables and/or HMVP candidates may be the same as HMVP for a normal motion vector.

b) In one example, the same table can be maintained for storing HMVP candidates and can be used for both non-TPM encoded blocks and TPM encoded blocks.

i) In one example, the traffic information used by the TPM does not fit into the HMVP storage. HMVP tables are not updated after encoding/decoding a TPM-encoded block.

c) In one example, separate tables may be maintained to store HMVP candidates for encoding TPM-encoded blocks.

i) Tables can be maintained to store HMVP candidates with traffic information exclusively from TPM-encoded blocks.

ii) Two tables can be maintained to store the uni- and bi-predicted HMVP candidates, respectively.

iii) Two tables can be maintained to store the movement information of the first and second partitions respectively.

iv) Two tables can be maintained to store traffic information from list 0 and list 1 of traffic information respectively.

v) Alternatively, three tables can be maintained to store uni-predicted candidates from list 0, uni-predicted candidates from L1 and bi-predicted HMVP candidates, respectively.

d) After encoding with a TPM-encoded block, the tables for storing HMVP candidates may not be updated.

i) Alternatively, one/more tables for storing HMVP candidates may be updated with information about the movement of the first partition.

ii) Alternatively, one/more tables for storing HMVP candidates may be updated with information about the movement of the second partition.

iii) Alternatively, one/more tables for storing HMVP candidates can be updated with traffic information from both partitions by adding two HMVP candidates.

iv) Alternatively, one/more tables for storing HMVP candidates can be updated with motion information from both splits by adding one HMVP candidate (eg, when two splits are predicted from two reference picture lists).

v) Whether one or two motion candidates are added to the HMVP table may depend on whether two splits are predicted from the same reference picture lists and/or the same reference picture.

vi) Whether to add motion information of the first or second split to the HMVP table may depend on the POC difference between the reference pictures and the current picture.

f) When separate tables are kept for encoding TPM-encoded blocks and non-TPM-encoded blocks, table updates may be caused on a single block mode basis.

i) In one example, after decoding a TPM-encoded block, the motion information may be used to update a table for encoding the TPM-encoded blocks.

ii) In one example, after decoding a non-TPM-encoded block, the motion information may be used to update a table for encoding the non-TPM-encoded blocks.

iii) Additionally, as an alternative, after decoding a non-TPM encoded block, the motion information may be used to update the table for encoding the TPM encoded blocks.

Using HMVP Candidates in TPM-Encoded Blocks

f) The HMVP candidate can be directly added to the merge list for TPM-encoded blocks.

i) Alternatively, one HMVP candidate can be used to obtain two motion candidates, for example, one is a uni-prediction with motion information from list 0, and the other is a uni-prediction with motion information from list 1 of the HMVP candidate.

ii) Truncation may be applied when inserting a motion candidate derived/inherited from an HMVP candidate with other motion candidates derived/inherited from spatial or temporal blocks.

iii) Truncation can be applied when inserting a motion candidate derived/inherited from an HMVP candidate with a motion candidate derived/inherited from another HMVP candidate.

iv) Truncation may be used when inserting a motion candidate derived/inherited from an HMVP candidate with another motion candidate derived/inherited from the same HMVP candidate.

g) In one example, one or more motion candidates derived from one or a plurality of HMVP candidates may be added after motion candidates derived from spatial and/or temporal blocks, such as block 1-7 shown in FIG. 14. In this case, the HMVP candidate is processed in the same way as other spatial/temporal merge candidates.

h) Motion candidates inherited/derived from HMVP candidates can be added to the merge list immediately after or before a specific step in the merge list creation process, which is based on the motion information of space/time blocks.

i) Alternatively, motion candidates inherited/derived from HMVP candidates can be added to the merge list immediately after or before several specified steps.

ii) In one example, one or more uni-prediction HMVP candidates may be added immediately after all originally predicted candidates with the current configuration.

iii) In one example, one or more HMVP candidates with a uni-prediction from L0 or a bi-prediction may be added immediately after all predicted candidates from the truncated list 0. Additionally, as an alternative, if the HMVP candidate has a bi-prediction, the storage may only traffic information from list 0.

iv) In one example, one or more HMVP candidates with uni-prediction from L1 or bi-prediction may be added immediately after all predicted candidates from truncated list 1. Additionally, as an alternative, if the HMVP candidate has a bi-prediction, the storage may only traffic information from list 1.

v) In one example, one or more HMVP candidates may be added immediately after the entire averaged uni-prediction of motion candidates from list 0 or list 1.

vi) In one example, one or more HMVP candidates may be added immediately before the entire averaged uni-prediction of list 0 or list 1 motion candidates.

i) Motion candidates inherited/derived from HMVP candidates may be added to an interleaved merge list with motion candidates inherited/derived from space/time block motion information.

j) In one example, motion candidates obtained from List 0 and List 1 of each HMVP candidate may be added in order before the motion candidates obtained from List 0 and List 1 of another HMVP candidate.

k) In one example, motion candidates obtained from List 0 of the first HMVP candidate set may be added in order before those motion candidates obtained from List 1 of the second HMVP candidate set.

i) In one example, the first set and the second set are the same set, that is, all available HMVP candidates.

ii) Alternatively, the first set and the second set may be different, for example, the first set of HMVP candidates includes HMVP candidates with uni-prediction from list 0 and bi-prediction; while the second set of HMVP candidates includes list 1 uni-prediction and bi-prediction HMVP candidates.

iii) The number of HMVP candidates associated with the first and second sets may be different.

l) HMVP candidates from the LX list can be first scaled to the L(1-X) list and then used for the L(1-X) list.

i) Scaled candidates can be inserted after all other candidates.

ii) Scaled candidates can be inserted after all other candidates except temporary candidates.

m) The order in which the HMVP candidates are checked to be used to obtain motion candidates to be added to the TPM merge list may depend on the HMVP candidate indices.

i) Alternatively, this order may depend on the prediction directions.

ii) Alternatively, this order may depend on motion information.

6. The number of HMVPs to check during the TPM merge list creation process can be defined in advance, for example, 5.

a) The number of HMVP candidates to be checked during the TPM merge list creation process may depend on the block size/block shape/number of available candidates before checking the HMVP candidates.

b) The number of HMVP candidates to be checked during the TPM merge list creation process may be signaled in the SPS/VPS/PPS/Image Header/Slice Header/Tile Group Header/CTU Lines/CTU/CTU Group.

c) When a portion of the available HMVP candidates must be checked during the merge list creation process in the TPM, the selection of HMVP candidates may depend on the prediction direction/MV information/reference picture index/POC distances of the reference picture and the current candidate picture and/or candidate index.

7. Whether and how to apply HMVP to a TPM-encoded block may depend on the size of the block (eg, width and/or height, aspect ratio) or the shape of the block (eg, square or non-square).

8. The proposed methods can also be applied to other types of motion candidate lists (for example, the AMVP list) for geometric partitions.

9. The size of the HMVP table (ie, the maximum number of motion candidates can be stored based on history) may depend on a function of one or multiple motion candidate list sizes.

a) In one example, the size of the HMVP table may depend on the size of the regular merge list, the size of the TPM merge list, the size of the IBC merge list, the size of the inter-AMVP regular list, the size of the inter-IBC regular list, etc.

b) In one example, the size of the HMVP table may depend on those motion candidate lists that access the same HMVP table.

c) In one example, the function may be a Max(input parameter values) function that returns the maximum value among a set of input parameter values.

d) In one example, the size of the HMVP table can be defined as Max(regular merge list size minus K0, TPM merge list size minus K1, IBC merge list size minus K2).

i) In one example, K0 = K1 = K2 = 1.

10. The size of the HMVP table (ie, the maximum number of history-based motion candidates can be stored) may depend on the function of one or more HMVP candidates (denoted numHMVP) that may be added to the motion candidate lists.

a) In one example, this HMVP table size may depend on the number of HMVP candidates that may be added to the movement's candidate lists (e.g., regular merge list, TPM merge list, IBC merge list, regular inter-AMVP list, regular inter-IBC list etc.).

b) In one example, the function may be a Max(input parameter values) function that returns the maximum value among a set of input parameter values.

c) In one example, the size of the HMVP table can be defined as Max(numHMVP for regular merge list minus K0, numHMVP for regular AMVP list minus K1).

ii) In one example, K0 = 1, and K1 = 0.

d) In one example, the size of the HMVP table can be defined as Max(numHMVP for regular merge list minus K0, numHMVP for TPM merge list minus K1, numHMVP for IBC merge list size minus K2).

iii) In one example, K0 = K1 = K2 = 1.

11. The size of the HMVP table for one encoding mode may be different from the size of the HMVP table for another encoding mode and may depend on the size of the motion candidate list.

a) In one example, the size of the HMVP table for the regular merge and TPM merge lists may depend on the size of the regular merge list.

iv) In one example, the size of the HMVP table could be equal to the size of the regular merge list minus K0, for example, K0 = 1.

b) In one example, the size of the HMVP table for the regular merge and TPM merge lists may depend on the size of the regular merge list and the size of the TPM merge list.

v) In one example, the size of the HMVP table can be defined as Max(regular merge list size minus K0, TPM merge list size minus K1).

vi) In one example, the HVMP table for normal inter-AMVP follows the table that is used for the regular merge and TPM merge lists.

c) In one example, the size of the HMVP table for IBC encoded blocks depends on that size for the regular merge list and/or TPM merge list.

vii) For example, the size of the HMVP table for IBC encoded blocks is equal to the size for a regular merge list and/or a TPM merge list.

viii) Alternatively, the size of the HMVP table for IBC encoded merge/AVMP blocks may depend on the size of the IBC/AMVP merge candidate list.

d) HMVP table size indications may be carried in a bit stream.

ix) When the signaled size is 0, HMVP candidates are not inserted into any motion candidate lists.

x) In one example, the size of the HMVP table cannot be 0.

xi) Alternatively, the size of the HMVP table minus K0 is signaled, e.g. K0 = 1.

12. Indications of the number of allowed HMVP candidates that may be added to motion candidate lists may be transmitted in the bit stream.

a) In one example, for each mode (eg, IBC or non-IBC; non-AMVP mergers), an indication of the number of allowed HMVP candidates may be signaled independently.

b) Alternatively, indications of the number of allowed HMVP candidates may be signaled via prediction.

c) Additionally, as an alternative, the HMVP table sizes may depend on the number of HMVP candidates allowed.

d) For example, the number of allowed HMVP candidates that can be added to a movement's candidate lists cannot be 0.

f) Alternatively, the number of allowed HMVP candidates that can be added to the motion candidate lists minus K0 can be signaled, e.g. K0 = 1.

f) Alternatively, predictive coding may be applied to the number of allowed HMVP candidates that can be added to the motion candidate list (denoted numHMVP) and the number of maximum motion candidates in the motion candidate list (denoted numMotionList).

12) One example could encode the difference between numMotionList and numHMVP.

13) One example could encode the difference between (numMotionList-K0) and numHMVP, e.g. K0 = 1.

13. The number of permitted HMVP candidates that can be added to a movement's candidate lists may depend on the size of the movement's candidate list.

a) In one example, it can be obtained from the appropriate size of the motion candidate list, such as (list size minus K0), e.g. K0 = 0 or 1.

b) In one example, this quantity may be derived from the size of the corresponding HMVP table that is used by the motion candidate list, e.g. (table size minus K0), e.g. K0 = 0 or 1.

14. The corresponding bitstream shall satisfy that the number of allowed HMVP candidates that can be added to the motion candidate list does not exceed the number of maximum motion candidates in the motion candidate list.

a) Alternatively, the corresponding bitstream shall satisfy that allowed HMVP candidates can be added to the motion candidate list, the number of which does not exceed (the number of maximum motion candidates in the motion candidate list minus K0), e.g., K0 = 1.

FIG. 29 shows a block diagram of a video processing apparatus 2900. Apparatus 2900 may be used to implement one or more of the methods described herein. The device 2900 may be embodied in a smartphone, tablet, computer, Internet of Things (IoT) receiver, etc. Apparatus 2900 may include one or more processors 2902, one or more memory devices 2904, and video processing hardware 2906. One or more processors 2902 may be configured to implement one or more of the methods described herein. One or more memory devices 2904 may be used to store data and code used to implement the methods and technologies described herein. Video processing hardware 2906 may be used to implement in hardware circuits some of the technologies described herein.

FIG. 31 is a flowchart of a video processing method 3100. Method 3100 includes determining (3105) that the first video block is internally coded or externally coded without merging, determining (3110) a first prediction subpart and a second prediction subpart for the first video block based on determining that the first video block is internally coded or externally coded without merging, wherein one or both of the first prediction sub-part or the second prediction sub-part are non-rectangular and non-square portions of the first video block, and further processing (3115) of the first video block using the first prediction sub-part and the second prediction sub-part.

FIG. 32 is a flowchart of a video processing method 3200. Method 3200 includes determining (3205) that the first video block is intra-encoded or externally encoded without merging, determining (3210) a first prediction sub-part and a second prediction sub-part for the first video block, wherein one or both of the first prediction sub-part or the second prediction sub-part are non-rectangular and non-square portions of the first video block, and further processing (3215) of the first video block using the first prediction sub-part and the second prediction sub-part, wherein at least one prediction part is externally encoded with or without merging using the current image as a reference image.

FIG. 33 is a flowchart of a video processing method 3300. Method 3300 includes determining (3305) that the first video block is intra-encoded or externally encoded without merging, determining (3310) a first prediction sub-part and a second prediction sub-part for the first video block, wherein one or both of the first prediction sub-part or the second prediction sub-part are non-rectangular and non-square portions of the first video block, and further processing (3315) of the first video block using a first prediction sub-part and a second prediction sub-part, wherein additional processing of the first video block using the first prediction sub-part and the second prediction sub-part is based on intra- or externally encoded information about non-adjacent spatial blocks in relation to the first video block.

FIG. 34 is a flowchart of a video processing method 3400. Method 3400 includes determining (3405) that the first video block is encoded in a triangular portion mode (TPM) using the triangular portion of the prediction of the first video block, and that the second video block is encoded in a non-TPM using the non-triangular portion of the prediction of the second video block, further processing (3410) the first video block and the second video block using the stored HMVP candidates; and storing (3415) the HMVP candidates associated with the first video block and the second video block.

FIG. 35 is a flowchart of a video processing method 3500. Method 3500 includes determining (3505) that the first video block includes prediction portions that are non-rectangular and non-square portions of the first video block, identifying (3510) an HMVP candidate, adding (3515) one or more motion candidates derived from the HMVP candidate , to a merge list associated with video blocks that include prediction portions that are non-rectangular and non-square, and further processing (3520) of the first video block using the merge list.

With respect to methods 3100, 3200, 3300, 3400, and 3500, some examples of determining encoding candidates and their use are described in section 4 of this document. For example, as described in Section 4, video blocks can be processed using prediction parts that are non-square and non-rectangular.

As for methods 3100, 3200, 3300, 3400 and 3500, a video block may be encoded in a video bitstream, in which bit efficiency can be achieved by using a bitstream generation rule that relates to motion information prediction.

The methods may include determining image information of a first prediction sub-part using a first inter prediction mode; and determining image information of the second prediction sub-part using a second inter prediction mode, wherein the first inter prediction mode is different from the second inter prediction mode.

The methods may include filtering prediction values that relate to a first prediction sub-part and a second prediction sub-part along an extreme boundary between the first prediction part and the second prediction part.

The methods may include: a processor determining first motion information of a first prediction sub-part; and determining by the processor second motion information for the second prediction sub-part, wherein the first motion information is determined without using the second motion information, and the second motion information is determined without using the first motion information.

The methods may include: a processor determining first motion information of a first prediction sub-part; and determining, by the processor, second motion information for the second prediction sub-part, wherein the second motion information is determined using the first motion information.

The methods may include processing the first prediction sub-part using intra prediction mode coding; and processing the second prediction sub-part using inter prediction mode coding.

The methods may include the processing of the second prediction sub-part including applying a bi-prediction.

The methods may include that processing the second prediction sub-part includes determining motion information signaled by the fusion index.

The methods may include filtering prediction values that relate to a first prediction sub-part and a second prediction sub-part along an extreme boundary between the first prediction part and the second prediction part.

The methods may include using a subset of inter-prediction mode coding.

The methods may include the first prediction subpart and the second prediction subpart being associated with different inter prediction mode encoding subsets.

The methods may include associating the first prediction sub-part and the second prediction sub-part with different encoding subsets in an intra-coding mode based on the positions of the first prediction sub-part and the second prediction sub-part.

The methods may include the first prediction subpart and the second prediction subpart being associated with different intra-mode encoding subsets based on one or a plurality of: the size of the first video block or the shape of the first video block.

The methods may include that the externally or internally encoded information of the non-contiguous spatial video blocks includes motion information of the non-contiguous spatial video blocks.

The methods may include using an inter-prediction mode of non-contiguous spatial video blocks.

The methods may include using encoded time block information.

The methods may include adding history-based motion vector prediction (HMVP) candidates to a list of motion vector prediction candidates, the HMVP candidates including motion information based on previously encoded video blocks; and decoding the first video block based on the HMVP candidates.

Methods may include HMVP candidates being stored in a table, and motion information used by the TPM not being stored in the table.

Methods may include the table not being updated after the first video block is encoded or decoded using the TPM.

The methods may include storing HMVP candidates in a first table and a second table, the first table storing HMVP candidates for TPMs and the second table storing HMVP candidates for non-TPMs.

The methods may include one of the first or second tables storing uni-predicted HMVP candidates and the other storing bi-predicted HMVP candidates.

The methods may include one of the first table or the second table storing information about the movement of the first subpartition, and the other storing information about the movement of the second subpartition.

The methods may include one of the first table or the second table storing motion information from list 0 and the other storing motion information from list 1.

The methods may include the first table storing uni-predicted HMVP candidates from list 0, the second table storing uni-predicted HMVP candidates from list 1, and the third table storing bi-predicted HMVP candidates.

The methods may include not updating tables for storing HMVP candidates based on the fact that one or both of the first prediction sub-part or the second prediction sub-part are non-rectangular and non-square geometry parts of the first video block.

The methods may include updating one or a plurality of the first table or the second table for storing HMVP candidates with information about the movement of the first subpartition.

The methods may include updating one or a plurality of the first table or the second table for storing HMVP candidates with information about the movement of the second subpartition.

The methods may include updating one or a plurality of the first table or the second table for storing HMVP candidates with motion information of the first subpartition and motion information of the second subpartition.

The methods may include determining that a first partition subpart and a second partition subpart are predicted from two reference picture lists; and updating one or more tables for storing HMVP candidates with information about the movement of one HMVP candidate based on determining that the first partition part and the second partition part are predicted from the two reference picture lists.

These methods may include determining that the first partition part and the second partition part are predicted using the same list of reference pictures or the same reference picture, wherein the tables are updated based on the determination that the first partition part and the second partition part of the partition are predicted using the same list of reference pictures or the same reference picture.

The methods may include determining a picture sequence (POC) difference between a reference picture and a current picture, wherein motion information of the first tile sub-part or the second tile sub-part is added to one or more tables based on the POC difference.

The methods may include determining characteristics regarding the encoding of the first video block; and updating one or both of the first table or the second table based on characteristics regarding the encoding of the first video block.

The methods may include using motion information to update a table associated with TPM video blocks after decoding the TPM video block.

The methods may include using motion information to update a table associated with non-TPM video blocks after decoding the non-TPM video block.

The methods may include using motion information to update a table associated with TPM video blocks after decoding a non-TPM video block.

The methods may include determining a first motion candidate and a second motion candidate from the HMVP candidate, wherein one of the first fusion candidate or the second fusion candidate is a uni-prediction with motion information from HMVP candidate list 0, and the other is a uni-prediction with information about the movement from the list of HMVP candidates.

The methods may include trimming based on insertion of a motion candidate obtained from an HMVP candidate with other motion candidates that are based on spatial or temporal video blocks.

The methods may include trimming based on insertion of a motion candidate derived from an HMVP candidate with other motion candidates that are based on other HMVP candidates.

The methods may include trimming based on insertion of a motion candidate derived from an HMVP candidate with other motion candidates that are based on the HMVP candidate.

The methods may include adding motion candidates obtained from an HMVP candidate associated with a first video block to a merge list after motion candidates obtained from one or both spatial or temporal video blocks.

The methods may include adding a motion candidate obtained from the HMVP candidate to a merge list after or before the process of creating a merge list based on motion information of spatial or temporal video blocks.

The methods may include adding a motion candidate derived from the HMVP candidate to the merge list after or before a given step.

The methods may include adding motion candidates derived from the uni-prediction HMVP candidate after the other initially uni-predicted candidates.

The methods may include adding motion candidates derived from the uni-prediction list 0 or bi-prediction HMVP candidate after the predicted candidates from the truncated list 0.

The methods may include adding motion candidates derived from the uni-prediction HMVP candidate from list 1 or bi-prediction after the predicted candidates from truncated list 1.

The methods may include adding motion candidates obtained from the HMVP candidate after averaging the uni-prediction from motion candidates from list 0 or list 1.

The methods may include adding motion candidates obtained from the HMVP candidate before the average uni-prediction from motion candidates from list 0 or list 1.

The methods may include adding motion candidates obtained from the HMVP candidates to a merge list in accordance with the interleave with motion candidates that are based on motion information of spatial or temporal video blocks relative to the first video block.

The methods may include adding motion candidates obtained from list 0 and list 1 of an HMVP candidate in order before motion candidates obtained from list 1 of another HMVP candidate.

The methods may include adding motion candidates obtained from list 0 of the first HMVP candidate set in order before motion candidates obtained from list 1 of the second HMVP candidate set.

The methods may include the first set and the second set being the same, wherein the first set and the second set include all available HMVP candidates.

The methods may include the first set and the second set being different, the first set including list 0 uni-prediction and bi-prediction HMVP candidates, the second set including list 1 uni-prediction HMVP candidates and bi-prediction.

The methods may include the first set including a first number of HMVP candidates, the second set including a second number of HMVP candidates, the first number and the second number being different.

The methods may include that the scaled motion candidate can be obtained from the HMVP candidate by scaling its motion vector from the LX list - the L(1-X) list, and the scaled motion candidate is assigned the scaled motion vector and the L(1-X) list. .

The methods may include having scaled motion candidates derived from the HMVP candidates inserted after other motion candidates derived from the HMVP candidates.

The methods may include having scaled motion candidates derived from the HMVP candidates inserted before the temporal candidates.

The methods may include having the order of checking HMVP candidates added to the merge list based on HMVP candidate indices.

The methods may include having the order of checking HMVP candidates added to the merge list based on prediction directions.

The methods may include a procedure for screening HMVP candidates added to the merge list based on the traffic information.

The methods may include having a predetermined number of HMVP candidates to be checked during the TPM merge list creation process.

The methods may include having the number of HMVP candidates to be checked during the TPM merge list creation process based on the block size of the first video block, the block shape of the first video block, or the number of candidates available before checking the HMVP candidates.

The methods may include that the number of HMVP candidates to be checked during the TPM merge list creation process is signaled in a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a picture header, a tile group header , slice header, coding tree unit (CTU) line group, CTU, or CTU group.

The methods may include the selection of HMVP candidates based on the prediction direction, motion vector (MV) information, reference picture index, POC distances of the reference picture, and the current picture of one or both of the candidate or the candidate index.

The methods may include applying the HMVP in conjunction with the first video block based on the block size of the first video block or the block shape of the first video block.

The methods may include the non-rectangular and non-square portions being triangular.

It should be appreciated that the disclosed technologies may be implemented in video encoders or decoders to improve compression efficiency when the compressed encoding blocks have a shape significantly different from traditional square blocks or rectangular blocks that are partially square in shape. For example, new encoding tools that use long or tall encoding blocks, such as 4x32 or 32x4 blocks, may benefit from the disclosed technologies.

FIG. 36 is a flowchart of a video processing method 3600. Method 3600 includes determining (3605) during conversion between a current block of visual media data and a corresponding encoded representation of visual media data a plurality of subparts for the current block; determining (3610) inter prediction information for a plurality of sub-parts; and transforming (3615) the current block using inter prediction information of the plurality of sub-parts; and where the current block is intra-coded and at least one of the plurality of sub-parts is a non-rectangular and non-square sub-part. It should be noted that this sub-part may be equivalent to the prediction part described above.

In some implementations, further modifications may be made to method 3600. For example, determining inter prediction information of a plurality of sub-parts comprises: determining first inter prediction information of a first sub-part in a plurality of sub-parts using a first inter prediction mode; determining second inter prediction information of a second sub-part in the plurality of sub-parts using a second inter prediction mode; where the first inter prediction mode is different from the second inter prediction mode. The first inter prediction information is determined using a first subset of inter prediction modes, and the second inter prediction information is determined using a second subset of inter prediction modes; where the first subset of inter prediction modes is different from the second subset of inter prediction modes. The subset of inter-prediction modes is based on at least one of the position, size and shape of the first sub-part and/or the second sub-part. Defining the external prediction information of the plurality of subparts further comprises: filtering the predicted values of the samples along an extreme boundary of at least one of the plurality of subparts. The first inter prediction information of the first sub-part of the plurality of sub-parts is determined without using the second inter prediction information of the second sub-part of the plurality of sub-parts, and the second inter prediction information of the second sub-part is determined without using the first inter prediction information of the first sub-part. The first inter prediction information of the first sub-part of the plurality of sub-parts is determined using the second inter prediction information of the second sub-part of the plurality of sub-parts, and/or the second inter prediction information of the second sub-part is determined using the first inter prediction information of the first sub-part.

FIG. 37 is a flowchart of a video processing method 3700. Method 3700 includes determining (3705), during conversion between a current block of visual media data and a corresponding encoded representation of visual media data, a plurality of subparts for the current block; determining (3710) information about the movement of the plurality of sub-parts; transforming (3715) the current block using the motion information of the plurality of sub-parts; and where the current block is not externally encoded without merging, and at least one of the plurality of subparts is a non-rectangular and non-square subpart.

In some implementations, further modifications may be made to the method 3700. For example, determining motion information of a plurality of sub-parts comprises: determining first motion information of a first sub-part in a plurality of sub-parts using a first inter-prediction mode; determining second motion information of the second sub-part in the plurality of sub-parts using a second inter-prediction mode. The definition of external prediction of a plurality of subparts further comprises: filtering the predicted values of the samples along the extreme boundary of at least one of the plurality of subparts. First motion information of the first sub-part is determined without using second motion information of the second sub-part, and second motion information of the second sub-part is determined without using first motion information of the first sub-part. First motion information of the first sub-part is determined using second motion information of the second prediction, and/or second motion information of the second sub-part is determined using first motion information of the first sub-part.

FIG. 38 is a flowchart of a video processing method 3800. Method 3800 includes a transformation (3805) between a current block of visual media data and a corresponding encoded representation of the visual media data, wherein the current block is split into multiple sub-parts according to a split pattern, the first sub-part having a non-rectangular, non-square shape; processing (3810) the first subpart in an internal encoding mode; and processing (3815) the second sub-part in an external encoding mode.

In some implementations, further modifications may be made to the method 3800. For example, the second subpart is processed using bi-prediction; signaling motion information using a merge index in response to encoding of the current block in merge mode; and determining motion information of the first prediction sub-part using the inter-prediction mode subset. The subset of inter-prediction modes is based on at least one of: a position of the first prediction sub-part, a size of the first prediction sub-part, and a shape of the first prediction sub-part.

FIG. 39 is a flowchart of a video processing method 3900. Method 3900 includes a transformation (3905) between a current block of visual media data and a corresponding encoded representation of the visual media data, wherein the current block is split into multiple sub-parts according to a split pattern, wherein the first sub-part has a non-rectangular, non-square shape; where at least one of the plurality of sub-parts is externally encoded with or without merging and uses the current image as a reference image.

FIG. 40 is a flowchart of a video processing method 4000. Method 4000 includes a transformation 4005 between a current block of visual media data and a corresponding encoded representation of the visual media data, the current block being partitioned into a plurality of sub-parts according to a partitioning pattern, wherein the first prediction partition has a non-rectangular, non-square shape; and transforming (4010) using inter- or intra-encoded information about one or more spatial blocks.

In some implementations, further modifications may be made to method 4000. For example, the inter- or intra-encoded information of one or more spatial blocks contains information about the movement of one or more spatial blocks. Converting a current block using inter- or intra-encoded information about one or more spatial blocks comprises: transforming a current block using an inter-prediction mode of one or more spatial blocks. In this case, encoded time block information is used.

FIG. 41 is a flowchart of a video processing method 4100. Method 4100 includes determining (4105), during conversion between a first block of visual media data and a corresponding encoded representation of the visual media data, a first block encoded in a geometric partitioning mode; determining (4110), based on at least one table storing history-based motion vector prediction (HMVP) candidates that include motion information based on previously encoded blocks, motion information of at least one sub-part of the first block; transforming (4115) the first block using the determined motion information.

In some implementations, further modifications may be made to method 4100. For example, the first block encoded in geometric partitioning mode contains: partitioning the first block into several subparts; wherein at least one of the plurality of portions is a non-rectangular and non-square portion. Determining, during conversion between the second block of visual media data and the corresponding encoded representation of the visual media data, motion information of the second block based on the same table that is used for the first block, and the second block does not use a geometric partitioning mode. The motion information used by the first block is refrained from being stored in at least one table. At least one table is not updated after the first block is converted. The at least one table includes a plurality of tables that are maintained to store HMVP candidates for geometric partitioning mode blocks. For at least one table, the stored HMVP candidates have only motion information in accordance with the information used by the blocks in the geometric partitioning mode. The at least one table includes two tables that are maintained for storing uni-predicted HMVP candidates and bi-predicted HMVP candidates, respectively. The at least one table includes two tables that are maintained for storing movement information of the first subpart and the second subpart of the first block, respectively. The at least one table includes two tables that are maintained for storing movement information from list 0 and list 1, respectively. The at least one table includes three tables that are maintained for storing uni-predicted HMVP candidates from list 0, uni-predicted HMVP candidates from list 1, and bi-predicted HMVP candidates, respectively. The at least one table is updated with information about the movement of a portion of the subparts of the first block. The at least one table is updated with movement information of the first sub-part and movement information of the second sub-part, wherein the first sub-part and the second sub-part are sub-parts of the first block. The at least one table is updated with the first subpart movement information and the second subpart movement information by adding one HMVP candidate. Whether to update the at least one table by adding one or two motion candidates depends on whether the first subpart and the second subpart use the same reference picture list or the same reference picture. Whether to update the at least one table by adding motion information of the first sub-part or the second sub-part depends on the picture sequence count (POC) difference between the reference picture and the current picture. After processing a non-rectangular and non-square block, the motion information of the non-rectangular and non-square block is used to update a table that is maintained to store HMVP candidates for the non-rectangular and non-square block. After processing a block without a non-rectangular and non-square part, the movement information of the block without a non-rectangular and non-square part is used to update a table that is maintained to store HMVP candidates for the block without a non-rectangular and non-square part. After processing a block without a non-rectangular and non-square part, the motion information of the block without a non-rectangular and non-square part is used to update a table that is maintained to store motion vector prediction (HMVP) candidates for the block with a non-rectangular and non-square part. The geometric partition mode contains a triangular partition mode. The method is applicable to other types of motion candidate lists.

FIG. 42 is a flowchart of a video processing method 4200. Method 4200 includes determining (4205), during conversion between a first block of visual media data and a corresponding encoded representation of the visual media data, a first block encoded in a geometric partitioning mode; determining (4210) motion information of at least one sub-part of the first block; transforming (4215) the first block using motion information of the at least one sub-part; wherein determining the motion information of the at least one sub-part comprises using at least one history-based motion vector prediction (HMVP) candidate that includes the motion information based on the previously encoded block to create a list of motion candidates and determine the motion information from list of movement candidates.

In some implementations, further modifications may be made to method 4200. For example, the first block encoded in geometric partition mode contains: a partition of the first block into several subparts; wherein at least one of the plurality of sub-portions is a non-rectangular and non-square portion. At least one HMVP candidate is added directly to the movement's candidate list. Using at least one HMVP candidate that includes motion information based on a previously encoded block to create a list of motion candidates comprises: using one HMVP candidate to determine a first motion candidate and a second motion candidate, wherein one of the first motion candidate and the second the motion candidate is a uni-prediction with motion information from HMVP candidate list 0, and the other of the first motion candidate and the second motion candidate is a uni-prediction with motion information from HMVP candidate list 1; truncation in response to insertion of a motion candidate derived from the HMVP candidate with other motion candidates that are based on spatial or temporal video blocks; truncation in response to insertion of a motion candidate derived from an HMVP candidate with other motion candidates that are based on other HMVP candidates; and truncation in response to inserting a motion candidate derived from the HMVP candidate with other motion candidates that are based on the HMVP candidate. Using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises: adding at least one motion candidate obtained from the HMVP candidate to the motion candidate list after the motion candidates obtained from one or both spatial or temporal blocks. Using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises: adding at least one motion candidate derived from the HMVP candidate to the motion candidate list after or before a predetermined step in the process of creating a list of motion candidates based on motion information of spatial or temporal blocks. Using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises: adding at least one motion candidate derived from the HMVP candidate to the motion candidate list after or before several in advance certain steps in the process of creating a list of motion candidates based on motion information of spatial or temporal blocks. Using at least one HMVP candidate that includes motion information based on the previously encoded block to create a motion candidate list comprises: adding at least one motion candidate derived from the uni-prediction HMVP candidate to the motion candidate list after all initially uni-predicted candidates. Using at least one HMVP candidate that includes motion information based on a previously encoded block to create a list of motion candidates comprises: adding at least one motion candidate derived from the HMVP candidate with uni-prediction from list 0 or bi-prediction, to the list of motion candidates after the truncated list of 0 predicted candidates. Using at least one HMVP candidate that includes motion information based on a previously encoded block to create a list of motion candidates comprises: adding at least one motion candidate derived from the HMVP candidate with uni-prediction from list 1 or bi -prediction, into the list of motion candidates after the truncated list of 1 predicted candidates. Using at least one HMVP candidate that includes motion information based on the previously encoded block to create a motion candidate list comprises: adding at least one motion candidate derived from the at least one HMVP candidate to the motion candidate list after an average uni-prediction from motion candidates from list 0 or list 1. Using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises: adding at least one candidate motion derived from at least one HMVP candidate to a list of motion candidates before averaging the uni-prediction from motion candidates from list 0 or list 1. Using at least one HMVP candidate that includes motion information based on a previously encoded block for creating a motion candidate list, comprises: adding at least one motion candidate obtained from the HMVP candidate to the motion candidate list in accordance with the interleaving with motion candidates that are based on motion information of spatial or temporal blocks with respect to the current block. The resulting motion candidate from the HMVP candidate is set equal to the HMVP candidate. Using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises: adding at least one motion candidate obtained from HMVP candidate list 0 and list 1 to the motion candidate list before the motion candidates obtained from list 0 and list 1 of another HMVP candidate. Using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises: adding at least one motion candidate obtained from list 0 of the first HMVP candidate set to the motion candidate list before motion candidates obtained from list 1 of the second set of HMVP candidates. The first set and the second set are the same. The first set and the second set are different, the first set includes HMVP candidates with list 0 uni-prediction and bi-prediction, and the second set includes HMVP candidates with list 1 uni-prediction and bi-prediction. The first set includes a first number of HMVP candidates, the second set includes a second number of HMVP candidates, and the first number and the second number are different. Using at least one HMVP candidate that includes motion information based on the previously encoded block to generate a list of motion candidates comprises: scaling the HMVP candidates from the LX-list L(1-X) to generate scaled HMVP candidates that are used for L(1-X). Using at least one HMVP candidate that includes motion information based on the previously encoded block to create a list of motion candidates comprises: adding scaled HMVP candidates after other motion candidates derived from the HMVP candidates. Using at least one HMVP candidate that includes motion information based on the previously encoded block to create a list of motion candidates comprises: adding the scaled HMVP candidates after other motion candidates derived from the HMVP candidates and before the temporary candidates. The screening order for HMVP candidates added to a movement's candidate list is based on the HMVP candidate indices. The order of checking HMVP candidates added to the motion candidate list is based on the prediction directions. The screening process for HMVP candidates added to a movement's candidate list is based on the movement's information. The number of HMVP candidates to be screened during the motion candidate list creation process is predetermined. The number of HMVP candidates to be checked during the motion candidate list creation process is based on the block size of the current block, the block shape of the current block, or the number of candidates available before checking the HMVP candidates. The number of HMVP candidates to be checked during the motion candidate list creation process is signaled in the Video Parameter Set (VPS), Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Picture Header, Tile Group Header, Slice Header, Tree Unit Row Group coding (CTU), CTU or CTU group. The HMVP candidates selected for verification in the motion candidate list are based on at least one of: prediction direction, motion vector (MV) information, reference frame index, DOC distances of the reference image and the current candidate image, and candidate index. Applying HMVP to the current block is based on the block size of the current block or the block shape of the current block. The motion candidate list contains a merge list. The motion candidate list is applicable to other kinds of motion candidate lists except the merge candidate list.

Certain features preferably implemented in some embodiments will now be disclosed in a clause-based format.

(1) A method for processing video, comprising the steps of:

determining, during conversion between the current block of visual media data and the corresponding encoded representation of the visual media data, a plurality of subparts of the current block;

determining inter prediction information for the plurality of sub-parts;

transforming the current block using inter prediction information of the plurality of sub-parts; and

the current block is internally encoded, and at least one of the plurality of subparts is a non-rectangular and non-square subpart.

(2) The method according to (1), in which the step of determining inter prediction information of a plurality of sub-parts comprises sub-steps of:

determining first inter prediction information of the first sub-part of the plurality of sub-parts using a first inter prediction mode;

determining second inter prediction information of a second sub-part of the plurality of sub-parts using a second inter prediction mode; and

the first inter prediction mode is different from the second inter prediction mode.

(3) The method of (2), wherein the first inter prediction information is determined using the first subset of inter prediction modes, and the second inter prediction information is determined using the second subset of inter prediction modes; wherein

the first subset of inter prediction modes is different from the second subset of inter prediction modes.

(4) The method of (3), wherein the subset of inter prediction modes is based on at least one of the position, size and shape of the first sub-part and/or the second sub-part.

(5) The method as in any one of (1) to (4), wherein the step of determining inter prediction information of the plurality of sub-parts further comprises a substep of:

filtering the predicted sample values along the extreme boundary of at least one of the plurality of subparts.

(6) The method as in any one of (1) to (5), wherein the first inter prediction information of a first sub-part of the plurality of sub-parts is determined without using the second inter prediction information of the second sub-part of the plurality of sub-parts, and the second inter prediction information of the second sub-part is determined without using the first external prediction information of the first sub-part.

(7) The method according to any one of claims 1 to 5, wherein the first inter prediction information of the first sub-part of the plurality of sub-parts is determined using the second inter prediction information of the second sub-part of the plurality of sub-parts, and/or the second inter prediction information of the second sub-part is determined using the first external prediction information of the first sub-part.

(8) A method for processing video, comprising the steps of:

determining, during conversion between the current block of visual media data and the corresponding encoded representation of the visual media data, a plurality of subparts of the current block;

determine information about the movement of the plurality of sub-parts;

transforming the current block using information about the movement of the plurality of sub-parts; and

the current block is not externally merge encoded, and at least one of the plurality of subparts is a non-rectangular and non-square subpart.

(9) The method according to (8), in which the stage of determining information about the movement of a plurality of subparts contains substages in which:

determining first motion information of the first sub-part of the plurality of sub-parts using a first inter-prediction mode;

determining second motion information of a second sub-part of the plurality of sub-parts using a second inter-prediction mode.

(10) The method according to any one of (8)-(9), wherein the external prediction definition of the plurality of sub-parts further comprises:

filtering the predicted sample values along an extreme boundary of at least one of the plurality of subparts.

(11) The method as in any one of (8) to (9), wherein the first motion information of the first sub-part is determined without using the second motion information of the second sub-part, and the second motion information of the second sub-part is determined without using the first motion information of the first sub-part.

(12) The method as in any one of (8) to (9), wherein the first motion information of the first sub-part is determined using the second motion information of the second prediction, and/or the second motion information of the second sub-part is determined using the first motion information of the first subparts

(13) A video processing method comprising:

converting between a current block of visual media data and a corresponding encoded representation of the visual media data, wherein the current block is divided into several sub-parts according to a partitioning pattern, where the first sub-part has a non-rectangular, non-square shape;

processing the first subpart in internal coding mode; And

processing of the second subpart in external coding mode.

(14) The method according to (13), in which the second subpart is processed by applying bi-prediction.

(15) The method according to (13) or (14), additionally contains:

signaling motion information with a merge index in response to encoding of the current block in merge mode.

(16) The method according to any of (13)-(15), additionally contains:

determining motion information of the first prediction sub-part using a subset of the inter-prediction mode.

(17) The method of (16), wherein the subset of inter prediction modes is based on at least one of: a position of the first prediction sub-part, a size of the first prediction sub-part, and a shape of the first prediction sub-part.

(18) A video processing method comprising:

converting between a current block of visual media data and a corresponding encoded representation of the visual media data, the current block being split into multiple sub-parts according to a partitioning pattern, wherein the first sub-part has a non-rectangular, non-square shape;

where at least one of the plurality of sub-parts is externally encoded with or without merging and uses the current image as a reference image.

(19) A video processing method comprising:

converting between a current block of visual media data and a corresponding encoded representation of the visual media data, wherein the current block is partitioned into multiple sub-parts according to a partitioning pattern, wherein the first prediction partition has a non-rectangular, non-square shape;

and transforming using the externally or internally encoded information of the one or more spatial blocks.

(20) The method according to (19), in which the externally or internally encoded information of one or more spatial blocks contains information about the movement of one or more spatial blocks.

(21) The method according to (19), in which the transformation of the current block using inter- or intra-encoded information about one or more spatial blocks contains:

transforming the current block using the inter-prediction mode of one or more spatial blocks.

(22) The method according to (19), in which the encoded information of time blocks is used.

(23) A video processing device comprising a processor configured to implement the method set forth in any of (1) to (22).

(24) The device according to (23), in which the device is a video encoder.

(25) The device according to (23), in which the device is a video decoder.

(26) A computer-readable recording medium on which a program containing code is recorded, the program being configured for a processor to perform the method set forth in any of (1) to (22).

Certain features preferably implemented in some embodiments will now be disclosed in a clause-based format.

(1) A method for processing video, comprising the steps of:

determining, during conversion between the first block of visual media data and the corresponding encoded representation of the visual media data, a first block encoded in a geometric partitioning mode;

determining, based on at least one table storing history-based motion vector prediction (HMVP) candidates including motion information based on previously encoded blocks, motion information of at least one sub-part of the first block;

transforming the first block using the determined motion information.

(2) The method according to (1), in which the first block encoded in the geometric partitioning mode contains:

splitting the first block into many subparts; wherein

at least one of the plurality of sub-portions is a non-rectangular and non-square portion.

(3) The method according to (1), further comprising the step of:

determining, during conversion between the second block of visual media data and the corresponding encoded representation of the visual media data, motion information of the second block based on the same table used for the first block, wherein

the second block does not use geometric partitioning mode.

(4) The method as in any one of (1) to (3), wherein the motion information used by the first block is not stored in the at least one table.

(5) The method as in any one of (1) to (4), wherein the at least one table is not updated after the first block is converted.

(6) The method as in any one of (1) to (5), wherein the at least one table includes a plurality of tables maintained for storing HMVP candidates for geometric partitioning mode blocks.

(7) The method according to (6), wherein, for at least one table, the stored HMVP candidates have only motion information according to the motion information used by the blocks in the geometric partitioning mode.

(8) The method as in any one of (1) to (7), wherein the at least one table includes two tables maintained for storing uni-predicted HMVP candidates and bi-predicted HMVP candidates, respectively.

(9) The method as in any one of (1) to (8), wherein the at least one table includes two tables maintained for storing movement information of the first sub-part and the second sub-part of the first block, respectively.

(10) The method as in any one of (1) to (9), wherein the at least one table includes two tables maintained for storing traffic information from list 0 and list 1, respectively.

(11) The method as in any one of (1) to (10), wherein the at least one table includes three tables maintained for storing uni-predicted HMVP candidates from list 0, uni-predicted HMVP candidates from list 1, and bi -predicted HMVP candidates, respectively.

(12) The method according to any one of (1) to (11), wherein the at least one table is updated with information about the movement of a portion of the subparts of the first block.

(13) The method as in any one of (1) to (12), wherein the at least one table is updated with movement information of the first sub-part and movement information of the second sub-part, wherein the first sub-part and the second sub-part are sub-parts of the first block.

(14) The method according to (13), wherein the at least one table is updated with the movement information of the first sub-part and the movement information of the second sub-part by adding one HMVP candidate.

(15) The method of (14), wherein whether to update the at least one table by adding one or two motion candidates depends on whether the first subpart and the second subpart use the same reference picture list or one and the same reference image.

(16) The method of (14), wherein whether to update the at least one table by adding motion information of the first sub-part or the second sub-part depends on a picture sequence count (POC) difference between the reference picture and the current picture.

(17) The method according to any one of (1) to (16), wherein the method further comprises the step of:

using, after the non-rectangular and non-square block processing step, the motion information of the non-rectangular and non-square block to update a table maintained to store HMVP candidates for the non-rectangular and non-square block.

(18) The method according to any one of (1)-(16), further comprising the step of:

using, after the step of processing the block without a non-rectangular and non-square part, the movement information of the block without a non-rectangular and non-square part to update a table maintained to store HMVP candidates for the block without a non-rectangular and non-square part.

(19) The method according to (1)-(16), additionally containing the step of:

using, after the step of processing the block without a non-rectangular and non-square portion, the motion information of the block without a non-rectangular and non-square portion to update a table maintained for storing motion vector prediction (HMVP) candidates for the block with a non-rectangular and non-square portion.

(20) The method according to any one of (1)-(19), in which the geometric partition mode contains a triangular partition mode.

(21) The method according to any of (1)-(20), in which the method is applicable to other types of motion candidate lists.

(22) A video processing device containing a processor configured to implement the method according to any of (1)-(21).

(23) The device according to (22), in which the device is a video encoder.

(24) The device according to (22), in which the device is a video decoder.

(25) A computer-readable recording medium storing a program containing code, wherein the program causes, when executed by a processor, the method of any one of (1) to (21).

Certain features preferably implemented in some embodiments will now be disclosed in a clause-based format.

(1) A video processing method comprising:

determining, during conversion between the first block of visual media data and the corresponding encoded representation of the visual media data, a first block encoded in a geometric partitioning mode;

determining motion information of at least one sub-part of the first block;

transforming the first block using motion information of the at least one sub-part; and

determining motion information of the at least one sub-part comprises using at least one history-based motion vector prediction (HMVP) candidate that includes motion information based on a previously encoded block to create a list of motion candidates and determine motion information from the list candidates of the movement.

(2) The method according to (1), in which the first block encoded in the geometric partitioning mode contains:

splitting the first block into several subparts; and

at least one of the plurality of sub-portions is a non-rectangular and non-square portion.

(3) The method according to (1), in which

at least one HMVP candidate is added directly to the movement's candidate list.

(4) The method as in any one of (1) to (3), wherein using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises:

using one HMVP candidate to determine a first motion candidate and a second motion candidate, wherein one of the first motion candidate and the second motion candidate is a uni-prediction with motion information from HMVP candidate list 0, and the other is a uni-prediction with motion information from list 1 candidate HMVP.

(5) The method according to any one of (1)-(4), further comprising:

truncation in response to inserting a motion candidate derived from an HMVP candidate with other motion candidates that are based on spatial or temporal video blocks.

(6) The method according to any one of (1)-(5), further comprising:

truncation in response to inserting a motion candidate derived from an HMVP candidate with other motion candidates that are based on other HMVP candidates.

(7) The method according to any one of (1)-(6), further comprising:

truncation in response to inserting a motion candidate derived from an HMVP candidate with other motion candidates that are based on the HMVP candidate.

(8) The method as in any one of (1) to (7), wherein using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises:

adding at least one motion candidate obtained from the HMVP candidate to the list of motion candidates after the motion candidates extracted from one or both of the spatial or temporal blocks.

(9) The method as in any one of (1) to (8), wherein using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises:

adding at least one motion candidate obtained from the HMVP candidate to the motion candidate list after or before a given step in the process of creating a motion candidate list based on the motion information of the spatial or time blocks.

10. The method as in any one of (1) to (9), wherein using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises:

adding at least one motion candidate obtained from the HMVP candidate to the motion candidate list after or before a plurality of predetermined steps in the process of creating a motion candidate list based on the motion information of the spatial or time blocks.

(11) The method as in any one of (1) to (10), wherein using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises:

adding at least one motion candidate derived from the uni-prediction HMVP candidate to the list of motion candidates after all initially predicted candidates.

(12) The method as in any one of (1) to (11), wherein using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises:

adding at least one motion candidate derived from the HMVP candidate with uni-prediction from list 0 or bi-prediction to the list of motion candidates after the predicted candidates from truncated list 0.

(13) The method as in any one of (1) to (12), wherein using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises:

adding at least one motion candidate derived from the uni-prediction candidate from list 1 or bi-prediction HMVP to the list of motion candidates after the predicted candidates from truncated list 1.

(14) The method as in any one of (1) to (13), wherein using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises:

adding at least one motion candidate derived from at least one HMVP candidate to the motion candidate list after averaging the uni-prediction from the motion candidates from list 0 or list 1.

(15) The method as in any one of (1) to (14), wherein using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises:

adding at least one motion candidate derived from at least one HMVP candidate to the motion candidate list before the average uni-prediction from the motion candidates from list 0 or list 1.

(16) The method as in any one of (1) to (15), wherein using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises:

adding at least one motion candidate obtained from the HMVP candidate to the list of motion candidates in accordance with the interleaving with motion candidates that are based on motion information of spatial or temporal blocks with respect to the current block.

(17) The method according to any one of (8)-(16), in which

The motion candidate obtained from the HMVP candidate is set equal to the HMVP candidate.

(18) The method as in any one of (1) to (17), wherein using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises:

adding at least one motion candidate obtained from list 0 and list 1 of the HMVP candidate to the motion candidate list before the motion candidates obtained from list 0 and list 1 of the other HMVP candidate.

(19) The method as in any one of (1) to (18), wherein using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises:

adding at least one motion candidate obtained from list 0 of the first HMVP candidate set to the motion candidate list before the motion candidates obtained from list 1 of the second HMVP candidate set.

(20) Method according to (19), in which the first set and the second set are the same.

(21) The method according to (19), in which the first set and the second set are different, and the first set includes HMVP candidates with uni-prediction from list 0 and bi-prediction, and the second set includes HMVP candidates with uni-prediction list 1 prediction and bi-prediction.

(22) The method according to (19), wherein the first set includes a first number of HMVP candidates, the second set includes a second number of HMVP candidates, and the first number and the second number are different.

(23) The method as in any one of (1) to (22), wherein using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises:

scaling HMVP candidates from the LX list - the L(1-X) list to produce scaled HMVP candidates that are used for L(1-X).

(24) The method of (23), wherein using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises:

adding scaled HMVP candidates after other motion candidates derived from HMVP candidates.

(25) The method of (23), wherein using at least one HMVP candidate that includes motion information based on a previously encoded block to create a motion candidate list comprises:

adding scaled HMVP candidates after other motion candidates derived from HMVP candidates and before temporal candidates.

(26) The method as in any one of (1) to (25), wherein the order of checking the HMVP candidates added to the motion candidate list is based on the indices of the HMVP candidates.

(27) The method as in any one of (1) to (25), wherein the order of checking HMVP candidates added to the motion candidate list is based on prediction directions.

(28) The method as in any one of (1) to (25), wherein the order of checking HMVP candidates added to the motion candidate list is based on the motion information.

(29) The method as in any one of (1) to (28), wherein the number of HMVP candidates to be checked in the motion candidate list creation process is determined in advance.

(30) The method as in any one of (1) to (28), wherein the number of HMVP candidates to be verified in the motion candidate list creation process is based on the block size of the current block, the block shape of the current block, or the number of candidates available before the HMVP candidates are verified .

(31) The method as in any one of (1) to (28), wherein the number of HMVP candidates to be checked in the motion candidate list creation process is signaled in a video parameter set (VPS), a sequence parameter set (SPS), an image parameter set ( PPS), image header, tile group header, slice header, coding tree unit (CTU) line group, CTU, or CTU group.

(32) The method as in any one of (1) to (28), wherein the HMVP candidates selected for checking in the motion candidate list are based on at least one of: prediction direction, motion vector (MV) information, reference frame index , the DOC distances of the reference image and the current candidate image and the candidate index.

(33) The method as in any one of (1) to (28), wherein applying the HMVP to the current block is based on the block size of the current block or the block shape of the current block.

(34) The method as in any one of (1) to (33), wherein the motion candidate list contains a merge list.

(35) The method as in any one of (1) to (33), wherein the motion candidate list is applicable to other kinds of motion candidate lists except the merger candidate list.

(36) A video processing device containing a processor configured to implement the method according to any of (1)-(35).

(37) Device according to (36), in which the device is a video encoder.

(38) Device according to (36), in which the device is a video decoder.

(39) A computer-readable recording medium on which a program containing code is recorded, the program being configured for a processor to execute the method of any one of (1) to (35).

The disclosed and other solutions, examples, embodiments, modules and functional operations described herein may be implemented in digital electronic circuits or in computer software, firmware or hardware, including the structures disclosed herein, and their structural equivalents, or combinations of one or more of them. The disclosed and other embodiments may be implemented as one or a plurality of computer program products, that is, one or multiple modules of computer program instructions encoded on a computer-readable storage medium for execution by a data processing device or for controlling the operation thereof. The computer-readable medium may be a computer-readable storage device, a computer-readable storage medium, a storage device, a composition of matter affecting the machine-readable propagation signal, or a combination of one or more of these. The term "data processing device" covers all hardware devices, devices and machines for processing data, including, by example, a programmable processor, a computer and multiple processors or computers. The apparatus may include, in addition to hardware, code that creates an execution environment for the computer program in question, such as code comprising processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these. The propagated signal is an artificially generated signal, such as a machine-generated electrical, optical, or electromagnetic signal, that is produced to encode information for transmission to a suitable receiving device.

A computer program (also known as a program, software, software application, script, or code) can be written in any kind of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or as a module , component, routine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file on a file system. A program may be stored in a portion of a file that contains other programs or data (for example, one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (for example, files in which stores one or more modules, subroutines, or pieces of code). A computer program may be deployed for execution on a single computer or on multiple computers that are located in one location or distributed over multiple locations and interconnected by a communications network.

The processes and logical flows described herein may be executed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and producing output data. Processes and logic flows may also be carried out by special purpose logic circuitry, such as a field programmable gate array (FPGA) or application specific integrated circuit (ASIC), and the device may also be implemented as these.

Processors suitable for executing a computer program include, by way of example, both general purpose and special purpose microprocessors, and any one or more digital computer processors of any type. In general, the processor will receive instructions and data from read only memory or random access memory, or both. The essential elements of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. In general, the computer will also include or be operably coupled to receive data from or transmit data to one or both of large capacity storage devices for storing data, such as magnetic, magneto-optical disks, or optical disks. However, such devices are not necessary in a computer. Computer readable storage media suitable for storing computer program instructions and data include all types of non-volatile memory, storage media and storage devices, including, for example, semiconductor storage devices such as EPROM, EEPROM and flash memory devices; magnetic disks, such as internal hard drives or removable drives; magneto-optical disks; and CD-ROMs and DVD-ROMs. The processor and memory may be supplemented or included with special-purpose logic.

Although this patent document contains many specific details, they should not be construed as limiting the scope of any invention or what may be claimed, but rather as describing features that may be specific to particular embodiments of particular technologies. Certain features that are described in this patent document in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of one embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as operating in certain combinations and even initially claimed as such, in some cases one or more features of a claimed combination may be excluded from the combination, and the claimed combination may be directed toward a subcombination or a variation of a subcombination.

Likewise, although operations are depicted in the drawings in a particular order, this should not be understood to require that such operations be performed in the particular order shown or in a sequential order, or that all illustrated operations be performed to achieve the desired results. Moreover, the partitioning of various system components in the embodiments described in this patent document should not be understood to require such partitioning in all embodiments

Only a few implementations and examples have been described above, and other implementations, improvements and changes may be made based on what is described and illustrated in this patent document.

Claims (37)

1. A method for processing video data, comprising the steps of: determining, during conversion between the first block of visual media data and the corresponding bitstream of visual media data, a first block encoded in a geometric partitioning mode; compiling, based on a table storing one or more motion candidates associated with motion information based on previously encoded blocks, a motion candidate list corresponding to the merge candidate list, said table comprising a history-based motion vector prediction (HMVP) table; determining, contained in the bit stream, a geometric partitioning mode merging index for a geometric partitioning of the first block, the geometric partitioning mode merging index and the motion candidate list referring to motion information of the first block; And transforming the first block using the motion information of the first block. 2. The method of claim 1, wherein the motion information used by the first block is not stored in said table. 3. The method of claim 1, wherein said table is not updated after the first block is converted. 4. The method according to claim 1, further comprising the step of: determining, during conversion between the second block of visual media data and the corresponding encoded representation of the visual media data, motion information of the second block based on the same table used for the first block, wherein the second block is encoded without using the geometric partitioning mode. 5. The method of claim 1, wherein the step of compiling a list of motion candidates further comprises compiling a list of motion candidates based on the motion candidates obtained from the spatial and/or time blocks. 6. The method of claim 1, wherein the step of constructing, based on a table storing one or more motion candidates associated with motion information based on pre-coded blocks, a motion candidate list corresponding to the merge candidate list, comprises the substep of: adding at least one motion candidate obtained from the table for the first block. 7. The method of claim 6, wherein the step of adding motion candidates obtained from the table for the first block comprises a substep of: adding at least one motion candidate obtained from the table to the list of motion candidates after receiving motion candidates from spatial blocks and/or time blocks. 8. The method of claim 6, wherein the at least one motion candidate obtained from the table is set equal to the corresponding at least one motion candidate in the table. 9. The method of claim 6, wherein the order of checking motion candidates in the table to be added to the motion candidate list is based on motion candidate indicators in the table. 10. The method of claim 1, wherein the geometric partitioning mode comprises a plurality of partitioning schemes, wherein at least one partitioning scheme divides the first block into two geometric parts, at least one of which is non-rectangular and non-square. 11. The method according to claim 1, in which the geometric partition mode contains a triangular partition mode. 12. The method of claim 1, wherein the transform step comprises decoding a first block from the corresponding bit stream. 13. The method of claim 1, wherein the conversion step comprises encoding the first block into a corresponding bitstream. 14. A video processing device comprising a processor and a computer-readable recording medium storing instructions that cause, when executed by the processor, execution by the processor: determining, during conversion between the first block of visual media data and the corresponding bitstream of visual media data, a first block encoded in a geometric partitioning mode; generating, based on a table storing one or more candidates associated with motion information based on previously encoded blocks, a motion candidate list corresponding to the merge candidate list, said table comprising a history-based motion vector prediction (HMVP) table; determining, contained in the bitstream, a geometric partitioning mode merging index for a geometric partitioning of the first block, wherein the geometric partitioning mode merging index and the motion candidate list refer to motion information of the first block; And performing a transformation of the first block using the motion information of the first block. 15. A computer-readable recording medium storing instructions causing the processor to execute: determining, during conversion between the first block of visual media data and the corresponding bitstream of visual media data, a first block encoded in a geometric partitioning mode; generating, based on a table of one or more candidates associated with motion information based on previously encoded blocks, a motion candidate list corresponding to the merge candidate list, said table comprising a history-based motion vector prediction (HMVP) table; determining, contained in the bit stream, a geometric partitioning mode merging index for a geometric partitioning of the first block, the geometric partitioning mode merging index and the motion candidate list referring to motion information of the first block; And performing a transformation of the first block using the motion information of the first block. 16. A method for storing a video bitstream, comprising the steps of: determining, while generating a visual media bitstream from the first visual media block, a first block being a coding unit encoded in a geometric partitioning mode being a prediction mode for the coding block; compiling, based on a table storing one or more motion candidates associated with motion information based on previously encoded blocks, a motion candidate list corresponding to the merge candidate list, said table comprising a history-based motion vector prediction (HMVP) table; determining, contained in the bit stream, a geometric partitioning mode merging index for a geometric partitioning of the first block, the geometric partitioning mode merging index and the motion candidate list referring to motion information of the first block; generating a bit stream from the first block using the motion information of the first block; And storing the bit stream on a non-volatile computer-readable recording medium.