KR102728144B1 - Method and apparatus for video decoder with motion vector refinement - Google Patents

Abstract

The present invention relates to a device and method including a method for selectively performing motion compensation on motion information received from an encoder in a video decoder and performing motion compensation using the same.

Description

{METHOD AND APPARATUS FOR VIDEO DECODER WITH MOTION VECTOR REFINEMENT}

The present invention relates to video encoding technology, and more particularly, to a method for effectively encoding/decoding a video by correcting a motion vector in a decoder.

Recently, the demand for high-resolution video services such as FHD (Full High Definition) and UHD (Ultra High Definition) and high-quality video services has increased.

The purpose of the present invention is to provide a method and device capable of effectively transmitting motion information in a video encoder/decoder for high-resolution images such as FHD (Full High Definition) and UHD (Ultra High Definition), by additionally correcting motion vector information transmitted from an encoder to a decoder.

In one embodiment of the present invention for solving the above problem, a video decoding device and method have an information extraction step for motion vector compensation, and can perform motion compensation in a decoder by adding a compensation motion vector to information of a motion vector directly transmitted from an encoder.

According to the problem solving means of the present invention described above, it is possible to enable detailed video decoding through additional correction of a motion vector transmitted through a motion information correction device or step in a decoder, thereby improving the efficiency of video encoding/decoding.

FIG. 1 is a block diagram of a decryption device according to an embodiment of the present invention.
FIG. 2 is a flowchart of a case in which a motion vector is obtained and motion compensation is performed using the motion vector in an embodiment in which the present invention is selectively applied in inter-screen prediction.
FIG. 3 is a flowchart of a case in which motion vector correction is performed repeatedly in the embodiment described in FIG. 2.
FIG. 4 is a diagram illustrating a case in which the size of a unit correcting a motion vector changes in a number of steps in which motion vector correction is performed repeatedly in the embodiment of FIG. 3.
Figure 5 is a diagram showing the size of a block of motion vector correction that changes as the block is divided and its size changes at each step while motion vector correction is performed repeatedly.
FIG. 6 is a diagram for an embodiment in which a differential motion vector is optionally transmitted in the embodiment of FIG. 2.
FIG. 7 is a flowchart of an embodiment in which a motion vector correction step is repeatedly performed in the embodiment of FIG. 6.

Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings. In describing embodiments of this specification, if it is determined that a detailed description of a related known configuration or function may obscure the gist of this specification, the detailed description will be omitted.

The description in the present invention of "comprising" a particular configuration does not exclude configurations other than the configuration, and means that additional configurations may be included within the scope of the practice of the present invention or the technical idea of the present invention.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

In addition, the components shown in the embodiments of the present invention are independently depicted to indicate different characteristic functions, and do not mean that each component is formed as a separate hardware or software configuration unit. That is, each component is included by listing each component for convenience of explanation, and at least two components among each component may be combined to form a single component, or one component may be divided into multiple components to perform a function, and such integrated embodiments and separated embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

In addition, some components may not be essential components that perform essential functions in the present invention, but may be optional components that are merely used to improve performance. The present invention may be implemented by including only essential components for implementing the essence of the present invention, excluding components that are merely used to improve performance, and a structure that includes only essential components, excluding optional components that are merely used to improve performance, is also included in the scope of the present invention.

The block used in the invention can be a basic block unit of decoding, a prediction block unit, or a transform block unit. In addition, the block boundary can be a boundary of a decoding block, a boundary of a prediction block, or a boundary of a transform block.

FIG. 1 is a block diagram of a decoding device according to an embodiment of the present invention. The decoder, which receives a bitstream from an encoder, performs decoding largely through inter-screen prediction (115) and intra-screen prediction (114). When performing inter-screen prediction during decoding, the inter-screen prediction may be performed using only the motion information and predicted motion vector information transmitted from the encoder (115-1) according to an embodiment of the present invention, or the motion vector may be obtained using the transmitted motion vector and predicted motion vector (115-2), and then the inter-screen prediction may be performed using the corrected motion vector after additional correction (131). The decoder may receive selection information regarding a method for performing inter-screen prediction, and may selectively apply motion vector correction (131) according to the selection information. If the currently decoded coding unit performs inter-screen prediction after additional correction (131), the decoding unit requires separate motion vector storage (130) to store the corrected motion vector. FIG. 2 is a flowchart of a case in which a motion vector is obtained and motion compensation is performed using the motion vector in an embodiment in which the present invention is selectively applied in inter-screen prediction. The decoder extracts information about the current coding unit from the bitstream (200) transmitted from the encoder. Information related to motion vector compensation, such as the unit for performing motion vector compensation, whether to select it, and the number of times to perform it, is extracted (210) from the information about the current coding unit, and if motion vector compensation is selected, motion vector compensation (220) is performed.

Information related to motion vector compensation can be transmitted from the encoder for each coding unit, but can also be transmitted at various levels such as a picture group, a picture, a frame group, a slice group, a slice, a tile, and a coding unit depending on the characteristics of the information or an agreement between a sub-processor and a decoder. Transmission can also be omitted depending on an agreement between a sub-processor and a decoder, and can also be transmitted via a high-level syntax (HLS). In the flowchart of Fig. 2, it can be sufficiently inferred from general knowledge in the field that information related to motion vector compensation in the current coding unit may not be extracted only at the decoding stage of the current coding unit, since it can be obtained through parsing/decoding in the unit to which the information is transmitted.

When performing motion vector compensation, the decoder primarily performs motion compensation (241) using the received motion vector differential value and predicted motion vector (PMV). At this time, the motion vector is calculated by the following equation.

(In Equation 1), i and j represent the position of the current coding unit, and t represents the time of the frame or slice in which decoding is currently performed.

Afterwards, motion vector correction (220) is performed using the image value on which motion compensation has been performed, and motion compensation (242) is performed using the changed motion vector (231). At this time, the motion vector is calculated using the following equation.

And when the motion vector is corrected (220), it is stored (130) in the decoder. Since the changed and stored motion vector is used when calculating the predicted motion vector (PMV) in the surrounding block or the next image, it is necessary to store (130) after performing the motion vector correction. For example, the corrected motion vector is used to determine the motion vector of the block or picture to be encoded/decoded in the subsequent order, and the motion vector before correction can be used to determine the motion vector of the surrounding block to be encoded/decoded in the subsequent order. If the motion vector correction is not selected, the current coding unit can be calculated by the existing motion vector calculation method (232), and since this can be calculated with the value of the predicted motion vector and the value of the transmitted differential motion vector, a separate storage step is not required.

FIG. 3 is a flowchart of a case where motion vector compensation is repeatedly performed in the embodiment described in FIG. 2. When motion vector compensation is repeatedly performed in the encoder, the encoder can transmit related information (310) to the decoder. This can be transmitted as depth information for motion vector compensation. If the motion vector compensation flag (311) is 1, that is, if the motion vector compensation method is selected, it is necessarily performed once, so the depth information repetition count (MV_refine_Count) becomes a value that adds 1 to the depth information. Of course, if the decoder and the encoder agree on the number of repetitions of motion vector compensation or if the decoder can calculate the condition for completing the motion vector compensation repetition, the transmission of motion vector compensation depth information (repetition count) information from the encoder to the decoder can be omitted. Since the description of other steps in FIG. 3 is the same as in FIG. 2, it is omitted. However, if the correction of the motion vector is performed repeatedly, not only the final motion vector but also the storage (130) depending on the implementation or embodiment may be performed repeatedly.

FIG. 4 is a diagram illustrating a case in which the size of a unit for correcting a motion vector changes in a number of steps in which motion vector correction is performed when motion vector correction is repeatedly performed in the embodiment of FIG. 3. When motion vector correction is repeatedly performed in a current coding unit that refers to two reference images and has a size of N×M (N and M may be the same depending on the convention of encoding/decoding), the size of the motion vector correction unit may change as in (400) for each repetition number step. In FIG. 4, in the first motion vector correction step, correction is performed in a size of (N/2)×(M/2), and in the second motion vector correction step, correction is performed in a form that is further divided into a size of (N/4)×(M/4) only for the upper left block (402), and is performed in the same size for other blocks (403). Whether motion compensation will be repeatedly performed on blocks of the same size as the previous compensation step (i.e., blocks for which segmentation has not been performed) can be transmitted from the encoder to the decoder through information related to motion vector compensation, or can be determined by a promise of the encoding/decoding period. Fig. 5 is a diagram showing the size of a block of motion vector compensation that changes as the block is segmented and its size changes at each step while motion vector compensation is repeatedly performed. The size of the block of performance may be constant according to the segmentation depth information, as in Fig. 5-(a), or may be rectangular, as in Fig. 5-(e). In addition, it may be a mixed shape of rectangular and square, as in (b), and not all blocks may be segmented the same number of times, as in 5-(c), or the ratio may vary, as in 5-(d). Figs. 5-(e), (f), and (g) are diagrams showing a case where the shape of the segmentation is not a square, but a triangle or a combination of a square and a triangle. FIG. 5 is a diagram illustrating a part of the embodiment, which may exist in various forms by combination thereof, and information such as shape, size, ratio, and number of times related to the division may be transmitted from the encoder to the decoder as motion compensation information, or may be performed by an agreement between the sub/decoder. In addition, the minimum unit of division may be down to a pixel unit, and blocks with the same motion compensation information may be merged again. In addition, even if the shape of the initial coding unit is not rectangular, it may be divided into various forms as in the embodiment of FIG. 5. In 4, two reference images were used to compensate for the motion vector, but depending on the embodiment, the reference images may be one or three or more. In addition, the reference images may change as they are repeatedly performed, and the method used to obtain the motion vector compensation information from the reference images may also change. All of the information may be transmitted from the encoder to the decoder, or may be performed by an agreement between the sub/decoder.

FIG. 6 is a diagram for an embodiment in which a differential motion vector is optionally transmitted in the embodiment of FIG. 2. In a typical decoder, the differential motion vector is added to the predicted motion vector as in the following equation and calculated as the actual motion vector of the corresponding coding unit.

However, in the present invention, if the motion vector of the coding unit changes through correction of the motion vector in the decoder and the value obtained in this process is identical to the differential motion vector, conventional decoding becomes possible without the differential motion vector, and encoding efficiency can be obtained while omitting transmission of the differential motion vector.

Fig. 6 is an example of selectively transmitting the value of the differential motion vector from the encoder to the decoder, and in each case, the motion vector can be calculated as follows (Equation 6) and (Equation 7).

(634), (633) can be calculated. The motion vector calculated in this way can be in the form of (640) and (641) through motion vector compensation, and this value is stored (130) for the next motion vector prediction. Thereafter, motion compensation is performed with the compensated motion vector value. In the embodiment of Fig. 6, the selection information for the transmission of the motion vector difference value can be transmitted at various levels such as an image group, an image, a frame group, a slice group, a slice, a tile, a coding unit, etc., and can be omitted if there is an agreement between the sub/decoder. For example, if there is an agreement between the sub/decoder not to unconditionally transmit the motion vector difference value for the motion vector compensation block, the corresponding selection information (620) can be omitted. The agreement and omission of the corresponding selection information (620) can be possible under various conditions, and since the conditions are agreements between the sub/decoder, the embodiments can also be diverse.

Fig. 7 is a flowchart of an embodiment in which the motion vector correction step is repeatedly performed in the embodiment of Fig. 6. The method is a combined form of the embodiments of Fig. 6 and Fig. 3, and when repeatedly performed, the shape of the block may be the same in all stages, or may be divided into various shapes as in Figs. 4 and 5.

Claims (5)

As an inter prediction method,
A step of obtaining motion vector correction information of the current block;
A step of generating a motion vector of the current block;
A step of correcting the motion vector of the current block based on the above motion vector correction information; and
A step of performing motion compensation for the current block using the above-mentioned compensated motion vector is included,
An inter prediction method in which the above-described compensated motion vector is derived using a reconstructed block decoded before the current block. In the first paragraph,
An inter prediction method in which the above-mentioned corrected motion vector is used to determine the motion vector of an image to be decoded later. In the first paragraph,
An inter prediction method in which the correction of the above motion vector is performed based on the size of the current block. As an inter prediction method,
Step of generating a motion vector of the current block;
A step of determining motion vector compensation information of the current block based on the motion vector of the current block; and
Comprising a step of encoding the above motion vector correction information,
The above motion vector compensation information is used to compensate the motion vector of the current block in the decoding process,
An inter prediction method, wherein the corrected motion vector of the current block in the decoding process is derived using a reconstructed block decoded before the current block in the decoding process. A non-transitory computer-readable recording medium storing a bitstream generated by an image encoding method using an encoding device, wherein the image encoding method comprises:
Step of generating a motion vector of the current block;
A step of determining motion vector compensation information of the current block based on the motion vector of the current block; and
Comprising a step of encoding the above motion vector correction information,
The above motion vector compensation information is used to compensate the motion vector of the current block in the decoding process,
A recording medium, wherein the corrected motion vector of the current block in the decoding process is derived using a restored block decoded before the current block in the decoding process.