KR100951301B1 - Method of inter-frame/intra block predictive coding in video coding - Google Patents

Abstract

The present invention relates to an inter-screen / in-picture predictive coding method in video encoding, wherein a coding block unit is extended to at least one super macro block having a size larger than 16 × 16 macro blocks, and each super macro block is variously sized. After inter-prediction is performed on the divided sub-blocks including each super macro block and selects and encodes any one of them as the inter-prediction encoding result, thereby improving encoding efficiency. It is possible to improve the coding efficiency by reducing the amount of residual signal generated by greatly improving the prediction accuracy by using pixel warping based on motion model as well as the conventional coding method using block matching. .

H.264, AVC, Super Macro Block, Block Segmentation, Motion Model, Inter-screen Prediction, Rate-Distortion Cost, Encoder

Description

Inter-screen / in-picture predictive coding method in video coding {METHOD OF INTER-FRAME / INTRA BLOCK PREDICTIVE CODING IN VIDEO CODING}

The present invention relates to a predictive encoding method using a super macro block or a motion model in video encoding. The present invention relates to a method of expanding a macroblock of 16 × 16 pixel units or more by dividing a size of a lower block into various sizes and then predicting encoding. Encoding efficiency can be further improved, and this can be further improved by reducing the amount of residual signal generated by greatly improving the prediction accuracy by using pixel warping based on motion model as well as the conventional coding method using block matching. The present invention relates to an inter-screen / intra-prediction coding method in a video encoding.

Generally, ISO / IEC 14496-10 (MPEG-4 Advanced Video Coding) or H.264 and ISO / IEC 14496-10 Amendment 3 (MPEG-4), a video compression encoding technology jointly standardized by ISO / IEC and ITU-T. Scalable Video Coding), the SMPTE standard VC-1, and the Audio Video coding Standard (AVS) specification have made significant progress in video data compression efficiency.

There are many factors that can improve the video compression efficiency, but in particular, existing video coding standards (MPEG-1 Video, MPEG-2 Video, MPEG-4 Part 2 Visual, H.261, H.263, etc.) Unlike dividing the size of the screen to be encoded into units of macroblocks (16 × 16) and then predictively encoding the macroblocks, the macroblocks are 16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 Further subdividing into x8 and 4x4 units, performing prediction coding on these sub-blocks, and encoding a block that generates cost minimization in terms of rate-distortion cost is an optimal block mode.

In this way, the compression efficiency can be greatly improved by greatly reducing the residual signal generated by performing the motion prediction of the fine motion or the complex image more effectively.

FIG. 1 is a diagram illustrating a partition block type of a 16 × 16 macroblock unit block for inter prediction encoding in an H.264 | AVC encoder according to the prior art. FIG. 1 illustrates seven motion prediction block partition types used in H.264. Indicates.

Referring to FIG. 1, mainly small blocks 8x8, 8x4, 4x8 and 4x4 provide a large improvement in prediction performance in the case of small or fine motion regions of an image.

On the other hand, large blocks 16 × 16, 16 × 8, and 8 × 16 are mainly selected for motion prediction coding in a homogeneous region, a region with little movement, or a motion region in which translational motion exists. Tends to As such, the use of a large block for motion prediction reduces the amount of motion vector generated at the time of encoding, thereby increasing the coding efficiency.

In conclusion, motion prediction coding is performed by using various block sizes in motion prediction coding, and among the blocks that generate the least rate-distortion cost as the following Equation 1, the optimal coding final block mode is determined. do.

Here, c and r represent the current block and the reference block, and D represents the distortion degree between the two blocks. λ _MODE is the Lagrangian Multiplier. In addition, MODE means all block modes (or sizes) for performing rate-distortion optimization used for predictive encoding.

For example, when the homogeneous region of the image occupies a large portion or when the portion of the region where no motion occurs is large, the frequency at which the macro block 16x16 is determined as the final motion prediction block increases greatly.

In particular, when the resolution of the input image to be encoded is very large, the homogeneous area of the image occupies a large portion, but since the macro block of 16 × 16 size is the largest block, that is, the size of the maximum processing block unit is limited to 16 × 16. There is a limit to increasing the motion prediction coding efficiency. For this reason, when a super high resolution image is to be encoded, using a 16 × 16 macroblock, which is the existing maximum size, has a problem in that the encoding efficiency is limited.

Meanwhile, in motion prediction encoding, existing video encoding methods locate the most similar pixel block through block matching in a search region in a reference picture and transmit the benefit as a motion vector.

In this case, the block matching technique used is basically a case of performing a motion prediction assuming a translational motion. However, the movement of the camera or the movement of an object in the image is not only a tracking movement, but also a zoom-in. The zoom lens is fixed by changing the focal length of the zoom lens as if approaching a subject. -out: Changes the focal length of the zoom lens while fixing the position of the camera and shoots it away from the subject), Roll (Roll: Shoots the subject by rotating the camera), and Panning: The vertical axis of the camera is fixed. Horizontal movement of the camera in the state), Boom-up: vertical upward movement of the camera, Boom-down: vertical downward movement of the camera, and tilting (vertical movement of the camera while the horizontal axis of the camera is fixed). Movement), otherwise (Dolly-in: the camera approaches the subject, Dolly-out: the camera withdraws from the subject). FIG. 2 is a diagram for describing a motion generation type by camera movement in a video image, and illustrates a diagram of a camera's movement pattern.

3 is a conceptual diagram illustrating a motion model projected in a two-dimensional image space, and illustrates a motion in a two-dimensional space projected when a motion of a subject in a real three-dimensional space is projected in the two-dimensional space. Equation 2 and Equation 3 express the movement in the projected space in the x-axis and y-axis by the equation.

Here, u and v denote motion vectors, and f denotes a focal length of the camera, and T _x , T _y , and T _z denote translational velocities in the X, Y, and Z axes in OXYZ space, respectively. . On the other hand, R _x , R _y , and R _z represent Angular Velocity, respectively. x and y mean coordinate values in the projection image space.

As the conventional predictive coding using motion compensation is a motion prediction considering only the translational motion, it does not properly predict the complex motion in the actual two-dimensional space, and thus has a limitation of improving the predictive coding performance.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problem, and an object of the present invention is to extend a prediction coding scheme through a maximum of 16 × 16 macroblocks in video encoding, thereby to block blocks having a size larger than 16 × 16 (hereinafter, referred to as “super Macroblock ") to perform motion prediction and compensation to provide a more efficient predictive encoding method.

Another object of the present invention is to predict a more complex complex motion such as a camera motion and an object motion by finding and predicting a maximum likelihood block in a search region in a reference screen by a conventional translational motion in block-based predictive coding. The present invention provides a method of improving encoding efficiency by reducing the amount of residual signals expected to be generated by more accurately predicting a motion using a motion model and performing predictive coding based on the motion model.

In order to achieve the above object, a first aspect of the present invention is an inter prediction encoding method in video encoding, wherein a coding block unit has a size larger than 16 × 16 macroblocks, and the length of the horizontal and the vertical is a power of two. Expanding to at least one super macro block of a square defined as; Dividing the super macro block into sub-blocks of various sizes, square or rectangular, defined as powers of two; And performing inter prediction on the super macro block and the lower blocks, selecting one of the super macro blocks, and encoding the selected result as the inter prediction encoding result. Decoding, which is an inverse process of encoding, is performed in units of at least one super macro block of a square whose coding unit is larger than a 16 × 16 macroblock and whose horizontal and vertical lengths are defined as powers of two. Extracting inter-prediction data encoded in units of lower blocks divided into various sizes of squares or rectangles defined by powers of two from inter-prediction encoded data; Decoding the inter prediction data of the extracted lower block unit; And restoring the decoded data in units of lower blocks and restoring the units in super macro blocks.

Here, it is preferable to perform transform and quantization on the residual signal through the divided lower blocks including each super macro block, and then entropy-encode the result.

delete

According to a second aspect of the present invention, in an inter prediction encoding method in video encoding, the input screen is divided into coding unit blocks, and at least one motion model for motion prediction is selectively selected for each of the divided coding unit blocks. The present invention provides an inter-prediction encoding method in video encoding, wherein the prediction encoding is performed by applying the L-P encoding, and then any one of the encoding is selected as an inter-screen prediction result.

Herein, the coding unit block is preferably a macroblock or a super macroblock having a size larger than that of a 16 × 16 macroblock.

Preferably, the motion model is modeled with parameters.

Preferably, the motion model may include at least one of a translational motion model, an affine motion model, a bilinear motion model, or a perspective motion model.

Preferably, the prediction encoding process applying the motion model estimates a motion model parameter, performs motion prediction from a reference picture using the estimated motion model parameter, transforms the residual signal, and quantizes the transform coefficients. In addition, entropy encoding may be performed on the motion model parameter and the quantized transform coefficient.

According to a third aspect of the present invention, there is provided a method of inter-prediction encoding in video encoding, comprising: (a) extending an input screen in a unit of a predefined super macro block; (b) dividing each super macro block into sub-blocks of various sizes; And (c) after performing motion prediction on the divided lower blocks in units of each super macro block, determining an optimal block mode and encoding a residual signal and a motion vector obtained through the prediction. An inter prediction encoding method in video encoding is provided.

A fourth aspect of the present invention provides a method for inter prediction encoding in video encoding, comprising: (a ') obtaining a motion model parameter by selecting a motion model for a block to be encoded; (b ') generating a motion compensation block pixel from a reference picture using the obtained motion model parameter; obtaining a residual signal of the generated motion compensation block pixel value and the pixel value of the block to be currently encoded; (d ') performing steps (b') and (c ') on a selectable motion model, and finally determining a motion model for a block to be currently encoded; And (e ') performing inter-screen motion prediction encoding by using the finally selected motion model.

In the step (d '), the final determination of the motion model is preferably made in consideration of the rate-distortion cost.

Preferably, the method may further include initializing a screen order and a block order for encoding before step (a) or step (a ').

Preferably, after performing the motion prediction in step (c) or after the step (a '), the screen is input in order for encoding, the input screen is divided in block units, and then one screen The method may further include initializing an order of blocks to be encoded in the inter prediction encoding.

Preferably, in step (c) or step (d '), the rate-distortion cost through the inter prediction may be calculated and summed into a macro block or a super macro block to be encoded.

According to a fifth aspect of the present invention, there is provided an inter prediction encoding method in video encoding, comprising: inputting a screen to be encoded; Dividing the input screen into units of a predefined super macro block; Dividing each super macro block into blocks having a lower size, performing inter-screen prediction on each super macro block and the lower block, and selecting any one of the block types; And performing inter prediction encoding by using the selected prediction result of the block type.

In a sixth aspect of the present invention, in an intra prediction encoding method in video encoding, a coding block unit is extended to at least one super macro block having a size larger than 16 × 16 macro blocks, and each super macro block is variously sized. The video encoding, characterized by dividing into sub-blocks of, performing intra-prediction on the divided sub-blocks including each super macro block, comparing the results, selecting a more advantageous encoding result, and performing final encoding. In the present invention, an intra prediction encoding method is provided.

In this case, the intra prediction encoding is performed on the sub-blocks including each super macro block by performing directional prediction, and encoding the directional prediction mode that generates the least rate-distortion cost as the optimal final intra prediction. desirable.

According to a seventh aspect of the present invention, in an inter prediction encoding method in video encoding, a coding block unit is extended to at least one super macro block having a size larger than 16 × 16 macro blocks, and each super macro block is expanded in various ways. After sub-blocks are divided into sub-blocks of size, intra-prediction is performed on the divided sub-blocks including each super macro block, and the intra-prediction result to be encoded is compared with the inter-prediction result to be encoded. It is an object of the present invention to provide an inter-prediction encoding method in video encoding, characterized by selecting and encoding an advantageous prediction.

An eighth aspect of the present invention provides a method of inter-prediction encoding in video encoding, comprising: (a ") inputting a screen to be encoded, (b") dividing an input screen into coding unit blocks; (c ") determining whether the current input screen is to perform inter-prediction encoding; (d") performing intra prediction encoding if the current input screen is not inter prediction; (e ") initializing the order of unit blocks encoded in the input screen when the current input screen is inter-screen prediction; (f") selecting a motion model to be applied to perform motion prediction on the unit block to be encoded. Selecting one and performing motion prediction; (g ") performing predictive encoding using the selected motion prediction result; (h") storing the predictive encoding result and the rate-distortion cost value in step (g "); (i") the Determining whether the applied motion model is a final model, determining an optimal motion model if the final motion model is selected, and finally selecting a corresponding encoding result; (j ″) if the applied motion model is not a final model, returning to step f ″; (k ") determining whether the current unit block is the last block in the current input screen; and (l") if the current unit block is the last block in the current input screen, determining whether the current input screen is the last screen, It provides a method of inter-prediction encoding in video encoding, comprising the step of repeating the steps (a ") to (k") until the final picture.

In the step (f ″), it is preferable to further include the step of estimating the motion model parameters when performing the motion prediction by applying the motion model.

A ninth aspect of the present invention provides a recording medium on which a program for executing the above-described inter prediction encoding method in video encoding is recorded.

According to the inter-screen / intra-prediction coding method of the video encoding of the present invention as described above, the size of the block to be encoded is extended beyond the size of the existing 16x16 macroblock, and this is further divided into lower blocks. By selectively using and effectively using an image having a large number of homogeneous regions, there is an advantage in that predictive encoding can be effectively performed by reducing the number of motion vectors to be coded and reducing the load of motion prediction.

In addition, according to the present invention, the coding efficiency can be increased by reducing the amount of residual signals generated by performing more accurate motion prediction using various motion models, and by warping the reference image by the motion model. By performing motion compensation from the refinery to sub-pel steps of 1 / 2-pixel, 1 / 4-pixel, and 1 / 8-pixel, it is possible to obtain a large amount of computational reduction of the motion prediction required. have.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the following embodiments of the present invention may be modified into various other forms, and the scope of the present invention is not limited to the embodiments described below. The embodiments of the present invention are provided to enable those skilled in the art to more fully understand the present invention.

First, in conventional inter prediction, a block is encoded by dividing a block to be encoded into macroblock units having a maximum size of 16 × 16. In particular, in ISO / IEC 14496-10 (MPEG-4 Advanced Video Coding) or H.264 standard, as shown in FIG. 1, the macro block is divided into seven sub-blocks to finalize a block that minimizes rate-distortion cost in encoding. Selectively perform prediction encoding.

As described above, limiting the block unit to be encoded to 16 × 16 macroblock units greatly increases the number of macroblocks when performing inter-prediction in homogeneous regions or regions without motion. In particular, in the encoding of an ultra high resolution image, the size of the macro block is relatively small, resulting in a decrease in the coding efficiency.

FIG. 4 is a diagram illustrating an example of a block and block division type with a super mat of 16 × 16 macroblock size or larger applied to an embodiment of the present invention. The macroblock is expanded in size and the lower block is divided into various sizes. Thus, the present invention can be effectively applied to the above-mentioned homogeneous region encoding or encoding of a motion-free image.

That is, the block size can be extended in the form of a 32x32 super macro block and its subdivided blocks, a 64x64 super macro block and its subdivided blocks, a 128x128 super macro block and its subdivided blocks, and the like.

In addition, in consideration of the complexity of the encoder, it is possible to limit the expansion of the block size or limit the number of block division types of the lower block. In actual encoding, as in Equation 1 described above, a block mode that minimizes the rate-distortion cost may be selected and encoded.

In addition, when transform coding is applied to a residual signal of an increased sized super macro block, encoding efficiency is increased by using a transform kernel larger than 4 × 4, 8 × 8, or 16 × 16. Can be increased.

In addition, coding efficiency may be further increased by selectively applying a motion prediction model to a block to be encoded in inter prediction encoding.

Hereinafter, the motion prediction model will be described in detail.

라고 하면 참조 영상으로부터 하기의 수학식 4와 같이 정의할 수 있다.In order to describe the motion prediction model in more detail, first, f _c and f _r are used as current and reference pictures to be encoded, respectively. (x, y) represents one pixel coordinate value of the current block to be encoded by f _c . In addition, the prediction of the current frame to be encoded

In this case, it can be defined as shown in Equation 4 below from the reference image.

Here, (u, v) represents a coordinate value of the reference image. Equation 4 denotes a geometric transformation in which motion compensation corresponds to a coordinate value of the current image, and may be generalized as shown in Equation 5 below.

Where g _x and g _y represent spatial transform functions. This spatial transformation is called texture mapping or image mapping.

A 2D translational motion model, which is a block matching method generally used for motion prediction, is expressed by Equation 6 below.

However, as described above, not only the translational movement but also the movement of zoom, rotation, panning, boom, tilting, and dolly are combined in the actual two-dimensional projection image. Are mixed.

More accurate motion models are needed to properly predict these complex movements. Equation 7 below represents an Affine motion model, Equation 8 represents a bilinear motion model, and Equation 9 represents a perspective motion model.

Here, a _i represents a motion model parameter. When performing prediction coding based on motion model, encoding efficiency may be increased by applying a model that minimizes the rate-distortion cost among the models.

In the present invention, the size of a unit block to which a motion model is applied may be a super macro block and a lower block thereof, and an optimal motion model may be selected according to coding efficiency. Meanwhile, the motion model applied to the embodiment of the present invention is preferably modeled by a parameter.

Meanwhile, although an embodiment of the present invention has been described mainly for the inter prediction encoding method in video encoding, the present invention is not limited thereto and may be applied to the intra prediction encoding in video encoding.

That is, when performing intra prediction encoding, first, a coding block unit is extended to at least one super macro block having a size larger than 16 × 16 macro blocks, and each super macro block is divided into sub-blocks having various sizes. Intra-picture prediction is performed on the divided lower blocks including each super macro block, and the results are compared to select a more advantageous encoding result and to perform final encoding.

In this case, intra prediction encoding is performed on the sub-blocks including each super macro block by performing directional prediction, and encoding the directional prediction mode that generates the least rate-distortion cost as the optimal final intra prediction. desirable.

FIG. 5 is a flowchart illustrating an inter prediction encoding method in video encoding according to an embodiment of the present invention, and illustrates an inter motion prediction encoding method based on a selective motion model in video encoding.

Referring to FIG. 5, first, the order of screen i for encoding is initialized (i = 0) (S100), and then the screen i is input in order for encoding (S101).

Then, after dividing the screen i input in step S101 into coding unit blocks (macro blocks or super macro blocks) (S102), it is determined whether the current screen i is to perform inter-prediction encoding. (S103).

If it is determined in step S103 that the current screen i is not inter-screen prediction, intra prediction coding is performed (S104). Otherwise, if the current screen i is inter-screen prediction, The order of the unit blocks j encoded in the screen i is initialized (j = 0) (S105).

Subsequently, one motion model to be applied to perform the motion prediction on the unit block j to be encoded is selected, motion prediction is performed on the motion block (S106), and prediction prediction is performed using the result (S107). The predictive encoding result and the rate-distortion cost value are stored (S108).

Then, it is determined whether the used motion model is the final motion model (S109), and if it is not the final motion model, the method returns to step S106 to select a motion model to be applied next, performs motion prediction, and repeats this process.

Otherwise, if the use motion model is the final motion model as a result of the determination in step S109, an optimal motion model is determined and the corresponding encoding result is finally selected (S110).

Subsequently, it is determined whether the current unit block j is the last block in the screen i (S111). If not, the next coding unit block j is input and the process returns to step S106.

Otherwise, as a result of the determination in step S111, if the current unit block j is the last block in the screen i, it is determined whether the current screen i is the last screen (S112). Returning to step S101, the next screen is input and the above-described processes are performed.

Meanwhile, the inter-screen / in-screen predictive encoding method in video encoding according to an embodiment of the present invention may also be implemented as computer readable codes on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored.

For example, the computer-readable recording medium may be a ROM, a RAM, a CD-ROM, a magnetic tape, a hard disk, a floppy disk, a removable storage device, a nonvolatile memory (Flash memory). Optical data storage, and the like, and also implemented in the form of a carrier wave (eg, transmission over the Internet).

The computer readable recording medium can also be distributed over computer systems connected over a computer network so that the computer readable code is stored and executed in a distributed fashion.

Although preferred embodiments of the inter-screen / in-screen prediction coding method in video encoding according to the present invention have been described above, the present invention is not limited thereto, but the claims and the detailed description of the invention and the scope of the accompanying drawings. Various modifications can be made therein and this also belongs to the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram showing a divided block type of a 16 × 16 macroblock unit block for inter prediction encoding in a conventional H.264 | AVC encoder.

FIG. 2 is a diagram for describing a motion generation type due to camera movement in a video image. FIG.

3 is a conceptual diagram illustrating a motion model projected on a two-dimensional image space.

4 is a diagram illustrating an example of a block and a block division type of a super mat having a size of 16 × 16 macroblocks or more applied to an embodiment of the present invention.

5 is an overall flowchart illustrating a method of inter prediction encoding in video encoding according to an embodiment of the present invention.

Claims (18)

In the inter prediction encoding method in video encoding, Extending the coding block unit into at least one super macro block of a square having a size larger than a 16 × 16 macro block and having a horizontal length and a vertical length defined as a power of two; Dividing the super macro block into sub-blocks of various sizes, including squares or rectangles, defined as powers of two; And Performing inter prediction on the super macro block and the lower blocks, and selecting one of the super macro block and the sub blocks as an inter prediction encoding result and encoding the same. The method of claim 1, wherein the encoding step After estimating a motion model parameter, performing motion prediction from a reference picture using the estimated motion model parameter, transforming the residual signal, quantizing transform coefficients, and entropy for the motion model parameter and quantized transform coefficient. An inter prediction encoding method in video encoding, characterized in that encoding is performed. delete The method of claim 1, wherein the encoding step A method of inter prediction encoding in video encoding, characterized by selectively applying at least one motion model for motion prediction. delete The method of claim 4, wherein the motion model An inter prediction encoding method in video encoding, characterized by a model. The method of claim 4, wherein the motion model A method of inter-prediction encoding in video encoding, which is selected from a translational motion model, an affine motion model, a bilinear motion model, and a perspective motion model. delete delete In the inter prediction encoding method in video encoding, (a) extending the input screen into at least one super macro block of a square having a coding block unit larger than a 16x16 macroblock and having a horizontal length and a vertical length defined as a power of two; (b) dividing the super macro block into sub-blocks of various sizes, including squares or rectangles, defined as powers of two; (c) selecting a motion model with respect to the encoding target block to obtain a motion model parameter; (d) generating a motion compensation block pixel from the reference picture using the obtained motion model parameter; obtaining a residual signal of the generated motion compensation block pixel value and the pixel value of the block to be currently encoded; (f) performing steps (d) and (e) on a selectable motion model, and finally determining a motion model for a block to be currently encoded; And and (g) performing inter prediction motion coding using the finally selected motion model. The method of claim 10, wherein step (f) And determining the final determination of the motion model in consideration of a rate-distortion cost. The method of claim 10, And initiating a picture order and a block order for encoding before step (a). In the intra prediction encoding method in video encoding, Extending the coding block unit into at least one super macro block of a square having a size larger than a 16 × 16 macro block and having a horizontal length and a vertical length defined as a power of two; Dividing the super macro block into sub-blocks of various sizes, including squares or rectangles, defined as powers of two; And And performing intra-picture prediction on the super macro block and the sub-blocks, comparing the results, selecting one prediction, and then performing final encoding on the super macro block and the sub-blocks. The method of claim 13, wherein the selected one prediction is An intra prediction encoding method in video encoding, characterized by performing directional prediction and generating the least rate-distortion cost. In the inter prediction encoding method in video encoding, Extending the coding block unit into at least one super macro block of a square having a size larger than a 16 × 16 macro block and having a horizontal length and a vertical length defined as a power of two; Dividing the super macro block into sub-blocks of various sizes, including squares or rectangles, defined as powers of two; And Performing inter prediction on the super macro block and the lower blocks, comparing the results, selecting one prediction, and performing final encoding on the super macro block and the lower blocks. In the inter-prediction decoding method in video encoding, From inter-prediction encoded data input in units of at least one super macro block of a square having a larger coding unit than a 16 × 16 macroblock, and having a horizontal and vertical length defined as a power of two, Extracting inter prediction data encoded in units of lower blocks divided into various sizes of squares or rectangles defined by powers of two; Decoding the extracted inter prediction data in units of lower blocks; And And superimposing the decoded data in units of the lower block and restoring the units in the super macro block unit. delete delete

Images