JP2004253826A - Adaptive macroblock scanning method for video and image compression - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an adaptive macroblock scanning method for video coding and decoding and an adaptive scanning of sub-blocks of macroblock. <P>SOLUTION: The scanning direction of a group of macroblocks (GMB) can be adaptively changed, and thus the normal scanning direction and the inverse scanning direction are used to code the groups of the macroblocks. In the methods to determine the groups of the macroblocks and the scanning direction of the groups, different modes of the size of the GMBs, including slice-size GMB, fixed-size GMB, or flexible-size GMB are used. Finally the rate distortion is used to determine the scanning direction of the GMBs, as well as the sub-blocks of a macroblock. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to moving picture coding and still picture coding, and more particularly to coding that can be used in intra-frame and inter-frame coding for video compression.
[0002]
[Prior art]
Video encoding is performed according to JPEG, MPEG-1, MPEG-2, MPEG-4, II. 261, II. H.263 has been widely used in many international standards. The purpose of video compression is to display a video sequence while maintaining high image quality with as few bits as possible. In video compression, efficient encoding within a frame is very effective for random access and error recovery, and attention to I-frames cannot be ignored.
[0003]
Intra prediction using I-frames removes the correlation between pixel values and thereby plays an important role in video compression. Residuals can be reduced by using good predictors, and the bit length when encoding these residues can be significantly reduced. Current intra-frame coding methods are based on two types of prediction. One is multi-mode spatial prediction (for example, intra-screen 4 × 4 and intra-screen 16 × 16 prediction), and the other is adaptive prediction (for example, AC / DC prediction).
[0004]
Since the transfer sequence of the macroblocks of a picture is always left-to-right and top-to-bottom, all intra prediction methods are based primarily on the top left available decoded macroblock (Macroblock). Therefore, the intra prediction method describes vertical, horizontal, and diagonal directions. However, they are not suitable for off-diagonal prediction. The reason is that according to the transfer sequence of the MB (macroblock) or block, the right MB always appears after decoding the current MB or block. This asymmetric prediction may not work with existing intra-picture coding methods for certain types of images, but will work for some types of images. Thus, different images have different properties, such as orientation of edges. To remedy this shortcoming, for example, in the current JVT (MPEG-4 Part 10) model, the original 6-mode intra prediction was replaced with a more complex 9-mode intra prediction that attempted non-diagonal prediction. . However, the improvement of 9-mode intra prediction is limited due to lack of information.
[0005]
Further, the adaptive macroblock scanning method can be executed for all pictures predicted to be inter-picture. For inter-picture coded pictures (inter pictures), the motion prediction of the sub-block is made from the previous picture and the subsequent pictures, and the prediction can be made from each of the up, down, left and right directions. In many cases, the design of motion vector prediction (eg, median prediction) is also asymmetric. One example is the motion vector prediction of the current JVT model.
[0006]
In the present invention, a method using an adaptive macroblock scan direction suitable for encoding all pictures, that is, I pictures, P pictures, and B pictures is described. The method solves the problems that exist in today's asymmetric prediction and standards in current prediction and coding methods. In addition, the application of the invention in various video technologies simplifies the current complex asymmetric intra-frame prediction while at the same time ensuring promising coding efficiency.
[0007]
[Problems to be solved by the invention]
Since the transfer sequence of a macroblock of an image is always left to right and top to bottom, the intra prediction method (including multi-mode adaptive spatial frequency domain prediction) is mainly based on the upper left available MB (macroblock). The intra prediction method describes vertical, horizontal, and diagonal directions. However, they are not suitable for off-diagonal prediction. The reason is that according to the transfer sequence of MBs or blocks, the right MB always appears after decoding of the current MB or block. This asymmetric prediction makes current video coding methods effective for certain types of pictures, which may not be valid for other types of pictures.
[0008]
In addition, there are also asymmetry problems in motion estimation.
[0009]
For some groups of MBs, a key concept of the present invention is to adapt adaptive MBs if they were predicted from the upper right MB (ie, using the reverse scan direction), since the prediction would be more efficient. The use of a scan direction, whereby the transfer sequence of a group of MBs in a line and a slice can be either left-to-right or right-to-left depending on the coding efficiency.
[0010]
The appropriate scanning direction of the GMB can be determined using a rate distortion technique.
[0011]
Using the methodology of the present invention, the scan direction of a sub-block of a macroblock can be adaptively determined according to coding efficiency.
[0012]
[Means for Solving the Problems]
The present invention discloses an adaptive MB scan. Adaptive MB scanning allows for a transfer sequence of macroblocks that changes adaptively for macroblocks that may be transmitted from left to right or right to left. Based on this transfer sequence, the prediction of the current macroblock is based on the left / top or right / top pixel. The encoder uses a rate-distortion algorithm to determine which transfer sequence (left to right or right to left) is better. Thus, the prediction is not symmetric, but the transfer sequence is selected adaptively to the characteristics of the coded picture.
[0013]
(Action)
The present invention discloses an adaptive MB scanning method. This provides an efficient solution for solving the asymmetric problem of the current intra- and inter-screen prediction methods. The method comprises the following concept or operation.
[0014]
A picture / frame is composed of a plurality of slices.
A slice is composed of a plurality of MB groups (GMB).
MBs in an MB group (GMB) are always in the same slice.
In order to describe the size of GMB, three cases of slice size GMB, fixed size GMB, and variable size GMB are used.
According to the adaptive MB scanning method, the scanning direction of GMB can be adjusted. That is, it can be scanned from left to right (forward) or right to left (reverse).
The scan direction within the MB corresponds to the direction of the GMB, otherwise it may be indicated using a codeword in the header of the MB.
If the scan direction is forward, the prediction of each MB of the GMB is often based on the previously decoded upper left MB.
If the scan direction is reverse, the prediction of each MB of the GMB is often based on the previously decoded upper right MB.
[0015]
The size of the MB is variable, such as 4x4, 8x8, 16x16, and so on. Common such as spatial intra prediction (4 mode 16 × 16 intra prediction, 9 mode 16 × 16 intra prediction, 6 mode 4 × 4 intra prediction and other intra prediction) and frequency domain intra prediction (AC / DC prediction etc.) Almost all of the optimal intra-frame prediction methods are suitable for use with this adaptive MB scanning method.
[0016]
The present invention also discloses an adaptive sub-block scanning method. A macro block may be composed of a plurality of sub blocks each having a different intra prediction mode. The scan direction of these sub-blocks may also be adaptive. In one adaptive mode, all sub-blocks are scanned using a forward or reverse scan direction. In another adaptive mode, the scan direction of the sub-block of each line of the macroblock may be determined adaptively.
[0017]
The present invention also discloses a method of determining an MB group (GMB), a method of determining a GMB scanning method, and a method of encoding GMB information and its scanning direction.
[0018]
The present invention can also be appropriately implemented so as not to affect the existing intra prediction module, inter prediction module, and motion prediction module. The present invention also has the option of improving those modules.
[0019]
BEST MODE FOR CARRYING OUT THE INVENTION
Preferred embodiments of the present invention will be described in the following four parts.
[0020]
1. GMB Adaptive Scan In the present invention, MBs can be scanned and encoded in the forward or reverse direction. The basic rule for forming a GMB is that the MBs of the GMB should belong to the same slice.
[0021]
Further, there are three cases of determining a GMB size: a slice size GMB, a fixed size GMB, and a variable size GMB.
[0022]
After determining the size of the GMB, the encoder encodes each GMB using both the forward scan direction (forward scan direction) and the reverse scan direction (reverse scan direction). It then determines which scan direction to use according to the rate distortion criteria (ie, shorter bit length and better image quality).
[0023]
Finally, the GMB information, that is, the size of the GMB, is encoded into the transmission stream together with the scanning direction of the GMB.
[0024]
2. Determination of GMB Size There are three methods for determining the size of GMB: slice size GMB, fixed size GMB, and variable size GMB.
[0025]
2.1 Slice size GMB
In this method, the frame / picture GMB is set the same as the slice. Note that a slice may be composed of a plurality of lines, and in this case, all MBs are scanned using the forward scan direction or the reverse scan direction.
[0026]
2.2 Fixed size GMB
In this method, the GMB size is fixed at a preselected number N. The number of MBs in one line is M, and the following rules are applied.
i) N is set such that M / N is a positive integer.
ii) If all MBs in one line belong to the same slice, the MBs are grouped into M / N GMBs. Therefore, the position of the MB where the GMB starts and ends is fixed.
iii) If the MB in one line belongs to two or more slices, the last MB of the slice must be the last MB of the GMB, and the first MB of the slice always starts a new GMB.
iv) If the MBs in a line belong to a very large number of slices (more than S (= M / N)), only rule iii) is used to group MBs and rule ii) does not apply.
[0027]
2.3 Variable size GMB
In this method, the size of the GMB is variable. The following method is used to determine the size of the GMB.
i) All MBs in the same line (and in the same slice) are first coded using the forward scan direction. Then, MB grouping is performed according to the prediction of the type and mode of the encoded MB. In the present invention, the MB is set to end the GMB by the vertical intra prediction and the DC intra prediction.
ii) All MBs in the same line (and in the same slice) are first coded using the reverse scan direction. Then, MBs are grouped according to the intra-picture prediction mode of the encoded MBs. In the present invention, the MB is set to end the GMB by the vertical intra prediction and the DC intra prediction.
iii) All MBs in the same line (and in the same slice) are first coded using both forward and reverse scan directions. Then, MB grouping is performed according to the intra-frame prediction mode of the encoded MB. In the present invention, the MB is set to end the GMB by the vertical intra prediction and the DC intra prediction.
[0028]
According to the above method, if there is no MB predicted vertically or in a DC picture in a line, the GMB is set as the entire line (if all MBs in the line are in the same slice). If an MB in a line belongs to more than one slice, in addition to the vertical and DC predicted MBs, the GMB is also determined by the first and last MB of the slice. Therefore, the MBs in the GMB should always belong to the same slice.
[0029]
Vertical and DC intra predictions are based on all MBs, not sub-blocks.
[0030]
Other methods may be used to determine the GMB according to the prediction type (intra-screen or inter-screen) and the prediction mode. The criterion is that the MB selected as the last MB of the GMB should not be predicted only from the MB above it or only from other MBs from the previous picture.
[0031]
3. Determining the GMB Scan Direction Once the GMB of the picture is defined and determined, it is easy to determine the GMB scan direction. In the present invention, rate distortion is used to obtain the optimal scan direction for shorter bit length and better image quality.
[0032]
The RD cost cost (normal) is calculated by encoding the GMB using the forward scan direction.
The RD cost cost (inverse) is calculated by encoding the GMB using the reverse scan direction.
[0033]
If cost (inverse) <cost (normal), a backward scan is used to encode the GMB; otherwise, a forward scan is selected.
[0034]
4. A different embodiment of the present invention will be described.
[0035]
4.1 Adaptive scan of slice size GMB In this method, the size of the GMB is the same as that of the slice. One codeword scanning_direction is used to indicate the scan direction of the slice size GMB (see FIGS. 1 and 6). In this case, it is not worthwhile that the GMB is often composed of MBs from multiple lines.
[0036]
4.2 Adaptive Scan of Fixed Size GMB In this method, the size of the GMB is fixed. Therefore, at the picture level, the size of the GMB should be signaled, and further before each GMB's data, the scan direction of that GMB should be encoded.
[0037]
4.3 Adaptive scanning of variable size GMB In this method, the size of the GMB is variably set by the encoder. Therefore, before the data of each GMB, the scan direction and the size of the GMB are inserted into the transmission stream.
[0038]
5. Adaptive Sub-Block Scan The present invention also discloses an adaptive sub-block scanning method that can be used for intra-screen sub-block prediction. A macroblock may be composed of multiple subblocks with different intra prediction modes, so that the scan direction of these subblocks may also be adaptive.
[0039]
One adaptive mode is a mode in which all sub-blocks are scanned using a forward scan direction or a reverse scan direction. Another mode is a mode in which the scan direction of the sub-block of each line of the macroblock can be determined adaptively by applying rate distortion.
[0040]
When the adaptive sub-block scanning method is used, first the codeword in the I-picture header should indicate that adaptive sub-block scanning is being used for the I-picture. Then, the codeword for each macroblock is used to indicate the scan mode of the sub-block. Finally, the scan direction is represented by a codeword in each line of the header or sub-block of the macroblock.
[0041]
6. Scan and prediction within a macroblock After the scan direction of each GMB is determined, the scan and prediction within each MB of the GMB should be further described.
[0042]
When the forward scan direction is used for GMB, the scanning and prediction of the pixels of each MB remains the same as the current method (ie, 4x4 in screen, 16x16 in screen, ABT, motion prediction, etc.).
[0043]
When the reverse scan direction is used for the GMB, the scanning and prediction of the pixels of each MB of the GMB are performed symmetrically with respect to the forward scan direction.
[0044]
【The invention's effect】
The use of adaptive MB scanning improves the coding efficiency within and between frames. For example, the bit rate is improved by about 5% compared to the current JVT, Foreman GIF sequence. Furthermore, by utilizing the present invention, the current intra-picture and inter-picture prediction methods can be simplified while maintaining good coding efficiency.
[Brief description of the drawings]
FIG. 1 is a diagram showing an example of a slice size GMB. GMB starts at MB A and ends at MB B.
FIG. 2 is a diagram illustrating an example of a fixed size GMB. This shows a case where the size of the GMB is set to 5 and all MBs in the line belong to the same slice.
FIG. 3 is a diagram illustrating an example of a fixed size GMB. When one or more slices are included in one line (in this case, MBs belong to two slices), the MB surrounded by a thick line belongs to one slice, and the others belong to another slice. GMB2 and GMB3 are divided by slice boundaries.
FIG. 4 is a diagram showing some examples of a variable size GMB. GMB1 is composed of MBs in the entire line, and the other GMBs (GMB2-GMB5) indicate the obtained variable size.
FIG. 5 shows prediction of current MBA in different directions. (A) The scanning direction is left to right, and (b) the scanning direction is right to left.
FIG. 6 is a diagram showing, for each I picture, a codeword GMB_type indicates which type of GMB is used for that picture.
FIG. 7 is a diagram illustrating a syntax when a slice size GMB is used for a scan direction (0 is a forward direction, 1 is a reverse direction) given for each slice.
FIG. 8 is a diagram illustrating syntax when a fixed size GMB in each slice is used. There are two or more GMBs, and a scan direction is given for each GMB.
FIG. 9 is a diagram illustrating a syntax when a variable size GMB in each slice is used. Not only the scan direction, but also the size of the GMB is given.
FIG. 10 is a diagram showing another embodiment when a variable size GMB is used.
FIG. 11 is a diagram illustrating an adaptive sub-block scanning method. (A) left-to-right scan for all sub-blocks, (b) right-to-left scan for all sub-blocks, (c) adaptive scan for sub-blocks of each line (first and last lines are An example is shown where scanning is performed from right to left and the other two lines are scanned from left to right).
FIG. 12 is a diagram related to encoding of a macroblock when adaptive sub-block scanning is used. Use or non-use of the adaptive sub-block scan is indicated in a codeword in the header of an I picture.
FIG. 13 is a diagram for explaining prediction of pixels in a macroblock (for example, 16 × 16 prediction in a screen), (a) prediction in a forward scan direction, and (b) prediction in a reverse scan direction.
FIG. 14 is a diagram illustrating 9-mode sub-block prediction in a JVT model in a forward scan direction.
FIG. 15 is a diagram showing 9-mode sub-block prediction in the JVT model in the reverse scan direction.
16A and 16B are diagrams showing a median prediction of a motion vector, (a) a forward scan direction, and (b) a reverse scan direction.

Claims (55)

In the adaptive macroblock scanning method in the encoder,
Determining the number of macroblocks in a macroblock group (GMB);
Determining a scan direction for the macroblock group;
Scanning the macroblocks in the macroblock group in the determined scanning direction,
Performing prediction of macroblocks in a macroblock group according to the scan direction;
Encoding the residual value of the macroblock,
Encoding the size of the macroblock,
An adaptive macroblock scanning method, wherein the scanning direction is encoded. 2. The adaptive macroblock scanning method according to claim 1, wherein when the picture to be coded is an intracoded picture, the intra prediction of the macroblock group is performed according to the scan direction. . 2. The adaptive macroblock scanning method according to claim 1, wherein when the picture to be coded is an inter-coded picture, the inter-picture prediction of the macroblock group is performed according to the scan direction. . 4. The adaptive macroblock scanning method according to claim 1, wherein a slice in the picture can include at least one macroblock group. 4. The adaptive macroblock scanning method according to claim 1, wherein the picture can include at least one slice. 4. The adaptive macroblock scanning method according to claim 1, wherein when the slice has two or more macroblock groups, the macroblocks in the macroblock group belong to the same line in the picture. . 4. The adaptive macroblock scanning method according to claim 1, wherein when the slice has only one macroblock group, macroblocks in the macroblock group belong to a plurality of lines in a picture. . 4. The adaptive macroblock scanning method according to claim 1, wherein a scan direction for the macroblock group is a forward direction from left to right. 4. The adaptive macroblock scanning method according to claim 1, wherein a scanning direction for the macroblock group is a reverse direction from right to left. 4. The adaptive macroblock scanning method according to claim 1, wherein the size of the macroblock group is the number of macroblocks in the group. 4. The adaptive macroblock scanning method according to claim 1, wherein the size of the macroblock group is set to the size of one slice. 4. The adaptive macroblock scanning method according to claim 1, wherein the size of the macroblock group is fixed for all macroblock groups in one picture. 4. The adaptive macroblock scanning method according to claim 1, wherein the size of the macroblock group is variable for all macroblock groups in one picture. 14. The adaptive macroblock scanning method according to claim 1, wherein the macroblocks in the macroblock group always belong to the same slice. 14. The adaptive macroblock scan according to claim 1, wherein the first macroblock in the slice must be the first macroblock in a macroblock group in the slice. Method. 14. The adaptive macroblock scan according to claim 1, wherein the last macroblock in the slice must be the last macroblock in a macroblock group in the slice. Method. 14. The adaptive macroblock scanning method according to claim 1, wherein the first macroblock in a line must be the first macroblock in a macroblock group in the line. 14. The adaptive macroblock scanning method according to claim 1, wherein the last macroblock in a line must be the last macroblock in a macroblock group in the line. 15. The adaptive macroblock scanning method according to claim 1, wherein a macroblock obtained by vertical prediction or DC prediction is selected as the last macroblock of the macroblock group. 20. The adaptive macroblock scanning method according to claim 1, wherein the number of macroblocks between the first and last macroblocks determines the size of the variable macroblock group. . 4. The adaptive macroblock scanning method according to claim 1, wherein the inter-frame and intra-frame prediction of the macroblock is performed in a spatial or frequency domain. 22. The adaptive macroblock scanning method according to claim 1, wherein a scanning direction of all luminance and chrominance pixels in the macroblock corresponds to a scanning direction of the macroblock. 23. The adaptive macroblock scanning method according to claim 1, wherein intra-frame or inter-frame prediction of a pixel value is performed in the forward direction or the backward direction. The adaptive macroblock scanning method according to claim 23, wherein intra-frame and inter-frame prediction in the forward direction is the same as the current prediction method. Intra-frame and inter-frame prediction in the backward direction is symmetric to forward prediction, and the current prediction module of the forward prediction can be applied without change by rearranging the input / output data buffer. 24. The adaptive macroblock scanning method according to claim 23. 26. The method of claim 21, wherein the prediction method includes macroblock and subblock spatial prediction, macroblock and subblock spectral inter-frame prediction, and motion prediction, and the encoding method is UVLC, VLC, or CABAC. Adaptive macroblock scanning method. 4. The adaptive macroblock scanning method according to claim 1, wherein the scan direction is determined using a rate distortion algorithm in both a forward direction and a backward direction for the macroblock group. 12. The adaptive macroblock scanning method according to claim 1, wherein the size of the macroblock group is represented by the number of macroblocks between the beginning and the end of the slice. 12. The adaptive macroblock scanning method according to claim 1, wherein the scan direction is represented by a codeword in a slice header. 13. The adaptive macroblock scanning method according to claim 1, wherein the size of the macroblock group is coded in a header of the picture. 13. The adaptive macroblock scanning method according to claim 1, wherein the scanning direction is represented by a flag in a header of the macroblock group. 14. The adaptive macroblock scanning method according to claim 1, wherein the size of the macroblock group is encoded in a header of each macroblock group. 27. The adaptive macroblock scanning method according to claim 1, wherein the scanning direction is represented by a flag in a header of one macroblock group. 20. The adaptive macroblock scanning method according to claim 1, wherein the scanning direction is represented by a flag in a header of each macroblock. 4. The adaptive macroblock scanning method in a decoder according to claim 1, 2 or 3,
Determine the size of the macroblock group,
Determine the scan direction of the macroblock,
Scanning macroblocks in the macroblock group in the determined direction,
Performing prediction on the macroblock according to the scanning direction,
An adaptive macroblock scanning method, comprising reconstructing pixel values of the macroblock. The mode for determining a size of a macroblock group according to claim 11, 12, or 13, is represented by a codeword in a header of the picture. 35. The adaptive macroblock scanning method according to 35. 36. The adaptive macroblock scanning method according to claim 1, wherein the size of the macroblock group is determined by the number of macroblocks in the slice. 38. The adaptive macroblock scanning method according to claim 1, wherein a scanning direction of the macroblock is determined by a codeword in a header of the slice. 36. The adaptive macroblock scanning method according to claim 1, wherein the size of the macroblock group is determined by a codeword in a header of the picture. 40. The adaptive macroblock scanning method according to claim 1, wherein a scanning direction of the macroblock is determined by a flag in a header of the macroblock group. 36. The adaptive macroblock scanning method according to claim 1, wherein the size of the macroblock group is determined by a codeword in a header of the macroblock group. 42. The adaptive macroblock scanning method according to claim 1, wherein a scanning direction of the macroblock is determined by a flag in a header of the macroblock group. 36. The adaptive macroblock scanning method according to claim 1, wherein a scanning direction of the macroblock is determined by a flag in a header of each macroblock. The adaptive macro according to claim 1, 2, 3, 3, 13, 34, 35, or 43, wherein the size of the macroblock group is determined by the number of macroblocks having the same direction as the determined scan direction. Block scan method. The adaptive macroblock scanning method according to claim 1 or 35, wherein the adaptive macroblock scanning is applicable to an adaptive subblock scanning. 36. The method of claim 35, wherein when adaptive sub-block scanning is used, all sub-blocks of the macroblock are scanned in a left-to-right forward scan or a right-to-left reverse scan. Or the adaptive macroblock scanning method according to 45. 47. The adaptive macroblock scanning method according to claim 1, 34, 45 or 46, wherein the codeword is used in the macroblock header to indicate the scanning direction of the sub-block. 46. The adaptive macroblock scanning method according to claim 1, 35 or 45, wherein when adaptive subblock scanning is used, subblocks in each line of the macroblock are scanned forward or backward. 49. The adaptive macroblock scanning method in both the encoder and the decoder according to claim 1, 35, 45 or 48, wherein the scanning direction of the subblock of each line of the macroblock is determined by a codeword at the head of each line of the subblock. An adaptive macroblock scanning method characterized by being represented. 50. An adaptive macroblock scanning method in both an encoder and a decoder according to claim 1, 35, 45, 48 or 49, wherein the codeword is used to indicate the scanning direction of the subblock in each line of the macroblock. An adaptive macroblock scanning method, comprising: 49. An adaptive macroblock scanning method in both an encoder and a decoder according to claim 46 or 48, wherein the scan direction is determined by applying a rate distortion algorithm in both the forward and reverse directions for the sub-block. An adaptive macroblock scanning method comprising: 49. An adaptive macroblock scanning method in both an encoder and a decoder according to claim 46 or 48, wherein a codeword is used to represent the scan mode according to claim 46 or 48 in the header of each macroblock. An adaptive macroblock scanning method, comprising: 36. The adaptive macroblock scanning method in both the encoder and the decoder according to claim 1 or 35, wherein the macroblock can be of any size. 36. The adaptive macroblock scanning method in both the encoder and the decoder according to claim 1 or 35, wherein the macroblock group is a region in a picture. 56. The adaptive macroblock scanning method in both the encoder and the decoder according to claim 1, 35 or 55, wherein a width of an area in the picture is equal to or smaller than a width of the picture. .

Images