JP3200199B2 - Video compression encoding method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture compression encoding method in which the quality of a decoded picture is improved.

[0002]

2. Description of the Related Art In recent years, MPs have been used
The EG method has been proposed. Encoding and decoding of a moving image is performed as follows. FIG. 5 shows encoding of a moving image,
FIG. 2 is a block diagram of a system that performs decoding. The input device 1 inputs moving image signals of various formats such as Y, Cb, and Cr. The preprocessor 2 converts the data into a format required by the encoder. The encoder 3
A bit stream is generated by reducing the amount of data without deteriorating the input moving image as much as possible.

[0005] The storage device 4 is composed of a CD, DAT, hard disk or the like, and stores a generated bit stream. The decoder 5 receives the bit stream and creates a reproduced image. The post-processor 6 performs processes such as line interpolation, pixel interpolation, rate conversion, frame field conversion, and conversion of the pixel aspect ratio according to the specifications of the output display. The output device 7 displays and outputs the reproduced moving image.

FIG. 6 shows a block diagram of the encoder 3. Before describing the encoder 3, the encoding mode will be briefly described. Coding modes are roughly classified into two modes: (1) intra coding mode, and (2) non-intra coding mode (prediction coding mode).

In the intra coding mode, an input image is coded as it is. On the other hand, in the non-intra coding mode, that is, in the predictive coding mode, a predicted image is referred to by referring to an image that has already been coded in one of three predictive modes: forward, backward, or interpolation between them. Generated and a difference image from the predicted image is encoded.
At this time, in the case of the prediction mode and the motion compensation prediction, the motion vector is also coded at the same time.

The encoder 3 will be described. The input image data (ID) is applied to the DCT without passing through the differentiator 301 in the intra-coding mode according to the coding mode.
In the case of a non-intra coding mode, that is, a mode for performing some prediction and coding, a prediction image (PID) according to the prediction mode and the motion vector
Is obtained by a differentiator 301, and the difference image (S
ID) is input to the DCT unit 302. DCT unit 302
Transform coefficient (C) as a result of discrete cosine transform
Are quantized by a quantizer 303, subjected to variable length coding by a VLC unit 304, and stored in a buffer 305. The encoded data stored in the buffer 305 is read at a constant rate and sent to a DSM (Digital Storage Media) or the like.

[0007] A predicted image in the non-intra coding mode is generated by the predictor from the locally decoded image stored in the frame memory of the frame memory & predictor 309. The local decoded image is encoded by the encoder at the same time as encoding.
This is an image locally decoded by performing a reverse decoding process. This is because, at the time of encoding, the data (QC) quantized by the quantizer 303 is inversely quantized by the inverse quantizer 306 and inverse DCT by the inverse DCT unit 307, and this is already stored in the frame memory 309. It is added to the predicted image generated by the predictor 309 from another local decoded image being generated by the adder 308, and is stored in the frame memory 309.

Note that two switches in FIG. 6, that is, a switch 311 between the difference unit 301 and the DCT unit 302
And the output of the frame memory & predictor 309 and the adder 30
The switch 312 between the inputs to 8 operates as follows.

In the intra coding mode, the switch 3
11 is connected to the lower side, ie, the ID side, and the switch 312 is connected to the upper side, ie, “0”. In the non-intra coding mode, the switch 311 is connected to the upper side, ie, the SID side, and the switch 312 is connected to the lower side, ie, the PID.

FIG. 7 is a diagram showing a configuration of the decoder 5.
The processing of the decoder is basically the same as that of the local decoder (310 in FIG. 6) in the encoder described above. The encoded data is input to the buffer 501 at a constant rate. The data read from the buffer 501 is stored in the inverse VLC unit 50.
2 and is inversely quantized by an inverse quantizer 503 to obtain an inverse D
The inverse DCT is performed by the CT unit 504. Thereafter, in the intra-encoding mode, the output is directly passed through the adder 505 (to which “0” is added). In the non-intra-encoding mode, the frame memory & predictor 506 outputs the simultaneously decoded encoding mode and motion. In accordance with the vector, the predicted image generated from the already decoded image in the frame memory is added by the adder 505 and output.

The switch 507 provided between the output of the frame memory & predictor 506 and the input to the adder 505 operates as follows.

In the intra coding mode, the switch 5
07 is connected to the left side, ie, “0”, and in the non-intra coding mode, the switch 507 is connected to the right side, ie, PI
D is connected.

The structure of an I / P / B picture will be described. FIG. 8A is a diagram showing some pictures in display order, and the arrows show the dependency between P pictures and B pictures.

There are four types of pictures. That is, an I (intra) picture is encoded without referring to another picture. P (predicted)
Pictures are coded using motion compensation from previous I or P pictures. B (bidirectionala
(ly predicted) pictures are encoded using motion compensation from previous or subsequent I or P pictures. The D (DC) picture is used in the high-speed sequential playback mode (not shown in FIG. 8A). Typical encoding mechanisms include a mixture of I, P, and B pictures.

Also, due to the dependence of pictures, the order of the bit stream, that is, the order in which pictures are transmitted, stored, and received, is not the order of display but the order necessary for the decoder to decode the bit stream. Become.

FIG. 8B shows a picture sequence in a typical display order, and FIG. 8C shows a bit stream sequence corresponding thereto. Since the B picture depends on the succeeding P picture in the display order, the P picture is transmitted and decoded before the dependent B picture as shown in FIG.

The MPEG coding technique has a layer structure corresponding to a hierarchical structure. FIG. 9 shows the data structure of the encoded data. The video sequence layer 11 is the top layer of the coding structure, and includes a header and some gross
It consists of ups of pictures (GOPs) layer 12. The sequence header initializes the state of the decoder so that the decoder can decode any sequence regardless of past decoding history.

The GOP layer 12 is the smallest coding unit that can be individually decoded in a sequence, and includes a header and several picture layers 13 and includes at least one or more I pictures. The GOP header contains time and editing information.

The picture layer 13 includes a header and one or more slice layers 14. If the bitstream becomes unreadable in the middle of a picture, the decoder can recover by waiting for the next slice layer 14 without losing the entire picture.

The slice layer 14 includes a header and one or more macroblock layers 15. The slice header includes information on the position and the quantization scale. This is sufficient for recovery from local errors.

The macroblock layer 15 is a basic unit for motion compensation and quantization scale change. Each macroblock layer 15 comprises a header and a six-component block layer 16. That is, four blocks of luminance and one block of Cb
The color difference is one block of Cr color difference. The macroblock header contains quantization scale and motion compensation information.

The block layer 16 is a basic coding unit, and DCT is applied to this block level. Each block is composed of 64 pixels in an 8 × 8 matrix. Pixel values are not individually coded, but are elements of a coded block. Each luminance pixel corresponds to one picture pixel, but since the chrominance information is sub-sampled both horizontally and vertically 2: 1, each chrominance pixel corresponds to four picture pixels.

[0023]

As a criterion for determining the intra-coding and the predictive coding in the above-mentioned MPEG coding, there is a criterion recommended by CCITT. In severe images, the image quality may be greatly deteriorated.

That is, the “intra-coding / prediction-coding determination criteria” when coding the currently used macroblock is as shown in FIG. Here, the horizontal axis represents the variance (VAR) of the prediction error, and the vertical axis represents the variance (VAROR) of the original signal. According to this standard, VAR>
In the range of 64, only the magnitude relation between VAR and VAROR is considered, and even if prediction is not performed well and VAR takes an extremely large value, the condition of VAR <VAROR (white background in the figure) is satisfied. , Predictive coding.

As described above, in the moving picture compression technique of the MPEG system, P pictures and B pictures, which are non-independent frames, are predicted from temporally preceding or succeeding pictures and their prediction errors are coded. On the other hand, in a macroblock in which prediction itself has not been performed well, there is a problem that distortion of a decoded image increases.

An object of the present invention is to provide a moving picture compression encoding method in which, when prediction is not performed well, intra coding is performed without performing predictive coding to improve the quality of a decoded picture. It is in.

[0027]

In order to achieve the above object, according to the first aspect of the present invention, intra coding or
Encode a macroblock using one of the predictive codings
Predicting during the encoding of said macroblock.
The variance of the error is calculated, and the variance of the calculated prediction error is
If it is larger than the threshold set when encoding the previous frame
A video encoding method for performing intra-coding.
I, the prediction of the macroblock lock during encoding on a frame-by-frame basis
The distribution of the variance of the error is calculated, and based on the calculated distribution,
The number of macroblocks to be intra-coded is constant
The threshold is adaptively set as follows, and the set threshold is
This is characterized in that the present invention is applied to a frame to be encoded as follows .

[0028] In the invention according to claim 2, the predetermined
The threshold value is set for P picture frames and B picture frames.
And different values .

[0029]

When coding a macroblock, if the variance (VAR) of the prediction error of the macroblock is larger than a certain threshold, intra coding is performed. This avoids performing predictive coding on a macroblock for which prediction was not successfully performed, so that the image quality of a decoded image can be improved.

[0030]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be specifically described below with reference to the drawings. As described above, where the quality of the decoded image is largely deteriorated, the distortion of the predicted image is often large. Therefore, in this embodiment, the intra-encoding is performed on the macroblock for which the prediction was not performed well.

FIG. 1 is a diagram showing "determination criteria for intra coding / prediction coding" of the present invention. When such a criterion is used, when the variance (VAR) of the prediction error is larger than a certain threshold value, intra coding is performed, so that coding can be performed without using a prediction that has not been performed well.

Since macroblocks having a large variance of prediction error (VAR) greatly affect the image quality deterioration of a decoded image, if such macroblocks are intra-coded, an improvement in the SN ratio is expected. it can. However, if too many macroblocks are intra-coded, the code amount increases and the amount of data accumulated in the buffer increases, so the quantization width increases due to the effect of the code amount control,
Conversely, the S / N ratio decreases. Therefore, a method of determining an optimum threshold becomes a problem.

The distribution of the variance (VAR) of the prediction error in the image sequence is generally as shown in FIG. In general, the frequency decreases exponentially with respect to the variance (VAR) of the prediction error, but the degree of the decrease varies with each image sequence and with time. Therefore, it is necessary to adaptively change the threshold value.

The hatched portion in FIG. 3 indicates the number of macro blocks to be intra-coded. If the threshold value is determined so that the area is constant, it is possible to prevent a decrease in the SN ratio due to an abnormal increase in the code amount.

FIG. 4 is a flowchart of a threshold value determining process according to the present invention. That is, from the 0th to the last frame (step 101), from the 0th macroblock to the 1320th macroblock in each frame (step 102), the variance of prediction error (VAR) during macroblock coding, VAROR is calculated, and intra-coding / prediction-coding is determined using a threshold set at the time of coding the previous frame (step 103).
A threshold value is set from the distribution of the calculated prediction error variance (VAR) (step 104).

That is, in step 103, the distribution of VAR and VAROR is calculated during encoding of the frame, and in step 104, a threshold value is determined from the distribution so that the number of macroblocks to be intra-coded becomes constant.
The threshold is used for encoding the next frame.

In this processing, the processing requiring a large amount of calculation is called "VAR".
Is calculated, which is the “intra-encoding /
This is processing that has been performed conventionally even if it is not a "criterion for determining predictive coding." The processes to be newly added include a process of “deciding intra-coding / prediction-coding by threshold” and a process of “V
The processing of “setting a threshold value from the distribution of AR” is the same. However, the former does not change much as the processing time because the latter is processing in the picture layer only by adding one line of if statement in the macroblock layer.

In the distribution of the variance (VAR) of the prediction error, the P picture is more scattered than the B picture. This is because, as described in the related art, the best mode among forward prediction / backward prediction / bidirectional prediction can be selected for a B picture, whereas only the forward prediction is available for a P picture. Therefore, the threshold value obtained for the B picture is P
When used in picture coding, many macroblocks are intra-coded, so that the code amount increases.
In addition, if the threshold value obtained for the P picture is used in the coding of the B picture, the number of macro blocks to be intra-coded becomes too small. The effect of intra-coding a macroblock that could not be predicted cannot be expected.

Therefore, in another embodiment of the present invention, threshold values are separately set for P pictures and B pictures. As a result, the effect of the criterion for determining intra-coding / prediction-coding according to the present invention can be obtained while avoiding an extreme increase in the coding amount for both P-pictures and B-pictures.

[0040]

As described above, according to the present invention , a macroblock that cannot be predicted well is intra-coded, so that the quality of a decoded image can be improved.

According to the present invention , the image quality can be stably improved for every image by adaptively changing the threshold value.

Further, according to the present invention , by making the number of macro blocks to be intra-coded to some extent constant, it is possible to prevent an extreme increase in the code amount and to improve the image quality.

Further, according to the present invention , a P picture and a B picture
Since different threshold values are set for pictures, it is possible to intra-code macroblocks that could not be predicted well, while avoiding an extreme increase in the code amount.

[Brief description of the drawings]

FIG. 1 is a diagram showing “determination criteria for intra-coding / prediction-coding” of the present invention.

FIG. 2 is a diagram showing a “criterion for determining intra-coding / prediction-coding” currently used.

FIG. 3 is a diagram showing a distribution of a variance of a prediction error in an image sequence.

FIG. 4 is a flowchart of a threshold value determination process according to the present invention.

FIG. 5 is a block diagram of a system for encoding and decoding moving images.

FIG. 6 is a block diagram of an encoder.

FIG. 7 is a diagram illustrating a configuration of a decoder.

8A is a diagram showing some pictures in display order, FIG. 8B shows a picture sequence in a typical display order,
(C) shows a bit stream sequence corresponding thereto.

FIG. 9 is a diagram showing a data structure of encoded data.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 input device 2 preprocessor 3 encoder 4 storage device 5 decoder 6 postprocessor 7 output device

Claims (2)

(57) [Claims] 1. A method of coding a macroblock using either intra-coding or prediction coding, wherein a variance of a prediction error is calculated during coding of the macroblock, and a variance of the calculated prediction error is calculated. When the variance is larger than a threshold value set at the time of encoding of the previous frame, a method of compressing and encoding a moving image to perform intra-encoding. Calculating and adaptively setting the threshold so that the number of macroblocks to be intra-coded based on the calculated distribution is constant, and applying the set threshold to a frame to be encoded next. Moving image compression encoding method. 2. The moving image compression encoding method according to claim 1, wherein the set threshold value is different for each of a P picture frame and a B picture frame.