JP2000236547A - Image information converter and image information conversion method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To decrease a bit rate of a bit stream resulting from compression coding in compliance with the moving picture experts group MPEG-2 system while minimizing degradation in image quality of a decoded image. SOLUTION: An inverse quantizer 26 applies inverse quantization to a bit stream in compliance with the MPG-2 and outputs a discrete cosine coefficient in the unit of discrete cosine transform DCT blocks. A band limit device 27 replaces a high frequency component of the discrete cosine coefficient in a horizontal direction with zero. A quantizer 28 quantizes the discrete cosine coefficient with a quantization width different from an original quantization width.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image compression system which performs an orthogonal transform on a predetermined pixel block basis and quantizes an orthogonal transform coefficient in the orthogonal transform block obtained by the orthogonal transform. The present invention relates to an image information conversion device and an image information conversion method for converting a bit rate of information. For example, the present invention is applied to a case where bits of image compression information compressed and encoded by MPEG-2 or the like are transmitted to a network such as a satellite broadcast or a cable television, or to a recording medium such as an optical disk or a magnetic disk. It is to reduce the rate.

[0002]

2. Description of the Related Art In recent years, image information is handled as digital data, and the digital data is compressed by orthogonal transformation and motion compensation using redundancy inherent in the image information, and transmitted to network media such as satellite broadcasting and cable television. In addition, devices that perform recording on storage media such as optical disks and magnetic disks have become widespread. In such an apparatus, MPEG-2 (Moving Picture Experts Group) using discrete cosine transform as an image compression method is generally used.
phase-2) is used.

In recent years, standardization of digital television broadcasting using an image compression method such as MPEG-2 has been promoted. Digital television broadcasting standards include:
Standard resolution image (for example, if the number of effective lines in the vertical direction is 57
6), a standard corresponding to a high-resolution image (for example, the number of effective lines in the vertical direction is 1152), and the like.

Incidentally, the image information of this high-resolution image is enormous, and even if it is compressed using an encoding method such as MPEG-2, a large amount of code (bit rate) is required to obtain a sufficient image quality. Required. For example, in the case of a 30 Hz interlaced scan image having an image frame of 1920 × 1080 pixels, a code amount (bit rate) of about 18 to 22 Mbps or more is required.

Therefore, when transmitting such a high-resolution image to network media such as satellite broadcasting and cable television, the code amount (bit rate) must be further reduced in accordance with the bandwidth of the transmission path. Must. Similarly, when recording such a high-resolution image on a storage medium such as an optical disk or a magnetic disk, the code amount (bit rate) must be further reduced in accordance with the recording capacity of the medium. Further, the necessity of reducing the code amount (bit rate) is not limited to a high-resolution image, but also to a standard-resolution image (for example, when the image frame is 7 frames).
It is also conceivable that this may occur in a 30 Hz interlaced scan image of 20 pixels × 480 pixels).

Conventionally, as shown in FIG. 41, an image information conversion device configured by connecting an image decoding device 110 and an image encoding device 120 in series has been used to provide a code amount of image compression information (bit stream). (Bit rate).

In this conventional image information conversion apparatus, high bit rate image compression information (bit stream) is input to the image decoding apparatus 110. This image decoding device 110
Once completely decodes the high bit rate image compression information and outputs baseband video data. And
The image encoding device 120 re-encodes baseband video data obtained as an output of the image decoding device 21, and outputs low bit rate image compression information (bit stream). At this time, the image encoding device 120
A target code amount (target bit rate) lower than the code amount (high bit rate) of the input image compression information is given in advance, and quantization processing according to the target code amount is performed.

[0008] By performing the above processing, the conventional image information conversion apparatus can reduce the code amount of the image compression information.

[0009]

However, such a conventional image information conversion apparatus requires all hardware components for decoding and encoding, so that the cost and power consumption increase, and It has been difficult to incorporate it into other devices or portable devices. Further, in such a conventional image information conversion device, since all processes associated with decoding and encoding must be performed, the amount of arithmetic processing becomes extremely large, and is realized as software using a general-purpose integrated circuit. In such a case, there is a possibility that a problem may occur that the processing does not end in real time with the arithmetic processing capability of the circuit.

Further, in the image encoding device 120 of the conventional image information conversion device, a lower target code amount (target bit rate) is given to the input high-bit-rate image compression information. The width became coarse, and the quantization noise generated by this appeared as block noise in the decoded image, and good image quality was not obtained.

SUMMARY OF THE INVENTION An object of the present invention is to solve the above-mentioned problems and reduce the deterioration of the image quality of a decoded image by minimizing the deterioration of the decoded image by inputting a large amount of code (high bit rate). It is an object of the present invention to provide an image information conversion apparatus and an image information conversion method for outputting image compression information with a small code amount (low bit rate) while reducing the code amount (bit rate).

[0012]

An image information conversion apparatus according to the present invention comprises a predetermined pixel block (orthogonal conversion block).
By performing orthogonal transform in units and quantizing orthogonal transform coefficients in the orthogonal transform block obtained by performing orthogonal transform, first image compression information of a first bit rate obtained by compression-coding an image signal is obtained. , An image information conversion device for converting the first image compression information into a second image compression information having a second bit rate lower than the first bit rate. In response, the inverse quantization means for inversely quantizing the orthogonal transform coefficient, and the inverse quantization means according to the quantization width such that the output second image compression information becomes the second bit rate. And a quantizing means for requantizing the inversely quantized orthogonal transform coefficient.

In this image information converter, the orthogonal transform coefficients are inversely quantized and then requantized by changing the quantization width.

The image information converting apparatus according to the present invention further comprises a band limiting unit for limiting a value of a high frequency component in a horizontal direction of the orthogonal transform coefficient inversely quantized by the inverse quantizing unit. Requantizes the orthogonal transform coefficient in which the high-frequency component in the horizontal direction is limited by the band limiting means according to the quantization width at which the output second image compression information becomes the second bit rate. It is characterized by the following.

In this image information conversion apparatus, after the orthogonal transform coefficient is dequantized, the value of the high-frequency component of the orthogonal transform coefficient in the horizontal direction is limited, and the quantization width is changed to perform requantization.

Further, the image information conversion apparatus according to the present invention performs orthogonal transformation on a predetermined pixel block (orthogonal transformation block) basis and quantizes orthogonal transformation coefficients in the orthogonal transformation block obtained by orthogonal transformation. Accordingly, the first image compression information of the first bit rate obtained by compressing and encoding the image signal is converted to the second image compression information lower than the first bit rate.
An image information conversion device for converting the orthogonal transform coefficient into a second image compressed information having a bit rate of, wherein the orthogonal transform coefficient is dequantized in accordance with the quantization width of the orthogonal transform coefficient of the input first image compressed information. A first inverse quantizing means for performing the operation, an adding means for adding the orthogonal transform coefficient inversely quantized by the first inverse quantizing means and the motion compensation error correction coefficient, and a second image compression information to be output. In accordance with the quantization width to achieve the second bit rate, the quantizing means for requantizing the orthogonal transform coefficient to which the motion compensation error coefficient has been added by the adding means and the requantization by the quantizing means. A second inverse quantizer for inversely quantizing the orthogonal transform coefficient, and an orthogonal transform obtained by adding a motion compensation correction coefficient by the adder from the orthogonal transform coefficient inversely quantized by the second inverse quantizer. Subtraction means for subtracting coefficients Motion compensation error correction means for orthogonally transforming the orthogonal transform coefficient subtracted by the subtraction means, performing motion compensation based on a motion vector, and performing inverse orthogonal transform on the value obtained by the motion compensation to generate the motion compensation error correction coefficient. And characterized in that:

In this image information converter, the orthogonal transform coefficient is inversely quantized, requantized by changing the quantization width, and the difference between the input orthogonal transform coefficient and the requantized orthogonal transform coefficient is calculated by moving the difference. Compensation corrects the error due to motion compensation.

The image information conversion apparatus according to the present invention further comprises a band limiting unit for limiting a value of a high frequency component in a horizontal direction of the orthogonal transform coefficient inversely quantized by the first inverse quantization unit. The quantizing means regenerates the orthogonal transform coefficient of which the high-frequency component in the horizontal direction is limited by the band limiting means according to the quantization width at which the output second image compression information has the second bit rate. It is characterized by being quantized.

In this image information conversion apparatus, after the orthogonal transform coefficients are dequantized, the values of the high-frequency components of the orthogonal transform coefficients in the horizontal direction are limited, the quantization width is changed, and requantization is performed. The difference between the coefficient and the requantized orthogonal transform coefficient is motion compensated to correct the error due to the motion compensation.

Further, in the image information conversion apparatus according to the present invention, the orthogonal transform coefficient is determined by calculating a coefficient in the horizontal direction and a coefficient in the vertical direction.
The motion compensation error correction means performs 4 × 8 inverse orthogonal transform on the quantization error coefficient including 8 horizontal coefficients and 8 vertical coefficients, and performs quantization in the spatial domain. 4 × 8 inverse orthogonal transform unit for generating a quantization error value, and performing motion compensation with a １ ／ pixel precision in the horizontal direction on the spatial domain quantization error value generated by the 4 × 8 inverse orthogonal transform unit. And perform motion compensation in the vertical direction with half-pixel accuracy,
A motion compensation unit that generates a quantization error correction value in the spatial domain,
A 4 × 8 orthogonal transformation unit that performs 4 × 8 orthogonal transformation on the spatial domain quantization error correction value generated by the motion compensation unit and generates the motion compensation error correction coefficient in the frequency domain. It is characterized by the following.

In this image information conversion apparatus, the orthogonal transform coefficient is inversely quantized, the value of the high-frequency component of the orthogonal transform coefficient in the horizontal direction is limited, the quantization width is changed, and requantization is performed. The four low-frequency coefficients in the horizontal direction and the eight coefficients in the vertical direction of the difference between the coefficient and the requantized orthogonal transform coefficient are motion-compensated to correct an error due to motion compensation.

An image information conversion method according to the present invention performs orthogonal transformation in units of a predetermined pixel block (orthogonal transformation block) and quantizes orthogonal transformation coefficients in the orthogonal transformation block obtained by orthogonal transformation. Image information conversion method for converting first image compression information of a first bit rate obtained by compression-coding an image signal into second image compression information of a second bit rate lower than the first bit rate And inputs the first image compression information of the first bit rate, and dequantizes the orthogonal transform coefficient according to the quantization width of the orthogonal transform coefficient of the input first image compression information. A second quantization coefficient generated by requantizing and requantizing the inversely-quantized orthogonal transform coefficient according to a quantization width at which the output second image compression information becomes the second bit rate. Image compression information And outputs a.

In this image information conversion method, the orthogonal transform coefficients are inversely quantized and then requantized by changing the quantization width.

Further, in the image information conversion method according to the present invention, the value of the high frequency component in the horizontal direction of the inversely quantized orthogonal transform coefficient is limited, and the second image compression information to be output is the second bit compression information. Depending on the quantization width that becomes the rate,
The orthogonal transform coefficient in which the high frequency component in the horizontal direction is restricted is requantized.

In this image information conversion method, after the orthogonal transform coefficient is dequantized, the value of the high frequency component of the orthogonal transform coefficient in the horizontal direction is limited, and the quantization width is changed to perform requantization.

In the image information conversion method according to the present invention, the orthogonal transform is performed in units of a predetermined pixel block (orthogonal transform block) and the orthogonal transform coefficients in the orthogonal transform block obtained by the orthogonal transform are quantized. Accordingly, the first image compression information of the first bit rate obtained by compressing and encoding the image signal is converted to the second image compression information lower than the first bit rate.
An image information conversion method for converting the first image compression information of the first bit rate into an image information conversion method of converting the first image compression information of the first bit rate into orthogonal image data. The orthogonal transform coefficient is inversely quantized in accordance with the quantization width of the coefficient, the inversely quantized orthogonal transform coefficient is added to the motion compensation error correction coefficient, and the second image compression information to be output is the second image compression information. In accordance with a quantization width such that the bit rate becomes, the orthogonal transform coefficient to which the motion compensation error coefficient has been added is requantized, and the second image compression information generated by requantization is output. The orthogonal transform coefficient is inversely quantized, the orthogonal transform coefficient to which the motion compensation correction coefficient has been added is subtracted from the inversely quantized orthogonal transform coefficient, and the subtracted orthogonal transform coefficient is orthogonally transformed to perform motion compensation based on the motion vector. And
The motion compensated value is subjected to inverse orthogonal transformation to generate the motion compensation error correction coefficient.

In this image information conversion method, the orthogonal transform coefficient is inversely quantized and then requantized by changing the quantization width, and the difference between the input orthogonal transform coefficient and the requantized orthogonal transform coefficient is changed. Compensation corrects the error due to motion compensation.

In the image information conversion method according to the present invention, the value of the high frequency component in the horizontal direction of the inversely quantized orthogonal transform coefficient is limited, and the second image compression information to be output is the second bit rate. The orthogonal transform coefficient whose horizontal high frequency component is restricted is requantized in accordance with a quantization width such that

In this image information conversion method, the orthogonal transform coefficient is inversely quantized, the value of the high frequency component of the orthogonal transform coefficient in the horizontal direction is restricted, the quantization width is changed, and requantization is performed. The difference between the coefficient and the requantized orthogonal transform coefficient is motion compensated to correct the error due to the motion compensation.

Further, in the image information conversion method according to the present invention, the orthogonal transform coefficient may be defined as a horizontal eight coefficient and a vertical eight coefficient.
A 4 × 8 inverse orthogonal transform is performed on a quantization error coefficient composed of 8 × 8 coefficients and composed of 8 horizontal coefficients and 8 vertical coefficients to generate a quantization error value in the spatial domain. 1/4 in the horizontal direction with respect to the quantization error value in the spatial domain
Motion compensation is performed with pixel precision, motion compensation is performed in the vertical direction with 1/2 pixel precision, and a quantization error correction value in the spatial domain is generated. 8
Is performed to generate the motion compensation error correction coefficient in the frequency domain.

In this image information conversion method, after the orthogonal transform coefficients are dequantized, the values of the high-frequency components of the orthogonal transform coefficients in the horizontal direction are limited, the quantization width is changed, and requantization is performed. The four low-frequency coefficients in the horizontal direction and the eight coefficients in the vertical direction of the difference between the coefficient and the requantized orthogonal transform coefficient are motion-compensated to correct an error due to motion compensation.

[0032]

BEST MODE FOR CARRYING OUT THE INVENTION As an embodiment of the present invention, an image information conversion apparatus for reducing the code amount (bit rate) of image compression information (bit stream) encoded by the MPEG-2 system will be described. . MPEG-2 (I
SO / IEC 13818-2) is a compression method of image information corresponding to interlaced scan images and progressive scan images, and both standard resolution images and high resolution images.

Before describing an image information conversion apparatus according to an embodiment of the present invention, an MPEG-2 image information encoding apparatus for encoding image compression information (bit stream) of the MPEG-2 system, and an MPEG. The configuration of the image compression information (bit stream) of the -2 system will be described.

(Image Information Encoding Apparatus) FIG.
2 shows a block diagram of an image information encoding device.

As shown in FIG. 1, the MPEG-2 image information encoding apparatus includes a screen rearrangement buffer 1, an adder 2, a discrete cosine transform unit 3, a quantization unit 4, a variable length encoding Device 5, code buffer 6, code amount control device 7, inverse quantization device 8, inverse discrete cosine transform device 9
, An adder 10, a video memory 11, and a motion prediction device 12.

The image rearranging buffer 1 receives baseband video data. The image rearrangement buffer 1 rearranges the order of frames arranged in chronological order in display time order into a frame order corresponding to the encoding order of MPEG-2.

FIG. 2 shows an example of a frame order rearrangement process performed by the screen rearrangement buffer 1.

Each frame of the input video data is
As shown in FIG. 2A, it is assumed that the images are arranged in chronological order in display time. At this time, the MPEG-2 image information encoding device first encodes the third frame in the display order by intra-frame encoding. Next, the third frame is used as a reference frame, the sixth frame is encoded by predictive coding, and the third frame and the sixth frame are used as reference frames, and the fourth and fifth frames are used as reference frames. Encoding processing such as encoding by predictive encoding is performed. Therefore, the image rearrangement buffer 1 rearranges the frames as shown in FIG. In MPEG-2, a GOP (Group Of Picture) composed of an image subjected to intra-frame encoding and an image group subjected to inter-frame encoding based on the image.
s) is defined. In this MPEG-2 image encoding apparatus, one image encoded by intra-frame encoding (hereinafter referred to as an I picture) and one reference frame (a frame that is processed earlier on the time axis) Are referred to as prediction frames, and four images (hereinafter referred to as P pictures) coded by inter-frame coding and two reference frames (on the time axis, past the frame to be processed) , And a picture in the future (hereinafter referred to as a B picture) encoded by inter-frame
It is assumed that a GOP composed of a total of 15 images of 0 is encoded.

Each frame rearranged by the image rearrangement buffer 1 is sent to the adder 2.

When a P-picture or a B-picture is sent, the adder 2 subtracts the frame and a predicted frame of this frame generated by a motion-compensated prediction unit 12 to be described later to obtain a discrete cosine. Send to conversion device 3. In addition, when an I picture is sent, the adder 2 sends the frame to the discrete cosine transform device 3 without subtracting any data from the frame.

The discrete cosine transform unit 3 divides each frame into blocks of 8 pixels × 8 pixels (DCT blocks), and performs an 8th-order discrete cosine transform on each DCT block in the horizontal and vertical directions. By performing the discrete cosine transform, a two-dimensional image signal is subjected to an orthogonal transform to be a spatial frequency coefficient (discrete cosine transform coefficient).
At this time, if the input video data is, for example, a Y: U: V = 4: 2: 0 signal as shown in FIG. 3, 16 pixels × 16 pixels of a luminance signal and 8 pixels × 8 pixels of a color difference signal are used. The processing is performed as one block (hereinafter, referred to as a macro block). In this case, the discrete cosine transform device 3 performs a discrete cosine transform of 8 × 8 pixels on a frame basis as shown in FIG. 4 (hereinafter, referred to as a frame discrete cosine transform mode) for the luminance signal. 5 is a mode in which the data of each macro block is separated for each field as shown in FIG. 5 and discrete cosine transform of 8 pixels × 8 pixels is performed (hereinafter referred to as a field discrete cosine transform mode). The more efficient discrete cosine transform is selected and performed.

The discrete cosine transform device 3 obtains data (discrete cosine transform coefficients) obtained by performing discrete cosine transform.
To the quantizer 4.

The quantizing device 4 has 8 horizontal and vertical directions.
The next discrete cosine transform coefficient is quantized using a predetermined quantization matrix. The quantization scale of the quantization matrix is feedback-controlled by the code amount control device 7 based on a predetermined target code amount (target bit rate) so that the subsequent code buffer 6 does not overflow or underflow. The quantization device 4 sends the quantized discrete cosine coefficients to the variable length coding device 5 and the inverse quantization device 8.

The variable length decoding device 5 performs variable length coding on the quantized discrete cosine transform coefficients so that the average code length becomes short. At this time, the variable-length decoding device 5 encodes the difference with the DC component coefficient of the previous block as a prediction value for the DC component of the discrete cosine transform coefficient, and encodes the difference for the other components using a preset scanning method. 1 based on (zigzag scan or alternate scan)
After rearrangement into the dimensional array data, encoding is performed using a pair of the number of consecutive 0 coefficients (run) and the non-zero coefficient (level) as an event.

FIG. 6A shows the scanning order for the 8th-order discrete cosine transform coefficients in the horizontal and vertical directions by zigzag scanning, and FIG. 6B shows the 8th-order discrete signals in the horizontal and vertical directions by the alternate scanning. 3 shows a scan order for cosine transform coefficients. Quantizer 4
EOB (End) when scanning in the DCT block and all coefficient values thereafter become 0
A code called “Of Block” is output, and the variable-length coding for the block ends.

The discrete cosine transform coefficients subjected to variable length coding by the variable length coding device 5 are sent to the code buffer 6 and temporarily stored in the code buffer 6.
A bit stream structure defined in PEG-2 is output as compressed image information.

On the other hand, the data quantized by the quantization device 4 is also sent to the inverse quantization device 8. The inverse quantization device 8 inversely quantizes the transmitted data and sends the data to the inverse discrete cosine transform device 9.

The inverse discrete cosine transform unit 9 performs an inverse discrete cosine transform on the inversely quantized discrete cosine transform coefficients. The pixel value obtained by performing the inverse discrete cosine transform is stored in the video memory 11 via the adder 10.

The adder 10 includes an inverse discrete cosine transform device 8
, A P-picture is sent from the video memory 11 and the frame is added to a reference frame of this frame generated by the motion compensation prediction device 12 described later and stored in the video memory 11. When an I picture is sent, the adder 10 stores the I picture in the video memory 11 without adding any data to the frame.

The motion-compensated prediction device 12 uses a half of a macroblock for a P picture and a B picture as a unit.
Pixel-accurate motion vector information is detected by a method such as a block matching method. The motion vector information is stored in the image compression information (bit stream) output from the code buffer 6 and sent out. At the same time, the motion compensation prediction device 12 performs motion compensation on the pixel data stored in the video memory 11 based on the detected vector information to generate prediction frames for P-pictures and B-pictures. Is supplied to the adder 2.

Since the I-picture is subjected to intra-frame prediction coding, no motion vector information is detected. However, for P-pictures and B-pictures, if the coding efficiency by inter-frame prediction based on motion vectors is not high, the macro It is also possible to perform intra-frame prediction in block units. Hereinafter, a macroblock encoded based on intra-frame prediction is called an intra macroblock, and a macroblock encoded based on inter-frame prediction is called an inter-macroblock. All macroblocks included in the I picture are intra macroblocks.

As described above, the MPEG shown in FIG.
The -2 image information encoding device encodes the input baseband video data and outputs image compression information (bit stream).

Next, the quantization processing in the quantization device 4 will be described in more detail.

First, the DC coefficient of an intra macroblock is quantized based on the following equation (1).

[0055]

(Equation 1)

"F" [0] shown in the equation (1)
[0] "is a quantization representative value of the DC coefficient value, and" QF
[0] [0] "is a quantization representative value level number of the DC coefficient value. The inverse quantization coefficient (intra dc mult) is a variable that can be set for each frame as shown in FIG. Intra DC precision (intra dc p) representing the bit precision of the DC coefficient
recision).

The coefficients other than the DC coefficient of the intra macroblock are quantized based on the following equation (2).

[0058]

(Equation 2)

"F" [u] shown in the equation (2)
[V] "is a quantization representative value of the (u, v) th coefficient value,
“QF [u] [v]” is a quantization representative value level number of the (u, v) th coefficient value. k is given by the following equation (3).

[0060]

(Equation 3)

"W [w] [v] [u]" is a quantization matrix, and "quantizer_scale" is a quantization scale.

The above-mentioned quantization matrix is a matrix provided for setting the relative quantization accuracy between the discrete cosine transform coefficient values in the DCT block. This quantization matrix is
It is possible to roughly quantize the discrete cosine transform coefficient of a high-frequency component whose deterioration is hardly noticeable visually, and finely quantize the discrete cosine transform coefficient of a low-frequency component whose visual deterioration is easily noticeable. Therefore, by using this quantization matrix, it is possible to reduce the code amount while minimizing image quality deterioration.

This quantization matrix can be set for each frame. If not set, a quantization matrix set to a default value as shown in FIG. 8A is used for an intra macroblock, and a default value as shown in FIG. 8B for an inter macroblock. The set quantization matrix is used.

Also, the above quantizer_scal
e (quantization scale) is q_scale_type (quantization scale type) defined for each frame in the syntax of MPEG2 image compression information (bit stream).
And q_scale_c defined for each macroblock
mode (quantization scale code) as shown in FIG. Q_s for each macroblock
The assignment of the call_code is performed by the code amount control device 7.

Next, the processing in the code amount control device 7 will be described in more detail.

The processing in the code amount control device 7 is not particularly defined in MPEG2 (ISO / IEC 13818-2), and the developer of the coding device does not cause overflow and underflow in the output code buffer 6. Under such a constraint condition, it is set freely so that better image quality can be obtained with the same code amount. Here, MPEG2 Test Model 5 (ISO
/ IEC JTC1 / SC29 / WG11 N040
The general processing described in (0) will be described with reference to the flowchart shown in FIG.

As shown in FIG. 10, the code amount control device 7 controls the target code amount (target bit rate) and the GOP
The components are set, and the target code amount and the GOP components are used as input variables. First, bit allocation to each picture is performed in step S1, and rate control using a virtual buffer is performed in step S2. As S3, adaptive quantization is performed in consideration of visual characteristics. The code amount control device 7 performs these processes on the output code buffer 6
While monitoring the occupancy (buffer fullness).

First, in step S1, a bit amount (hereinafter, referred to as R) to be allocated to a picture which has not been coded in the GOP including the picture to be allocated is allocated, and each bit in the GOP is allocated. A code amount is assigned to a picture. This code amount allocation is repeated in the order of the pictures in the GOP. At this time, a code amount is assigned to each picture based on the first and second assumptions described below.

First, it is assumed that the product of the average quantization scale code and the generated code amount (complexity of the screen) used when encoding each picture is a fixed value for each picture type unless the screen changes. I do. After encoding each picture, the variables Xi, Xp, and Xb (global complexity measure) indicating the complexity of the screen are updated for each picture type based on the following equation (4).

[0070]

(Equation 4)

The "Si" and "S" shown in the equation (4)
“p” and “Sb” are generated code bit amounts at the time of picture coding, and “Qi”, “Qp” and “Qb” are average quantization scale codes at the time of picture coding.

Variables Xi, Xp, Xb indicating screen complexity
Is the target code amount (target bit rate) b
Using it_rate [bits / sec], the values are represented by the following equations (5), (6), and (7).

[0073]

(Equation 5)

[0074]

(Equation 6)

[0075]

(Equation 7)

Second, when the ratios Kp and Kb of the quantized scale codes of the P and B pictures with respect to the quantized scale code of the I picture become the values defined by the following equation (8), the whole is always obtained. Assume that the image quality is optimized.

[0077]

(Equation 8)

From the above first and second assumptions, GO
The amount of bits allocated to each picture of P (Ti, T
p, Tb) are calculated by the following equations (9), (10), and (1).
1).

[0079]

(Equation 9)

[0080]

(Equation 10)

[0081]

[Equation 11]

"N" shown in equations (9) to (11)
“p” and “Nb” are the numbers of P and B pictures that have not been encoded in the GOP.

Based on the allocated code amount thus obtained, the bit amount R allocated to the uncoded picture in the GOP is updated by the following equation (12) every time each picture is coded. I do.

[0084]

(Equation 12)

When encoding the first picture of the GOP, R is updated by the following (13).

[0086]

(Equation 13)

"N" shown in equation (13) is the number of pictures in the GOP. Also, set the first R in the sequence to 0.
And

Subsequently, in step S2, in order to match the actual generated code amount with the allocated bit amount (Ti, Tp, Tb) for each picture obtained in step 1, three types independently set for each picture type are set. , The quantization scale code is obtained by feedback control in units of macroblocks based on the capacity of the virtual buffer.

First, prior to the j-th macroblock coding, the occupancy of the virtual buffer is obtained by the following equations (14), (15), and (16).

[0090]

[Equation 14]

[0091]

(Equation 15)

[0092]

(Equation 16)

“D ₀ ⁱ ” shown in the equations (14) to (16),
“D ₀ ^p ” and “d ₀ ^b ” are the initial occupancy of the virtual buffer of each of the I, P, and B pictures, and “Bj” is the amount of generated bits from the head of the picture to the j-th macroblock. , “MB_cnt” is the number of macroblocks in one picture. The virtual buffer occupancy at the end of picture encoding (d _{MB_cnt} ⁱ , d _{MB_cnt} ^p , d _{MB_cnt} ^b ) is the same as the picture type, and the initial value (d ₀ ⁱ , d) of the virtual buffer occupancy for the next picture ₀ ^p , d ₀ ^b ).

Next, the quantization scale code for the j-th macroblock is calculated by the following equation (17).

[0095]

[Equation 17]

"R" shown in the equation (17) is a variable called a reaction parameter for controlling the response of the feedback loop, and is given by the following equation (18).

[0097]

(Equation 18)

The initial value of the virtual buffer at the start of encoding is given by the following equation (19).

[0099]

[Equation 19]

Finally, in step 3, the quantization was made finer in a flat portion where the deterioration is visually conspicuous, and coarsely quantized in a complicated portion of the pattern where the deterioration is relatively inconspicuous. Quantization scale code,
It is changed by a variable called activity for each macroblock.

The activity act _j uses the pixel values of the luminance signal of the original image, and uses the pixel values of a total of eight blocks of four blocks in the frame discrete cosine transform mode and four blocks in the field discrete cosine transform mode. Therefore, the following expressions (20), (21), and (2)
Given in 2).

[0102]

(Equation 20)

[0103]

(Equation 21)

[0104]

(Equation 22)

Pk shown in Expressions (20), (21), and (22) is a pixel value in a luminance signal block of an original image. The reason for taking the minimum value in the equation (20) is to make the quantization finer when there is a flat portion even in only a part of the macroblock.

Further, according to the following equation (23), the normalized activity Nact whose value ranges from 0.5 to 2.
_{Find j} .

[0107]

(Equation 23)

Avg_act shown in the equation (23) is the average value of act _j in the picture coded immediately before.

The quantization scale code mquant _j considering the visual characteristics is given by the following equation (24) based on the quantization scale code Qj obtained in step 2.

[0110]

(Equation 24)

The encoding control device 7 performs the processing of steps SS1 to S3 as described above so that overflow and underflow do not occur in the output code buffer 6 and a better image quality can be obtained with the same code amount. Set freely.

Next, the constraint conditions in the output code buffer 6 will be described.

MPEG2 (ISO / IEC 13818)
In -2), a virtual buffer called a VBV (Video Buffer Verifier) buffer is defined for the decoding device. The code amount control device 7 allocates codes so that the image compression information (bit stream) output from the image encoding device satisfies the constraint condition. VBV
The buffer has a first ideal condition that the encoding apparatus and the VBV buffer operate completely synchronously, and a second ideal condition that decoding of each picture is performed instantaneously and data of each picture is instantaneously extracted from the VBV buffer. It is assumed to operate under two ideal conditions and two ideal conditions.
Note that the purpose of defining the VBV buffer is to add a definition for the image compression information (bit stream) based on this model, but does not specify an actual implementation method of the decoding device.

A VBV buffer has a constant code amount per unit time (fixed rate: CBR [Constant Bit R]).
ate]) and the case where the code amount per unit time is not constant (variable rate: VBR [Variable Bit Rate]), and the fixed rate is defined as a special case of the variable rate.

Further, in order to cope with various applications, VBV
Data input to the buffer can be defined by the following two methods, a first method and a second method. In the first method, the input rate is defined to change on a picture-by-picture basis. In the second method, the input rate is set to the peak rate or 0 according to the state of the VBV buffer.
Is defined to take one of the following values. In either method, the peak rate Rmax [bit / sec] is
In the syntax (syntax) of image compression information according to MPEG2, it is specified by a parameter bit rate of a part called a sequence header. The distinction between the two methods is made by a variable called vbv_delay defined for each frame.

If the vbv_delay of each picture is not the maximum value “0xFFFF”, vbv_delay
Indicates the time from when the start code of each picture is input to the VBV buffer until it is decoded. in this case,
The input rate of data to the VBV buffer is variable on a picture data basis. Rate R (n) [bit at which the n-th picture data is input to the VBV buffer
/ Sec] is determined by the following equation (25).

[0117]

(Equation 25)

Dn ^* shown in the equation (25) is the code amount from the start code for the nth picture to the start code for the (n + 1) th picture. τ (n) is the value of the variable vbv_delay set for the nth picture, and t (n) is the time at which the value for the nth picture is extracted from the VBV buffer. The fixed rate (CBR) data is R
A special case where (n) is a constant value irrespective of n is included in this specification.

FIG. 11 shows changes in the bit occupancy of the VBV buffer. The VBV buffer is initially
The bit occupancy is 0. The VBV buffer starts from the time when the start code of the first picture is input.
Decoding starts after the lapse of the time indicated by the variable vbv_delay. The decoding of each picture and the verification of the VBV buffer are continued at the determined decoding time interval.

When the vbv_delay of each picture is the maximum value “0xFFFF”, the data input rate to the VBV buffer is determined by the bit occupancy of the VBV buffer. This input rate becomes Rmax when there is a vacancy in the VBV buffer, and becomes 0 when there is no vacancy.

In this case, the bit occupancy of the VBV buffer is 0 in the initial state. Decoding starts when the buffer becomes full. Subsequently, decoding of each picture and verification of the VBV buffer are continued at a predetermined decoding time interval.

As for the image compression information (bit stream), data is input to the VBV buffer in accordance with the above rules. At that time, image compression information (bit stream)
The condition is that the VBV buffer does not overflow immediately before picture data is instantaneously extracted from the VBV buffer (condition (1)), and that the VBV buffer does not underflow immediately after picture data is instantaneously extracted from the VBV buffer (condition (1)). (2)), verification is performed so as to satisfy the two conditions, and encoding is performed.

Incidentally, regarding the above condition (2), the variable l
The verification content differs between the case of the low delay mode specified by ow-delay and the case of not being the low delay mode. What is low-latency mode?
This mode prohibits the use of B pictures. This low delay mode prevents delay caused by rearrangement of pictures at the time of encoding and decoding, and allows skipping (dropping of frames) in encoded pictures to reduce delay in the VBV buffer.

If the mode is not the low-delay mode, the VBV
It is verified that the V-buffer must not underflow.

In the low delay mode, if picture data is extracted, the VBV buffer is allowed to underflow. In this case, without extracting the picture data at that time, verification of whether or not all the picture data has arrived is repeated every two-field interval, and the picture data is extracted when the picture data arrives for the first time.

Therefore, the data amount of one picture must be smaller than the VBV buffer size including the case of the low delay mode.

(Configuration of MPEG-2 Image Compression Information) Next, the data configuration of MPEG2 image compression information (bit stream) encoded by the MPEG-2 image information encoding apparatus shown in FIG. 1 will be described.

FIG. 12 schematically shows the data structure of MPEG2 image compression information (bit stream).

The MPEG2 image compression information (bit stream) has a hierarchical structure as shown in FIG. 12, and a sequence (Sequence) layer, GO
P (Group of Pictures) layer, Picture (Picture) layer,
Slice layer, macro block (Macro Block)
Layer and block layer.

In each layer from the sequence layer to the slice layer, a unique start code of 8 bits (1 byte) following “0x000001” (total 32 bits (4 bytes))
Bytes)) are predefined. In the MPEG2 image compression information (bit stream), each layer is distinguished by this start code, and this start code is an error recovery point. FIG. 13 shows the code values of the start code in each layer.

FIG. 14 shows a block configuration of each layer of the MPEG2 image compression information (bit stream). Hereinafter, this MPEG2 image compression information data configuration will be described in more detail.

As shown in FIG. 14, the sequence layer is the highest layer of the image compression information (bit stream), and starts with a sequence header and ends with a sequence end code (sequence_end_code). This sequence layer basically includes one or more GOP layers. This sequence layer includes a sequence extension part and extension part & user data (1), and a start code (extension_start_co.
Based on the presence or absence of de), MPEG1 and MPEG2 image compression information (bit stream) is classified.

As shown in FIG. 15, a sequence header code (sequence_header_cod
e), horizontal pixel count information (horizontal_size_value),
Vertical line number information (vertical_size_value), pixel aspect ratio information (aspect_ratio_information), frame rate code (frame_rate_code), bit rate information (bit_rate_value), VBV buffer size information (vbv_buffer_size_value), quantization matrix for intra macroblock (intra_quantiser_matrix [64]) ]), Inter-macroblock quantization matrix (non_intra_quantiser_
matrix [64]) and other information set in sequence units.

After the start code (extension_start_code) of the sequence extension unit, as shown in FIG. 16, the profile extension and level display information (profile_and_level_indicati) used in MPEG2 are added to the sequence extension unit.
on), progressive scan image flag (progress
ive_sequence), chroma format (chroma_forma)
t), low-delay mode (low_delay), and additional data for extending the upper limit of the number of pixels and code amount (bit rate) in the sequence header shared with MPEG1 for MPEG2.

FIG. 17 shows the extension and user data (1).
As shown in, the extension data (1) (extension_data
(1)), sequence display extension () (sequence_display_
extension ()), sequence scalable extension () (s
equence_scalable_extension), a sequence scalable extension ID (extension_start_code_identifier), scalability mode information (scalable_mode), and a layer ID (layer_id) of a scalable hierarchy.
The sequence display extension section () contains information such as the RGB conversion characteristics of the original signal and the display image size. In the sequence scalable extension unit (), a scalability mode (space, SNR, temporal, data partition) and a scalability layer are set.
Further, the following variables are set in the extension part & user data (1) for spatial scalability and temporal scalability. In the case of spatial scalability, the horizontal size of the lower layer for prediction (lower_layer_pred
iction_horizontal_size), prediction lower-layer vertical size (lower_layer_prediction_vertical_size), vertical up-sample divisor (vertical_subsampling_fac)
tor_n), and in the case of temporal scalability, the number of additional layer images before the first base layer image (pi
cture_mux_order), the number of pictures in the additional layer between the base layers (picture_mux_factor), user data () (user
_data ()) and user data (user_data). User data at the sequence level can be freely set in the user data () (user_data ()).

Subsequently, as shown in FIG. 14, the GOP layer is located in the lower layer of the sequence layer, starts with the GOP header, and includes an extension & user data (2) and one or more picture layers. Is done. The first picture in the GOP layer is an I picture that is independently coded (intra-coded) without using a reference screen. Therefore, this GOP
Are used as entry points for random access by using this I picture. For example, when one GOP is composed of 10 to 15 pictures, if the video data is 30 frames per second, the GOP is 0.3 to 15 pictures.
There will be a random access point every 0.5 seconds. Further, since the length of the GOP is arbitrary, it is possible to start or end the GOP at the scene switching point to increase the coding efficiency. In addition, since the I picture has a very large code amount and a large occupied buffer amount, MPEG2 uses an I-picture so that it can be used in applications such as communication requiring low delay characteristics.
The GOP layer can be omitted. In this case, for example, a slice for performing intra-frame encoding (hereinafter, referred to as a slice)
Intra-encoded data of each slice is divided into several screens and circulated, thereby making it possible to substitute for I-pictures and reduce the buffer occupancy of any screen on average. is there.

As shown in FIG. 18, in the GOP header,
After the start code (group_start_code) of the GOP header, a time code (time_code), a flag (closed_gop) indicating the independence of the GOP, and a B before the I picture in the GOP
A flag (broken_link) indicating the validity of the picture is included. The flag (broken_link) indicating the validity of the B picture before the I picture in the GOP indicates that the B picture before the first I picture in the GOP cannot be decoded correctly due to, for example, the previous GOP being cut by editing. It is shown that. A flag (closed_gop) indicating the independence of a GOP indicates that the GOP does not depend on an image in another GOP. These codes can be used to avoid illegal B picture display at random access and
Used as a flag when the OP is cut.

The extension part and user data (2) include FIG.
As shown in, the extension data (2) (extension_data
(2)), user data () (user_data ()), and user data (user_data) are included, and user data is set at the GOP level.

Subsequently, the picture layer is a layer located below the GOP layer and corresponding to each screen, as shown in FIG. The picture layer includes a picture header, a picture coding extension, an extension & user data (3), and picture data. This picture layer is divided into one or more slice layers as shown in FIG.

The picture header includes a picture start code (picture_start_code), as shown in FIG.
It includes a temporal reference (temporal_reference), picture coding type information (picture_coding_type), and VBV delay amount information (vbv_delay) until the start of coding. Temporal reference (temporal_reference)
Is an image number indicating the display order of the picture in the GOP, and the initial value is reset to 0 for each GOP. In the picture coding type information (picture_coding_type), the distinction between I, P, and B for each picture is set. Also, VBV delay amount information until code start (vbv_delay)
Is set to the amount of delay until the decoding start time by the VBV buffer described above.

As shown in FIG. 21, the picture coding extension unit includes motion vector range information (f_code [s] [t]) and DC coefficient precision information (intra_dc_pre
cision), picture structure information (picture_structure),
Display field designation information (top_field_first), frame prediction and frame DCT flag (frame_pred_frame)
_dct), concealment motion vector flag for intra macroblock (concealment_motion_vector), quantization scale type information (q_scale_type), variable length code type information for intra macroblock (intra_vlc_form)
at), scan type information (alternate_scan), 2: 3
Field repeat information for pulldown (repeat_first_f
ield), 420 type information (chroma_420_type), and a progressive scan flag (progressive_frame). In the motion vector range information (f_code [s] [t]), the motion vector ranges in the forward / backward and horizontal / vertical directions are set.
In the picture structure information (picture_structure), whether encoding / decoding processing is performed for each field (hereinafter, referred to as a field structure) or encoding / decoding processing is performed for each frame (hereinafter, referred to as a frame structure) Is set. In addition, the linear / non-linear quantization scale is set in the variable length code type information for intra macroblock (intra_vlc_format), and the scan type information (alternate_sca
In n), the selection of the type of the variable-length encoding process for the picture (selection between alternate scan and zigzag scan) is set.

The extension part & user data (3) includes FIG.
As shown in, the extension data (3) (extension_data
(3)), quantization matrix extension part () (quant_matrix_extensio
n ()), intra macroblock quantization matrix (intra_qua
ntiser_matrix [64]), inter macroblock quantization matrix (non_intra_quantiser_matrix [64]), chroma intra macroblock quantization matrix (chroma_intra_quantiser)
_matrix [64]), chroma inter-macroblock quantization matrix (chroma_non_intra_quantiser_matrix [64]), copyright extension () (copyright_extension ()), picture display extension () (picture_display_extension ()), picture space scalable extension ( ) (Picture_spatial_sc
alable_extention ()), spatial weighting table for upsampling (spatial_temporal_weight_code_table_inde
x), lower layer progressive scan image flag (lower_layer_pro
gressive_frame), lower layer field selection information (lower_layer_deinterlaced_field_select), picture temporal scalable extension () (picture_tempor
al_scalable_extention ()), reference screen selection code (r
eference_select_code), the image number of the lower layer for forward prediction (forward_temporal_reference), the image number of the lower layer for backward prediction (backward_tempral_referen
ce), user data () (user_data ()), and user data (user_data).

Quantization matrix extension () (quant_matrix_ext)
In the extension (), a quantization matrix is set. Therefore, the quantization matrix of the intra, inter, luminance, and chrominance blocks can be changed for each picture, and quantization according to the characteristics of each screen can be performed. A copyright number is set in the copyright extension () (copyright_extension ()). Picture display extension () (pict
In ure_display_extension ()), a display area is set. Thus, for example, a 16: 9 high-resolution image (H
DTV) and decode the 4: 3
, A pan-scan function such as displaying a part of a high-resolution image on a standard resolution television by designating the area with the standard resolution television size can be realized. Picture space scalable extension () (picture_spatial_scalab
le_extention ()) and picture temporal scalable extension () (picture_temporal_scalable_extention ())
Is a space scalability or a temporal scalability, and information such as an image number of a lower layer used for prediction of an upper layer is set.

Subsequently, the slice layer is a layer located at a lower layer than the picture layer, as shown in FIG. As shown in FIG. 23, the slice layer is composed of a horizontally long band-like area (A, B, C... Q area in FIG. 23) divided in the screen, and is defined as slice data as shown in FIG. Is done. By configuring the screen with multiple slice layers,
Even if an error occurs in a certain slice layer, the error can be recovered by synchronization from the start code (slice_start_code) of the next slice layer. The slice layer is composed of one or more macroblocks, and is arranged in a raster scan order from left to right and from top to bottom. Its length and starting position are free,
It can be changed for each screen. However, for the purpose of parallel processing and effective error tolerance, one slice extends only to the right and does not extend downward.

The slice data includes a slice layer start code (slice_start_cod) as shown in FIG.
e), slice vertical position extension information (slice_vertical_posi)
tion_extension), partitioning point information for data partitioning (priority_breakpoint), quantization scale code (quantizer_scale_code), intra slice flag (intra_slice), macroblock data (macrobloc)
k ()). Start code of slice layer (sli
ce_start_code), slice vertical position extension information (slice_v
In “ertical_position_extension”, a vertical position at the start of the slice is set. Quantization scale code (quanti
In ser_scale_code), the quantization scale code shown in FIG. 9 is set. Intra slice flag (intra_sl
In ice), it is set whether or not all the macroblocks in the slice are intra macroblocks.

Subsequently, the macroblock layer is a layer located at a lower layer than the slice layer, as shown in FIG.
The macroblock layer includes macroblock information and block data.

As shown in FIG. 26, the macro block data includes address extension data (macrob
lock_escape), the difference between the current and previous macroblock addresses (macroblock_address_increment),
Macroblock modes () (macroblock_modes ()), macroblock coding type information (macroblock_type),
Spatio-temporal weighting code for upsampling (spatial_temp
oral_weight_code), frame structure motion compensation type information (frame_motion_type), field structure motion compensation type information (field_motion_type), DCT type information (dct_type), macroblock quantization scale code (quantiser_scale_code), motion vector (s) (moti
on_vectors [s]), reference field selection information used for prediction (motion_vertical_field_select [r] [s]), motion vectors (r, s) (motion_vectors [s] [r]), basic difference motion vectors (motion_code [r]) [s] [t]), residual vector (motion_residual [r] [s] [t]) dual-prime difference vector (dmvector [t]), CBP (coded_block_pat
tern) and block data (block (i)).

The difference between the current macroblock and the previous macroblock address (macroblock_address_increment) includes:
Current macroblock location (macroblock_address)
Is the position of the previous macroblock (previous_macrobloc
k_address). The difference between the current macroblock and the previous macroblock address (macrob
If lock_address_increment is greater than 1, the macroblock between the current macroblock and the immediately preceding coded macroblock is a skip macroblock,
Encoding information such as a motion vector and a discrete cosine transform coefficient does not exist in the image compression information (bit stream).
The condition for a skipped macroblock is that the motion vector value of the P picture is 0 for both the horizontal and vertical components and the discrete cosine transform coefficients are all 0. For the B picture, the previous macroblock and the motion compensation prediction mode ( (Forward, backward, bidirectional) and the motion vector are the same, and the discrete cosine transform coefficients are all zero. MPE
In G2, unnecessary macroblock information is reduced by introducing skipped macroblocks. Note that I
There are no skip macroblocks in pictures, and even in P and B pictures, the first and last macroblocks of a slice are always encoded without being skipped macroblocks.

The macroblock coding type information (macrob
In lock_type), a motion compensation prediction mode and a discrete cosine transform coefficient encoding mode are determined. Although possible modes differ depending on the picture type, basically, an MC / No_MC selection mode indicating whether a motion vector is 0 (in the case of a P picture), a direction of motion compensation prediction (in the case of a B picture, a forward direction, Backward or bidirectional), C
oded / Not_Coded (indicating presence / absence of discrete cosine transform coefficients), intra-frame encoding, and updating of the quantization scale code are set.

The frame type motion compensation type information (fram
In the “e_motion_type” and the motion compensation type information of the field structure (field_motion_type), it is set whether to perform motion compensation on a field basis or on a frame basis. Hereinafter, the former is called a field motion compensation mode, and the latter is called a frame motion compensation mode.

The motion vectors (s) (motion_vectors)
In [s]), a motion vector is set. Spatio-temporal weighting code for upsampling (spatial_temporal_weight_
In code), a spatio-temporal weighting coefficient for up-sampling the lower layer and predicting the upper layer is set.
Macroblock quantization scale code (quantiser_scal
In e_code), the size of the quantization scale for each macroblock is set. Note that the quantization scale is controlled by the code amount control device 7 as described above.

Subsequently, as shown in FIG. 14, the block layer is a layer located at a lower layer than the macro block layer.
Block data (block (i)) composed of quantized discrete cosine transform coefficients is defined in the block layer. The block data (block (i)) is composed of 8 lines × 8 pixels of a luminance signal or a color difference signal, and discrete cosine transform and inverse discrete cosine transform are performed in this unit.

As shown in FIG. 27, DCT luminance DC coefficient difference size (dct_dc_size_lumi)
nance), DCT luminance DC coefficient difference value (dct_dc_differen
cial), DCT color difference DC coefficient difference size (dct_dc_size_
chrominance), DCT color difference DC coefficient difference value (dct_dc_di
fferencial), the first nonzero coefficient of the non-intra block (Fi
rst_DCT_coefficients) and subsequent DCT coefficients (Subsquen
ce_DCT_coefficients), and a DCT coefficient end flag (End_of_Block) in the block.

DCT luminance DC coefficient difference size (dct_dc_s
ize_luminance) is given the magnitude of the difference between the DC component coefficient in the intra macroblock of the luminance component and the adjacent block. DCT luminance DC coefficient difference value (dct_dc_d
Ifferencial), difference information of the coefficient of the DC component of the intra macroblock of the luminance component is given. DCT color difference DC coefficient difference size (dct_dc_size_chrominance),
The magnitude of the difference between the DC component coefficient in the intra macroblock of the color difference component and the adjacent block is given. DC
In the T color difference DC coefficient difference value (dct_dc_differencial),
The difference information of the coefficient of the DC component of the intra macroblock of the color difference component is provided. In the first non-zero coefficient (First_DCT_coefficients) of the non-intra block, the length of a zero coefficient up to the non-zero quantized discrete cosine transform coefficient of the inter macroblock (hereinafter, referred to as a run length) is given. Subsequent DCT coefficients (Subsquence_DCT_coefficients)
The magnitude of the non-zero coefficient of the inter macroblock is given. DCT coefficient end flag in block (End_of_Bloc
In k), the end information is given to the discrete cosine transform coefficient in each block.

The MPEG2 image compression information (bit stream) configured as described above is encoded by the MPEG-2 image information encoding device and transmitted to a decoding device or the like.

(Image Information Converter of First Embodiment of the Present Invention) Next, an image information converter of the first embodiment to which the present invention is applied will be described.

FIG. 28 is a block diagram showing an image information conversion apparatus according to the first embodiment of the present invention. This image information conversion apparatus is an apparatus that outputs a low bit rate image compression information by reducing the code amount (bit rate) of the image compression information (bit stream) encoded by the MPEG-2 system.

An image information conversion device 20 shown in FIG. 28 includes a code buffer 23, a compression information analysis device 24, a variable length decoding device 25, an inverse quantization device 26, and a band limiting device 2.
7, a quantization device 28, a variable length coding device 29, a code buffer 30, and a code amount control device 31.

The code buffer 23 stores image compression information (bit stream) having a large code amount (high bit rate).
Is entered. Image compression information (bit stream)
As described above, since the encoding is performed so as to satisfy the constraint condition of the VBV buffer, no overflow or underflow occurs in the code buffer 23. Image compression information (bit stream) stored in the code buffer 23
Is sent to the compression information analyzer 24.

[0160] The compression information analysis device 24 has a
The information is extracted from the compressed image information (bit stream) according to the MPEG-2 syntax (syntax) as shown in (1), and the extracted information is sent to the variable length decoding device 25 and each of the subsequent devices.

The variable-length decoding device 25 first performs variable-length decoding on the DC component of the intra macroblock, which is encoded as a difference value from the adjacent block.
For other coefficients, the data encoded by the run and the level are subjected to variable length decoding, and quantized one-dimensional discrete cosine transform coefficients are detected. Next, the variable length decoding device 2
Reference numeral 5 denotes an inverse scan of the discrete cosine transform coefficients arranged one-dimensionally on the basis of information on the scanning method (zigzag scan or alternate scan) extracted by the compression information analysis device 24, thereby obtaining a quantized two-dimensional discrete cosine. Rearrange them into transform coefficients. The two-dimensionally arranged quantized discrete cosine transform coefficients are sent to the inverse quantization device 26.

The inverse quantization device 26 is a compression information analysis device 2
4, the quantized discrete cosine transform coefficients are inversely quantized based on the information on the quantization width and the quantization matrix extracted in step 4. The inversely quantized discrete cosine transform coefficients are sent to the band limiting device 27.

The band limiting device limits the band of the high frequency component coefficient in the horizontal direction for each DCT block with respect to the discrete cosine transform coefficient sent from the inverse quantization device.

FIG. 29 shows an example of the band limiting process of the high frequency component in the horizontal direction in the band limiting device 27. For example,
The band limiting device 27 controls the luminance signal as shown in FIG.
As shown in (A), of the 8 × 8 discrete cosine transform coefficients, only the values of the 6 × 8 coefficients, which are the low-frequency components in the horizontal direction, are stored, and the rest are replaced with 0. Also, the band limiting device 27
With respect to the color difference signal, as shown in FIG.
Of the 8 × 8 discrete cosine transform coefficients, only the values of 4 × 8 coefficients, which are low-frequency components in the horizontal direction, are stored, and the rest are replaced with 0.

By limiting the band of the high-frequency component of the discrete cosine transform coefficient, the amount of code (bit rate) can be reduced in the frequency domain.

If the input image compression information (bit stream) is that of an interlaced image, the information on the time difference between fields is included in the vertical high-frequency component of the discrete cosine transform coefficient. Become. Therefore, limiting the band of the discrete cosine transform coefficient in the vertical direction leads to a significant deterioration in image quality. Therefore, the band limiting device 27 does not perform band limiting in the vertical direction.

The band limiting device 27 limits the band of a color difference signal which is less likely to be seen by humans than a luminance signal which is more easily seen by humans. As a result, this band limiting device 27
Thus, distortion of requantization can be reduced while minimizing image quality deterioration. When the amount of code (bit rate) to be reduced is small or when there is a circuit limitation, the band limitation of the luminance signal and the color difference signal may be the same.

The band limiting process of the discrete cosine transform coefficient in the horizontal direction in the band limiting device 27 is the same as that shown in FIG.
Is not limited to the process of setting the coefficient to 0 as shown in FIG. For example, instead of replacing with 0, it is possible to similarly reduce the code amount (bit rate) by multiplying a previously prepared weight coefficient by the horizontal high-frequency component of the discrete cosine transform.

The discrete cosine transform coefficient band-limited by the band limiting device 27 is sent to the quantization device 28.

The quantizing device quantizes the 8 × 8 discrete cosine transform coefficients sent from the band limiting device 27. The quantization width used at that time is controlled by the code amount control device 31 as described below. As the quantization matrix used for quantization, the quantization matrix used in the input image compression information (bit stream) having a high code amount (high bit rate) may be used as it is. A new matrix suitable for conversion may be used.

A method of controlling the quantization width in the coding control device 31 will be described below.

As described with reference to FIG.
-2 MPEG applied in image information coding apparatus
2 Test Model 5 (ISO / IEC JT
In the method used in C1 / SC29 / WG11 N0400), first, a target code amount (target bit rate) for each picture based on information about pictures (I picture, P picture, B picture) constituting a GOP.
Is calculated, and an encoding process is performed. However, in the image information conversion device 20 of the present embodiment, since image compression information (bit stream) is input, GOP information cannot be read from header information, and MPEG-2
The technique used in Test Model 5 cannot be applied as it is.

Therefore, in the image information conversion device 20, as shown in FIG. 30, first, the compression information analysis device 24 analyzes the code amount assigned to one picture in advance (step S11). Then, information on the code amount assigned to the analyzed one picture is provided to the code amount control device 31. The code amount control device 31 sets the given code amount to B ₁ , the bit rate of the input bit stream to R ₁ , the bit rate of the output bit stream to R ₂ , and sets the target code amount (target bit rate) of the picture. ) T is calculated by the following equation (26), and a target code amount is set (step S12).

[0174]

(Equation 26)

Note that R ₁ and R ₂ can be extracted by analyzing the information in the sequence header shown in FIG.

Then, the code amount control device 31 performs the following steps S1, S2, S2 shown in FIG.
The same processing as in step 3 is performed.

As described above, in the image information conversion apparatus 20, 1
The code amount assigned to the picture is analyzed in advance, and the analyzed code amount is multiplied by the bit rate ratio of the input / output image compression information, so that even if there is no information on the GOP, the code amount is determined by MPEG-2 Test Model 5. It is possible to control the code amount in the specified method.

The code amount control device 31 also allocates a quantization width to each macro block in the code buffer 30 so that the above-described constraint condition of the VBV buffer is satisfied.

The discrete cosine transform coefficients requantized as described above are supplied from the quantizer 29 to the variable-length encoder 29.
Sent to

The variable length decoding device 29 encodes the quantized discrete cosine transform coefficients so that the average code length becomes short. At this time, the variable-length decoding device 5 encodes the difference with the DC component coefficient of the previous block as a prediction value for the DC component of the discrete cosine transform coefficient, and encodes the difference for the other components using a preset scanning method. After rearrangement into one-dimensional array data based on (zigzag scan or alternate scan), variable-length coding is performed using a pair of the number of consecutive 0 coefficients (runs) and the non-zero coefficient (level) as an event. Then, the quantization device 4 outputs a code called EOB (End Of Block) when scanning all the coefficients after that during the scan in the DCT block, and outputs a code for the block. End long coding.

Note that the variable-length coding device 29 arranges discrete cosine transform coefficients into one-dimensional data by an alternate scan method regardless of the scan method of the inputted high-code-rate (high bit rate) image compression information. You may. The reason why the discrete cosine transform coefficients are arranged in one-dimensional data by the alternate scan method is as follows.

That is, it is assumed that the discrete cosine transform coefficient of a certain block of the input image compression information (bit stream) is as shown in FIG. 31A, for example. In FIG. 31, coefficients indicated by ● are non-zero coefficients,
The circles indicate the 0 coefficient. Assuming that the horizontal high frequency component of the discrete cosine transform coefficient is 0 for such a discrete cosine transform coefficient, the distribution of the non-zero coefficient is, for example, as shown in FIG.
The result is as shown in FIG. When the discrete cosine transform coefficient in which the horizontal high frequency component shown in FIG. 31B is set to 0 is re-encoded by zigzag scan, the scan number of the last non-zero coefficient becomes 50 (see FIG. 6A). For it,
When the scan conversion is performed and the encoding is performed again by the alternate scan, the scan number of the last non-zero coefficient becomes 44 (see FIG. 6B). From this, when performing variable length coding on the discrete cosine transform coefficient with the horizontal high frequency component set to 0, if the scan is performed by the alternate scan method, the EOB signal is set with a scan number earlier than that in the zigzag scan. It becomes possible. Therefore, a finer value can be assigned as the quantization width, and quantization distortion accompanying re-quantization can be reduced.

The discrete cosine transform coefficients subjected to variable length coding by the variable length coding device 5 are sent to the code buffer 6 and temporarily stored in the code buffer 6.
A bit stream structure defined in PEG-2 is output as compressed image information.

As described above, in the image information conversion apparatus 20 according to the first embodiment of the present invention, the code amount (bit rate) can be reduced by transferring the data of each block in the frequency domain. As compared with a conventional image information conversion apparatus that decodes and encodes baseband video data, the amount of calculation is reduced, and the circuit configuration can be significantly reduced.

In the image information converter 20, a band limiting device 27 is provided between the inverse quantization device 26 and the quantization device 28. However, depending on the amount of code (bit rate) reduction, this band The restriction device 27 need not be provided.

(Image Information Conversion Apparatus According to Second Embodiment of the Present Invention) Next, an image information conversion apparatus according to a second embodiment of the present invention will be described.

FIG. 32 is a block diagram showing an image information conversion apparatus according to the second embodiment of the present invention. In describing the image information conversion device of the second embodiment, the same components as those of the image information conversion device 20 of the first embodiment are denoted by the same reference numerals in the drawings. Detailed description is omitted.

The image information converter 40 shown in FIG. 32 comprises a code buffer 23, a compression information analyzer 24, a variable length decoder 25, an inverse quantizer 26, an adder 41,
Band limiting device 27, quantizing device 28, variable length coding device 29, code buffer 30, code amount control device 31
And a motion compensation error correction device 42.

The adder 41 is provided between the inverse quantization device 26 and the band limiting device 27. The adder 41 subtracts the motion compensation error correction coefficient generated by the motion compensation error correction device 42 from the discrete cosine transform coefficient obtained by the inverse quantization performed by the inverse quantization device 26.

A motion compensation error correction device 42 corrects a motion compensation error generated when the quantized device 28 requantizes the discrete cosine transform coefficient inversely quantized by the inverse quantizer 26. Generate

First, the cause of the motion compensation error will be described.

Let the pixel value of the original image be O, and let Q be the quantization width of the input image compression information (bit stream) of a high code amount (high bit rate) with respect to the pixel value O of this original image.
_1, and the quantization width with respect to the pixel value O in the original image of the image compression information of a low code amount after re-encoding (low bit rate) (bit stream) is referred to as Q _2. Also, let the pixel values of the reference image decoded with the quantization width Q ₁ and the quantization width Q ₂ be L (Q ₁ ) and L (Q ₂ ), respectively.

At the time of encoding, the difference value “OL (Q ₁ )” of the pixel of the inter macroblock is obtained by the adder 2 of the MPEG-2 image information encoding apparatus shown in FIG.
Is calculated, and a discrete cosine transform is applied to the difference value “OL (Q ₁ )”. Pixels of the thus coded inter macroblock, during decoding, inverse discrete cosine transform is applied to the difference value _{"O-L (Q 1)} ", the difference value "O-L (Q ₁₎ Is subtracted from the reference image “L (Q ₁ )” generated by the motion compensation, and the pixel value O of the original image is decoded.

On the other hand, the pixels of the inter macroblock are
When the code amount (bit rate) is reduced by the image information conversion device 20 shown in FIG. 28, the quantization width of the difference value “OL (Q ₁ )” is set to Q1 by the inverse quantization device 26 and the quantization device 28. To Q2. The pixels of the inter macro block whose code amount has been reduced in this way are decoded at the time of decoding, assuming that the difference value “OL (Q ₂ )” is coded with the quantization width Q _2. .

Here, since the amount of code is reduced by changing the quantization width in the image information converter 20, Q ₁ = Q ₂
Does not hold, and a quantization error occurs at the time of decoding the inter macroblock. Therefore, an error accompanying the motion compensation occurs in the P picture and the B picture encoded by the inter macro block.

The error generated in the P picture propagates to a P picture or a B picture using the P picture as a reference picture, which leads to further deterioration of the picture quality.

The motion compensation error correction device 42 of the image information compression device 40 according to the second embodiment generates a motion compensation error correction coefficient and subtracts it from the discrete cosine transform coefficient inversely quantized by the inverse quantization device 26. Then, the above-described motion compensation error is corrected.

Next, the motion compensation error correction device 42 will be described.

The motion compensation error correction device 42 includes an inverse quantization device 43, an adder 44, and an inverse discrete cosine transform device 45.
, A video memory 46, a motion compensation prediction device 47, and a discrete cosine transform device 48.

The inverse quantization device 43 inversely quantizes the discrete cosine transform coefficients requantized by the quantization device 28 based on the quantization matrix used in the quantization device 28. The discrete cosine transform coefficient inversely quantized by the inverse quantization device 43 is sent to the adder 44.

The adder 44 subtracts the discrete cosine transform coefficient, from which the motion compensation error correction coefficient has been subtracted by the adder 41, from the discrete cosine transform coefficient inversely quantized by the inverse quantizer 43, to obtain a frequency domain. A quantization error coefficient is generated and sent to the inverse discrete cosine transform unit 45.

The inverse discrete cosine transform unit 45 includes the adder 4
4 for the quantization error coefficient in the frequency domain sent from
Perform inverse discrete cosine transform. The result obtained by performing the inverse discrete cosine transform is stored in the video memory 46 as a quantization error value in the spatial domain.

The motion-compensated prediction device 47 calculates the motion-compensated prediction mode information (field motion-compensated prediction mode or frame motion-compensated prediction mode) in the input image compression information (bit stream) having a high code amount (high bit rate). , Forward prediction mode, backward prediction mode, or bidirectional prediction mode) and motion vector information, and performs motion compensation on the quantization error value in the spatial domain in the video memory 46. The data subjected to the motion compensation becomes a motion compensation error correction value in the spatial domain. This motion compensation error correction value is sent to the discrete cosine transform unit 48.

The discrete cosine transform unit 48 performs a discrete cosine transform on the sent motion compensation error correction value in the spatial domain, and generates a motion compensation error correction coefficient in the frequency domain. The motion compensation error correction coefficient is sent to the adder 41.

In the adder 41, the motion compensation error is corrected by subtracting the motion compensation error correction coefficient from the discrete cosine transform coefficient dequantized by the dequantizer 26. You.

As described above, in the image information conversion device 40 according to the second embodiment of the present invention, the code amount (bit rate) can be reduced by transferring the data of each block in the frequency domain. As compared with a conventional image information conversion apparatus that decodes and encodes baseband video data, the amount of calculation is reduced, and the circuit configuration can be significantly reduced. At the same time, the image information conversion device 40 can reduce the code amount without deteriorating image quality due to accumulation of motion compensation errors.

The inverse discrete cosine transform unit 45 and the discrete cosine transform unit 4 of the motion compensation error correction unit 42
8, a fast algorithm can be applied.

FIG. 33 shows a document "A fast comp"
national algorithm fort
he discrete cosine transf
orm "(IEEE Trans. Commun., v
ol. 25, no. 9 pp. 1004-109,1
977) is a flowchart when the high-speed algorithm shown in FIG.

In this method, the variables [X (0),..., X (0),.
X (7)], it is possible to execute the discrete cosine transform. In addition, by processing in the direction opposite to the arrow in FIG. 33 (direction from variable [X (0)... X (7)] to variable [x (0)... X (7)]). It is possible to perform an inverse discrete cosine transform. Note that “C _l ⁱ ” shown in FIG.
(Iπ / l).

In the inverse discrete cosine transform unit 45 and the discrete cosine transform unit 48, when the coefficient of the horizontal high frequency component is replaced with 0 in the band limiting unit 37,
By omitting the inverse discrete cosine transform and the discrete cosine transform for the coefficient replaced with 0, it is possible to reduce the circuit scale and the amount of arithmetic processing.

Furthermore, since the deterioration of the color difference signal in the image has a characteristic color that is hard to be recognized by human eyes as compared with the deterioration of the luminance signal, the above-described motion compensation error correction is applied only to the luminance signal. Thus, the circuit scale and the amount of arithmetic processing can be significantly reduced while keeping the image quality deterioration to a minimum. Further, the error in the P picture propagates to the B picture, but the error in the B picture does not propagate any more. On the other hand, a B picture includes a bidirectional prediction mode and requires a large amount of calculation processing. Therefore, by performing the motion compensation error correction only on the P picture, it is conceivable to significantly reduce the circuit scale and the amount of arithmetic processing while keeping the image quality deterioration to a minimum. By not performing the processing on the B picture, the capacity of the video memory 46 can be reduced.

Although the above description has been directed to image compression information (bit stream) by MPEG2 as input, image compression information (bit stream) encoded by orthogonal transformation and motion compensation has the same configuration as that of the present apparatus. Thus, it is possible to reduce the code amount (bit rate).

(Image Information Conversion Apparatus According to Third Embodiment of the Present Invention) Next, an image information conversion apparatus according to a third embodiment of the present invention will be described.

FIG. 32 is a block diagram showing an image information conversion apparatus according to the third embodiment of the present invention. In describing the image information conversion apparatus according to the third embodiment, the same image information conversion apparatus 20 as the image information conversion apparatus 20 according to the first embodiment and the image information conversion apparatus 40 according to the second embodiment will be described. The same reference numerals are given to the components in the drawings, and detailed description thereof will be omitted.

An image information conversion device 50 shown in FIG. 34 includes a code buffer 23, a compression information analysis device 24, a variable length decoding device 25, an inverse quantization device 26, an adder 41,
Band limiting device 27, quantizing device 28, variable length coding device 29, code buffer 30, code amount control device 31
And a motion compensation error correction device 51.

The adder 41 is provided between the inverse quantization device 26 and the band limiting device 27. The adder 41 subtracts the motion compensation error correction coefficient generated by the motion compensation error correction device 51 from the discrete cosine transform coefficient obtained by the inverse quantization performed by the inverse quantization device 26.

The motion compensation error correction device 51 corrects a motion compensation error generated when the quantized device 28 requantizes the discrete cosine transform coefficient inversely quantized by the inverse quantizer 26. Generate The motion compensation error correction device 51 corrects the motion compensation error by subtracting the motion compensation error correction coefficient from the discrete cosine transform coefficient inversely quantized by the inverse quantization device 26.

Next, the motion compensation error correction device 51 will be described.

The motion compensation error correction device 51 includes an inverse quantization device 43, an adder 44, a 4 × 8 inverse discrete cosine transform device 52, a video memory 53, an interpolation device 54, and a motion compensation prediction device 55. 4 × 8 discrete cosine transform unit 56
It is composed of

[0220] The inverse quantization device 43 inversely quantizes the discrete cosine transform coefficients requantized by the quantization device 28 based on the quantization matrix used in the quantization device 28. The discrete cosine transform coefficient inversely quantized by the inverse quantization device 43 is sent to the adder 44.

The adder 44 subtracts the discrete cosine transform coefficient, from which the motion compensation error correction coefficient has been subtracted by the adder 41, from the discrete cosine transform coefficient inversely quantized by the inverse quantizer 43, and performs frequency domain quantization. A generated error coefficient is generated and sent to the 4 × 8 inverse discrete cosine transform device 52.

The 4 × 8 inverse discrete cosine transform unit 52 converts the quantization error coefficient sent from the adder 44 into a 4 × 8
Is performed, and a 4 × 8 quantization error value in the spatial domain is calculated. 4 × 8 inverse discrete cosine transform device 52
Stores in the video memory 53 the 4 × 8 quantization error value in the spatial domain obtained as a result of performing the inverse discrete cosine transform.

The video memory 53 stores 4 × 8 quantization error values in the spatial domain. This 4 × 8 quantization error value has a capacity capable of storing data having a resolution of に 対 し て of the resolution of the input image compression information, since the horizontal data is decimated by １ ／. It should just be.

[0224] The interpolation device 54 performs an interpolation process on the horizontal data of the 4x8 quantization error value stored in the video memory 53. This interpolator 54 allows the horizontal 4 ×
By interpolating the 8 quantization error values, the motion compensation prediction unit 55 at the subsequent stage
Pixel-precision motion prediction can be performed. Interpolator 54
Performs interpolation by performing linear interpolation or interpolation using a digital filter having several taps such as a half-band filter. It should be noted that the interpolating device 54 may generate only necessary values by interpolating according to the value of the motion vector.

FIG. 35 shows that 1 in the horizontal direction is obtained by linear interpolation.
The processing content when performing interpolation with / 4 pixel accuracy is shown. When the motion vector indicates a pixel position having the same phase as the original pixel, the value of the original pixel becomes the value of the interpolated pixel as shown in FIG. When the motion vector indicates a pixel position shifted by 1/4 phase from the original pixel, FIG.
As shown in FIG. 35 (C) and FIG. 35 (D), the value of the interpolated pixel is obtained by multiplying the two pixel values adjacent in the horizontal direction by the ratio of the distance to the pixel.

The motion-compensated prediction device 55 calculates the motion-compensated prediction mode information (field motion-compensated prediction mode or frame motion-compensated prediction mode) in the input compressed image information (bit stream) having a high code amount (high bit rate). , Forward prediction mode, backward prediction mode, or bidirectional prediction mode), and performs motion compensation on a 4 × 8 quantization error value in a spatial region in the video memory 53 based on the motion vector information. At this time, motion compensation is performed in the horizontal direction with 1/4 pixel accuracy using the value interpolated by the interpolation device 54, and motion compensation is performed in the vertical direction with 1/2 pixel accuracy.

The data subjected to the motion compensation becomes a 4 × 8 motion compensation error correction value in the spatial domain. This 4 × 8 motion compensation error correction value is sent to a 4 × 8 discrete cosine transform device 56.

The 4 × 8 discrete cosine transform unit 56 performs a 4 × 8 discrete cosine transform on the received 4 × 8 motion compensation error correction value, and generates a motion compensation error correction coefficient in the frequency domain. This motion compensation error correction coefficient is calculated by the adder 4
Sent to 1.

In the adder 41, the motion compensation error is corrected by subtracting the motion compensation error correction coefficient from the discrete cosine transform coefficient dequantized by the dequantizer 26. You.

As described above, in the image information conversion apparatus 50, 8
Of the error components due to motion compensation represented as × 8 discrete cosine coefficients, the arithmetic processing is performed only on the horizontal low-frequency components and all the vertical frequency components that greatly contribute to the error correction, The arithmetic processing is not performed on the high frequency component in the horizontal direction. As for the high frequency component of the error in the horizontal direction, the frequency of distribution of the error is originally low, and the band limiting device 27 performs a process of reducing the high frequency component.
Even if the arithmetic processing is not performed, the image quality hardly deteriorates. However, the error component in the vertical direction is, for example, when the image compression information to be input is based on interlaced scanning, since information on the time difference between fields is included in the high frequency component, Perform arithmetic processing.

As described above, in the image information conversion apparatus 50 according to the third embodiment of the present invention, the code amount (bit rate) can be reduced by transferring the data of each block in the frequency domain. As compared with a conventional image information conversion apparatus that decodes and encodes baseband video data, the amount of calculation is reduced, and the circuit configuration can be significantly reduced. At the same time, the image information conversion device 50 can reduce the code amount without causing image quality deterioration due to accumulation of motion compensation errors. Also,
Motion compensation error correction device 5 for obtaining motion compensation error correction coefficient
1, the arithmetic processing is not performed on the high frequency component in the horizontal direction, so that the amount of operation can be further reduced.

The 4 × 8 inverse discrete cosine transform unit 52
As the processing in the vertical direction of the 4 × 8 discrete cosine transform device 56, normal 8th-order inverse discrete cosine transform and 8th-order discrete cosine transform are performed. At this time, the 4 × 8 inverse discrete cosine transform device 52 and the 4 × 8 discrete cosine transform device 56 can apply a high-speed algorithm.

FIG. 36 shows a document “A fast comp”.
national algorithm fort
he discrete cosine transf
orm "(IEEE Trans. Commun., v
ol. 25, no. 9 pp. 1004-109,1
977) is a flowchart when the high-speed algorithm shown in FIG.

In this method, the variables [X (0),..., X (0),...
X (7)], it is possible to execute the discrete cosine transform. Also, by processing in the direction opposite to the arrow in FIG. 36 (the direction from the variable [X (0)... X (7)] to the variable [x (0)... X (7)]). It is possible to perform an inverse discrete cosine transform. Note that “C _l ⁱ ” shown in FIG.
(Iπ / l).

The 4 × 8 inverse discrete cosine transform unit 52
As the processing in the horizontal direction of the 4 × 8 discrete cosine transform device 56, there are the following two methods.

In the first method, in the 4 × 8 inverse discrete cosine transform device 52, of the quantization error coefficients in the frequency domain, which are the 8th-order discrete cosine coefficients, only the 4th order inverse Perform discrete cosine transform. The 4 × 8 discrete cosine transform unit 56 performs a fourth-order discrete cosine transform in the horizontal direction on the motion compensation error correction value in the 4 × 8 spatial region generated by the motion compensation. This makes it possible to output a motion compensation error correction coefficient in the 4 × 8 frequency domain. Here, a high-speed algorithm can be applied to the fourth-order inverse discrete cosine transform and the fourth-order discrete cosine transform. FIG. 37 shows a flowchart in the case where the fourth-order high-speed algorithm is applied. This fourth-order high-speed algorithm takes F (0) to F (3) as inputs and f
By setting (0) to f (3) as outputs, it is possible to execute inverse discrete cosine transform. Also, f (0)
To f (3) as input and F (0) to F (3) as output, it is possible to execute discrete cosine transform.

In the second method, the 4 × 8 inverse discrete cosine transform unit 52 replaces the high-frequency 4 coefficients of the frequency-domain quantization error coefficients, which are the 8th-order discrete cosine coefficients, with 0, and replaces them with the 8th-order discrete cosine coefficients. A discrete cosine transform is performed to obtain a quantization error value of an eighth-order spatial domain. Subsequently, the 4 × 8 inverse discrete cosine transform device 52 performs an averaging process or a thinning process in the horizontal direction on the quantization error value of the eighth-order spatial domain to obtain a fourth-order quantization error value of the spatial domain. Generate The 4 × 8 discrete cosine transform device 56 converts the four error correction values of the pixel area obtained by the motion compensation into eight points by interpolation processing. Subsequently, the 4 × 8 discrete cosine transform device 56 performs discrete cosine transform on the motion compensation pixel correction values in the spatial region that has been made into eight points by the interpolation process, and then extracts low-order components up to the fourth order. This makes it possible to output an error correction coefficient in a 4 × 8 frequency domain.

In the second method, the 4 × 8 inverse discrete cosine transform unit 52 performs the inverse discrete cosine transform and the averaging process or the inverse discrete cosine transform and the thinning process.
A series of processes may be performed using one matrix. FIG. 38 shows an arithmetic expression (iD ₄ _ave) in the case where the inverse discrete cosine transform and the averaging process are performed using one matrix. Also,
FIG. 39 shows an arithmetic expression (iD ₄ _deci) in the case where the inverse discrete cosine transform and the thinning-out process are performed with one matrix.

The 4 × 8 discrete cosine transform unit 56
Similarly, the interpolation process and the discrete cosine transform may be calculated as a series of processes using one matrix. An arithmetic expression (D ₄ _ave) in the case where the interpolation process corresponding to the averaging process and the discrete cosine transform are calculated using one matrix, and the interpolation process corresponding to the thinning process and the inverse discrete cosine transform are calculated using one matrix. An arithmetic expression (D ₄ _deci) for performing the arithmetic operation is represented by the following expression. Note that ^t () indicates a transposed matrix.

[0240]

[Equation 27]

Further, the deterioration of the color difference signal in the image has a characteristic feature that it is difficult for the human eye to understand the deterioration of the luminance signal compared to the deterioration of the luminance signal. For this reason, as shown in FIG. 40, the 4 × 8 inverse discrete cosine transform device 52 replaces the vertical high-frequency component 4 coefficients of the discrete cosine coefficients of the color difference signal with 0, performs inverse orthogonal transform, and reduces the amount of operation. Reduction may be performed.

Although the above description has been directed to image compression information (bit stream) by MPEG2 as input, image compression information (bit stream) encoded by orthogonal transformation and motion compensation has the same configuration as that of the present apparatus. Thus, it is possible to reduce the code amount (bit rate).

[0243]

In the image information conversion apparatus and method according to the present invention, the orthogonal transform coefficients are inversely quantized and then requantized by changing the quantization width. As a result, according to the present invention, the image signal can be processed in the frequency domain without being processed in the baseband. Therefore, the bit rate can be reduced with a small amount of calculation and a small circuit scale. Further, according to the present invention, distortion due to requantization can be reduced, image quality degradation can be reduced, and the bit rate can be reduced.

In the image information conversion apparatus and method according to the present invention, after the orthogonal transform coefficient is dequantized, the value of the high frequency component of the orthogonal transform coefficient in the horizontal direction is limited, and the quantization width is changed to change the requantization. Become As a result, in the present invention, distortion due to requantization can be reduced, image quality degradation can be reduced, and the bit rate can be reduced. In the present invention,
Since information in the vertical direction including difference information between fields is not limited, it is possible to reduce image quality degradation and reduce the bit rate.

In the image information conversion apparatus and method according to the present invention, the orthogonal transform coefficient is inversely quantized and then requantized by changing the quantization width, and the input orthogonal transform coefficient and the requantized orthogonal The difference from the transform coefficient is motion-compensated to correct the error due to the motion compensation. As a result, the accumulation of motion compensation errors due to requantization is reduced, and the bit rate can be reduced with less deterioration in image quality.

Further, in the image information converting apparatus and method according to the present invention, after the orthogonal transform coefficient is dequantized, the value of the high frequency component of the orthogonal transform coefficient in the horizontal direction is limited, and the quantization width is changed to perform requantization. At the same time, the four low-frequency coefficients in the horizontal direction and the eight coefficients in the vertical direction of the difference between the input orthogonal transform coefficient and the requantized orthogonal transform coefficient are motion-compensated to correct an error due to motion compensation. As a result, the accumulation of errors in motion compensation due to requantization is reduced, and the bit rate can be reduced by reducing the deterioration of image quality.
The circuit scale can be reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram of an MPEG-2 image information encoding device.

FIG. 2 is a diagram illustrating a frame order rearrangement process performed by a screen rearrangement buffer of the MPEG-2 image information encoding apparatus. (A) is a diagram showing a frame order before rearrangement, and (B) is a diagram showing a frame order after rearrangement.

FIG. 3 is a diagram illustrating a configuration of a macroblock when video data is a 4: 2: 0 signal.

FIG. 4 shows DCT in the frame discrete cosine transform mode
It is a figure explaining a block.

FIG. 5 shows DC in field discrete cosine transform mode
It is a figure explaining a T block.

FIG. 6 is a diagram illustrating a scan order of discrete cosine transform coefficients when performing variable length coding. (A) is a figure which shows the scan order of a zigzag scan, (B) is a figure which shows the scan order of an alternate scan.

FIG. 7 is a diagram illustrating a relationship between an inverse quantization coefficient and intra DC precision.

FIG. 8 is a diagram showing default values of a quantization matrix.
(A) is a figure which shows the quantization matrix set by default used about an intra macroblock.
(B) is a diagram showing a quantization matrix set to a default value used for an inter macroblock.

FIG. 9 shows MPEG2 image compression information (bit stream).
Quantizer_scale defined in the syntax of
It is a figure explaining (quantization scale).

FIG. 10 is a flowchart showing an operation content of a code amount control device of the MPEG-2 image information encoding device.

FIG. 11 is a diagram illustrating a change in a bit occupancy of a VBV buffer.

FIG. 12 is a diagram showing a hierarchical structure of MPEG2 image compression information (bit stream).

FIG. 13 is a diagram showing a code value of a start code in each layer of MPEG2 image compression information (bit stream).

FIG. 14 is a diagram illustrating a block configuration of each layer of MPEG2 image compression information (bit stream).

FIG. 15 is a diagram illustrating information included in a sequence header.

FIG. 16 is a diagram illustrating information included in a sequence extension unit.

FIG. 17 is a diagram illustrating information included in an extension part and user data (1).

FIG. 18 is a diagram illustrating information included in a GOP header.

FIG. 19 is a diagram illustrating information included in an extension part and user data (2).

FIG. 20 is a diagram illustrating information included in a picture header.

FIG. 21 is a diagram illustrating information included in a picture coding extension unit.

FIG. 22 is a diagram illustrating information included in an extension part and user data (3).

FIG. 23 is a diagram illustrating a slice layer divided in a screen.

FIG. 24 is a diagram illustrating slice data defining a slice layer.

FIG. 25 is a diagram illustrating information included in slice data.

FIG. 26 is a diagram illustrating information included in macroblock data.

FIG. 27 is a diagram illustrating information included in block data.

FIG. 28 is a block diagram of an image information conversion device according to the first embodiment of the present invention.

FIG. 29 is a diagram illustrating an example of band limitation of horizontal high-frequency components of discrete cosine transform coefficients by the band limitation device of the image information conversion device. (A) is a figure which shows the example of a band limitation of the discrete cosine transform coefficient with respect to a luminance signal, (B) is a figure which shows the example of a band limitation of the discrete cosine transform coefficient with respect to a chrominance signal.

FIG. 30 is a flowchart showing the operation of a code amount control device of the image information conversion device.

FIG. 31 regardless of the scanning method of the input signal.
FIG. 5 is a diagram illustrating scanning of a discrete cosine transform coefficient by an alternate scan method. (A) is a diagram showing discrete cosine transform coefficients before band limitation,
(B) is a diagram showing discrete cosine transform coefficients after band limitation.

FIG. 32 is a block diagram of an image information conversion device according to a second embodiment of the present invention.

FIG. 33 is a diagram showing a processing flow when a high-speed algorithm is applied to discrete cosine transform and inverse discrete cosine transform.

FIG. 34 is a block diagram of an image information conversion device according to a third embodiment of the present invention.

FIG. 35 is a diagram showing processing contents when performing interpolation with １ ／ pixel accuracy in the horizontal direction by linear interpolation.

FIG. 36 is a diagram showing a processing flow when a high-speed algorithm is applied to 4 × 8 discrete cosine transform and 4 × 8 inverse discrete cosine transform.

FIG. 37 is a diagram showing a processing flow when another high-speed algorithm is applied to 4 × 8 discrete cosine transform and 4 × 8 inverse discrete cosine transform.

FIG. 38 is a diagram illustrating an arithmetic expression in a case where the inverse discrete cosine transform and the averaging process are performed using one matrix.

FIG. 39 is a diagram illustrating an arithmetic expression in the case where the inverse discrete cosine transform and the thinning-out process are performed using one matrix.

FIG. 40 is a diagram illustrating a process of replacing 4 coefficients in the vertical direction of the discrete cosine coefficient of the color difference signal with 0.

FIG. 41 is a block diagram of a conventional image information conversion device.

[Explanation of symbols]

20, 40, 50 image information conversion device, 24 compression information analysis device, 25, variable length decoding device, 26 inverse quantization device, 27 band limiting device, 28 quantization device, 29 variable length coding device, 31 code amount Control device, 41, 44
Adder, 42, 51 motion compensation error correction device, 43 inverse quantization device, 45 inverse discrete cosine transform device, 46, 53
Video memory, 47, 55 motion compensation prediction device, 48
Discrete Cosine Transformer, 52 4 × 8 Inverse Discrete Cosine Transformer, 54 Interpolator, 564 × 8 Discrete Cosine Transformer

Continued on the front page (72) Inventor Shintaro Okada 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Ryuik 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation In-house (72) Inventor Naofumi Yanagihara 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo F-term in Sony Corporation (reference) 5C059 KK01 KK22 MA00 MA23 MC11 MC38 ME01 NN01 PP04 PP16 SS01 SS02 SS13 TA53 TB08 TC10 TD02 TD03 UA02 UA05 UA11

Claims (27)

[Claims] 1. An image signal is compressed and encoded by performing orthogonal transformation in a predetermined pixel block (orthogonal transformation block) unit and quantizing orthogonal transformation coefficients in the orthogonal transformation block obtained by orthogonal transformation. An image information conversion apparatus for converting the first compressed image information of the first bit rate into second compressed image information of a second bit rate lower than the first bit rate. An inverse quantization means for inversely quantizing the orthogonal transform coefficient according to the quantization width of the orthogonal transform coefficient of the image compression information; and a quantizer for outputting the second image compression information having the second bit rate. An image information conversion device comprising: a quantization unit that requantizes the orthogonal transform coefficient inversely quantized by the inverse quantization unit according to the quantization width. 2. The image processing apparatus according to claim 1, further comprising a band limiting unit configured to limit a value of a high-frequency component in a horizontal direction of the orthogonal transform coefficient inversely quantized by the inverse quantization unit, wherein the quantization unit outputs second image compression information to be output. 2. The method according to claim 1, wherein the orthogonal transform coefficient whose horizontal high-frequency component is limited by the band limiting means is re-quantized in accordance with a quantization width at which the second bit rate becomes the second bit rate. Image information conversion device. 3. The apparatus according to claim 2, wherein said band limiting means limits the orthogonal transform coefficient of the chrominance signal to the orthogonal transform coefficient of a lower component than the orthogonal transform coefficient of the luminance component.
The image information conversion device described in the above. 4. An image compression information analyzing means for analyzing a code amount of each picture of the input first image compression information, and a second image compression means for controlling and outputting a quantization width of the quantization means. Code amount control means for controlling the code amount of the information, wherein the code amount control means sets the code amount per picture of the input first image compression information to B, the first bit rate to R1, When the second bit rate is R2, a target code amount (target bit) T of each picture of the second image compression information is obtained by T = B (R2 / R1) to control the quantization width. The image information conversion device according to claim 1, wherein: 5. The orthogonal quantizer inversely quantized by the inverse quantizer using a quantization matrix different from a quantization matrix obtained by quantizing an orthogonal transform coefficient of the first image compression information. 2. The image information conversion device according to claim 1, wherein the conversion coefficient is requantized. 6. Variable-length decoding means for performing variable-length decoding on the input first image compression information, and variable-length coding means for performing variable-length coding on the image compression information requantized by the quantization means. 2. The image information conversion apparatus according to claim 1, wherein the variable length coding means converts the orthogonal transform block into a one-dimensional signal by an alternate scan method and performs variable length coding. 7. An image signal is compressed and encoded by performing orthogonal transformation in units of a predetermined pixel block (orthogonal transformation block) and quantizing orthogonal transformation coefficients in the orthogonal transformation block obtained by orthogonal transformation. An image information conversion apparatus for converting the first compressed image information of the first bit rate into second compressed image information of a second bit rate lower than the first bit rate. First inverse quantizing means for inversely quantizing the orthogonal transform coefficient according to the quantization width of the orthogonal transform coefficient of the image compression information; and orthogonal transform coefficient inversely quantized by the first inverse quantizing means. Means for adding the motion compensation error coefficient and the motion compensation error correction coefficient, and the motion compensation error coefficient is added by the addition means in accordance with the quantization width such that the output second image compression information has the second bit rate. Sa Quantizing means for requantizing the orthogonal transform coefficient obtained, second inverse quantizing means for inversely quantizing the orthogonal transform coefficient requantized by the quantizing means, and second inverse quantizing means Subtraction means for generating a quantization error coefficient in the frequency domain by subtracting the orthogonal transformation coefficient to which the motion compensation correction coefficient has been added by the addition means from the orthogonal transformation coefficient inversely quantized by the addition means; Motion compensation based on a motion vector by orthogonally transforming the orthogonal transformation coefficient, and a motion compensation error correction means for generating the motion compensation error correction coefficient by performing an inverse orthogonal transformation on a value obtained by the motion compensation. Image information conversion device. 8. A band limiting means for limiting a value of a high frequency component in a horizontal direction of the orthogonal transform coefficient dequantized by the first dequantizing means, wherein the quantizing means outputs The orthogonal transform coefficient whose horizontal high-frequency component is limited by the band limiting means is re-quantized according to a quantization width at which image compression information becomes the second bit rate. 7. The image information conversion device according to 7. 9. The apparatus according to claim 8, wherein said band limiting means limits the orthogonal transform coefficient of the chrominance signal to the orthogonal transform coefficient of a lower component than the orthogonal transform coefficient of the luminance component.
The image information conversion device described in the above. 10. The image information conversion apparatus according to claim 8, wherein said motion compensation error correction means does not perform inverse orthogonal transform and orthogonal transform on the orthogonal transform coefficient set to 0 by said band limiting means. 11. An image compression information analysis means for analyzing the code amount of each picture of the input first image compression information, and a second image compression means for controlling and outputting a quantization width of the quantization means. Code amount control means for controlling the code amount of the information, wherein the code amount control means sets the code amount per picture of the input first image compression information to B, the first bit rate to R1, When the second bit rate is R2, a target code amount (target bit) T of each picture of the second image compression information is obtained by T = B (R2 / R1) to control the quantization width. The image information conversion device according to claim 7, wherein: 12. The orthogonal quantizer inversely quantized by the inverse quantizer using a quantization matrix different from a quantization matrix obtained by quantizing the orthogonal transform coefficient of the first image compression information. The image information conversion device according to claim 7, wherein the conversion coefficient is requantized. 13. Variable-length decoding means for performing variable-length decoding on the input first image compression information, and variable-length coding means for performing variable-length encoding on the image compression information requantized by the quantization means. Wherein the variable length coding means converts the orthogonal transform block into a one-dimensional signal by an alternate scan method and performs variable length coding irrespective of the scan method of the input first image compression information. The image information conversion device according to claim 7, wherein 14. The image information according to claim 7, wherein said motion compensation error correction means generates a motion compensation error correction coefficient for a P picture, and generates a motion compensation error correction coefficient for a B picture. Conversion device. 15. The image information according to claim 7, wherein said motion compensation error correction means generates a motion compensation error correction coefficient for the luminance signal and a motion compensation error correction coefficient for the color difference signal. Conversion device. 16. The image information conversion apparatus according to claim 7, wherein said motion compensation error correction means performs inverse orthogonal transformation and orthogonal transformation based on a high-speed algorithm. 17. The orthogonal transformation coefficient comprises 8 × 8 coefficients of 8 coefficients in a horizontal direction and 8 coefficients in a vertical direction, and the motion compensation error correction means comprises a quantization error comprising 8 coefficients in a horizontal direction and 8 coefficients in a vertical direction. A 4 × 8 inverse orthogonal transform unit for performing a 4 × 8 inverse orthogonal transform on the coefficients to generate a quantization error value in the spatial domain; and a quantum in the spatial domain generated by the 4 × 8 inverse orthogonal transform unit. Motion compensating unit for performing motion compensation on the quantization error value in the horizontal direction with １ ／ pixel accuracy and motion compensation in the vertical direction with 画素 pixel accuracy to generate a quantization error correction value in the spatial domain A 4 × 8 orthogonal transform unit that performs 4 × 8 orthogonal transformation on the spatial domain quantization error correction value generated by the motion compensation unit, and generates the motion compensation error correction coefficient in the frequency domain. 8. The image information conversion device according to claim 7, comprising: . 18. The inverse orthogonal transform unit according to claim 17, wherein the 4 × 8 inverse orthogonal transform unit performs an inverse orthogonal transform by replacing a value of a vertical high-frequency component of an orthogonal transform coefficient of the color difference signal with 0. Image information conversion device. 19. The motion compensation error correction means according to claim 4, wherein
An interpolation unit that performs pixel interpolation with 1/4 pixel accuracy in the horizontal direction on the quantization error value in the spatial domain generated by the x8 inverse orthogonal transform unit; 18. The image information conversion device according to claim 17, wherein horizontal motion compensation is performed using the interpolated interpolation value. 20. The 4 × 8 inverse orthogonal transform unit performs a fourth-order inverse orthogonal transform on only four low-frequency coefficients among the eighth-order quantization error coefficients in the frequency domain in the horizontal direction, Is 8
The following inverse orthogonal transform is performed. The 4 × 8 orthogonal transform unit performs a fourth-order orthogonal transform on the motion compensation error correction value in the spatial domain in the horizontal direction and an eighth-order orthogonal transform on the vertical direction. 18. The image information conversion device according to claim 17, wherein: 21. The 4 × 8 inverse orthogonal transform section, in the horizontal direction, outputs a high-frequency 4th-order quantized error coefficient in the frequency domain.
After the coefficient is replaced with 0 and an 8th-order inverse orthogonal transform is performed, a thinning process or an averaging process is performed to generate a 4th-order spatial domain quantization error value. Has 4 in the spatial domain
18. The image information conversion device according to claim 17, wherein discrete cosine transform is performed after the motion compensation error correction values of the points are interpolated into eight motion compensation error correction values. 22. The 4 × 8 inverse orthogonal transform unit performs an inverse orthogonal transform and thinning process or an inverse orthogonal transform and an averaging process by one matrix, and the 4 × 8 orthogonal transform unit performs an interpolation process. 22. The image information conversion device according to claim 21, wherein the image information and the orthogonal transform are calculated using one matrix. 23. An image signal is compressed and encoded by performing orthogonal transformation in units of a predetermined pixel block (orthogonal transformation block) and quantizing orthogonal transformation coefficients in the orthogonal transformation block obtained by orthogonal transformation. An image information conversion method for converting the first compressed image information of the first bit rate into second compressed image information of a second bit rate lower than the first bit rate. Is input according to the quantization width of the orthogonal transformation coefficient of the input first image compression information, and the second image compression information to be output is The dequantized orthogonal transform coefficient is requantized in accordance with a quantization width that becomes the second bit rate, and the second image compression information generated by requantization is output. Do Image information conversion method. 24. A value of a high-frequency component in the horizontal direction of the inversely quantized orthogonal transform coefficient is limited, and the second image compression information to be output is determined according to a quantization width such that the second bit rate is obtained. 24. The image information conversion method according to claim 23, further comprising requantizing the orthogonal transform coefficient in which high frequency components in the horizontal direction are limited. 25. Compression encoding of an image signal by performing orthogonal transformation on a predetermined pixel block (orthogonal transformation block) basis and quantizing orthogonal transformation coefficients in the orthogonal transformation block obtained by orthogonal transformation. An image information conversion method for converting the first compressed image information of the first bit rate into second compressed image information of a second bit rate lower than the first bit rate. Is input, and the orthogonal transform coefficient is inversely quantized according to the quantization width of the orthogonal transform coefficient of the input first image compressed information. And the motion compensation error correction coefficient, and the orthogonal transform coefficient to which the motion compensation error coefficient has been added is reproduced according to the quantization width such that the output second image compression information has the second bit rate. And outputs the second image compression information generated by requantization, dequantizes the requantized orthogonal transform coefficient, and adds a motion compensation correction coefficient from the dequantized orthogonal transform coefficient. Subtracting the orthogonal transformation coefficient, orthogonally transforming the subtracted orthogonal transformation coefficient, performing motion compensation based on the motion vector, and performing inverse orthogonal transformation on the value subjected to the motion compensation to generate the motion compensation error correction coefficient. Image information conversion method. 26. A value of a high-frequency component in the horizontal direction of the inversely quantized orthogonal transform coefficient is limited, and the second image compression information to be output is set in accordance with a quantization width such that the second bit rate is obtained. 26. The image information conversion method according to claim 25, further comprising requantizing an orthogonal transform coefficient in which high frequency components in the horizontal direction are restricted. 27. The orthogonal transform coefficient is composed of 8 × 8 coefficients of 8 coefficients in the horizontal direction and 8 coefficients in the vertical direction, and is 4 × 8 with respect to a quantization error coefficient composed of 8 coefficients in the horizontal direction and 8 coefficients in the vertical direction. And generates a quantization error value in the spatial domain. The quantization error value in the spatial domain is 1 /
Motion compensation is performed with 4-pixel accuracy, and motion compensation is performed with 1 / 2-pixel accuracy in the vertical direction to generate a quantization error correction value in the spatial domain. 26. The image information conversion method according to claim 25, wherein an orthogonal transformation of x8 is performed to generate the motion compensation error correction coefficient in the frequency domain.