JP4383240B2 - Intra-screen predictive coding apparatus, method thereof and program thereof - Google Patents

Description

  The present invention relates to a moving picture coding technique, and more particularly, an intra-picture prediction coding apparatus that suppresses a flicker phenomenon caused by intra-picture prediction of an intra picture (encoded picture within a picture) of a moving picture, and a method thereof. And the program.

  Currently, MPEG-1, MPEG-2 and MPEG-4, ISO / IEC JTC1 / SC29 / WG1 (standardized by ISO / IEC JTC1 / SC29 / WG11 (MPEG)), and ISO / IEC JTC1 / SC29 / WG1 JPEG) standardized by JPEG) and H.264 recommended by ITU-T. 261 and H.H. H.263 (see Non-Patent Document 1, for example).

  In these compression encoding methods, a moving image is divided into regions of a specific size called blocks (for example, 16 × 16 pixels), and motion compensation prediction and conversion processing (discrete cosine transform [DCT: DCT: Discrete Cosine Transform] and the like, and further, quantization is performed so as to greatly reduce high-frequency components, and variable length coding is performed to realize moving image coding. In addition, in the above-described compression coding method, a transform coefficient such as DCT in a block to be coded is applied to an intra image (intra-screen coded image) for which motion compensation prediction is not performed. Encoding efficiency is improved by predictive encoding using a difference format with the block.

As another method for compressing and encoding moving images, H.264 standardized by ITU-T and ISO / IEC JTC1 / SC29 / WG11. H.264 / MPEG-4 AVC (hereinafter referred to as H.264 / AVC) (see Non-Patent Document 2).
This H. In H.264 / AVC, in an intra image in which motion compensation prediction is not performed, the pixel value of the image (screen) is directly predicted in the screen (intra-screen prediction), thereby improving the coding efficiency. That is, H.I. In H.264 / AVC, for an area (block) of 16 × 16 pixels called a macroblock, pixel values close to the macroblock in an already encoded macroblock at the top or the left of the macroblock are calculated. Utilizing this, the pixel value of the macroblock is predicted in a plurality of prediction modes, and the pixel value of the macroblock is predicted by selecting the prediction mode most similar to the pixel value of the macroblock.

  Further, H.C. H.264 / AVC has a mode in which a macroblock is divided into regions (blocks) each having a size of 4 × 4 pixels, and intra prediction is performed for each 4 × 4 pixel block. In this case, H.C. H.264 / AVC uses a pixel value close to the block in the already encoded block at the upper, upper right, left, and upper left of the 4 × 4 pixel block, and sets the pixel value of the block. In this prediction mode, the pixel value of the block is predicted by selecting the prediction mode most similar to the pixel value of the block. And H. H.264 / AVC selects a macro block that is more similar to the macro block from the prediction results of the 16 4 × 4 pixel blocks and the prediction result of the macro block (16 × 16 pixel block). Are predicted.

In general, in a compression encoding method that performs motion compensation prediction, an inter image that is encoded by inter prediction (inter prediction) in which motion compensation prediction is performed with reference to a temporally previous or subsequent image (screen). In between, an intra image is inserted (for example, about once every several seconds). At this time, H.C. H.264 / AVC is a mode in which a previous image is copied in block units, called a skip mode, in a portion where there is almost no motion in an inter image (below a certain amount) when a moving image is encoded at a low bit rate. The coding efficiency is improved by using a lot of.
As a result, H.C. H.264 / AVC has significantly improved compression efficiency compared to compression encoding schemes such as MPEG-1, MPEG-2, and MPEG-4.
"ISO / IEC 13818-2: 2000 Information technology-Common coding of video and associated audio information: Video", December 21, 2000 “ISO / IEC 14496-10: 2003 Information Technology—Audio—Visual Object Coding—Part 10: Information Technology-Coding of Audio-Visual Objects—Part 10: Advanced Video Coding”, 11 November 2003 27th of May

  The compression encoding method described above reduces the amount of information of encoded data to be generated by roughly performing quantization when encoding a moving image at a low bit rate. However, when performing intra prediction with an intra image, the quantization target is residual information (prediction error information) that is a difference between pixel values of the original macroblock and the predicted macroblock. Even if a macroblock is continuous with images of approximately the same pixel value in time, depending on the prediction method (prediction mode), if the quantization is rough, the quantization is completely different from the time direction. May result. Therefore, moving images that have been compression-encoded at a low bit rate according to the conventional compression-encoding method are temporally different by being decoded based on different quantization results even if they are macroblocks at the same position. There is a problem that the image becomes an image, flicker occurs in the decoded image, and the image quality deteriorates.

  In addition, the aforementioned H.P. When H.264 / AVC encodes an intra image in a moving image, even if a macro block is continuous with images having substantially the same pixel values in time, other macro blocks close to the macro block When the pixel value of a macroblock is temporally different, the prediction mode of the macroblock may be temporally different. As a result, even though the images have substantially the same pixel value at the same macroblock position, they are predicted as greatly different pixel values, and flicker occurs when the encoded moving image is decoded. However, there is a problem that the image quality deteriorates.

  In such a problem, when all frames of a moving image are encoded as intra images by intra prediction (intra encoding), flicker may occur in all frames (images). H. In H.264 / AVC, when compression coding of a moving image is performed at a low bit rate, a copy of the previous image in time is used in the inter image between the intra images with a little motion in the skip mode. . For this reason, H.C. H.264 / AVC may be encoded as a different image from the previous intra image due to the difference in prediction mode and quantization as described above when encoding the next inserted intra image. As a result, H.C. In H.264 / AVC, flicker occurs in a short cycle in which an intra image is inserted.

  The present invention has been made in view of the above-described problems. In the compression encoding of moving images, the occurrence of flicker in a decoded image accompanying intra prediction of an intra image is suppressed, and deterioration of image quality is prevented. It is an object of the present invention to provide an intra-screen predictive encoding apparatus, a method thereof, and a program thereof that can be performed.

  The present invention was devised to achieve the above object. First, the intra prediction encoding apparatus according to claim 1 sets a pixel value in a screen constituting a moving image to a specific size. In the intra prediction encoding apparatus that encodes the screen by predicting in units of blocks, a prediction error information generation unit, a time difference information generation unit, a prediction evaluation value calculation unit, and a prediction mode determination unit are provided. It was set as the structure provided.

  According to such a configuration, the intra prediction encoding apparatus predicts the pixel value of the encoding target block that is the encoding target block by the prediction error information generation unit based on a plurality of predetermined prediction modes, By taking the difference from the original image, prediction error information that is a prediction error is generated. This prediction error information may vary greatly depending on the pixel value and prediction mode of a block adjacent to the encoding target block. Therefore, here, a plurality of prediction error information are generated by a plurality of prediction modes without specifying one prediction mode.

  In addition, the intra prediction encoding apparatus uses the time difference information generation means to quantize and locally decode the prediction error information generated by the prediction error information generation means, and to store the past local data that existed at the same position as the encoding target block. By taking the difference from the decoded image, time difference information that is a difference in the time direction is generated. That is, the time difference information generation means reproduces a decoded image (local decoded image) decoded on the decoding device side by decoding (local decoding) the prediction error information at the current time, and further reproduces the past (for example, one frame). It is possible to detect how much error has occurred in the time direction by taking the difference from the decoded image that has been locally decoded (before) and at the same position on the screen. The error in the time direction may vary greatly depending on the prediction mode, similar to the prediction error in the screen. Therefore, here, a plurality of pieces of time difference information are generated for each prediction mode.

  Then, the intra prediction encoding apparatus calculates the prediction evaluation value indicating the degree of similarity in the prediction mode from the prediction error information and the time difference information by the prediction evaluation value calculation means. A smaller error between the prediction error information and the time difference information means that the prediction in the screen and in the time direction (between screens) is more accurate. Therefore, the intra prediction encoding apparatus determines the prediction mode of the encoding target block based on the prediction evaluation value calculated by the prediction evaluation value calculation means by the prediction mode determination means. As a result, the intra prediction encoding apparatus can encode a moving image in a prediction mode in which prediction errors in the screen and in the time direction are reduced.

  The intra prediction encoding apparatus according to claim 2 predicts pixel values of a block obtained by dividing a screen constituting a moving image into a specific size in a plurality of predetermined prediction modes, and the prediction error thereof. An intra-screen predictive encoding device that encodes the screen by determining the prediction mode based on: a block dividing unit, a prediction error information generating unit, a time difference information generating unit, a prediction evaluation value The calculation unit, the local decoding unit, the prediction mode determination unit, and the encoding unit are provided.

  According to such a configuration, the intra prediction encoding apparatus divides the screen constituting the moving image into blocks each divided into a specific size (for example, 16 × 16 pixels) by the block dividing unit. Then, the intra prediction encoding apparatus predicts the pixel value of the encoding target block based on a plurality of predetermined prediction modes by the prediction error information generation unit, and obtains a difference from the original image. Prediction error information that is an error is generated. Here, the pixel value of the encoding target block means the pixel values of a plurality of pixels constituting the block.

  Also, the intra prediction encoding apparatus quantizes and locally decodes the prediction error information for the prediction mode generated by the prediction error information generation unit by the time difference information generation unit and exists at the same position as the block to be encoded. By taking a difference from the past local decoded image, time difference information that is a difference in the time direction is generated. Then, the intra prediction encoding apparatus calculates the prediction evaluation value indicating the degree of similarity in the prediction mode from the prediction error information and the time difference information by the prediction evaluation value calculation means. The prediction evaluation value becomes an index indicating that the prediction in the screen and in the time direction is more accurate as the errors between the prediction error information and the time difference information are smaller.

Then, the intra prediction encoding apparatus generates quantized data and a locally decoded image by quantizing and locally decoding the prediction error information by the local decoding unit.
Furthermore, the intra prediction encoding apparatus determines the prediction mode by the prediction mode determination unit based on the prediction evaluation value calculated by the prediction evaluation value calculation unit. As a result, a prediction mode in which the prediction error in the screen and in the time direction is reduced is determined.

  Then, the intra prediction encoding apparatus encodes the quantized data generated based on the prediction mode determined by the prediction mode determination means by the encoding means, and generates encoded data. As a result, the intra prediction encoding apparatus can encode a moving image by appropriately determining and predicting a prediction mode for each block in which the prediction error in the screen and in the temporal direction is small.

  Furthermore, the intra prediction encoding apparatus according to claim 3 is the intra prediction encoding apparatus according to claim 2, wherein the prediction evaluation value calculating means includes a code amount calculating means, and the code amount calculating means The prediction evaluation value is calculated based on the generated code amount, the prediction error information, and the time difference information.

  According to such a configuration, the intra prediction encoding apparatus calculates the code amount of the quantized data by the code amount calculation unit. And a prediction evaluation value calculation means calculates a prediction evaluation value based on the code amount, prediction error information, and time difference information. As a result, the prediction evaluation value calculation means calculates a prediction evaluation value in consideration of which ratio of the information amount (code amount) and the error amount (prediction error information and time difference information) is to be encoded. be able to. The predicted evaluation value can be calculated by defining in advance a cost function having the information amount and the error amount as arguments. As a result, even in the most suitable prediction mode, the amount of code increases, so another prediction mode is adopted, and a prediction mode suitable for the bit rate is selected.

  The intra prediction encoding apparatus according to claim 4 is the intra prediction encoding apparatus according to claim 2 or 3, wherein the prediction mode determination means includes a decoded image comparison means, a quantization parameter, Changing means, and reducing the value of the quantization parameter until the local decoded image of the block to be encoded and the past local decoded image are similar within a predetermined range. .

  According to such a configuration, the intra prediction encoding apparatus uses the decoded image comparison unit to generate a local decoded image generated by the local decoding unit and a past local decoded image that is an image at the same position as the encoding target block. Compare Thereby, the decoded image comparison unit detects an error in the block between the screens in the time direction. If the detected error is larger than a predetermined threshold, the quantization parameter indicating the degree of quantization in the time difference information generating means and the local decoding means is changed by the quantization parameter changing means. As a result, the local decoding means generates quantized data in which the code amount increases, but the block error between screens in the time direction is reduced.

  Furthermore, the intra prediction encoding apparatus according to claim 5 is the intra prediction encoding apparatus according to any one of claims 2 to 4, wherein the sub-block dividing means and the second prediction error are provided. An information generation means, a second time difference information generation means, and a second prediction evaluation value calculation means, wherein the prediction mode determination means, based on the prediction evaluation value and the sub-block prediction evaluation value, It is characterized by determining which block of the block or the sub-block is to be encoded and which prediction mode is to be used.

According to such a configuration, the intra prediction encoding apparatus divides the encoding target block into a plurality of sub blocks (for example, 4 × 4 pixels) by the sub block dividing unit.
Then, the intra prediction encoding apparatus uses the second prediction error information generation unit to set the pixel values of the sub blocks divided by the sub block dividing unit based on sub block prediction modes that are a plurality of predetermined prediction modes. Prediction error information for the sub-block prediction mode is generated by taking a difference from the sub-block of the original image.

  In addition, the intra prediction encoding apparatus quantizes and locally decodes the prediction error information for the sub-block prediction mode generated by the second prediction error information generation unit by the second time difference information generation unit, By taking the difference between the block and the past local decoded image at the same position, time difference information that is a difference in the time direction is generated. Then, the intra prediction encoding apparatus uses the second prediction evaluation value calculation unit to calculate the prediction error information generated by the second prediction error information generation unit and the time generated by the second time difference information generation unit. A sub-block prediction evaluation value indicating the degree of similarity in the sub-block prediction mode is calculated from the difference information. The prediction evaluation value becomes an index indicating that the prediction in the screen and in the time direction is more accurate as the errors between the prediction error information and the time difference information are smaller.

Thereby, the intra prediction encoding apparatus performs prediction evaluation values when prediction is performed in a plurality of prediction modes in block units (for example, 16 × 16 pixel units) and sub block units (for example, 4 × 4 pixel units). A prediction evaluation value when prediction is performed in a plurality of prediction modes is calculated, and in which prediction mode of which division mode (block unit or sub-block unit) the prediction is performed, the error is minimized. Can be determined.
Therefore, the intra prediction encoding apparatus determines the division mode and the prediction mode based on the prediction evaluation value by the prediction mode determination unit.

The intra prediction encoding apparatus according to claim 6 is the intra prediction encoding apparatus according to claim 5, wherein the second prediction evaluation value calculation means includes a second code amount calculation means. The sub-block prediction evaluation value is calculated based on the code amount generated by the second code amount calculation means and the prediction error information and temporal difference information for the sub-block prediction mode.

  According to such a configuration, the intra prediction encoding apparatus calculates the code amount of the quantized data by the second code amount calculation unit. Then, the second prediction evaluation value calculation unit is configured to generate a sub-block prediction evaluation value based on the code amount generated by the second code amount calculation unit, the prediction error information for the sub-block prediction mode, and the time difference information. Is calculated. As a result, the second prediction evaluation value calculation means calculates the prediction evaluation value in consideration of which ratio of the information amount (code amount) and the error amount (prediction error information and time difference information) is to be encoded. Can be calculated. The predicted evaluation value can be calculated by defining in advance a cost function having the information amount and the error amount as arguments. As a result, even in the most suitable prediction mode, the amount of code increases, so another prediction mode is adopted, and a prediction mode suitable for the bit rate is selected.

  Furthermore, the intra prediction encoding method according to claim 7 predicts an intra picture prediction that encodes the picture by predicting a pixel value in the picture constituting the moving picture in units of a block having a specific size. The encoding method includes a prediction error information generation step, a time difference information generation step, a prediction evaluation value calculation step, and a prediction mode determination step.

  According to this procedure, in the intra prediction encoding method, the prediction error information generation step predicts the pixel value of the encoding target block based on a plurality of predetermined prediction modes, and obtains a difference from the original image. Thus, prediction error information that becomes a prediction error is generated.

  Subsequently, in the intra prediction encoding method, the prediction error information generated in the prediction error information generation step is quantized and locally decoded in the time difference information generation step, and the past prediction information existing in the same position as the encoding target block is detected. By taking the difference from the local decoded image, time difference information that is a difference in the time direction is generated.

  Then, the intra prediction encoding method calculates a prediction evaluation value indicating the degree of similarity in the prediction mode from the prediction error information and the time difference information in the prediction evaluation value calculation step. The smaller the error between the prediction error information and the time difference information, the more accurate the prediction in the screen and in the time direction. Therefore, in the intra prediction encoding method, the prediction mode determination step determines the prediction mode of the encoding target block based on the prediction evaluation value calculated in the prediction evaluation value calculation step. As a result, the intra prediction encoding method can encode a moving image in a prediction mode in which prediction errors in the screen and in the time direction are reduced.

  The intra prediction encoding program according to claim 8 predicts a pixel value of a block obtained by dividing a screen forming a moving image into a specific size in a plurality of predetermined prediction modes, and the prediction error thereof. In order to encode the screen by determining the prediction mode based on the above, a computer is divided into a block division unit, a prediction error information generation unit, a time difference information generation unit, a prediction evaluation value calculation unit, and a local decoding unit. And functioning as a prediction mode determining means and an encoding means.

  According to such a configuration, the intra-screen predictive coding program divides the screen constituting the moving image into block units divided into specific sizes by the block dividing unit. The intra-screen prediction encoding program predicts the pixel value of the encoding target block based on a plurality of predetermined prediction modes by the prediction error information generation unit, and obtains a difference from the original image. Prediction error information that is an error is generated.

  In addition, the intra-screen prediction encoding program quantizes and locally decodes the prediction error information for the prediction mode generated by the prediction error information generation unit by the time difference information generation unit, and exists in the same position as the encoding target block. By taking a difference from the past local decoded image, time difference information that is a difference in the time direction is generated. Then, the intra prediction encoding program calculates the prediction evaluation value indicating the degree of similarity in the prediction mode from the prediction error information and the time difference information by the prediction evaluation value calculation means.

Then, the intra prediction encoding program generates quantized data and a local decoded image by quantizing and local decoding the prediction error information by the local decoding unit.
Further, the intra prediction encoding program determines the prediction mode by the prediction mode determination means based on the prediction evaluation value calculated by the prediction evaluation value calculation means.
Then, the intra-screen prediction encoding program encodes the quantized data generated based on the prediction mode determined by the prediction mode determination unit by the encoding unit, and generates encoded data.

  According to the invention of claim 1, claim 2, claim 7 or claim 8, by predicting a moving image in units of blocks in a prediction mode in which a prediction error in the screen and in the time direction becomes small. The moving image can be encoded. As a result, when the encoded moving image is decoded, the occurrence of flicker in the decoded image can be suppressed, and deterioration in image quality can be suppressed. In the encoding according to the present invention, encoded data can be generated within a range of a general encoding standard (for example, H.264 / AVC, etc.). Can be decrypted.

  According to the invention described in claim 3 or claim 6, it is determined in advance which of the error amount (prediction error information and time difference information) and the information amount (code amount) is to be preferentially encoded. It can be determined by a cost function. This makes it possible to determine the most suitable prediction mode for the code amount by using a cost function determined in advance by the characteristics of the moving image (small movement, many of the same color, etc.) and the bit rate, and within the code amount range. Thus, the occurrence of flicker in the decoded image can be suppressed.

  According to the fourth aspect of the present invention, when the block difference between the screens in the time direction is larger than a predetermined threshold, the quantization parameter can be reduced to reduce the quantization. By finely quantizing in this way, when the prediction result is different for a block with little motion, the block can be encoded more faithfully to the original image, so that the error of the block between screens in the time direction can be reduced. be able to. As a result, occurrence of flicker in the decoded image can be suppressed, and deterioration in image quality can be suppressed.

  According to the fifth aspect of the present invention, since the block is performed in sub-block units, a plurality of prediction modes performed in block units and a plurality of prediction modes performed in sub-block units are used. In addition, prediction can be performed in a prediction mode that minimizes the prediction error in the time direction, and the moving image can be encoded. As a result, when the encoded moving image is decoded, the occurrence of flicker in the decoded image can be suppressed, and deterioration in image quality can be suppressed.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[Configuration of intra prediction encoding apparatus]
First, the configuration of the intra prediction encoding apparatus according to the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing the configuration of the intra prediction encoding apparatus.
As shown in FIG. 1, the intra prediction encoding apparatus 1 predicts pixel values in a screen in units of a specific size for moving image data in which the screen is continuous in time series, Encoding is performed for each screen to generate encoded data. That is, the intra prediction encoding apparatus 1 encodes an intra image (intra encoded image) that is not subjected to motion compensation prediction.

  This intra prediction encoding apparatus 1 includes an image (screen) as a 16 × 16 pixel block called a macro block and a 4 × 4 pixel block (hereinafter referred to as a sub block) obtained by dividing the macro block into 16 blocks. Intra-screen prediction is performed independently in each of a plurality of prediction modes, and encoded data is generated based on a prediction result most similar to a block at the same position on a past (for example, one frame before) screen. Here, the intra prediction encoding apparatus 1 includes a macroblock division unit 2, an original image storage unit 3, a similarity determination unit 4, a macroblock prediction evaluation unit 5, a subblock division unit 6, and a subblock prediction. An evaluation unit 7, a division mode determination unit 8, a quantization / local decoding unit 9, a division / prediction mode determination unit 10, a decoded image storage unit 11, and an encoding unit 12 are provided.

  The macro block dividing means (block dividing means) 2 divides input moving image data into blocks of a specific size. The blocks divided here are sequentially stored in the original image storage unit 3 and are output to the similarity determination unit 4. Here, the macroblock dividing means 2 converts the moving image data to H.264. The data is divided into macro block units of 16 × 16 pixels in accordance with the H.264 / AVC coding standard. The size of the block to be processed here is a size according to other encoding standards such as 16 × 8, 8 × 16, 8 × 8, and 8 × 4, in addition to the size of 16 × 16 (pixels). It does not matter.

  The original image storage unit 3 stores the macroblocks divided by the macroblock division unit 2 and is a general storage device such as a hard disk. The original image storage means 3 stores past images (intra images) for at least one image (one screen).

  The similarity determination unit 4 includes a macro block divided by the macro block dividing unit 2 and a macro block stored in the original image storage unit 3 and having the same position in the screen as the macro block in the past intra image. Are similar to each other. The similarity determination means 4 detects a macroblock with little motion in a moving image by comparing a macroblock to be encoded with a past macroblock. As a result, a macroblock that has less motion and is predicted as a pixel value different from the past macroblock can be detected as a macroblock candidate that causes flicker.

  The similarity determination means 4 notifies the division / prediction mode determination means 10 described later as information indicating whether or not the macroblock is a candidate for a macroblock that causes flicker. Further, the macroblock input to the similarity determination unit 4 is output to the macroblock prediction evaluation unit 5 and the subblock division unit 6 as it is.

  The determination method in the similarity determination means 4 is a macroblock in which the difference between all pixel values in the macroblock is calculated for the macroblock and the past macroblock, and the sum is equal to or less than a predetermined threshold. Is determined by determining that the macro block has less motion in the moving image. Alternatively, an average value of all the pixel values in the macro block is calculated for the macro block and the past macro block, and a macro block whose difference is equal to or less than a predetermined threshold is a macro with less motion in the moving image. It may be determined as a block.

The macroblock (16 × 16 pixel block) prediction evaluation unit 5 predicts the pixel value of the macroblock to be encoded based on a plurality of prediction modes from the macroblock decoded image locally decoded by the past macroblock. At the same time, an evaluation value indicating the degree of similarity of the prediction is calculated. The macroblock prediction evaluation means 5 will be described in detail with reference to FIG. FIG. 2 is a block diagram showing the configuration of the macroblock prediction evaluation means of the intra prediction encoding apparatus. Note that FIG. 1 is appropriately referred to for the configuration of the entire apparatus.
As shown in FIG. 2, the macroblock prediction evaluation unit 5 includes a prediction error information generation unit 51, a time difference information generation unit 52, and a prediction evaluation value calculation unit 53.

  The prediction error information generation unit 51 predicts the pixel value of the macroblock to be encoded from the macroblock adjacent to the macroblock to be encoded, and the predicted pixel value and the pixel value of the macroblock to be encoded Is generated as prediction error information. Here, the prediction error information generation unit 51 includes a macroblock prediction image generation unit 51a and a macroblock difference calculation unit 51b.

  The macroblock predicted image generation means 51a predicts pixel values of a macroblock to be encoded from macroblocks close to the macroblock to be encoded based on a plurality of predetermined prediction modes, and generates a predicted image. To do. The prediction image generated here is output to the macroblock difference calculation unit 51b and the time difference information generation unit 52.

  Note that the macroblock predicted image generation unit 51a performs coding based on an image (proximity decoded image) that is close to the macroblock to be encoded among the macroblock decoded images stored in the decoded image storage unit 11. The pixel value of the macroblock to be converted is predicted. Here, the prediction of the macroblock in the macroblock predicted image generation means 51a is performed as H.264. It is assumed that the operation is performed by four types of prediction modes standardized by H.264 / AVC.

  Here, referring to FIG. An outline of a macroblock prediction mode (intra 16 × 16 prediction mode) standardized by H.264 / AVC will be described. FIG. 3 is an explanatory diagram for explaining each prediction method in the intra 16 × 16 prediction mode, where (a) is vertical prediction, (b) is horizontal prediction, and (c) is DC (direct current) prediction. , (D) conceptually show each prediction method of plane prediction.

  As shown in FIG. 3, the intra 16 × 16 prediction modes include four types of prediction modes (a) to (d). “H” and “V” indicate pixels adjacent to the encoding target macroblock among the upper and left macroblocks adjacent to the encoding target macroblock.

As shown in FIG. 3A, in the vertical prediction, the value of the pixel “H” adjacent to the upper part is used as it is in the vertical direction, thereby obtaining the pixel value of the macroblock to be encoded. Note that when the pixel “H” does not exist, the prediction mode is not used.
As shown in FIG. 3B, in the horizontal direction prediction, the value of the pixel “V” adjacent to the left part is used as it is in the horizontal direction to obtain the pixel value of the macroblock to be encoded. Note that when the pixel “V” does not exist, the present prediction mode is not used.

As shown in FIG. 3C, in the DC (direct current) prediction, the values of the pixels “H” and “V” adjacent to the upper and left portions are added and averaged to obtain a macro to be encoded. The pixel value of the block. In addition, when either the pixel “H” or the pixel “V” does not exist, the value of either the existing pixel “H” or the pixel “V” is averaged to obtain a macroblock to be encoded. Pixel value. If neither the pixel “H” nor the pixel “V” exists, “128” is set as the pixel value of the macroblock to be encoded.
As shown in FIG. 3D, in the plane prediction, the pixel value of the macroblock to be encoded is predicted in a planar manner from the pixel “H” and the pixel “V” adjacent to the upper part and the left part. Is. Note that when either the pixel “H” or the pixel “V” does not exist, the present prediction mode is not used.

In general, a macro block is composed of a 16 × 16 pixel block indicating a luminance signal and two 8 × 8 pixel blocks indicating a color difference signal. The 8 × 8 pixel block indicating the color difference signal is predicted in the same manner as in the intra 16 × 16 prediction mode.
Returning to FIG. 2 (see FIG. 1 as appropriate), the description of the macroblock prediction evaluation unit 5 will be continued.

The macroblock difference calculation unit 51b is an original image (original image block) that is a block to be encoded among the prediction image generated by the macroblock prediction image generation unit 51a and the macroblock output from the similarity determination unit 4. Is used to calculate prediction error information for the prediction mode. The prediction error information for the prediction mode calculated here is output to the time difference information generation means 52, the prediction evaluation value calculation means 53, and the division mode determination means 8.

  The time difference information generation unit 52 decodes the prediction error information for the prediction mode generated by the prediction error information generation unit 51, and takes a difference from the past decoded image at the same position, so that the macro block to be encoded The difference in the time direction is generated for the prediction mode as time difference information. Here, the time difference information generation unit 52 includes a conversion unit 52a, a quantization unit 52b, an inverse quantization unit 52c, an inverse conversion unit 52d, a reconstruction unit 52e, and a difference calculation unit 52f. .

  The conversion means 52a converts the prediction error information generated by the prediction error information generation means 51 into a conversion coefficient that is frequency domain information. The transform coefficient transformed here is output to the quantization means 52b. Examples of the frequency conversion performed by the conversion means 52a include MPEG-1, MPEG-2, MPEG4, H.264, and the like. The discrete cosine transform (DCT) used in H.263, H.263, etc. The conversion method used in H.264 / AVC can be used. H. The frequency conversion used in H.264 / AVC is obtained by converting a floating point calculation of a discrete cosine transform into an integer (or a decimal number).

  The quantization unit 52b quantizes the transform coefficient generated by the transform unit 52a to generate a quantized transform coefficient. The quantization means 52b performs quantization based on a quantization parameter Qp indicating the degree of quantization input from the outside so as to greatly reduce high-frequency components with low visual sensitivity. The larger the value of this quantization parameter Qp, the more coarsely the transform coefficient is quantized. The quantized transform coefficient generated here is output to the inverse quantization means 52 c and the predicted evaluation value calculation means 53.

  The quantization means 52b quantizes the same transform coefficient stepwise with a plurality of quantization parameters based on the quantization control information Qc notified from the division / prediction mode determination means 10 described later. That is, each time the quantization means 52b is notified as the quantization control information Qc that the value of the quantization parameter is decreased by one step (or more), the quantization parameter obtained by decreasing the same transform coefficient. And sequentially output to the inverse quantization means 52 c and the predicted evaluation value calculation means 53.

  The inverse quantization unit 52c performs inverse quantization of the quantization performed by the quantization unit 52b on the quantization transform coefficient generated by the quantization unit 52b, thereby dequantizing the quantization transform coefficient. This is converted to a coefficient. The inverse quantization transform coefficient transformed here is output to the inverse transform means 52d.

  The inverse transformation unit 52d performs inverse transformation of the transformation performed by the transformation unit 52a on the inverse quantization transform coefficient transformed by the inverse quantization unit 52c. The information generated by the inverse conversion by the inverse conversion unit 52d is information (prediction error decoding information) in which the prediction error information generated by the prediction error information generation unit 51 is restored. However, the prediction error information and the prediction error decoding information differ depending on the degree of quantization. The prediction error decoding information generated here is output to the reconstruction unit 52e.

  The reconstructing unit 52e generates a local decoded image by adding the prediction error decoding information generated by the inverse converting unit 52d and the prediction image generated by the macroblock predicted image generating unit 51a. The local decoded image generated by the reconstructing unit 52e is a reproduction of the image when the macroblock is decoded on the decoding device side. The locally decoded image generated here is output to the difference calculation means 52f.

  The difference calculation unit 52f and the local decoded image generated by the reconstructing unit 52e and the local decoded image at the same position, which is stored in the decoded image storage unit 11 and locally decoded in the past (for example, one frame before). The time difference information indicating the difference in the time direction for each macroblock is generated by taking the difference from the macroblock decoded image). The time difference information generated here is sequentially output to the prediction evaluation value calculation unit 53 for each prediction mode.

  The difference calculation means 52f generates time difference information by taking the difference between the average values for each macroblock. Or it is good also considering the sum of the difference of all the pixels in a macroblock as time difference information. This time difference information means that the smaller the difference is, the less the change in the image in the time direction is, and the flicker in the decoded image is not noticeable. That is, this time difference information can be used as an index for suppressing flicker.

The prediction evaluation value calculation unit 53 is based on the prediction error information for each prediction mode (four types) generated by the prediction error information generation unit 51 and the time difference information generated by the time difference information generation unit 52. A prediction evaluation value indicating the degree of similarity of prediction is calculated. Note that the predicted evaluation value calculation means 53 is an H.264 standard. According to a rate distortion (RD) optimization mode determination method used in H.264 / AVC, a predicted evaluation value when moving image data is divided in units of macroblocks is calculated by a predetermined cost function. . Also, here, the prediction evaluation value calculation means 53 is a prediction evaluation value (division mode evaluation value) obtained by dividing into macroblocks, and a prediction evaluation value obtained by prediction using four types of prediction modes (see FIG. 3) ( (Prediction mode evaluation value).
Here, the prediction evaluation value calculation unit 53 includes a code amount calculation unit 53a, a division mode evaluation value calculation unit 53b, and a prediction mode evaluation value calculation unit 53c.

  The code amount calculation unit 53a encodes the prediction error information generated by the prediction error information generation unit 51 by encoding the quantized transform coefficient quantized by the quantization unit 52b of the time difference information generation unit 52. A code amount for calculating data is calculated. The code amount calculated here is output to the division mode evaluation value calculation unit 53b.

  The division mode evaluation value calculation unit 53b calculates a prediction evaluation value (division mode evaluation value) obtained by dividing the moving image data in units of macroblocks. The division mode evaluation value calculated here is output to the division mode determination means 8, and is used to determine whether it is appropriate to divide the moving image data in macroblock units or subblock units. Is done.

In the rate distortion optimization mode determination method, the information function (R: Rate) and the error amount (D: Distortion) are expressed as the cost function C for calculating the division mode evaluation value by the following equation (1), which is a linear sum. We are going to calculate.
C = D + λR (λ is a coefficient) (1)

Therefore, the division mode evaluation value calculation unit 53b uses the linear sum of the prediction error information and the time difference information as the error amount D, and uses the code amount calculated by the code amount calculation unit 53a as the information amount R. The split mode evaluation value is calculated.
H. In H.264 / AVC, the coefficient λ of the cost function C is defined as a function of the quantization parameter (Qp) as shown in the following equation (2). The quantization parameter Qp is input from the outside.
λ = 0.85 × 2 ^{Qp / 3} (2)

In this division mode evaluation value calculation means 53b, the division mode evaluation value is calculated by the rate distortion optimization mode determination method. However, the rate distortion optimization is not performed and the following equation (3) is used. The split mode evaluation value may be calculated using only the error amount D (prediction error information and time difference information). In this case, the code amount calculation unit 53 a can be omitted from the predicted evaluation value calculation unit 53. Here, λ is the same as the equation (2).
C = D + 6 × λ (λ is a coefficient) (3)

The prediction mode evaluation value calculation means 53c calculates a prediction evaluation value (prediction mode evaluation value) when a macroblock is predicted for each of the four types of prediction modes. The prediction mode evaluation value calculation unit 53c calculates the linear sum of the prediction error information and the time difference information for each of the four types of prediction modes, and sets the value having the smallest value as the prediction mode evaluation value. The prediction mode evaluation value calculated here is output to the division mode determination unit 8, and is used by the division / prediction mode determination unit 10 described later to determine which prediction mode is appropriate for prediction.
Returning to FIG. 1, the description of the overall configuration of the intra prediction encoding apparatus 1 will be continued.

  The sub-block dividing means 6 divides the macro block to be encoded into further subdivided blocks (sub blocks). Here, the sub-block dividing means 6 divides the 16 × 16 pixel macroblock output from the similarity determining means 4 into 16 blocks of 4 × 4 pixels. The 4 × 4 pixel blocks (sub-blocks) divided here are output to the sub-block prediction evaluation means 7 in units of sub-blocks.

The sub-block (4 × 4 pixel block) prediction evaluation unit 7 selects a target sub-block in a plurality of prediction modes from a macro-block decoded image (a set of sub-block decoded images) locally decoded with a past macro block. In addition to prediction, an evaluation value indicating the degree of similarity of the prediction is calculated. The sub-block prediction evaluation means 7 will be described in detail with reference to FIG. FIG. 4 is a block diagram showing the configuration of the sub-block prediction evaluation means of the intra prediction encoding apparatus. Note that FIG. 1 is appropriately referred to for the configuration of the entire apparatus.
As shown in FIG. 4, the sub-block prediction evaluation unit 7 includes a prediction error information generation unit 71, a time difference information generation unit 72, and a prediction evaluation value calculation unit 73.

  The prediction error information generation unit (second prediction error information generation unit) 71 predicts the pixel value of the encoding target sub-block from the sub-blocks close to the encoding target sub-block, and the predicted pixel value The difference from the pixel value of the sub-block to be encoded is generated as prediction error information. Here, the prediction error information generation unit 71 includes a sub-block prediction image generation unit 71a and a sub-block difference calculation unit 71b.

  The sub-block prediction image generation unit 71a predicts the pixel value of the sub-block to be encoded from sub-blocks close to the sub-block to be encoded based on a plurality of predetermined prediction modes (sub-block prediction modes). Then, it is generated as a predicted image. The predicted image generated here is output to the sub-block difference calculating unit 71b and the time difference information generating unit 72.

  The sub-block predicted image generation unit 71a is an image (proximity to the encoding target sub-block) among the macroblock decoded images (a set of sub-block decoded images) stored in the decoded image storage unit 11. Based on the decoded image), the pixel value of the sub-block to be encoded is predicted. Here, the prediction of the sub-block in the sub-block predicted image generating means 71a is performed as H.264 It is assumed that the operation is performed by nine kinds of prediction modes standardized by H.264 / AVC.

Here, referring to FIG. An outline of a prediction mode (intra 4 × 4 prediction mode) of a 4 × 4 pixel block (sub block) standardized by H.264 / AVC will be described. FIG. 5 is an explanatory diagram for explaining a prediction method in the intra 4 × 4 prediction mode, where (a) is vertical direction prediction, (b) is horizontal direction prediction, (c) is DC (direct current) prediction, (D) Diagonal lower left direction prediction, (e) Diagonal lower right direction prediction, (f) Vertical right direction prediction, (g) Horizontal down direction prediction, (h) Vertical left direction prediction, (i) Shows conceptually each prediction method of horizontal upward prediction.
As illustrated in FIG. 5, the intra 4 × 4 prediction mode includes nine types of prediction modes (a) to (i). Note that “A” to “M” indicate the pixels of the upper, left, upper left, and upper right subblocks adjacent to the subblock.

As shown in FIG. 5A, in the vertical prediction, the values of the pixels “A”, “B”, “C”, and “D” adjacent to the upper part are used as they are in the vertical direction, so that the encoding target The pixel value of the sub-block. Note that, when the pixels “A”, “B”, “C”, and “D” do not exist, the present prediction mode is not used.
As shown in FIG. 5B, in the horizontal prediction, encoding is performed by using the values of the pixels “I”, “J”, “K”, and “L” adjacent to the left part as they are in the horizontal direction. The pixel value of the target sub block is used. Note that, when the pixels “I”, “J”, “K”, and “L” do not exist, the present prediction mode is not used.
As shown in FIG. 5C, in the DC (direct current) prediction, the pixels “A”, “B”, “C”, “D”, “I”, “J”, “J”, The values of “K” and “L” are added and averaged to obtain the pixel value of the sub-block to be encoded. In addition, when any of the upper and left pixels does not exist, the pixel value of the sub-block to be encoded is obtained by averaging the values of either the existing upper or left pixels. If neither the upper pixel nor the left pixel exists, “128” is set as the pixel value of the sub-block to be encoded.

As shown in FIG. 5D, in diagonal lower left direction prediction, the corresponding pixel value is determined based on the pixels “A” to “H” existing in the diagonal upper right direction of the pixel to be predicted. For example, the value of the upper right pixel (circle mark in the figure) in the sub-block is determined by the following equation (4).
{(D pixel value) + 2 × (E pixel value) + (F pixel value) +2} / 4 (4)
Here, “+2” is a value for rounding off when dividing by “4”.
As shown in FIG. 5 (e), in the diagonally lower right direction prediction, the pixel “A” to “D” and “L” to “M” existing in the diagonally upper left direction of the pixel to be predicted correspond. The pixel value to be determined is determined. For example, the value of the upper left pixel (circle mark in the figure) in the sub-block to be encoded is determined by the following equation (5).
{(Pixel value of A) + 2 × (pixel value of M) + (pixel value of I) +2} / 4 (5)

As shown in FIG. 5F, in the vertical right direction prediction, the corresponding pixel value based on the pixels “A” to “D” and “L” to “M” existing in the vertical upper left direction of the predicted pixel. To decide. For example, the value of the upper left pixel (circle mark in the figure) in the sub-block to be encoded is determined by the following equation (6).
{(Pixel value of M) + (pixel value of A) +1} / 2 (6)
Here, “+1” is a value for rounding off when dividing by “2”.
Also, for example, the value of the pixel (（mark in the figure) immediately below the upper left pixel (marked in the figure) in the sub-block to be encoded is determined by the following equation (7).
{(Pixel value of I) + 2 × (pixel value of M) + (pixel value of A) +2} / 4 (7)

As shown in FIG. 5G, in the horizontal downward prediction, the corresponding pixel value is based on the pixels “A” to “D” and “I” to “M” existing in the horizontal upper left direction of the pixel to be predicted. To decide. For example, the value of the upper left pixel (circle mark in the figure) in the sub-block to be encoded is determined by the following equation (8).
{(Pixel value of M) + (pixel value of I) +1} / 2 (8)
Further, for example, the value of the right pixel (の in the figure) of the upper left pixel (◯ in the figure) in the sub-block to be encoded is determined by the following equation (9).
{(Pixel value of I) + 2 × (pixel value of M) + (pixel value of A) +2} / 4 (9)

As shown in FIG. 5H, in the vertical left direction prediction, the corresponding pixel value is determined based on the pixels “A” to “F” existing in the vertical upper right direction of the pixel to be predicted. For example, the value of the upper left pixel (circle mark in the figure) in the sub-block to be encoded is determined by the following equation (10).
{(Pixel value of A) + (pixel value of B) +1} / 2 (10)
Also, for example, the value of the pixel (（mark in the figure) immediately below the upper left pixel (◯ mark in the figure) in the sub-block to be encoded is determined by the following equation (11).
{(Pixel value of A) + 2 × (pixel value of B) + (pixel value of C) +2} / 4 (11)

As shown in FIG. 5I, in the horizontal upward prediction, the corresponding pixel value is determined based on the pixels “I” to “L” existing in the horizontal lower right direction of the pixel to be predicted. For example, the value of the upper left pixel (circle mark in the figure) in the sub-block to be encoded is determined by the following equation (12).
{(Pixel value of I) + (pixel value of J) +1} / 2 (12)
Further, for example, the value of the right pixel (（in the figure) of the upper left pixel (◯ in the figure) in the sub-block to be encoded is determined by the following equation (13).
{(I pixel value) + 2 × (J pixel value) + (K pixel value) +2} / 4 (13)
Note that the value of the pixel marked with x in the figure is the value of the pixel L.
In FIG. 5, the prediction modes (d) to (i) are used only when the pixels “A” to “D” and “I” to “M” are all present.
Returning to FIG. 4 (see FIG. 1 as appropriate), the description of the sub-block prediction evaluation means 7 is continued.

  The sub-block difference calculating unit 71b obtains the difference between the predicted image generated by the sub-block predicted image generating unit 71a and the sub-block (original image block) divided by the sub-block dividing unit 6 to obtain prediction error information. Is calculated. The prediction error information calculated here is output to the time difference information generation means 72, the prediction evaluation value calculation means 73, and the division mode determination means 8.

  The time difference information generation means (second time difference information generation means) 72 decodes the prediction error information for the prediction mode generated by the prediction error information generation means 71, and calculates the difference from the past decoded image at the same position. Thus, the difference in the time direction in the sub-block to be encoded is generated for the prediction mode as the time difference information. Here, the time difference information generation means 72 includes a conversion means 72a, a quantization means 72b, an inverse quantization means 72c, an inverse conversion means 72d, a reconstruction means 72e, and a difference calculation means 72f. .

  The configuration of the time difference information generating unit 72 is the same as that of the time difference information generating unit 52 described in FIG. 2, and the handled block is from a 16 × 16 pixel block (macro block) to a 4 × 4 pixel block. (Sub-block), and only the block size is different. Therefore, a detailed description of the time difference information generating unit 72 is omitted here.

Prediction evaluation value calculation means (second prediction evaluation value calculation means) 73 is generated by prediction error information generation means 71 and prediction error information for each prediction mode (9 types) and time difference information generation means 72. Based on the time difference information, a prediction evaluation value indicating a degree of similarity of prediction is calculated. Also in the prediction evaluation value calculation means 73, the prediction evaluation values (division mode evaluation value and prediction mode evaluation value) when the macroblock is divided in units of subblocks by the above-described rate distortion optimization mode determination method are as follows. The calculation is made by a predetermined cost function.
Here, the prediction evaluation value calculation unit 73 includes a code amount calculation unit 73a and a division / prediction mode evaluation value calculation unit 73b.

  The code amount calculation unit 73a encodes the prediction error information generated by the prediction error information generation unit 71 by encoding the quantized transform coefficient quantized by the quantization unit 72b of the time difference information generation unit 72. A code amount for calculating data is calculated. The code amount calculated here is output to the division / prediction mode evaluation value calculation unit 73b.

  The division / prediction mode evaluation value calculation unit 73b calculates a prediction evaluation value obtained by performing prediction in units of subblocks obtained by further subdividing the macroblock. Here, the division / prediction mode evaluation value calculation unit 73b calculates the division mode evaluation value and the prediction mode evaluation value using the same cost function determined by the rate distortion optimization mode determination method. . Here, the cost function of the above equation (1) is used as the cost function.

Note that the prediction evaluation value calculation unit 73 calculates the prediction evaluation value by the rate optimization mode determination method, but only the error amount (prediction error information and time difference information) without optimization of the rate distortion. A prediction mode evaluation value may be calculated.
For example, the following equation (14) is used as the cost function C for calculating the split mode evaluation value without considering the rate distortion. Here, the error amount D is a linear sum of the prediction error information and the time difference information, and λ is a function value of the same quantization parameter as in the equation (2).
C = D + 6 × λ (λ is a coefficient) (14)

Further, the following equation (15) is used as the cost function C for calculating the prediction mode evaluation value that does not consider the rate distortion. Here, the error amount D is a linear sum of prediction error information and time difference information, and MPM (Most Probable Mode) is “0” when the prediction mode is most frequently used in the macroblock, otherwise Is “4 × λ”.
C = D + MPM (15)
If the optimization of the rate distortion is not performed, the code amount calculation unit 73 a can be omitted from the prediction evaluation value calculation unit 73.
Returning to FIG. 1, the description of the overall configuration of the intra prediction encoding apparatus 1 will be continued.

Based on the division mode evaluation value output from the macroblock prediction evaluation unit 5 and the division mode evaluation value output from the subblock prediction evaluation unit 7, the division mode determination unit 8 determines the macroblock in the subsequent processing. It is determined whether to use a sub-block obtained by further subdividing a macro block.
Here, the division mode determination means 8 determines that a block having a small division mode evaluation value is used in the subsequent processing, and indicates a division mode indicating whether the block is a macro block or a sub block, and the division mode. The prediction mode evaluation value and the prediction error information are output to the quantization / local decoding means 9.

The quantization / local decoding unit (local decoding unit) 9 quantizes the prediction error information output from the division mode determination unit 8 in units of blocks indicated by the division mode determined by the division mode determination unit 8. By performing local decoding, quantized data and a locally decoded image are generated. The quantization / local decoding unit 9 generates a transform coefficient obtained by transforming the prediction error information into frequency domain information, and quantizes the transform coefficient based on the quantization parameter Qp input from the outside. Then, the quantization / local decoding means 9 decodes the prediction error information by inversely quantizing the quantized transform coefficient (quantized data), which is the quantization result, and further inversely transforming it, and the decoded image accumulating means 11 is added (reconstructed) to a locally decoded image (locally decoded image) by adding it to the past same-position macroblock decoded signal accumulated in 11. The quantized data and the locally decoded image generated here are output to the division / prediction mode determination unit 10.

The transform process, quantization process, inverse quantization process, inverse transform process, and reconstruction process described here are performed by the transform unit 52a, the quantizer 52b, the inverse quantizer 52c, and the inverse transform unit illustrated in FIG. It can process by setting it as the structure similar to 52d and the reconfiguration | reconstruction means 52e.
When the quantization / local decoding means 9 is notified of the quantization control information Qc when quantizing the transform coefficient, the quantization / local decoding means 9 sequentially decreases the value of the quantization parameter to quantize the transform coefficient. Do. As a result, the quantization / local decoding means 9 sequentially generates quantized data and a locally decoded image having a larger amount of information than when quantized with the initial quantization parameter.

  The division / prediction mode determination unit (prediction mode determination unit) 10 compares the local decoded image output from the quantization / local decoding unit 9 with a previously decoded image (macroblock decoded image). Thus, it is determined whether or not the division mode and the prediction mode used when generating the local decoded image are appropriate, and the final division mode and the prediction mode are determined. Here, the division / prediction mode determination unit 10 includes a decoded image comparison unit 10a and a quantization parameter change unit 10b.

The decoded image comparison unit 10a is the same as the local decoded image output from the quantization / local decoding unit 9 and the local decoded image stored in the decoded image storage unit 11 and decoded in the past. The macroblock decoded image is compared.
When the difference between the local decoded image and the macroblock decoded image is equal to or smaller than a predetermined threshold, the decoded image comparison unit 10a uses the division mode and the prediction mode used when the local decoded image is generated ( The division mode and the prediction mode are determined as final modes. Then, the decoded image comparison unit 10a outputs the division mode and the prediction mode determined here and the quantized data generated in this mode to the encoding unit 12. Also, the decoded image comparison unit 10a stores the local decoded image (macroblock decoded image) at this time in the decoded image storage unit 11 for reference to the subsequent macroblock.
On the other hand, when the difference between the local decoded image and the macroblock decoded image is larger than a predetermined threshold, the decoded image comparison unit 10a notifies the quantization parameter changing unit 10b that the quantization parameter is changed.

When there is a notification from the decoded image comparison unit 10a that the quantization parameter is to be changed, the quantization parameter changing unit 10b performs an operation by reducing the quantization parameter for each unit that performs quantization. Quantization control information Qc indicating an instruction to that effect is notified. For example, the quantization parameter changing unit 10b outputs an instruction to lower the quantization parameter value by one step to the macroblock prediction evaluation unit 5, the subblock prediction evaluation unit 7, and the quantization / local decoding unit 9, By operating with the quantization parameter lowered by one step, the information amount of the quantized data output from the quantization / local decoding means 9 increases, but an image similar to the original image is generated. become.
Even in this process, when the difference between the locally decoded image and the macroblock decoded image does not become the threshold value or less in the decoded image comparison unit 10a, the fact that the quantization parameter is changed further indicates that the quantization parameter change unit 10b. And the operation is repeatedly executed with the quantization parameter sequentially reduced.

As described above, since the division / prediction mode determination unit 10 can determine the division mode and the prediction mode in which the local decoded image and the past macroblock decoded image are similar, the moving image data is encoded at a low bit rate. Even when this is done, the difference between intra images can be reduced.
Note that the division / prediction mode determination unit 10 receives input from the similarity determination unit 4 as control information indicating that the macroblock to be processed is not a candidate for a macroblock that causes flicker. The divided mode and prediction mode and the quantized data are output to the encoding means 12 as they are.

  The decoded image accumulating unit 11 accumulates a local decoded image (macroblock decoded image) generated by the division mode and the prediction mode determined by the division / prediction mode determination unit 10, and is, for example, a storage device such as a hard disk. is there. The decoded macroblock image stored in the decoded image storage unit 11 is referred to by the macroblock prediction evaluation unit 5, the subblock prediction evaluation unit 7, the quantization / local decoding unit 9, and the division / prediction mode determination unit 10. .

  The encoding unit 12 encodes the moving image data by entropy encoding and multiplexing each value of the division mode and the prediction mode output from the division / prediction mode determination unit 10 and the quantized data. Coded data is generated.

By configuring the intraframe prediction encoding apparatus 1 in this way, a code using a partition mode and a prediction mode that is most similar to a past macroblock decoded image by each block predicted by a plurality of partition modes and prediction modes. Data can be generated. As a result, it is possible to reduce image quality degradation due to flicker when decoding encoded data.
In addition, the intra prediction encoding apparatus 1 can also implement each unit as a function program, and can also combine the function programs to operate as an intra prediction encoding program. This intra-screen prediction encoding program can be distributed via a communication line, or can be distributed by writing in a recording medium such as a CD-ROM.

[Operation of intra prediction encoding apparatus]
Next, the operation of the intra prediction encoding apparatus according to the present invention will be described with reference to FIGS. 6 to 8 (refer to FIG. 1 as appropriate). FIG. 6 is a flowchart showing the operation of the intra prediction encoding apparatus. FIG. 7 is a flowchart showing the operation of the macroblock prediction evaluation unit of the intra prediction encoding apparatus. FIG. 8 is a flowchart showing the operation of the sub-block prediction evaluation unit of the intra prediction encoding apparatus.

(Block division step)
First, the intraframe prediction encoding apparatus 1 divides a moving image (moving image data) into blocks (macroblocks) of a specific size by the macroblock dividing unit 2 (step S1). Here, the macroblock dividing unit 2 divides the moving image data into 16 × 16 pixel blocks and sequentially stores them in the original image storage unit 3.

  Then, the intra prediction encoding apparatus 1 compares the divided macro blocks with the past macro blocks at the same position stored in the original image storage unit 3 by the similarity determination unit 4 (step S2). Here, if the divided macro block is similar to the past macro block at the same position, it can be determined that the macro block is a candidate for a macro block that causes flicker. The comparison result in step S2 is used in step S8 described later.

(Macroblock prediction evaluation step)
Then, the intra prediction encoding apparatus 1 uses the macroblock prediction evaluation unit 5 to divide the moving image data in units of macroblocks, and calculates an evaluation value when pixel values are predicted in units of macroblocks (steps). S3: Macroblock prediction evaluation process).
Here, the operation of the macroblock prediction evaluation unit 5 will be described in detail with reference to FIG.

(Prediction error information generation step)
First, the macroblock prediction evaluation unit 5 uses the macroblock prediction image generation unit 51a of the prediction error information generation unit 51 to approach the macroblock to be encoded and store the past macros stored in the decoded image storage unit 11. A prediction image is generated by predicting the pixel value of the encoding target macroblock from a block decoded image in a plurality of prediction modes (here, four types of prediction modes: see FIG. 3). And the macroblock prediction evaluation means 5 produces | generates the prediction error information for prediction mode by taking the difference of the prediction image and an original image (step S31).

(Time difference information generation step)
Further, the macroblock prediction evaluation unit 5 converts, quantizes, and decodes the prediction error information generated in step S31 by the time difference information generation unit 52, and then the difference from the past decoded image at the same position. By taking this, the difference in the time direction in the macroblock to be encoded is generated as time difference information (step S32).

(Predictive evaluation value calculation step)
Then, the macroblock prediction evaluation unit 5 calculates the code amount by encoding the quantized transform coefficient generated at the time of quantization in step S32 by the code amount calculation unit 53a (step S33).
Further, the macroblock prediction evaluation unit 5 uses the division mode evaluation value calculation unit 53b to generate the prediction error information generated in step S31, the time difference information generated in step S32, and the code amount calculated in step S33. Then, the division mode evaluation value is calculated by a predetermined cost function (the above formula (1)) (step S34).
In addition, the macroblock prediction evaluation unit 5 calculates the linear sum of the prediction error information generated in step S31 and the time difference information generated in step S32 by the prediction mode evaluation value calculation unit 53c. The value is calculated every time (see FIG. 3), and the value having the smallest value is calculated as the prediction mode evaluation value (step S35).
Returning to FIG. 6 (see FIG. 1 as appropriate), the description of the overall operation will be continued.

(Sub-block division step)
The intra prediction encoding apparatus 1 uses the sub-block dividing unit 6 to divide the macro block divided by the macro block dividing unit 2 into further sub-blocks (here, 4 × 4 pixel blocks) (steps). S4).

(Sub-block prediction evaluation step)
Then, the intra prediction coding apparatus 1 divides the macroblock into subblocks by the subblock prediction evaluation unit 7 and calculates an evaluation value when the pixel value is predicted in the subblock (step S5). : Sub-block prediction evaluation process).
Here, with reference to FIG. 8 (refer to FIG. 4 as appropriate), the operation of the sub-block prediction evaluation means 7 will be described in detail.

(Prediction error information generation step)
First, the sub-block prediction evaluation unit 7 uses the sub-block prediction image generation unit 71a of the prediction error information generation unit 71 to close the past decoding stored in the decoded image storage unit 11 in the vicinity of the sub-block to be encoded. A prediction image is generated by predicting the pixel value of the encoding target sub-block from the image in a plurality of prediction modes (here, nine types of prediction modes: see FIG. 5). Then, the sub-block prediction evaluation unit 7 generates prediction error information for the prediction mode by taking the difference between the predicted image and the original image (step S51).

(Time difference information generation step)
Further, the sub-block prediction evaluation unit 7 converts, quantizes and decodes the prediction error information generated in step S51 by the time difference information generation unit 72, and then the difference from the past decoded image at the same position. By taking this, the difference in the time direction in the sub-block to be encoded is generated as time difference information (step S52).

(Predictive evaluation value calculation step)
Then, the sub-block prediction evaluation unit 7 calculates the code amount by encoding the quantized transform coefficient generated at the time of quantization in step S52 by the code amount calculation unit 73a (step S53).
Further, the sub-block prediction evaluation unit 7 uses the division / prediction mode evaluation value calculation unit 73b to generate the prediction error information generated in step S51, the time difference information generated in step S52, and the code calculated in step S53. From the quantity, a split mode evaluation value and a prediction mode evaluation value are calculated by a predetermined cost function (the above formula (1)) (step S54).
Returning to FIG. 6 (see FIG. 1 as appropriate), the description of the overall operation will be continued.

Subsequently, the intra prediction encoding apparatus 1 uses the division mode determination unit 8 to determine whether to divide the moving image into macroblocks or further subdivided subblocks (step S6).
Then, the intra prediction encoding apparatus 1 generates quantized data and a locally decoded image by the quantization / local decoding unit 9 using the prediction error information generated by the division mode determined in step S6. (Step S7).
Here, the intra prediction encoding apparatus 1 determines whether or not the macroblock is a candidate of a macroblock that causes flicker by the division / prediction mode determination unit 10 based on the comparison result in step S2. (Step S8) If it is the candidate (Yes in Step S8), the process proceeds to Step S9. On the other hand, if it is not a candidate (No in step S8), the process proceeds to step S12 without operating the following quantization parameter changing step.

(Quantization parameter change step)
Then, the intra-screen predictive encoding device 1 stores the local decoded image generated in step S7 by the decoded image comparison unit 10a of the division / prediction mode determination unit 10 and the decoded image storage unit 11 in the past. The local decoded image being decoded is compared with the macroblock decoded image at the same position (step S9).
Here, when the difference between the local decoded image and the macroblock decoded image is larger than a predetermined threshold (step S9:> threshold), the quantization parameter changing means 10b lowers the quantization parameter by one step (step S10). ), The process returns to step S3.

(Prediction mode determination step)
On the other hand, when the difference between the local decoded image and the macroblock decoded image is equal to or smaller than a predetermined threshold (step S9: ≦ threshold), the division mode and the prediction mode at the current stage are set in the block (macroblock or subblock). Finally, the optimum division mode and prediction mode are determined (step S11).

(Encoding step)
Then, the intra prediction encoding apparatus 1 uses the encoding means 12 to entropy encode and multiplex each value of the division mode and the prediction mode and the quantized data generated in step S7, The encoded image data is generated as encoded data (step S12).
And the intra prediction encoding apparatus 1 returns to step S2, and continues operation | movement, when the macroblock to continue exists (Yes in step S13). Further, if there is a screen (frame) that follows (Yes in step S14), the process returns to step S1 to continue the operation. If there is no subsequent screen (frame) (No in step S14), the operation ends.

By the above operation, the intra-screen predictive coding apparatus 1 compresses a moving image by the division mode and the prediction mode in which the decoded image is most similar when images having substantially the same pixel value in time are consecutive. Therefore, a decoded image having substantially the same pixel value is generated from an image having substantially the same pixel value in terms of time, and flicker in the decoded image can be reduced.
In addition, the intra prediction encoding apparatus 1 changes the degree of quantization when images having substantially the same pixel value in time are continuous, thereby producing different quantization results in time. This can be avoided, and flicker in a low bit rate decoded image can be reduced.

As mentioned above, although the embodiment of the present invention has been described as the best mode, the present invention is not limited to this.
For example, as shown in FIG. 9, the original image storage unit 3, the similarity determination unit 4, and the division / prediction mode determination unit 10 are omitted from the intra-screen prediction encoding apparatus 1 of FIG. 1, and the quantization parameter is lowered step by step. It is good also as a structure of the intra prediction encoding apparatus 1B which is not. In this case, when a moving image is encoded at a low bit rate, if the skip mode is frequently used in a portion where there is almost no motion in the inter image (below a certain amount), flicker occurs. In an encoding method that does not use the skip mode, flicker can be sufficiently reduced.

  As another configuration, as shown in FIG. 10, the video data is divided in one block division unit (here, macroblock), predicted by a plurality of prediction modes, and predicted most similar to the time direction. It is good also as a structure of 1 C of intra prediction encoding apparatuses which encode by mode. Further, in the configuration of the intra prediction encoding apparatus 1C, the macroblock prediction evaluation unit 5 may be configured as the subblock prediction evaluation unit 7 (FIG. 1).

It is a block diagram which shows the structure of the intra prediction encoding apparatus which concerns on this invention. It is a block diagram which shows the structure of the macroblock prediction evaluation means of the intra prediction encoding apparatus which concerns on this invention. It is explanatory drawing for demonstrating each prediction method of intra 16x16 prediction mode, (a) is vertical direction prediction, (b) is horizontal direction prediction, (c) is DC (direct current) prediction, (d). Is a conceptual illustration of each prediction method for plane prediction. It is a block diagram which shows the structure of the subblock prediction evaluation means of the intra prediction encoding apparatus which concerns on this invention. It is explanatory drawing for demonstrating the prediction method of intra 4x4 prediction mode, Comprising: (a) is vertical direction prediction, (b) is horizontal direction prediction, (c) is DC (direct current) prediction, (d) is Diagonal lower left direction prediction, (e) Diagonal lower right direction prediction, (f) Vertical right direction prediction, (g) Horizontal lower direction prediction, (h) Vertical left direction prediction, (i) Horizontal upper direction prediction Each prediction method of prediction is shown conceptually. It is a flowchart which shows operation | movement of the intra prediction encoding apparatus which concerns on this invention. It is a flowchart which shows operation | movement of the macroblock prediction evaluation means of the intra prediction encoding apparatus which concerns on this invention. It is a flowchart which shows operation | movement of the subblock prediction evaluation means of the intra prediction encoding apparatus which concerns on this invention. It is a block diagram which shows the other structure of the intra prediction encoding apparatus which concerns on this invention. It is a block diagram which shows the other structure of the intra prediction encoding apparatus which concerns on this invention.

Explanation of symbols

1, 1B, 1C Intra prediction encoding device 2 Macroblock dividing means (block dividing means)
DESCRIPTION OF SYMBOLS 3 Original image storage means 4 Similarity determination means 5 Macroblock prediction evaluation means 51 Prediction error information generation means 52 Time difference information generation means 53 Prediction evaluation value calculation means 53a Code amount calculation means 6 Subblock division means 7 Subblock prediction evaluation means 71 Prediction error information generation means (second prediction error information generation means)
72 Time difference information generating means (second time difference information generating means)
73 Predictive evaluation value calculating means (second predictive evaluation value calculating means)
73a Code amount calculation means (second code amount calculation means)
8 Division mode determination means 9 Quantization / local decoding means (local decoding means)
10 division / prediction mode determination means (prediction mode determination means)
10a Decoded image comparison means 10b Quantization parameter change means 11 Decoded image storage means 12 Encoding means

Claims (8)

In the intra prediction encoding device that encodes the screen by predicting the pixel value in the screen constituting the moving image in units of a block having a specific size,
Prediction that generates prediction error information as a prediction error by predicting pixel values of a block to be encoded that is a block to be encoded based on a plurality of predetermined prediction modes and taking a difference from the original image Error information generating means;
The prediction error information generated by the prediction error information generating means is quantized and locally decoded, and a difference with the past local decoded image existing at the same position as the encoding target block is obtained. Time difference information generating means for generating time difference information,
The predetermined cost function, the minimum value of the linear sum of the prediction error information and the prior SL time difference information of the prediction mode for each prediction evaluation value calculation for calculating a prediction evaluation value indicating the degree of similarity in the prediction mode Means,
Based on the prediction evaluation value calculated by the prediction evaluation value calculation means, prediction mode determination means for determining the prediction mode of the coding target block;
An intra-screen predictive coding apparatus comprising: The pixel value of a block obtained by dividing a screen constituting a moving image into a specific size is predicted in a plurality of predetermined prediction modes, and the prediction mode is determined based on the prediction error, thereby encoding the screen. An intra-screen predictive encoding device for
Block dividing means for dividing the screen into the block units;
Prediction which becomes a prediction error by predicting the pixel value of the encoding target block which is the encoding target block divided by the block dividing means based on the plurality of prediction modes and taking the difference from the original image Prediction error information generating means for generating error information;
The prediction error information for the prediction mode generated by the prediction error information generation unit is quantized and locally decoded, and a time difference is obtained from the past local decoded image existing at the same position as the encoding target block. Time difference information generating means for generating time difference information which is a difference in direction;
Prediction evaluation value calculation means for calculating the minimum value of the linear sum of the prediction error information and the time difference information for each prediction mode as a prediction evaluation value indicating the degree of similarity in the prediction mode by a predetermined cost function When,
Local decoding means for generating quantized data and the locally decoded image by quantizing and locally decoding the prediction error information generated by the prediction error information generating means;
Prediction mode determination means for determining the prediction mode based on the prediction evaluation value calculated by the prediction evaluation value calculation means;
Encoding means for encoding the quantized data generated based on the prediction mode determined by the prediction mode determining means, and generating encoded data;
An intra-screen predictive coding apparatus comprising: The prediction evaluation value calculation means includes code amount calculation means for calculating a code amount for encoding the prediction error information,
The minimum value of the linear sum of the code amount generated by the code amount calculation means and the prediction error information and the time difference information is calculated as the prediction evaluation value by a predetermined cost function. The intra prediction encoding apparatus according to claim 2. The prediction mode determination means includes
A decoded image comparison unit that compares the local decoded image of the encoding target block generated by the local decoding unit with a past local decoded image that existed at the same position as the encoding target block;
A quantization parameter changing means for changing a value of a quantization parameter indicating a degree of quantization in the time difference information generating means and the local decoding means;
The value of the quantization parameter is decreased until the local decoded image of the encoding target block and the past local decoded image are similar within a predetermined range. 4. The intra prediction encoding apparatus according to 3. Sub-block dividing means for dividing the encoding target block into a plurality of sub-blocks;
The sub-block prediction is performed by predicting the pixel value of the sub-block divided by the sub-block dividing unit based on a sub-block prediction mode which is a plurality of predetermined prediction modes, and taking a difference from the sub-block. Second prediction error information generating means for generating prediction error information for the mode;
The prediction error information for the sub-block prediction mode generated by the second prediction error information generation unit is quantized and locally decoded, and a difference from a past local decoded image existing at the same position as the sub-block is obtained. Then, a second time difference information generating means for generating time difference information that becomes a difference in the time direction,
The predetermined cost function, and the sub-block the second prediction error information generated prediction error information generated by the unit and the previous SL second time difference information generating means time difference information generated by the respective prediction modes A second prediction evaluation value calculating means for calculating a minimum value of the linear sum as a sub-block prediction evaluation value indicating a degree of similarity in the sub-block prediction mode;
The prediction mode determining means determines which block of the block or the sub-block is to be encoded and which prediction mode is to be used based on the prediction evaluation value and the sub-block prediction evaluation value. The intra prediction encoding apparatus according to any one of claims 2 to 4, wherein the intra prediction encoding apparatus is characterized. The second prediction evaluation value calculating means includes second code amount calculating means for calculating a code amount when encoding the prediction error information generated by the second prediction error information generating means,
The minimum value of the linear sum of the code amount generated by the second code amount calculation means and the prediction error information and the time difference information for the sub-block prediction mode is determined by the predetermined cost function using the sub-block prediction. 6. The intra prediction encoding apparatus according to claim 5, wherein the prediction encoding apparatus calculates the evaluation value. In the intra prediction encoding method for encoding the screen by predicting the pixel value in the screen constituting the moving image in units of a block having a specific size,
Prediction that generates prediction error information as a prediction error by predicting pixel values of a block to be encoded that is a block to be encoded based on a plurality of predetermined prediction modes and taking a difference from the original image An error information generation step;
The prediction error information generated in this prediction error information generation step is quantized and locally decoded, and by taking a difference from the past local decoded image existing at the same position as the encoding target block, A time difference information generating step for generating time difference information,
The predetermined cost function, the minimum value of the linear sum of the prediction error information and the prior SL time difference information of the prediction mode for each prediction evaluation value calculation for calculating a prediction evaluation value indicating the degree of similarity in the prediction mode Steps,
Based on the prediction evaluation value calculated in the prediction evaluation value calculation step, a prediction mode determination step for determining a prediction mode of the encoding target block;
An intra-screen predictive encoding method characterized by comprising: The pixel value of a block obtained by dividing a screen constituting a moving image into a specific size is predicted in a plurality of predetermined prediction modes, and the prediction mode is determined based on the prediction error, thereby encoding the screen. Computer to
Block dividing means for dividing the screen into block units;
Prediction which becomes a prediction error by predicting the pixel value of the encoding target block which is the encoding target block divided by the block dividing means based on the plurality of prediction modes and taking the difference from the original image Prediction error information generating means for generating error information;
The prediction error information for the prediction mode generated by the prediction error information generation unit is quantized and locally decoded, and a time difference is obtained from the past local decoded image existing at the same position as the encoding target block. Time difference information generating means for generating time difference information which is a difference in direction;
Prediction evaluation value calculation means for calculating the minimum value of the linear sum of the prediction error information and the time difference information for each prediction mode as a prediction evaluation value indicating the degree of similarity in the prediction mode by a predetermined cost function ,
Local decoding means for generating quantized data and the locally decoded image by quantizing and locally decoding the prediction error information generated by the prediction error information generating means;
A prediction mode determination unit that determines the prediction mode based on the prediction evaluation value calculated by the prediction evaluation value calculation unit;
Encoding means for encoding the quantized data generated based on the prediction mode determined by the prediction mode determining means, and generating encoded data;
An intra-screen predictive coding program characterized in that it is made to function as:

Images