JP2006086863A - Image stream transforming apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image stream transforming apparatus (transcoder) capable of generating a satisfactory encoded stream at the time of re-encoding in an image stream transformation processing. <P>SOLUTION: A header extracting circuit 4 extracts header information concerning an image stream to be an object of image transformation, and a syntax evaluation circuit 5 refers to the header information to determine whether or not an encoded syntax of the image stream is right. In this case, if it is determined that the encoded syntax is not right (wrong), for example, evaluation concerning change in encoding characteristics such as change in picture type is performed to search a proper encoded syntax. Then, a syntax re-constructing circuit 6 and an encoding information transformation circuit 9 re-construct re-encoding conditions in order to generate more satisfactory re-encoding stream. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention performs a re-conversion process on an image-encoded bit stream (sometimes called an image bit stream, an image stream, an encoded stream, etc.) to obtain a resolution, an encoding bit rate, an encoding method, and the like. More particularly, the present invention relates to an image stream conversion apparatus capable of realizing a good conversion method in consideration of image characteristics.

  In recent years, services that distribute information through various transmission paths such as satellite waves, terrestrial waves, and telephone lines using information compressed by high-efficiency coding for digitized image signals have been put into practical use. Yes. In such a service, when distributing information such as moving images / sounds, MPEG2 (Moving Picture Experts Group Phase 2), which is an international standard, is used as a high-efficiency encoding method for moving images / sounds. MPEG2 is an encoding method that compresses the information amount of an image signal by using a correlation between adjacent pixels (spatial direction) of an image signal and a correlation between adjacent frames or adjacent fields (time direction).

  Image encoding in the MPEG2 standard is processed by the following algorithm. First, temporally continuous image frames are divided into a reference frame and a prediction frame. By encoding the reference frame using only the spatial correlation, the original image can be restored using only the encoded data of the frame. On the other hand, the prediction frame is encoded using the correlation in the time direction and the correlation in the spatial direction from the reference frame, thereby realizing higher encoding efficiency than the reference frame. Note that the encoded data of the prediction frame is recovered by the recovered reference frame and the encoded data of the prediction frame.

  Next, a specific encoding system used in MPEG2 image encoding will be described with reference to FIG. In FIG. 4, numbers are assigned to the picture types as necessary so that they can be identified. An I picture (I frame), which is a reference frame indicated as “I” in FIG. 4A, is periodically present and is information serving as a reference for decoding processing. On the other hand, in the prediction frame, a P picture (P frame) encoded by only prediction from a temporally previous (past) reference frame, indicated by “P” in FIG. There is a B picture (B frame) that is predictively encoded from two reference frames before and after (past and future) in time, which are indicated as “B” in FIG. Note that arrows in FIG. 4A indicate prediction directions related to the P picture and the B picture. The P picture itself is a prediction frame and is also used as a reference frame for other P pictures and B pictures.

  An I-picture image signal is divided into processing units called macroblocks of horizontal 16 pixels × vertical 16 pixels based on the luminance signal. The divided macroblock data is further divided into two-dimensional blocks of 8 pixels × 8 pixels and subjected to DCT (Discrete Cosine Transform) processing which is a kind of orthogonal transform.

  Since the signal after the DCT processing shows a value according to the frequency component of the two-dimensional block, the component is concentrated in a low band in a general image. In addition, information degradation of high frequency components has a property that is visually less noticeable than information degradation of low frequency components. Therefore, the amount of information is compressed by finely quantizing the low frequency components and coarsely quantizing the high frequency components, and variable length coding the coefficient component and the continuous length of coefficient 0 having no component.

  Similarly to the I picture, the P picture image signal is also divided into units of macroblocks of horizontal 16 pixels × vertical 16 pixels based on the luminance signal. In the P picture, a motion vector between the reference frame and each macroblock is calculated. Motion vector detection is generally obtained by block matching. In this block matching, the difference between each pixel of the macroblock and each pixel obtained by blocking the reference frame where the horizontal / vertical position where the macroblock exists by the motion vector value is moved into 16 horizontal pixels × 16 vertical pixels. The absolute value sum (or the sum of squared differences) is obtained, and the value of the motion vector taking the minimum value is output as the detected motion vector.

  Each pixel of the macro block is subjected to a difference from each pixel of the two-dimensional block cut out by the motion vector. When an accurate motion vector is detected, the information amount of the difference block is significantly smaller than the information amount of the original macroblock, so that coarser quantization processing than that of the I picture is possible. Actually, it is selected whether to encode a differential block or a non-differential block (intra block) (prediction mode determination), and DCT similar to an I picture is selected for the selected block. A variable length encoding process is performed to compress the amount of information.

  In addition, the same processing as that for the P picture is performed for the B picture, but the I and P pictures that are the reference frames exist before and after the time frame, and motion vectors are detected between the reference frames. Is called. In the B picture, prediction options are prediction from a previous reference frame (forward prediction), prediction from a subsequent reference frame (backward prediction), and an average value (average (average (2)) of two prediction blocks. There are three types of (average) prediction), and prediction mode determination is performed from among four types of schemes including a scheme for decoding only by intra blocks.

  B pictures can be predicted from temporally preceding and following reference frames, so that prediction efficiency is further improved than P pictures. Therefore, in general, a B picture is quantized more coarsely than a P picture. The block selected as the B picture is subjected to the same encoding process as the I and P pictures.

  In the decoding process of a B picture, a prediction process from a later reference frame is also performed, and thus this reference frame needs to be encoded before the B picture. For this reason, in the encoding process, as shown in FIG. 4B, the image signal recorded and input is arranged such that the B picture is arranged after the I picture or P picture which is the reference frame of the B picture. Then, the order is rearranged and encoded. That is, during the encoding process, the order of the input order of the original image is rearranged in view of the encoding order during the decoding process. On the other hand, in the decoding process, as shown in FIG. 4C, the decoded image is reproduced in the order of the input image signals by performing the reverse rearrangement with respect to the order of FIG. It becomes possible.

  Next, a general encoding device and decoding device for realizing MPEG2 image encoding will be described. First, a general encoding apparatus in the prior art will be described. FIG. 5 is a block diagram showing an example of a general encoding apparatus according to the conventional technique. In FIG. 5, the digital image signal (input image signal) input from the input terminal 201 is supplied to and stored in the input image memory 202 and is delayed in order to be rearranged in the encoding order according to the encoding syntax. Is done. The digital image signal output from the input image memory 202 is subjected to a macroblock cutout process in the two-dimensional block conversion circuit 203.

  Macroblock data relating to the reference frame is supplied to the orthogonal transform circuit 205 via the subtractor 204, where DCT processing is performed in units of horizontal 8 pixels × vertical 8 pixels to calculate DCT coefficients. The DCT coefficients are further collected in units of macroblocks of horizontal 16 pixels × vertical 16 pixels based on the luminance signal, and sent to the quantization circuit 206. In the quantization circuit 206, for example, the quantization process is performed by dividing by a different value for each DCT coefficient by a quantization matrix having a different value for each frequency component. The quantized DCT coefficient is sent to the encoding circuit 214, and variable length or fixed length encoding is performed by referring to an address corresponding to the coefficient of the encoding table 215. Then, the multiplexer 216 multiplexes the encoded data after the processing in the encoding circuit 214 and the additional information indicating the location of the macroblock in the screen from the two-dimensional block conversion circuit 203, and the image stream After being stored in the buffer 218 once, it is output from the output terminal 219 as a bit stream (output image bit stream).

  Also, the DCT coefficients quantized by the quantization circuit 206 are subjected to inverse quantization processing and inverse DCT processing by the inverse quantization circuit 212 and the inverse orthogonal transform circuit 213, and the quantized DCT coefficients are decoded. The data is supplied to and stored in the reference image memory 209 via the adder 210 and the deblocking circuit 211. The image stored in the reference image memory 209 is used at the time of predictive frame encoding processing.

  On the other hand, for the predicted frame, a motion vector between images is obtained by the motion vector detection circuit 207 between the macroblock data cut out from the input image memory 202 and the image stored in the reference image memory 209. The motion vector obtained in the motion vector detection circuit 207 is supplied to the motion compensation prediction circuit 208, where a prediction block is cut out from the reference image from the reference image memory 209. The motion compensated prediction circuit 208 selects an optimal prediction mode according to the plurality of extracted prediction blocks, and sends a difference signal from the input image block to be encoded to the orthogonal transformation circuit 205. The difference signal is processed in the same manner as each block of the reference frame described above, the DCT coefficient is quantized, and the image stream buffer 218 is output from the multiplexer 216 as an output image bit stream together with the motion vector and the prediction mode. Then, it is output from the output terminal 219.

  Regarding the control of the code amount, the code amount control circuit 217 compares the code amount of the bit stream output from the multiplexer 216 with the target code amount (target code amount) to obtain the target code amount. In order to make it closer, the quantization fineness (quantization scale) of the quantization circuit 206 is controlled. Then, for the above-described three types of picture types (frame types) having different information amounts, the target code amount for each frame is calculated using the nature and appearance frequency of each picture type for the set encoding bit rate. .

  Also, the target code amount is limited so that a buffer overflow / underflow does not occur in a stream buffer (called a VBV (Video Buffer Verifier) buffer) virtually simulated by a decoding device. In addition, the quantization scale is calculated by calculating the quantization scale value for the target code amount for each frame type by utilizing the fact that the scale and the output code amount are generally inversely proportional. Done. Then, by controlling the quantization scale in a direction approaching the target code amount for each block, control is performed to suppress the encoded stream within the target code amount.

  Next, a general decoding device in the prior art will be described. FIG. 6 is a block diagram illustrating an example of a general decoding device according to the related art. In FIG. 6, first, an image bit stream (image stream) input from the input terminal 101 is stored in the image stream buffer 102. Note that a virtually simulated buffer value is written in the image bit stream, and the following decoding process is performed after the image bit stream is stored in the image stream buffer 102 by the buffer value. By doing so, it is possible to prevent the decoding process from stopping due to the failure of the buffer. In the image bit stream output from the image stream buffer 102, the variable length decoding circuit 103 separates additional information such as a quantization scale, a prediction mode, and a motion vector, and decodes the quantized DCT coefficient. .

  The decoded DCT coefficients are processed in the same manner as the inverse quantization circuit 212 and the inverse orthogonal transform circuit 213 in the encoding circuit (encoding apparatus shown in FIG. 5), and the inverse quantization circuit 105 and the inverse orthogonal transform are performed. In the circuit 111, inverse quantization processing and inverse DCT processing are performed, and an intra block or a difference block is decoded and supplied to the adder 107. In the case of a prediction block, a reference image signal read from the reference image memory 109 in the motion compensated prediction circuit 106 based on the prediction mode and the motion vector value decoded by the variable length decoding circuit 103 (of the process). A prediction block is cut out from a previously stored image signal of an I picture or a P picture. As a result, the decoded intra block or difference block and the prediction block cut out by the motion compensation prediction circuit 106 are added by the adder 107, and the image signal of the macro block is restored.

  The macroblock data (macroblock image signal) restored by the addition processing in the adder 107 is supplied to the deblocking circuit 108 and returned to the image signal in the order of image scanning. At this time, in the case of an I or P picture, it is written in the reference image memory 109, and in the case of a B picture, it is once stored in the output frame memory 110 and then output as an image signal (output image signal). Note that the I or P picture image data stored in the reference image memory 109 is once stored in the output frame memory 110 in accordance with the image output timing as shown in FIGS. In the same manner as described above, an image signal (output image signal) is output.

  Further, in a system that distributes image information using the above-described image encoding technology, there is a need to perform encoding processing again after decoding a once encoded bitstream. For example, when transmitting information from a place where information is collected / recorded to a place where information is distributed, a wired / wireless communication line or recording medium may be considered as a transmission path, but information is distributed from the place where information is collected / recorded. If the bandwidth of the transmission path for transmitting to the location is different from the bandwidth of the transmission path of the system that distributes information, it is necessary to re-encode and change the bit rate of the bit stream There is. For example, such a condition applies to a case where a broadcast station conducts interviewing and edits image data recorded on a VTR and then broadcasts. In addition, when it is desired to record an encoded stream sent by broadcasting or the like on a predetermined recording medium, it is necessary to re-encode the image stream according to the recording capacity and recording rate of the recording medium. There is also. An apparatus that performs the re-encoding process as described above is called a transcoder (image stream conversion apparatus). The basic configuration of the decoding device and the encoding device in the transcoder includes, for example, an image output of the decoding device (for example, the output terminal 112 shown in FIG. 6) and an image input of the encoding device (for example, the input terminal 201 shown in FIG. 5). Can be realized by a directly connected configuration.

  In such a transcoder, in order to reduce encoding degradation at the time of re-encoding, the picture type at the time of encoding on the image signal (decoded baseband signal) transmitted by the transcoder A scheme is considered in which frame information and macroblock additional information (hereinafter referred to as encoded information) are transmitted in a superimposed manner. For example, in MPEG2 encoding, the quality of an image signal generally deteriorates in the order of I picture, P picture, and B picture. Since the I picture is a reference frame, it is quantized more finely than other frames and is not affected by reference frame deterioration because there is no reference from other images. On the other hand, the P picture and the B picture are roughly quantized in the order of the P picture and the B picture, and are easily affected by reference frame deterioration.

  In the transcoder, it is possible to improve the quality of encoded data after the re-encoding process and to efficiently perform the re-encoding process by effectively using the above-described encoded information. For example, at the time of re-encoding in the transcoder, referring to the encoding information, the picture type in the encoded data before decoding in the transcoder and the picture type at the time of re-encoding are matched, thereby Factors can be reduced. Also, by referring to the macroblock additional information, the processing amount for motion vector detection can be reduced, and by referring to the quantization scale and the number of encoded bits, a guideline for the information amount possessed by the macroblock can be obtained. As a result, good code amount control can be performed at the time of bit rate conversion.

  Here, the configuration of the decoding apparatus and the encoding apparatus in the transcoder that transmits the encoding information as described above will be described. FIG. 7 is a block diagram showing an example of a general stream conversion recording apparatus according to the prior art. Note that the stream conversion recording apparatus shown in FIG. 7 basically has a configuration in which the encoding apparatus shown in FIG. 5 and the decoding apparatus shown in FIG. 6 are connected. Only the functional blocks added in the stream conversion recording apparatus shown in FIG. 7 will be described, and only the part related to image coding will be described.

  The variable-length decoding circuit 2 (corresponding to the variable-length decoding circuit 103 shown in FIG. 6) of the decoding device 100, which is a component of the image stream conversion device, performs frame information, macroblock additional information, and macroblock when the image stream is decoded. Are calculated, and the calculated data is supplied to the encoded information generation circuit 7. The encoded information generation circuit 7 formats the frame information and macroblock additional information for each frame and stores them in the encoded information memory 8.

  The encoded information memory 8 has a storage capacity for a plurality of frames stored in the output frame memory 3 (corresponding to the output frame memory 110 shown in FIG. 6) so as to correspond to output rearrangement of I, P, and B pictures. Yes. The encoded information composed of the frame information and the macroblock additional information stored in the encoded information memory 8 is supplied to the encoded information superimposing circuit 10, and the order of the encoded information extracted according to the picture type in the encoded information superimposing circuit 10. Are changed and output in synchronization with the image data (output image signal) output from the output frame memory 3 in the decoding apparatus 100. That is, the data output from the encoding information superimposing circuit 10 is encoding information attached to the image data output from the output frame memory 3 in the decoding device 100. The encoded information output from the encoded information superimposing circuit 10 is stored in the encoded information memory 11. At this time, in the encoding device 200 which is a component of the image stream conversion device, the image data (image signal) output from the output frame memory 3 corresponds to the input frame memory 14 (the input image memory 202 shown in FIG. 5). ).

  The encoded information stored in the encoded information memory 11 is read by the encoded information separation circuit 12. Then, the encoded information separation circuit 12 separates the frame information and the macroblock additional information from the encoded information, and sends them to the encoding syntax control circuit 13 and the macroblock information generation circuit 15. The encoding syntax control circuit 13 performs control to detect the frame type from the received frame information and rearrange the input images in the encoding order. When the encoded information separation circuit 12 cannot extract the encoded information of the frame, a non-detection signal is sent from the encoded information separation circuit 12 to the encoding syntax control circuit 13, and the encoding information is separated. In the syntax control circuit 13, an encoding syntax is configured in the encoding device 200 in the same manner as a normal encoding process, and the encoding process is performed. On the other hand, the macroblock information generation circuit 15 receives from the encoding information separation circuit 12 the macroblock additional information whose order has been changed in accordance with the encoding syntax in the encoding information separation circuit 12.

  The macroblock information generation circuit 15 converts the encoded information received from the encoded information separation circuit 12 into a motion compensation prediction circuit 16 (corresponding to the motion compensation prediction circuit 208 shown in FIG. 5) and code amount control in the encoding device 200. This is supplied to the circuit 17 (corresponding to the code amount control circuit 217 shown in FIG. 5). The motion compensation prediction circuit 16 cuts out a prediction block from the reference image memory 18 (corresponding to the reference image memory 209 shown in FIG. 5) using the motion vector and the prediction mode existing in the encoded information, and subtracter 19 (FIG. 5). The difference signal with the input image block from the input frame memory 14 to be encoded is generated, and the difference signal is converted into the orthogonal transformation circuit 20 (orthogonal shown in FIG. 5). Corresponding to the conversion circuit 205). If the encoded information separation circuit 12 cannot extract the encoded information of the picture or the encoded information for each macroblock, the encoded information is not detected by the macroblock information generation circuit 15 from the encoded information separation circuit 12. The signal is sent, and the motion vector detection and prediction mode selection processing similar to the normal encoding processing is performed in the encoding device 200.

  Further, the code amount control circuit 17 is supplied with the quantization scale for each macroblock at the time of encoding and the number of bits required for this processing as encoded information, and the supplied information, the current control target, Comparison processing with the set code amount at the encoding bit rate is performed. As a result, the code amount control circuit 17 determines an appropriate quantization scale and performs the encoding process. In the above description, the encoded information memories 8 and 11 are provided separately. However, in a transcoder in which the decoding apparatus 100 and the encoding apparatus 200 are integrated, these encoded information memories 8 and 11 are provided. , 11 can be combined into one.

Also, unlike the process in the transcoder described above, encoding information is not extracted, and the picture type of the input image stream is always inherited as the picture type at the time of re-encoding. There is also a method for reducing the rearrangement processing of the decoding circuit and the encoding circuit. This method is disclosed, for example, in Patent Document 1 below.
Japanese Unexamined Patent Publication No. 2000-92497 (paragraphs 0019 to 0025, FIG. 1)

  In a conventional image stream conversion apparatus, when re-encoding is performed, encoding information and pictures extracted from an image stream are based on the assumption that the encoding process of an input image stream is close to an ideal encoding process. In general, the type is handled as correct information, and the information is reflected as it is when re-encoding. For example, it can be said that the technique disclosed in Patent Document 1 described above is also based on this concept.

  However, for example, when the input image stream has poor characteristics and the best encoding process is not performed, if the encoding information related to the input image stream is taken over at the time of re-encoding, re-encoding is performed. Even in the converted image stream, a bad influence (an influence of quality degradation, etc.) seen in the input image stream is taken over, and there is a possibility that an inaccurate image stream conversion process is performed. . That is, when re-encoding an original image stream as in the conventional technique, if parameters relating to the original image stream are taken over and reflected as they are, in some circumstances, the original image stream has inconvenience. There is a problem that parameters are also carried over as they are.

  In addition, for example, when compressing an image stream received through broadcasting or the like and recording it on a recording medium, the encoding bit rate is significantly changed for the input image stream to comply with the encoding bit rate. When re-encoding is not performed (especially when converting to a low encoding bit rate), appropriate and sufficient compression cannot be performed, and image quality is greatly reduced during re-encoding. There is a problem of deterioration.

  In view of the above problems, the present invention decodes an encoded stream of an image that has been encoded once, and converts it again into a necessary code amount, image size, etc., and re-encodes the image stream conversion process An object of the present invention is to provide an image stream conversion apparatus capable of generating a good encoded stream at the time of re-encoding.

In order to achieve the above object, according to the present invention, in an image stream conversion apparatus for performing decoding processing of an image stream and performing re-encoding processing of the image stream decoded by the decoding processing,
Header information extraction means for extracting header information relating to the image stream;
When the header information extracted by the header information extraction unit is referenced and the image stream is re-encoded based on the encoding syntax of the image stream, a good encoded stream is generated. A syntax evaluation unit that determines whether or not to change the encoding syntax when it is determined that it is not possible to generate the good encoded stream;
Syntax reconstructing means for reconstructing the coding syntax of the image stream based on a determination result relating to the appropriate coding syntax by the syntax evaluation means;
Encoding information conversion for converting encoding information related to the image stream used for the re-encoding process of the image stream based on the encoding syntax reconstructed by the syntax reconstructing means Means,
There is provided an image stream conversion device characterized by comprising:

  Furthermore, according to the present invention, in the above invention, the syntax evaluation means includes a required number of bits for each picture in the image stream, a motion vector value, an intra-screen distribution state of a prediction mode, and a frame sum of a quantization scale. Whether or not a good encoded stream can be generated when re-encoding processing of the image stream based on the picture type of each picture of the image stream is performed using at least one of the parameters 2. The image stream according to claim 1, further comprising: determining whether or not to change the picture type when it is determined that the good encoded stream cannot be generated. A conversion device is provided.

  The image stream conversion apparatus according to the present invention generates a good encoded stream at the time of re-encoding by re-determining the encoding information of the image stream before conversion and reconfiguring the encoded information as necessary. It has the effect that it can be done.

  Hereinafter, an image stream conversion apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of an image stream conversion apparatus according to an embodiment of the present invention. The image stream conversion apparatus shown in FIG. 1 has components common to the above-described conventional image stream conversion apparatus (see FIG. 7). Here, description of these common components is provided. Is omitted.

  The image stream conversion apparatus shown in FIG. 1 analyzes an encoded stream before decoding and re-encodes the encoded stream of the image that has been subjected to encoding processing, and re-encodes the encoded stream. By determining changes or inheritance of various parameters related to encoding information under predetermined conditions, appropriate and good encoding syntax control processing at the time of re-encoding, motion vector detection processing, code amount control processing, etc. It is configured to be able to be done. Hereinafter, the configuration and operation of the image stream conversion apparatus shown in FIG. 1 will be specifically described.

  When an input bit stream (image stream) is stored in the image stream buffer 1 (corresponding to the image stream buffer 102 shown in FIG. 6) of the decoding device 100 that is a component of the image stream conversion device, the variable length decoding of the decoding device 100 is performed. Before being output to the circuit 2 (corresponding to the modulation modulation decoding circuit 103 shown in FIG. 6), it is immediately supplied to the header extraction circuit 4. In the header extraction circuit 4, the header information of the image stream is extracted prior to the decoding process (or independently of the decoding process). In this header information extraction process, basically, the picture type and the required bit amount of each picture of the MPEG image stream are extracted and calculated. The required number of bits can be extracted by calculating the interval between picture headers.

  Further, the header extraction circuit 4 may be provided with a decoding processing function similar to that of the variable length decoding circuit 2 of the decoding device 100. In this case, a motion vector, a prediction mode, a quantization scale, etc. can be calculated. It is. In the following description, it is assumed that the variable length decoding circuit 2 is provided with a decoding processing function and the above parameters can be calculated.

  The syntax evaluation circuit 5 evaluates the coding characteristics of the coded stream by referring to the header information extracted by the header extraction circuit 4. At this time, basically, I-pictures are not subjected to syntax change processing (also called coding syntax) so as to be encoded with I-pictures as they are (inheritance of syntax), while P For the picture / B picture, the validity of the syntax is determined from the header information. It should be noted that, from the input encoded stream and its header information, the image code such as the required number of bits for each picture in the encoded stream, the motion vector value, the intra-screen distribution state and the frequency of the prediction mode, and the quantization scale value are included. It is possible to acquire various parameters related to the encoding characteristics, and the syntax evaluation circuit 5 can evaluate the encoding characteristics using any one of these parameters or a combination thereof. It is.

  The P picture / B picture evaluation algorithm in the syntax evaluation circuit 5 can be performed, for example, by the following method. Here, the evaluation is performed by an evaluation unit in which one reference picture (I picture or P picture) and a B picture therebetween are paired.

  Here, with reference to FIG. 2, the correctness determination of the reference picture in the syntax evaluation circuit 5 of the image stream conversion apparatus according to the present invention will be described. FIG. 2 is a flowchart showing the reference picture correctness determination operation of the syntax evaluation circuit of the image stream conversion apparatus according to the embodiment of the present invention.

  First, it is determined whether the reference picture is an I picture or a P picture (step S1001). When the reference picture is an I picture, the position of the I picture is not changed. Therefore, it is determined that the reference picture determination is correct (step S1003).

  On the other hand, when the reference picture is a P picture, a determination is made by measuring the predictive medium accuracy (prediction efficiency) of the P picture. In this case, for example, the number of bits Bits (P) required for the P picture and the quantization scale value AvgQ (P) are determined, and Bits (P) is less than or equal to the threshold value α and AvgQ (P) is less than or equal to the threshold value β. In the case (“Yes” in step S1005), it is determined that the reference picture determination is correct. The threshold values α and β are variables controlled by the encoding bit rate of the input stream. Further, the quantization scale value AvgQ (P) can be said to be a parameter indicating the possibility of a prediction error. As described above, when it is determined that the reference picture determination is correct, the syntax of the corresponding frame is inherited and re-encoded.

  On the other hand, if it is not determined to be correct in the above determination (“No” in step S1005), the bit number Bits (PrevI) of the last input I picture (the reference I picture when decoded) The quantization scale value AvgQ (PrevI) is compared with the bit number Bits (P) of the P picture and the quantization scale value AvgQ (P). In this comparison, for example, the product of the number of bits and the quantization scale value can be used.

For example, in comparing the P picture with the last input I picture,
Bits (P) * AvgQ (P) * γ> Bits (PrevI) * AvgQ (PrevI)
If PrevI is in the notation representing the last I picture (“Yes” in step S1007), the reference picture determination is determined to be erroneous, and this P picture is corrected to an I picture (step S1009). . Note that γ used in the above equation is a variable calculated from the number of intra modes in the prediction mode of the P picture.

  If the condition of the above equation is not satisfied (“No” in step S1007), it is determined that the reference picture determination is correct, and the syntax of this P picture is retained unchanged and inherited as the P picture. (Step S1111). As described above, the inheritance or change of the syntax relating to the reference picture determination is determined.

  Next, the B picture determination in the syntax evaluation circuit 5 of the image stream conversion apparatus according to the present invention will be described with reference to FIG. FIG. 3 is a flowchart showing the B picture determination operation of the syntax evaluation circuit of the image stream conversion apparatus according to the embodiment of the present invention.

  In the determination of the B picture, the appearance degree of the prediction mode can be used as a determination material together with the number of bits and the quantization scale value as in the determination of the P picture. The determination of the B picture is performed based on the result of the correctness determination operation of the reference picture determination described above. That is, in the above-described correct / incorrect determination operation of the reference picture determination, a determination result of whether the reference picture determination is correct or incorrect is acquired, and the syntax evaluation circuit 5 performs a B picture determination operation according to the determination result. . In the following description, the case where the reference picture is a P picture will be described as an example, but the same processing is performed when the reference picture is an I picture.

  When determining a B picture, first, a determination result of whether the reference picture determination is correct or incorrect is acquired (step S2001). If it is determined that the reference picture determination is correct (“Yes” in step S2001), it is determined whether or not the B picture is an isolated frame (a frame in which there is no frame that deserves prediction). Is called. In this isolated frame determination, for example, the product of the number of bits and the quantization scale value can be used.

For example, in a comparison between a reference picture (here, a P picture) and a determination-target B picture,
Bits (B) * AvgQ (B) * Δ> Bits (P) * AvgQ (P)
And the number of intra modes in the prediction mode is greater than or equal to the threshold ε (that is, the number of in-plane predictions is greater than or equal to ε) (“Yes” in step S2003), the B picture is determined to be an isolated frame. Then, the picture type of this B picture is changed to an I picture (step S2005). Note that Δ used in the above equation is a variable calculated from the number of intra modes in the prediction mode of the B picture.

  If the condition of the above equation is not satisfied (“No” in step S2003), it is determined that the B picture is not an isolated frame, and the syntax of this B picture is retained without being changed. (Step S2007).

  On the other hand, when it is determined that the reference picture determination of the P picture is incorrect (“NO” in step S2001), for example, assuming that a scene change (scene change) has occurred in the B picture, The search for the switching point and the holding / changing of the syntax according to the search result are performed (step S2009).

  Here, an example of a search for a switching point (processing in step S2009) in the case where the period M = 3 as illustrated in FIG. 4 will be described. When the period M = 3, two frames of B pictures exist between P pictures (or between I pictures and P pictures) (referred to as B1 and B2). In the search for switching points, first, the appearance frequencies of the preceding and following prediction modes are compared for each of these B pictures B1 and B2.

  At this time, when the appearance frequency of forward prediction (forward prediction) is greater than or equal to the threshold in both B pictures B1 and B2 (that is, when forward prediction is determined to be dominant for both B pictures B1 and B2). Is expected to have a scene change after the B picture B2. In this case, it is desirable to change B picture B2 to P picture.

  Also, when the appearance frequency of backward prediction (backward prediction) is greater than or equal to the threshold in both B pictures B1 and B2, (ie, when it is determined that backward prediction is dominant for both B pictures B1 and B2). Is expected to have a scene change before the B picture B1. In this case, it is desirable to maintain the same syntax for both B pictures B1 and B2. Note that it is possible to make the B pictures B1 and B2 independent of the previous GOP by making the GOP (Group of Pictures) a closed GOP at the transition of the scene. In this case, the closed GOP flag Outputs information to set.

  Further, when the appearance frequency of the forward prediction is equal to or higher than the threshold value in the B picture B1, and the appearance frequency of the backward prediction is equal to or higher than the threshold value in the B picture B2 (that is, the forward prediction is determined to be dominant in the B picture B1). In the case of B picture B2, when it is determined that backward prediction is dominant), it is expected that there is a scene change between B picture B1 and B picture B2. In this case, it is desirable to change the B picture B1 to a P picture. In this case as well, it is possible to set a scene change point to a closed GOP. For each condition other than the above, it is basically desirable to keep the syntax as it is.

  As described above, the syntax evaluation circuit 5 performs syntax holding or change determination processing, and the determination result information is supplied from the syntax evaluation circuit 5 to the syntax reconfiguration circuit 6. In the syntax reconfiguration circuit 6, the syntax is updated according to the determination result information, and is supplied to the encoded information conversion circuit 9.

  The encoded information conversion circuit 9 compares the picture type supplied from the encoded information memory 8 with the picture type generated with the updated syntax supplied from the syntax reconstruction circuit 6. If the picture types are different, conversion processing is performed on the motion vector and the prediction mode. For example, if it is determined that the B picture is a P picture as a result of the comparison of the picture types, the motion vector is used as it is when the prediction mode is forward prediction (prediction from the previous reference image), In the case of backward prediction or average prediction, it is converted to intra mode.

  Also, for a B picture (for example, B picture B1) in which a B picture (for example, B picture B2) immediately after is changed to a P picture, the back reference image of this B picture B1 is B picture B2 at the time of the input image stream. It will change to the image that was. In this case, when the B picture B1 is predicted backward, the motion vector is reduced in proportion to the distance between the prediction frames. As described above, basically, there is a scene change between the B pictures B1 and B2, and it is determined that the forward prediction is dominant for the B picture B1 (there is almost no backward prediction). In this case, since the B picture B2 immediately after the B picture B1 is changed to the P picture, it is expected that the adverse effect due to the reduction of the motion vector hardly occurs.

  As described above, the encoding information conversion circuit 9 performs conversion processing on the motion vector and the prediction mode, and also updates the encoding information such as rewriting of the picture type. Then, the encoded information updated in the encoded information conversion circuit 9 is supplied to the encoded information superimposing circuit 10, and thereafter, based on the updated encoded information, an encoding process similar to the conventional technique is performed. Will be performed.

  In the above-described embodiment, only the encoding information of the input image stream is used as the determination material supplied to the syntax evaluation circuit 5, but for example, the encoding process is terminated at a predetermined reproduction time. When the I-picture is inserted into a specific frame, or when it is desired to switch the GOP at a specific frame, the frame position specifying information for specifying these positions is input from the outside. Similar to the determination material, syntax switching processing using this frame position designation information can be performed. In this case, since it is possible to determine the effective portion of the encoding information of the input image stream and to perform information correction regarding the correctable portion, it is compared with a case where the encoding device 200 simply switches the syntax. Thus, the coding efficiency is improved.

  Note that the present invention can also be realized, for example, by performing syntax evaluation processing simultaneously with encoding / decoding processing (conversion processing) of an input image sequence. Also, the present invention, for example, in a state where an input image stream to be re-encoded is recorded on a recording medium, a syntax evaluation process is performed on the entire sequence in advance, and the result information (entire sequence) This information can be realized by referring to the result information when the input image sequence is converted, after the information on the retention or change of the syntax is stored in a recording medium or a buffer. .

  In the above-described embodiment, hardware elements such as a circuit and a schematic block are illustrated as examples of components of the image stream conversion apparatus according to the present invention. Similar to the apparatus, these hardware elements can be realized by software (program) executable by a computer.

  The image stream conversion apparatus according to the present invention has an effect that a good encoded stream can be generated at the time of re-encoding, and performs re-conversion (transcoding) of an image-encoded bit stream. It is applicable to the technical field of

It is a block diagram which shows an example of the image stream conversion apparatus in embodiment of this invention. It is a flowchart which shows the correctness determination operation | movement of the reference | standard picture of the syntax evaluation circuit of the image stream converter in embodiment of this invention. It is a flowchart which shows the determination operation | movement of the B picture of the syntax evaluation circuit of the image stream converter in embodiment of this invention. It is a figure which shows typically the arrangement | sequence of the image at the time of the process in MPEG2 image coding based on the prior art, and an output, (A) is a figure which shows the encoding system used by MPEG2 image coding, ) Is a diagram showing rearrangement of the encoding order at the time of MPEG2 image encoding, and (C) is a diagram showing the stream arrival order and the decoded image output order at the time of MPEG2 image decoding. It is a block diagram which shows an example of the general encoding apparatus which concerns on a prior art. It is a block diagram which shows an example of the general decoding apparatus concerning a prior art. It is a block diagram which shows an example of the general stream conversion recording device based on a prior art.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,102,218 Image stream buffer 2,103 Variable length decoding circuit 3,110 Output frame memory 4 Header extraction circuit 5 Syntax evaluation circuit 6 Syntax reconstruction circuit 7 Encoding information generation circuit 8,11 Encoding information memory 9 Encoding information conversion circuit 10 Encoding information superposition circuit 12 Encoding information separation circuit 13 Encoding syntax control circuit 14 Input frame memory 15 Macroblock information generation circuit 16, 106, 208 Motion compensation prediction circuit 17, 217, 403 Code amount Control circuit 18, 109, 209 Reference image memory 19, 204 Subtractor 20, 205 Orthogonal transformation circuit 100 Decoding device 101, 201 Input terminal 104, 215 Coding table 105, 212 Inverse quantization circuit 107, 210 Adder 108, 211 Deblock circuit 11 , 213 inverse orthogonal transform circuit 112,219 output terminal 200 encoder 202,404 input image memory 203 2-dimensional block transform circuit 206 quantization circuit 207 motion vector detection circuit 214 encoding circuit 216 multiplexer

Claims (2)

In the image stream conversion apparatus for performing the decoding process of the image stream and performing the re-encoding process of the image stream decoded by the decoding process,
Header information extraction means for extracting header information relating to the image stream;
When the header information extracted by the header information extraction unit is referenced and the image stream is re-encoded based on the encoding syntax of the image stream, a good encoded stream is generated. A syntax evaluation unit that determines whether or not to change the encoding syntax when it is determined that it is not possible to generate the good encoded stream;
Syntax reconstructing means for reconstructing the coding syntax of the image stream based on a determination result relating to the appropriate coding syntax by the syntax evaluation means;
Encoding information conversion for converting encoding information related to the image stream used for the re-encoding process of the image stream based on the encoding syntax reconstructed by the syntax reconstructing means Means,
An image stream converter characterized by comprising: The syntax evaluation means uses the image stream using at least one parameter of a required number of bits for each picture in the image stream, a motion vector value, an intra-screen distribution state of a prediction mode, and a frame sum of quantization scales. When the re-encoding process of the image stream based on the picture type of each picture is performed, it is determined whether or not a good encoded stream can be generated and the good encoded stream is generated 2. The image stream conversion apparatus according to claim 1, wherein if it is determined that the picture type cannot be changed, it is determined whether or not to change the picture type.

Images