JP2006295705A - Animation encoder, animation decoder, animation encoding method, animation decoding method, animation encoding program, and animation decoding program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To avoid the data transfer of the background image which little change as much as possible. <P>SOLUTION: The VLC18 removes a static block from a slice layer. When a flag "0" is set, for example, the a more significant 100th bit corresponding to the location of a static block MB100, data of the static block MB100 are removed. On the other hand, the macro-block data other than the static block to which the flag "0" is set and a flag of each one bit are transmitted, while a data capacity required for the transmission of I picture can be saved because the macro-block data of the static block are not transmitted. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to encoding and decoding of moving images, and particularly to data transmission / reception of background images that do not change.

  As a system for compressing moving images and audio, there is an “MPEG2” system. In moving picture compression using MPEG2, an image is divided into 16 × 16 pixel rectangular blocks called macroblocks, and an area (reference area) similar to the macroblock to be compressed is extracted from the temporally preceding and following images. Then, the spatial distance and direction (motion vector) from the reference area and the difference information between the reference area and the area to be compressed are calculated, and these pieces of information are converted into DCT (discrete cosine transform) and variable length coding. The amount of data is reduced by using it to compress into a bitstream and reducing temporal redundancy. However, since an image compressed by the difference information cannot be expanded without an image to be referred to, in order to cope with a use in which the bitstream is expanded from the middle (hereinafter referred to as “random access”), It is necessary to periodically provide images that do not refer to images. This image is called an “I picture” (Intra Picture). The next image is compressed using the I picture as a reference image, and the subsequent images are further compressed using the already compressed image as a reference image.

  Among the images to be compressed using the reference image, a “P picture” (Predictive-coded Picture) in which only the previous image in time is used as a reference image and a “B picture in which images before and after in time are used as reference images” "Picture" (Bidirectionally predictive-coded Picture). A P picture can be a reference image of another image, like an I picture. In MPEG, a GOP (Group Of Pictures) can be configured from an I picture and a series of image aggregates that are directly and indirectly related thereto. A GOP header is added to the GOP. GOP is used as a unit of random access.

  In compression using a B picture, in order to use temporally previous and subsequent images as reference images, processing is performed to switch the order of input images and the order of images on the bitstream obtained as a result of compression.

In moving picture compression by MPEG2, the compression efficiency is further increased by a method called skipping. In B picture and P picture coding, if the macroblock of the current picture being coded is the same as the macroblock of the previous picture in time, the data is sent as a skipped macro. Do not. In decoding, the same image data as the image data decoded from the immediately preceding I picture or P picture is restored.
JP 2001-103429 A JP 2001-224028 A JP 2002-10214 A JP-A-8-126021

  By the way, in a moving picture transmission apparatus such as a videophone, the background image often changes little. In the conventional MPEG2 compression, even in such a case, it is necessary to periodically send I pictures that hardly change for each GOP unit, which is equivalent to continuously sending useless data. Especially on telephone lines where transmission capacity is limited, the necessary data transmission is under pressure, and image data transfer for important parts (such as human faces and hands) where image quality is most important is also restricted. May reach. Even on the receiving side, it is necessary to continue to decode an image with almost no motion in vain, and the waste in image processing is also significant. The present invention has been made in view of such problems, and an object thereof is to avoid data transfer of a background image that hardly changes as much as possible.

  In order to solve the above-described problem, a moving image encoding apparatus according to the present invention refers to an input unit that inputs frame moving image data and an intra (I) picture that constitutes certain frame moving image data input from the input unit. An I picture storage unit for storing as an I picture for comparison, an I picture for reference stored in the I picture storage unit and an I picture constituting the current frame moving image input from the input unit, A still block detecting unit that detects a still block that is a block that constitutes the current I picture that is substantially unchanged from the blocks that constitute the frame, a flag indicating the still block, and data of blocks other than the still blocks that constitute the current I picture A compression / encoding unit that compresses and encodes the frame moving image data including

  According to the present invention, since the flag is output instead of the still block data, it is possible to save the data capacity necessary for transmitting the almost still background image data included in the I picture.

  The flag may be set in the header part of the slice layer of the current I picture.

  The flag can be 1-bit data indicating the position of a still block.

  A static block includes a macroblock. The still block itself may be a macro block, but may be a division unit of one screen smaller than the macro block or a division unit of one screen larger than the macro block.

  It is preferable that the I picture storage unit periodically updates the reference I picture. This is because the background image itself changes due to panning of the imaging device or the like, and all still blocks may change.

  A video decoding device according to the present invention includes an input unit that inputs encoded frame video data, a decoding unit that decodes encoded frame video data, and a decoded frame video A reproduction unit that reproduces a frame moving image including an intra (I) picture from data, an I picture storage unit that stores an I picture included in the reproduced frame moving image data, and a decoded frame moving image of the decoding unit A flag detection unit for detecting a flag indicating a still block from data, and a still block of an I picture of frame moving image data input based on the flag indicating a still block by a still block of an I picture stored in an I picture storage unit A stationary block complementing unit for complementing.

  The moving picture decoding apparatus reproduces a frame moving picture from the frame moving picture data input from the moving picture encoding apparatus according to the present invention, detects a flag from the input frame moving picture data, and based on the flag The still block of the I picture is complemented by the still block of the I picture stored in the I picture storage unit.

  That is, even if the frame moving image data output from the moving image encoding apparatus according to the present invention does not include still block data constituting the I picture, it can be complemented by the stored still block and does not change. There is no need to repeatedly output the still block from the moving picture encoding apparatus.

  A step of inputting frame moving image data; a step of storing an intra (I) picture constituting a certain frame moving image data input from the input unit as a reference I picture; and a reference stored in the I picture storage unit The I picture and the I picture constituting the current frame moving image input from the input unit are compared, and the still block which is the block constituting the current I picture having substantially no change from the block constituting the reference I picture There is also a moving image encoding method including a step of detecting, and a step of compressing and encoding frame moving image data including a flag indicating a still block and block data other than the still block constituting the current I picture. It is included in the present invention.

  The step of inputting the encoded frame moving image data, the step of decoding the encoded frame moving image data, and the playback of the frame moving image including the intra (I) picture from the decoded frame moving image data A step of storing an I picture included in the reproduced frame moving image data, a step of detecting a flag indicating a still block from the decoded frame moving image data of the decoding unit, and a flag indicating the still block Complementing the still block of the I picture of the frame moving image data input based on the I block with the still block of the I picture stored in the I picture storage unit is also included in the present invention.

  A program for causing a computer to execute the above method is also included in the present invention.

  According to the moving picture coding apparatus of the present invention, since a flag is output instead of still block data, it is possible to save a data capacity necessary for transmitting almost still background image data included in an I picture.

  According to the moving picture decoding apparatus according to the present invention, the frame moving picture data output from the moving picture encoding apparatus according to the present invention is stored even if the data of the still block constituting the I picture is not included. Therefore, there is no need to repeatedly output a still block of an I picture that can be complemented by a still block and does not change from the moving picture coding apparatus.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

  FIG. 1 is a block diagram of an encoding apparatus 100 according to a preferred embodiment of the present invention. The encoding apparatus 100 includes an image input unit 11, a motion vector detection circuit 14, a motion compensation circuit 15, a DCT 16, a quantization circuit 17, a variable length encoder (VLC) 18, an encoding control unit 20, a buffer 19, and data transmission. Unit 21, a stationary block detection unit 24, a stationary block storage unit 25, and the like. This apparatus partially includes a configuration of an MPEG2 system encoding apparatus that combines motion compensation predictive encoding and compression encoding by DCT.

  The image input unit 11 inputs a moving image signal from an image pickup apparatus including an image pickup device such as a CCD (not shown).

  The motion vector detection circuit 14 detects a motion vector by comparing the current frame image represented by the data input from the image input unit 11 with the previous frame image stored in the frame memory 22. This motion vector detection is performed by dividing the input current frame image into a plurality of macro blocks, and appropriately searching the macro block to be searched within the search range set on the previous frame image in units of individual macro blocks. By repeating the error calculation while moving, the macro block that is most similar to the macro block to be searched (the macro block with the smallest error) is searched from the search range, the amount of deviation between the macro block and the macro block to be searched, and The direction of deviation is taken as the motion vector for the searched macroblock. Then, by synthesizing the motion vector obtained for each macroblock in consideration of the error for each macroblock, a motion vector that minimizes the prediction difference in predictive coding can be obtained.

  The motion compensation circuit 15 generates motion picture data by performing motion compensation on the prediction reference picture based on the detected motion vector, and outputs the data to the subtracter 23. The subtracter 23 generates difference data representing a prediction difference by subtracting a prediction image represented by data input from the motion compensation circuit 15 from a current frame image represented by data input from the image input unit 11. To do.

  A DCT (discrete cosine transform) unit 16, a quantization circuit 17, a VLC 18, and a buffer 19 are sequentially connected to the subtracter 23. The DCT 16 orthogonally transforms the difference data input from the subtractor 23 for each arbitrary block and outputs the result. The quantization circuit 17 quantizes the difference data input from the DCT 16 after the orthogonal transformation in a predetermined quantization step. And output to VLC18. A motion compensation circuit 15 is connected to the VLC 18, and motion vector data is also input from the motion compensation circuit 15. The VLC 18 encodes the difference data that has undergone orthogonal transformation / quantization using a two-dimensional Huffman code, encodes the input motion vector data using the Huffman code, multiplexes both, and outputs the encoded moving image data. . The encoded moving image data is stored in the buffer 19 and is transmitted as a packet from the data transmission unit 21 to a network such as the Internet as image compression information. The code amount of the quantization circuit 17 is controlled by the code amount control device 20.

  The data structure of the encoded moving image data created by the VLC 18 has a hierarchical structure, and is, from the lower order, a block layer, a macroblock layer, a slice layer, a picture layer, a GOP layer, and a sequence layer.

  The block layer is composed of DCT blocks that are units for performing DCT. The macroblock layer is composed of a plurality of DCT blocks. The slice layer is composed of a header part and one or more macroblocks. The picture layer is composed of a header part and one or more slice layers. A picture corresponds to one screen. The GOP layer includes a header part, an I picture that is a picture based on intra-frame coding, and a P and B picture that are pictures based on predictive coding. The I picture can be decoded only with its own information, and the P and B pictures require previous or previous pictures as predicted pictures and are not decoded alone.

  Also, at the beginning of the sequence layer, GOP layer, picture layer, slice layer, and macroblock layer, an identification code having a predetermined bit pattern is arranged, and the encoding parameters of each layer are stored following the identification code. A header part is arranged.

  The macroblock included in the slice layer is a set of a plurality of DCT blocks, and is obtained by dividing a screen (picture) into a lattice shape (for example, 8 pixels × 8 pixels). The slice is formed by, for example, connecting the macro blocks in the horizontal direction. When the screen size is determined, the number of macroblocks per screen is uniquely determined.

  In the MPEG format, the slice layer is one variable length code sequence. A variable-length code sequence is a sequence in which a data boundary cannot be detected unless the variable-length code is decoded. When decoding the MPEG stream, the header part of the slice layer is detected to find the start point and end point of the variable length code.

  The still block detection unit 24 according to the present invention includes a macroblock existing in the slice layer of the I picture of the previous frame and a macro existing in the slice layer of the I picture of the current frame at the same position in the image. In accordance with the comparison with the block, a macro block (referred to as a still block) having substantially no change is detected. Specifically, the still block detection unit 24 selects the macro block Mp (n) of the I picture of the previous frame arranged in the same position as each macro block Mc (n) of the I picture of the current frame. The search range is set in the motion vector detection circuit 14, and the difference (change amount) between the macroblock Mc (n) and the macroblock Mp (n) is calculated, and this difference is substantially 0 (or less than a predetermined lower limit threshold). Detect macroblocks as static blocks. Here, n is a subscript indicating the position of the macro block on the screen. Here, as an example, the order determined by sequentially scanning the macro block from left to right in the horizontal direction from the upper left to the lower right of the screen. Shall be shown. The position of the detected still block is stored in the still block storage unit 25.

  Note that still block detection by the still block detection unit 24 may be performed every time a predetermined period elapses (for example, every 10 seconds), not every time a new frame image is input. This is because a stationary block rarely changes between temporally adjacent frames, and it is sufficient to detect a stationary block change periodically. If still block detection is performed at too short intervals, data transmission is delayed, so it is preferable to perform the detection every appropriate period in consideration of the shooting situation and the calculation speed of the encoding device 100.

  Alternatively, an I picture of a frame input at a certain timing is stored in the frame memory 22 as a reference I picture, and a still block is detected according to a comparison between the current I picture and the reference I picture. May be.

  In this way, the reference I picture to be compared with the current I picture does not change, and it is not necessary to update the I picture every time a frame image is input. However, since the background itself may change as described above, it is preferable to update the reference I picture stored in the frame memory 22 at some timing (for example, every time a predetermined period elapses).

  The VLC 18 sets a flag in the header part of the slice layer arranged below the I picture layer according to the position of the still block stored in the still block storage unit 25. This flag indicates the position of the still block detected by the still block detection unit 24.

  For example, as shown in FIG. 2, it is assumed that a square screen of 800 pixels in length and 800 pixels in width is divided so that 100 × 100 macroblocks of 8 × 8 pixels are arranged in a grid pattern. An image input from the image input unit 11 captures a dynamic subject V such as a person and a stationary background S, and an I picture includes the subject V and a stationary background S. And It is assumed that the dynamic subject V is moving and the background S basically remains stationary.

  In this case, the still block detection unit 24 detects a macro block corresponding to the region of the background S that is still as a still block, and the position is stored in the still block storage unit 25. In the I picture as shown in FIG. 2, the still blocks are MB100, MB200,. The background S may be hidden by the movement of the subject V or may appear again.

  The VLC 18 sets a flag indicating the position of a still block in the header part of the slice layer arranged below the picture layer of the I picture.

  FIG. 3 illustrates the data structure of the flag provided in the header part. As shown in FIG. 3A, the flag is composed of a data string having the same number of bits as the number of macroblocks, and each bit from the most significant bit to the least significant bit is determined in the scanning order of the macroblocks. Corresponds to the position of the macroblock. Here, the VLC 18 sets 0 for the bit corresponding to the position of the still block and 1 for the bit not corresponding to the position of the still block. That is, the position of the still block can be identified from the bit set to 0.

  Needless to say, if the total number of macro blocks dividing one image is small, the number of bits of the flag corresponding to the position of the macro block also decreases.

  The VLC 18 excludes static blocks from the slice layer. In the slice layer illustrated in FIG. 3B, the flag “0” is set in the upper 100 bits corresponding to the position of the still block MB100, and the MB100 data is excluded and missing. That is, macro block data other than still blocks for which the flag “0” is set and 1-bit flags are sent, while macro block data for still blocks are not sent, so the data capacity necessary for I picture transmission is reduced. Can save.

  In order to reproduce an I picture from the slice layer configured as described above, the following decoding apparatus is used.

  FIG. 4 is a block diagram of a decoding device 200 according to a preferred embodiment of the present invention. The decoding apparatus 200 includes a data input unit 51, a buffer memory 52, a variable length decoding circuit 53, an inverse quantization circuit 54, an inverse DCT circuit 55, a motion compensation circuit 56, a frame memory 57, an image output unit 58, and a flag detection unit 59. , An I picture storage unit 60, a still block complementing unit 61, and the like. The decoding device 200 includes a part of the configuration of a conventional MPEG-2 decoding device.

  When encoded moving image data sent from the encoding device 100 is input from the data input unit 51, the encoded moving image data is temporarily stored in the buffer memory 52 and then transferred to the variable length decoding circuit 53.

  The variable length decoding circuit 53 decodes the quantized data from the encoded data and outputs it to the inverse quantization circuit 54 and the flag detection unit 59.

  The inverse quantization circuit 54 inversely quantizes the quantized data from the variable length decoding circuit 53, and the inverse DCT circuit 55 performs inverse DCT transform on the inversely quantized data to generate inverse DCT data.

  The motion compensation circuit 56 reproduces image data of I, P, and B pictures using the inverse DCT data generated by the inverse DCT circuit 55 and transfers the image data to the frame memory 57. The frame image in the frame memory 57 is converted into the NTSC format or the like by the image output unit 58 and output to the image display device.

  The I picture reproduced by the motion compensation circuit 56 is also transferred to the I picture storage unit 60. The I picture storage unit 60 stores the transferred I picture. If an I picture has already been stored, the newly transferred I picture is updated and stored.

  The flag detection unit 59 according to the present invention identifies the slice layer of the I picture by the identification code included in the quantized data, and detects a flag indicating the position of the still block from the header part of the identified slice layer of the I picture. .

  The still block complementing unit 61 extracts a still block at the position indicated by the flag from the I picture stored in the I picture storage unit 60, and complements the missing static block in the slice layer of the I picture with the extracted still block. . That is, as shown in FIG. 3B, even if no still block exists in the slice layer of the I picture, it is completely supplemented by the still block stored in the I picture storage unit 60, and the P picture and B picture Playback is also possible.

Block diagram of encoding device The figure which shows the macroblock which comprises I picture The figure which shows notionally the structure of the slice layer of I picture Block diagram of decoding device

Explanation of symbols

11: input unit, 22: frame memory, 24: still block detection unit, 17: quantization circuit 18: VLC, 100: encoding device, 200: decoding device

Claims (10)

An input unit for inputting frame moving image data;
An I picture storage unit for storing an intra (I) picture constituting a certain frame moving image data input from the input unit as a reference I picture;
The reference I picture stored in the I picture storage unit is compared with the I picture constituting the current frame moving image input from the input unit, and there is almost no change from the block constituting the reference I picture. A still block detector for detecting a still block that is a block constituting the current I picture
A compression / encoding unit that compresses and encodes frame moving image data including a flag indicating the still block and data of a block other than the still block constituting the current I picture; and
A video encoding device comprising:   The moving image encoding apparatus according to claim 1, wherein the flag is set in a header part of a slice layer of the current I picture.   The moving image encoding apparatus according to claim 2, wherein the flag is 1-bit data indicating a position of the still block.   The moving image encoding apparatus according to claim 1, wherein the still block includes a macro block.   The moving picture coding apparatus according to claim 1, wherein the I picture storage unit periodically updates the reference I picture. An input unit for inputting encoded frame moving image data;
A decoding unit for decoding the encoded frame moving image data;
A reproducing unit for reproducing a frame moving image including an intra (I) picture from the decoded frame moving image data;
An I picture storage unit for storing an I picture included in the reproduced frame moving image data;
A flag detection unit for detecting a flag indicating a still block from the frame video data decoded by the decoding unit;
A still block complementing unit that complements a still block of an I picture of the input frame moving image data with a still block of an I picture stored in the I picture storage unit based on a flag indicating the still block;
A video decoding device comprising: Inputting frame moving image data;
Storing an intra (I) picture constituting a certain frame moving image data input from the input unit as a reference I picture;
The reference I picture stored in the I picture storage unit is compared with the I picture constituting the current frame moving image input from the input unit, and there is almost no change from the block constituting the reference I picture. Detecting still blocks that are blocks constituting the current I picture;
Compressing and encoding frame moving image data including a flag indicating the still block and data of a block other than the still block constituting the current I picture, and outputting the compressed data
A moving picture encoding method including: Inputting encoded frame moving image data;
Decoding the encoded frame video data;
Replaying a frame moving image including an intra (I) picture from the decoded frame moving image data;
Storing an I picture included in the reproduced frame moving image data;
Detecting a flag indicating a still block from the frame moving image data decoded by the decoding unit;
Complementing a still block of an I picture of the input frame moving image data with a still block of an I picture stored in the I picture storage unit based on a flag indicating the still block;
A moving picture decoding method including:   The program for making a computer perform the moving image encoding method of Claim 7.   A program for causing a computer to execute the moving picture decoding method according to claim 8.

Images