KR100335606B1 - Image encoder and/or decoder using recursive motion prediction/compensation - Google Patents

Abstract

PURPOSE: An image encoder and/or a decoder using a recursive motion prediction/compensation are provided to improve the efficiency of information compression and storing by selectively adjusting a computational complexity and amount of data according to information. CONSTITUTION: A motion compensation error frame memory(103) stores a computed motion compensation error signal at every time. A second motion compensator(104) compensates the stored motion compensation error once again by using a motion vector calculated by a first motion estimator(3). A subtracter(2) generates a differential signal by subtracting a motion compensated signal from a first motion compensator(10) from an original image signal of a current frame. A second estimator(101) estimates a motion for the differential signal by using the stored motion compensation error signal from the motion compensation error frame memory(103) to calculate a new motion vector. A mode selector(110) selects an optimal value capable of reducing amount of data.

Description

Image Encoder and / or Decoder Using Iterative Motion Prediction / Compensation

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image encoder and / or a decoder, and more particularly, to an image encoder and / or decoder capable of efficiently compressing and storing an amount of information by compressing an image signal using repetitive motion prediction and compensation.

Recently, video signal transmission and database construction have been shifted from analog to digital. Especially in the multimedia system environment, as the demand for integrated data that can be used with mutual compatibility in the fields of film, communication, and computers increases, a large amount of image data can be transferred through existing transmission lines. In order to transmit a large amount of data, a data processing technique capable of sending as much data as possible while having minimum quality deterioration is required. This is collectively referred to as a data compression technique.

Among the data compression technologies, especially the most efficient video compression and internationally compatible technology, HTU is a method that enables two-way video communication on a low transmission path (typically 64 Kbps or less) made by ITU-T SG15. There is also a standard method called .263 (aka TMN), also called WG11 (MPEG; Moving Picture Experts Group) under the Joint Technical Committee (JTC) 1 jointly established by the International Standardization Organization (ISO) and the International Electrotechnical Committee (IEC). ) Has created an international standard called MPEG-1 in digital storage media that can compress and store digital video and audio together in the 1.5 Mbps band. Furthermore, the MPEG-1 system has been improved, and the international standard of MPEG-2 in the 4 to 15 Mbps band has been enacted.

Although these technologies provide very good and high quality services, there is a limit to efficient compression and storage of information volume in order to simultaneously provide a large amount of information to meet various consumer demands in the information society of the future.

FIG. 1 illustrates a conventional video encoder.

As shown in FIG. 1, the input moving picture sequences 1 are applied to the first motion estimator 3 and the motion vector MV _{k +} through accurate motion estimation from the decoded previous frame 9 (P ′ _K ). ₁ ) The first motion compensator 10 performs motion compensation using the motion vector MV _{K + 1} to generate a signal P ′ _{K + 1} whose motion is compensated from the previous frame.

The subtractor 2 subtracts the motion compensation signal P ' _{K + 1} from the original video sequence ₁ to obtain an error signal EMC _{K + 1} according to the motion compensation. This error signal (EMC _{K + 1} ) is input to a converter (4) that concentrates important data components in one place, converted from the values in the spatial domain to the coefficients in the frequency domain, and then adapted to the capacity of the transmission path. It is quantized by the quantizer 5.

The quantized signal is subjected to lossless compression in consideration of statistical characteristics in the entropy encoder 11 together with the motion vector, and is transmitted to a decoder (not shown) and input to the dequantizer 6. The inverse quantizer 6 inversely quantizes the output signal of the entropy encoder 11 in the reverse order of quantization, and then inverse transformer 7 converts the frequency coefficient components into data in the spatial domain.

The adder 8 adds the motion compensation signal P ′ _{K + 1} and the output signal of the inverse transformer 7 and stores the decoded image for estimating the motion of the next frame in the frame memory 9. The operation of the video encoder for the frame is completed.

2 shows a video decoder used in the related art, respectively.

As shown in FIG. 2, the variable length coded signal transmitted from the encoder in the statistical manner is converted into a signal before variable length coding through the entropy decoder 21, and then the motion vector MV _{K + 1} and the image are obtained. Classify as data.

The image data is input to the inverse quantizer 22 and inversely quantized into a signal before quantization, and then input to the inverse transformer 23 to convert the frequency coefficient component into data EMC _{K + 1} in the spatial domain. At the same time, the first motion compensator 26, motion vectors (MV _{K + 1)} is the previous frame data restoration that has been input are stored in the frame memory (25) (P _'K) to the motion vector using the (MV _{K + 1} Generates a signal P ′ _{K + 1} compensated by

The adder 24 adds the motion compensation signal P ′ _{K + 1} and the output signal of the inverse transformer 23 to obtain a final reconstructed image 27 (RP _{K + 1} ) for the K + 1 th frame. The signal is stored in the frame memory 25 to be used as a reference image for reconstructing the K + 2th frame.

However, such a conventional method is known as a very efficient method for compressing and decompressing a moving image having a motion amount every frame, but a high quality cannot be provided in a transmission path requiring a very small bit amount or a medium for storing a lot of information. Because of the limitations, we need a way to compress more effectively.

SUMMARY OF THE INVENTION The present invention has been made in an effort to provide an image encoder and / or a decoder capable of efficiently compressing and storing an amount of information by compressing an image signal by repeatedly performing motion prediction and compensation.

1 is a block diagram showing the configuration of a video encoder used in the prior art.

2 is a block diagram showing the configuration of a video decoder used in the prior art.

3A and 3B are block diagrams illustrating the structure of an image encoder using iterative motion prediction / compensation according to the present invention.

4 is a block diagram showing the configuration of an image decoder using iterative motion prediction / compensation according to the present invention.

5 is a flowchart for explaining the operation of the present invention.

Fig. 6 is a diagram showing a configuration of a frame and a relationship with a block.

7 is a diagram illustrating a process of estimating motion.

8 shows a program syntax in the form of a pseudo program representing an operation required for the operation of the present invention.

9A to 9D are diagrams for comparing the amounts of error signals generated according to an embodiment of the present invention for a relatively simple input image.

10A to 9D are diagrams comparing amounts of error signals generated according to an exemplary embodiment of the present invention for a relatively complicated input image.

11a to 11d are graphs showing the results of experiments in the prior art and the technique according to the present invention.

In order to achieve the above object, an image encoder using repetitive motion prediction / compensation according to the present invention predicts a motion from a current frame and a previous frame among video signals input to be encoded, and compensates for the motion by using the motion prediction result. A motion error frame in which an error signal after compensation is transformed by using transform coding and then quantized and transmitted, and the sum signal is stored in a predetermined region of a frame memory and used as a reference image for subsequent frame processing. A motion compensator for repeatedly compensating a motion error signal from the motion vector obtained from the memory and the motion prediction result, obtaining a new motion vector for the motion compensation error signal of the current frame from the motion compensation error signal of the previous frame, This It can be used to compensate for the motion again, characterized in that the above process can be selected by the mode selector appropriate.

Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings.

Figure 3a is a block diagram showing the configuration of a video encoder to operate according to the present invention. As shown in FIG. 3A, the core of the image encoder according to the present invention is a portion 100 indicated by a dotted line, and compared to the conventional method, one motion estimator 101 and a local memory motion compensation (MC) error frame memory ( 103, two motion compensators 102 and 104 and a mode selector 110 are added.

4 is a block diagram showing the configuration of an image decoder operated by the present invention. As shown in FIG. 4, the core of the image decoder according to the present invention is a portion 200 indicated by a dotted line, and the MC error frame memory 201. Two motion compensators 202 and 203, a data selector 204 and one adder 205 are added.

First, as shown in FIG. 3A, the system according to the present invention additionally installs the MC error frame memory 103 in the conventional method shown in FIG. 1 to store the MC error signal SEMC _K calculated every time. A second motion compensator 104 is used to compensate this MC error signal once more using the motion vector MV _{K + 1} obtained by the one motion estimator 3. At the same time, a subtractor (2) by subtracting the first motion compensator 10, motion compensation is a signal (P _{'K + 1)} by the previously from the original image signal (P _{K + 1)} of the current frame difference signal ( EMC _{K + 1} ).

The second motion estimator 101 estimates the motion with respect to the difference signal EMC _{K + 1} by using the MC error signal SEMC _K input from the MC error frame memory 103 to generate a new motion vector for the MC error signal. Find (EMV _{K + 1} ). Next, an error signal after motion compensation is obtained through the third motion compensator 102 and input to the mode selector 110 to select an optimal value SEMC _{K + 1 that} can further reduce the amount of data.

3B is a block diagram showing the detailed configuration of the mode selector 110. The mode selector 110 receives the outputs of the subtractor 2, the second motion compensator 104, and the third motion compensator 102, and receives the data EMC _K1 and the second motion compensator from the subtractor 2. Data input from the 104 and third motion compensators 102 are subtracted by the first subtractor 111 and the second subtractor 112, respectively. Next, their outputs are input to the first complexity calculator 113 and the second complexity calculator 114 and simultaneously to the first selector 116 and the second selector 117. The minimum value determiner 115 determines a minimum value among the results of the first and second complexity calculators 113 and 114. According to the result of the minimum value determining unit 115, the first selector 116 selects one of the outputs of the first subtractor 111 and the second subtractor 112, outputs the output to the converter 4, and selects the second selector ( 117 selects one of the outputs of the second motion compensator 104 and the third motion compensator 102 and outputs it to the adder 105.

Meanwhile, the signal EMC ' _{K + 1} output to the adder 105 to correct the MC error output from the mode selector 110 is the signal P ′ _{K + 1} that was compensated by the first motion compensator 10. ) To generate a more accurate signal (P '' _{K + 1} ) with motion correction. The final error signal SEMC _{K + 1} after correcting the Mc error is output to the converter 4 and the quantizer 5 to produce a modified signal through the conversion process and the quantization process. The signal is again reproduced as a signal before entering the converter 4 through the inverse quantizer 6 and the inverse transformer 7 while simultaneously generating a signal to be finally transmitted or stored through the entropy encoder 11. The addition is performed with the final motion compensated signal P " _{K + 1} previously made in the adder 8, and the reproduced signal is stored in the frame memory 9 to produce the most optimal signal.

As shown in FIG. 4, the image decoder according to the present invention is provided with an MC error frame memory 201 in addition to the conventional configuration, according to the state of a flag provided in the data stream transmitted from the encoder. The data selector 204 selects one of the results of the second motion compensator 202 or the third motion compensator 203. The output signal of the data selector 204 is further added in the adder 25 to the signal compensated from the first motion compensator 26 in a conventional manner, and then the minimum error signal SEMC _{K + 1} of the encoder is added. The addition process in the adder 24 obtains the final reproduced signal 27 and at the same time stores it in the frame memory 25 to terminate the decoding process for one frame.

The operation principle of the present invention having the configuration as described above will be described in more detail with reference to FIGS.

FIG. 5 is a flowchart illustrating a process of performing the most effective data compression and decompression process shown by the present invention. FIGS. 6 and 7 are diagrams illustrating a process of performing a block unit operation, and FIG. Syntax of the bitstream configuration of the compressed data produced by the present invention is shown in pseudo code for each unit process.

First, in FIG. 5, after initializing to K = 0 (step 51), the K-th frame is read from the moving image sequence 1 (step 52; P _K ). After the operation on the K th frame is finished, the K + 1 th frame is read (step 53; P _{K + 1} ). Motion estimation is performed by selecting a search section of a predetermined area for each block using two frames (step 54). At this time, the definition of each frame and block is as shown in FIG.

As shown in FIG. 6, when the size of the frame 71 is M × N, each block (for example, the same unit as the portion 72 indicated by the comb pattern in the figure) is horizontal 73 and vertical in the frame. It consists of blocks having a size of several tens to hundreds of P × Q in the (74) direction.

7 is a diagram illustrating a process of estimating motion using blocks between frames. In FIG. 7, assuming that the large square on the left is the previous frame 81 (K) and the large square on the right is the current frame 82 (K + 1), the motion is estimated for any block 83 in the current frame. In this case, a predetermined search area in the previous frame is determined. In FIG. 7, an area 84 denoted by S _P × S _Q corresponds to this.

For each block, the search area is moved up, down, left, and right, and the absolute value of the difference between pixels is sequentially added. Then, the difference between the coordinate value of the minimum part and the coordinate value of the current block is defined as a motion vector. That is, if the block 83 in the current frame is the same as the position 85 indicated by the dotted line in the previous frame, the point 86 where the result of calculation for each search area is the minimum is 86 in gray. In this case, the X coordinate of the motion vector is MV _X (87), and the Y coordinate is MV _Y (88). The motion vector thus obtained is referred to as a motion vector MV _{K + 1} as shown in FIG. 5 (step 55).

The motion compensation is performed on the K + 1 th current frame using this motion vector. This is performed by the first motion compensator 10 in FIG. 3A. The signal whose motion is compensated for is referred to as P ′ _{K + 1} (step 56). Next, calculate the current frame is a K + 1 th frame and the motion compensated signal (P _{'K + 1)} with the error signal (EMC _{K + 1)} (step 57).

The conventional scheme ends the processing for one frame by transmitting the motion vector MV _{K + 1} and the error signal EMC _{K + 1} . However, the present invention is characterized by determining whether the K value is greater than 1 (step 59), that is, performing recursive motion compensation from the third frame. This is the portion 300 indicated by dotted lines in FIG. 5, which can be referred to as a core part of the present invention.

First, the motion compensation for the MC error signal of the current frame is performed using the MC error signal for the previous frame stored in the MC error frame memory 103 in FIG. 3A by using the motion vector MV _{K + 1} . In step 301, a motion compensated signal EMC ' _{K + 1} is obtained. In this way, the motion-compensated signal EMC ' _{K + 1} to the error signal is added to the motion-compensated signal P' _{K + 1} through the first motion compensator 10 in FIG. The compensation signal P '' _{K + 1} is obtained (step 302).

Next, the final motion-compensated in the _{(K + 1} signals (P current frame signal P) "" _{K + 1)} is obtained by subtracting the compensation error (DEMC _{K + 1)} (The 303 step). Among the obtained DEMC _{K + 1} and EMC ' _{K + 1} , the _one having the smallest error is selected (step 304; SEMC _{K + 1} ). Another feature of the present invention is that the process of step 304 may be omitted as necessary.

The final selected compensation error (SEMC _{K + 1} ) obtained is transform-coded, quantized, and transmitted together with the motion vector (MV _{K + 1} ) and mode information to complete processing for one frame, and then the K value is the last frame (last frame) is reached (step 60). If the processing is completed until the last frame (step 61), if the frame to be processed remains, the process moves to the process of reading the next frame (step 53).

8 shows the bit stream syntax of the compressed signal produced after performing the above process. The part to be added by the present invention is indicated by a dotted line. That is, it is stored in two bits of data called Prediction_Type_Code (91), which determines whether the WPS predictive method will use the conventional method or the special effect of the present invention. .

Prediction_Type_Code Description

00 conventional method

01 repetitive motion compensation

10 repetitive movement predictions

11 Reserved

That is, if it is '00', the conventional method is used as it is, and if it is '01', the motion vector obtained for the original image is applied to the MC error signal as it is, and only repetitive motion compensation is added. New motion prediction for MC error signal is performed to obtain additional motion vector and then compensate for motion by this motion vector. In case of '11', it is reserved for additional use. not. If the Prediction_Type_Code is '00', the portion 92 indicated by the dotted line is omitted, and in the case of '01', the operation principle has been described in detail above. This completes the detailed operation of the video encoder according to the present invention.

The decoder classifies the data coming from the entropy decoder 21 as shown in FIG. 4, and the final MC error signal is converted into the original signal through the inverse quantizer 22 and the inverse transformer 23. The result is processed and added to adder 24 to produce the final reconstructed signal. The processing unit according to the present invention is a portion 200 indicated by a dotted line. First, the error signal of the previous frame is stored in the MC error frame memory 201, and the second motion compensator 202 according to the mode information of the current incoming bit stream. The error signal is compensated using the motion vector obtained by the motion prediction of the original image. In addition, the third motion compensator 203 compensates for the motion by using the motion vector for the newly transmitted error signal. In the data selector 204, one of the outputs of the second motion compensator 202 and the third motion compensator 203 is selected by the mode specified in the bit stream, and the output of the data selector 204 is added by the adder 205. In a conventional manner, the first motion compensator 26 adds the compensated signal of the original image calculated in advance and inputs it to another adder 24.

9A to 9D show examples of one specific frame among moving images, in particular, an input image having a relatively simple form. FIG. 9A shows an original image and FIG. 9B shows a case where a conventional method is applied. 9c shows the amount of error signal, and FIG. 9c shows the amount of error signal when repetitive motion compensation is performed for the error signal in the present invention. FIG. 9d shows a motion vector by newly estimating a motion with respect to the error signal. Calculate and then show the amount of error when motion compensation is performed once more. The black ones in the figure indicate the amount of error, and it can be seen that the effect of the present invention is excellent.

10A to 10D show an example of a specific frame among moving images, in particular, an input image having a relatively complicated shape. FIG. 10A shows an original image and FIG. 10B shows a case where a conventional method is applied. Figure 10c shows the amount of the error signal, Figure 10c shows the amount of the error signal when the repetitive motion compensation for the error signal in the present invention, Figure 10d is a motion vector by estimating the new motion with respect to the error signal After calculating, we show the amount of error when the motion compensation is performed once more. As can be seen in Figures 10b to 10d it can be seen that the amount of error is much higher. However, it can be seen that the tendency to gradually decrease is the same as in FIG.

11A to 11D show statistical results obtained when the present invention and the conventional method are applied to 10 frames, and FIG. 11A shows the amount of bits generated after each frame process. From the third frame to which the invention is applied, it can be seen that the bit amount is significantly reduced, so that only about 50% occurs. In addition, Figure 11b shows the peak SNR (PSNR) for the luminance component, it can be seen that the proposed method is slightly dropped, but the big difference is not seen, similarly to the chrominance component (Chrominance) of Figures 11c and 11d It can be seen that for PSNR, there is no big difference.

The present invention is not limited to the above embodiments, and it is apparent that many modifications are possible by one of ordinary skill in the art within the technical idea of the present invention.

According to an image encoder and / or decoder using iterative motion prediction / compensation according to the present invention, only optimal data can be compressed when a large amount of information is efficiently transmitted in a small amount. In addition, it is possible to selectively adjust the complexity of data amount adjustment and calculation as necessary according to all the information. In addition, it is designed to reduce the amount of bits as much as possible by adding only the minimum necessary circuit. In addition, the existing method can be used as it is, and in particular, it is designed to maintain a small bit amount in the highest quality for use in computer networks, wireless communication networks, and the like.

Claims (7)

Predict the motion from the current frame and the previous frame among the video signals input for encoding, compensate for the motion using the motion prediction result, convert the error signal after the compensation using transform coding, and then quantize and transmit the same. In the image encoder that is added to the motion-compensated signal and stored in a predetermined region of the frame memory to use as a reference image of the next frame processing, A motion compensator for repeatedly compensating the motion error signal from the motion vector obtained from the motion prediction frame memory and the motion prediction result, and a new motion vector for the motion compensation error signal of the current frame from the motion compensation error signal of the previous frame. The image encoder using iterative motion prediction / compensation can be used to compensate for the motion, and then select the appropriate ones by using a mode selector. The method of claim 1, wherein in repeatedly compensating for the motion compensation error signal, the error signal of the current frame is repeatedly compensated by using the motion vector obtained by the motion prediction of the original image as it is, and then the motion compensation is more accurately used. And an image encoder using iterative motion prediction / compensation, which minimizes a compensation error. 2. The method of claim 1, wherein in repeatedly compensating for the motion compensation error signal, the motion signal for the error signal of the current frame is newly executed by using the error signal stored by the previous frame and the error signal is obtained by the motion vector. An image encoder using iterative motion prediction / compensation, characterized in that it can be repeatedly compensated. 2. The image encoder according to claim 1, further comprising a mode selector for selecting an optimal state and designed to select to generate a minimum amount of bits. The image encoder according to claim 4, wherein the minimum value selection method calculates the complexity of the error signal and selects a low complexity one. An image decoder using iterative motion prediction / compensation for decoding a bitstream encoded by an image encoder such as claim 1. Predict the motion from the current frame and the previous frame among the video signals input for encoding, compensate for the motion using the motion prediction result, convert the error signal after the compensation using transform coding, and then quantize and transmit the same. The motion compensation signal is stored in a predetermined area of the frame memory and used as a reference image for the next frame processing, and includes a motion error frame memory and a motion compensator for repeatedly compensating the motion error signal from the motion vector obtained from the motion prediction result. By obtaining a new motion vector for the motion compensation error signal of the current frame from the motion compensation error signal of the previous frame, the motion compensation can be compensated again by using the same. So that Image encoder; And And an image decoder for decoding the bitstream encoded by the image encoder by iterative motion prediction / compensation.