DE102011100498A1 - Method for coding signal i.e. video signal, for generating binary data stream for transmitting e.g. video data, involves carrying out prediction of transformation coefficients of blocks based on quantized transformation coefficients - Google Patents

Abstract

The method involves dividing a signal i.e. image signal (1), into signal portions, where each portion is divided into blocks of defined size. Each block is transformed by a discrete signal transformation, and transformation coefficients of the signal transformation are determined. A prediction of transformation coefficients of the blocks is executed based on quantized transformation coefficients determined for the blocks. Parameters of prediction procedure are transmitted from an encoder (2) to a decoder (4) over a transmission channel (3). Independent claims are also included for the following: (1) a method for decoding a binary data stream (2) an encoder for generating a binary data stream (3) a decoder for decoding a binary data stream.

Description

The invention relates to a method for coding a signal according to claim 1. The invention further relates to a method for decoding a binary data stream according to claim 13. Further, the invention relates to an encoder for generating a binary data stream according to claim 14 and a decoder for decoding of a binary data stream according to claim 15. The invention further relates to a binary data stream according to claim 16.

In general, the invention relates to the digital signal transmission, as z. B. is used for transmission of video and audio data. Currently used hybrid video coder, such. B. H.264 / AVC, use various intra- and inter-prediction methods to determine spatial and temporal dependencies between pixels or blocks of pixels and to use for an increase in efficiency of data transmission. The results of the prediction are subtracted from the original signal, and only the remainder, the so-called residual, is transmitted. This reduces the bandwidth required to transmit the video information as compared to transmitting the full original information.

Such a method is for example from WO 2007/079964 A1 or the EP 0 831 660 B1 known.

The invention has for its object to further improve the signal coding, so that even more bandwidth can be saved.

This object is achieved by a method for coding a signal which is divided into signal sections, wherein each signal section is divided into blocks of defined size and each block is transformed by means of a discrete signal transformation, wherein transformation coefficients of the discrete signal transformation are determined, wherein a prediction at least a transformation coefficient of a block in dependence on one or more for the same block already determined and possibly quantized transformation coefficients is performed. By means of the invention, it is advantageously possible to use spatial dependencies within a block in order to carry out a block-internal prediction of transformation coefficients and, as a result, to further compress the information to be transmitted, so that less bandwidth is required for the transmission. As a result, in particular the block-internal linear or nonlinear dependencies generated by the transformation method used are exploited, the z. B: reflect in correlations or covariances and z. B. in pairs between two transformation coefficients can be measured.

The inventive method is suitable for coding signals of all kinds, in particular for video and audio signals. The invention is particularly suitable for hybrid video coding methods. As a discrete signal transformation z. The discrete cosine transform, a hair transform, an integer transform as in H.264 / AVC or the discrete Fourier transform. In the field of video transmission, the discrete cosine transformation (DCT) is preferably used as the discrete signal transformation. In this transformation method, a gain of 0.5 dB can be achieved with the block-internal prediction according to the invention. In other words, with regard to the data rate to be transmitted, a savings of about 20% can be achieved, if not particularly unfavorable data constellations exist.

Compared with known prediction methods, it is therefore not a prediction between different blocks that is proposed, but within one and the same block. In addition, the method according to the invention can advantageously be combined with known, block-overlapping prediction methods in order to achieve the best possible overall data compression in this way.

As a signal section is z. A single video image, a video sequence, d. H. a sequence of video images, a frame, a slice or a GOP (Group of pictures according to MPEG definition) understood. As a block z. For example, an image section of 8x8 or 16x16 pixels may be defined, or any other suitable block size.

According to an advantageous development of the invention, the prediction is carried out as a function of covariances between the transformation coefficients at the corresponding positions in the block.

According to an advantageous development of the invention, the prediction comprises a determination of prediction coefficients, one prediction coefficient each being associated with a pair of transformation coefficients.

According to an advantageous development of the invention, a determination of prediction coefficients takes place in the process of quantization of the transformation coefficients, wherein already quantized transformation coefficients are used for the prediction. In this case, quantized transformation coefficients of the same block or of previously coded blocks can be used. For the prediction additional encoding parameters can be used in addition, such. As the block size of the current and the surrounding blocks, as well as other information such as autocorrelation functions of adjacent residual data or results of a statistical evaluation of the signal to be encoded.

According to an advantageous development of the invention, scanning of the transformation coefficients is provided. The purpose of the scanning is to convert the two-dimensional transform coefficients present after the transformation into a one-dimensional structure, which is subsequently coded for transmission over the transmission channel. The scanning can z. B. according to the known zig-zag scheme, as in 4 shown.

According to an advantageous development of the invention, the prediction and a quantization of the transformation coefficients are carried out before an ordered read in of the transformation coefficients. As a result, the prediction and the quantization become independent of the scanning order in the subsequent lossless coding and transmission of the transformation coefficients.

According to an advantageous development of the invention, the prediction and a quantization of the transformation coefficients take place after a scan of the transformation coefficients. This has the advantage that a prediction of the transformation coefficients can take place successively, in each case after previous transformation coefficients have been quantized. This has the advantage that in the prediction on already obtained information of the currently processed block can be used and the efficiency of the prediction can thus be improved wertwert.

According to an advantageous development of the invention, more than one prediction scheme and / or a variable prediction scheme with variable parameters is provided. In this case, the prediction scheme used for the prediction of the transformation coefficients is called the prediction scheme. Through the use of multiple prediction schemes or a variable prediction scheme, the prediction can be flexibly adapted to the characteristics of the signal to be coded and to the block size.

According to an advantageous development of the invention, the prediction scheme to be used or the parameters of the prediction scheme to be used are transmitted from the encoder to the decoder via the transmission channel. This provides the decoder with the information needed to decode the received bitstream.

According to an advantageous development of the invention, the prediction scheme to be used or the parameters of the prediction scheme to be used are automatically determined in the decoder on the basis of already decoded data. This has the advantage that no information about the prediction scheme or the prediction parameters has to be transmitted via the transmission channel, so that the bandwidth required for this purpose can also be saved. For the determination of the applicable prediction scheme or the parameters to be used in the decoder z. B. the information from already decoded data such as adjacent blocks, previously decoded transform coefficients of the same block and previously decoded frames are used. So z. For example, the decoder determines autocorrelation functions based on the residuals of adjacent blocks, and from this automatically selects a prediction scheme to be used from a plurality of predefined prediction schemes or determines the parameters of the prediction scheme. It is also possible, in the case of the successive prediction of the transformation coefficients, to use the already defined order of scanning the transformation coefficients for determining the prediction scheme or the parameters of the prediction scheme.

According to an advantageous development of the invention, information about which temporal and / or local prediction was used before the transformation is transmitted from the encoder to the decoder via the transmission channel or obtained automatically in the decoder on the basis of already decoded data. This has the advantage that, with the additional use of further prediction methods, on a prediction between different blocks, corresponding parameters of the cross-block prediction in the decoder are available.

According to an advantageous development of the invention, fixed coefficients which are dependent on the position in the block are used for the prediction.

According to an advantageous embodiment of the invention, the prediction is carried out with coefficients which are calculated in the encoder in each case for a signal section and transmitted as side information to the decoder.

According to an advantageous development of the invention, only one prediction coefficient is used for the prediction of a transformation coefficient, the position of which is selected as a function of preceding processing mechanisms. The bit stream to be transmitted from encoder to decoder can thus be further reduced by reducing the necessary page information.

The object mentioned at the outset is also achieved by an encoder for generating a binary data stream, which is set up to carry out a method of the type described above.

The object is further achieved by a binary data stream generated by a method of the kind described above.

The object is also achieved by a method for decoding a binary data stream which has been generated by a method according to one of the preceding claims, with a reconstruction of the transformation coefficients depending on the prediction carried out during coding, in particular by means of already decoded transform coefficients of respectively assigned prediction coefficients.

The object is further achieved by a decoder for decoding a binary data stream, which is set up to carry out the aforementioned method.

The invention will be explained in more detail by means of embodiments using drawings.

Show it:

1 a video signal transmission over a transmission channel and

2 a first embodiment of an encoder and

3 a second embodiment of an encoder and

4 a scan order and

5 a predictor with different prediction schemes and

6 an embodiment of a decoder.

In the figures, like reference numerals are used for corresponding elements.

The 1 shows a video signal 1 , z. B. an image consisting of a two-dimensional image pixel array, the an encoder 2 is supplied. The encoder 2 performs a video coding of the image signal 1 by the method according to the invention. The one from the encoder 2 generated binary data stream is transmitted through a transmission channel 3 to a decoder 4 transfer. The decoder performs a decoding of the binary data stream and outputs a reconstructed image signal 5 from.

The 2 shows the relevant to the explanation of the invention details of an encoder 2 according to a first embodiment. A transformation unit 7 become two-dimensional image blocks 6 fed. The picture blocks 6 can z. B. pixel blocks of defined size, z. B. 8 × 8, from the video signal 1 be. The transformation unit 7 supplied image blocks 6 can also be pre-processed in addition, z. B. by a cross-block prediction. In this case, from the preceding prediction, the data of the residuum are used as image blocks 6 fed.

The transformation unit 7 performs a discrete signal transformation, e.g. A DCT. Output variable of the transformation unit 7 is a block of transform coefficients that usually has the same block size as the injected image block 6 , However, on the output side, the transformation coefficients are present in the block.

The transformation coefficients are over a summer 8th in which the with a prediction unit 15 certain prediction is subtracted from the transform coefficients, a quantization unit 9 fed. In the quantization unit 9 there is a quantization of the fed data. Quantization reduces the irrelevance of the data on the basis of a human psycho-visual model. This is based on the finding that higher frequency components are recognized by the eye substantially worse than low-frequency components of the image, so that a higher restriction of the value range can be made for the higher-frequency components. In quantization, every value that enters the quantization unit 9 is divided by an associated quantization value. This results in a restriction of the value range. For the quantization values z. For example, a standard quantization matrix can be used, as already used in known video encodings.

The of the quantization unit 9 generated data are on the one hand via a summer 14 to which the output values of the prediction unit 15 additionally supplied, the prediction unit 15 supplied as input data. The prediction unit 15 performs a prediction of the transform coefficients for the respective processed image block 6 by. The details of the prediction carried out there will be explained in more detail below. In the prediction are in the prediction unit 15 additional data 16 considered, such. B. the block size of the image block 6 ,

The output data of the quantization unit 9 become a scanning unit 10 fed. In the scan unit 10 a scan of the supplied data takes place. Scanning has the purpose of converting the data supplied as a two-dimensional data block (matrix) into a one-dimensional structure (vector). Scanning in the scan unit 10 can z. B. such that the supplied data, each frequency components of the original image block 6 represent, be sorted according to their frequency and in this order as a vector 11 an encoding unit 12 be supplied. The coding unit 12 can z. B. perform entropy coding, z. B. in the manner of run-length coding. The coding unit 12 outputs the binary data stream on the output side 13 from.

According to 2 Thus, a prediction of transformation coefficients on the basis of an image block is performed 6 , where the prediction unit 15 already quantized transformation coefficients of previously coded blocks and other parameters 16 can use. The prediction is done interleaved with the quantization on the basis of two-dimensional data blocks.

The 3 shows an alternative embodiment of the relevant parts of the invention for an encoder, in which the prediction of the transformation coefficients takes place in successive manner. Significant difference to the embodiment of the 2 is doing that scanning in the scan unit 10 already done before the prediction and quantization, ie the scan unit 10 scans directly from the transformation unit 7 given, yet unmodified transformation coefficients. As in the embodiment according to 2 is also in the embodiment according to 3 provided that the scanning unit 10 a one-dimensional data structure in the form of a vector 11 emits. The processing of the vector 11 in the quantization unit 9 as well as in the prediction unit 15 takes place in a comparable manner as in the embodiment according to 2 However, due to the one-dimensional data structure, the prediction of the transformation coefficients can take place successively if preceding transformation coefficients have already been quantized. In this way, the prediction unit can use information already quantized transformation coefficients for further prediction.

The scan unit 10 has a scan order. The scan order describes how the two-dimensional transform coefficient matrix is mapped into a one-dimensional vector form. A set of several different scan orders can be used. One of the possible scan sequences is in the 4 displayed. This is the zigzag scan order in this example.

The scan order used can be used for data transmission over the transmission channel 3 once for the entire sequence, or for a frame, slice or macroblock level become. The decoder can decide which scan order to apply, even without appropriate information about the broadcast channel 3 be transmitted by using already decoded data such as adjacent blocks and previously decoded frames. So z. For example, the autocorrelation functions of the adjacent block residuals are computed, processed, and a scan order selected therefrom.

The 5 shows a further embodiment of the prediction unit 15 in which not a single, fixed prediction scheme is provided, but rather several prediction schemes 19 . 20 . 21 that have a selector 18 can be selected. A control unit of the prediction unit 15 controls the selector 18 in such a way that a prediction scheme which is as efficient as possible for the data to be coded is selected. The number of available prediction schemes 19 . 20 . 21 can be set according to need and use case.

In a further embodiment, the prediction unit 15 also have a prediction scheme with variable parameters a, b, c .... In this case, appropriate sets of parameters for respective coding cases are selected by a control of the prediction unit.

The 6 shows by way of example the parts of a decoder which are relevant for the description of the invention 4 , where the in 6 shown construction a counterpart to the embodiment of an encoder according to 2 represents.

The over the transmission channel 3 received binary data stream 22 becomes a decoding unit 23 supplied, which performs an entropy decoding. This will output data in vector form to a distribution unit 24 issued. The distribution unit 24 represents the counterpart to the scan unit, so to speak 10 through the distribution unit 24 the one-dimensional data is converted into a two-dimensional matrix form, taking into account the scan order used on the encoder side. The from the distribution unit 24 delivered two-dimensional data blocks are via a summer 25 in the output data of a reconstruction unit 30 be added. The resulting data becomes a scaling unit 26 fed. The scaling unit 26 performs the inverse function of quantization and thus provides a counterpart to the quantization unit 9 dar. The reconstruction unit 30 performs the inverse function for prediction and thus provides a counterpart to the prediction unit 15 dar. The reconstruction unit 30 can, similar to the prediction unit 15 , additional data 31 use. The reconstruction unit 30 receives as input data the output data of the scaling unit 26 of which in a summer 29 the initial data of the reconstruction unit 30 be subtracted.

The output data of the scaling unit 26 also become an inverse transformation unit 27 carried out. The inverse transformation unit 27 performs the inverse discrete signal transformation to that of the transformation unit 7 performed transformation, ie, for example, an inverse discrete cosine transformation. From the inverse transformation unit 27 becomes an image block 28 issued.

The prediction in prediction unit 15 can for example be carried out as follows: It is assumed that the in the publication of S. Klomp, M. Munderloh, J. Ostermann, "Block size dependent error model for motion compensation", Proceedings ICIP 2010 Differences of the variances described by multiplying a signal of constant variance r by a window function w, arise: x _n = r _n w _n (1)

und u_k = √ 1/2 für k = 0 und u_k = 1 in allen übrigen Fällen ist.As a result, correlations are generated in the output data X _{k of} the DCT. The following example assumes that the frequency domain correlations are determined for a one-dimensional discrete cosine transform of length N, where the spectral coefficients are:

and u _k = √ 1.2 for k = 0 and u _k = 1 in all other cases.

Since discrete Fourier transform (DFT) is related to normalization and symmetric expansion with respect to the behavior in the time and frequency domain, this will be used in the following to investigate the behavior of the discrete cosine transformation. For this purpose, a symmetrical extension is first to be carried out, which is subsequently identified by a tilde: n = x ~ _2N-1-n = x _n, 0 ≤ n ≤ n ≤ N - 1 (3) With x ~ _n = r _n w _n , r _n = r _2N-1-n = r _n, w _n = w _2N-1-n = w _n

In a further step, a DFT is performed whose basic function has a time offset of half a sampling interval, which is also referred to as odd-time DFT, or SDFT for short. The SDFT is 2N long:

This leads to the following relationship of the coefficients:

Symmetry relationships in the SDFT basis function follow: X ~ _k = X ~ _{4N + k} = X ~ _4N-k , X ~ _{2N + k} = -X ~ _k X ~ _N = X ~ ₃ N = 0

Therefore, X ~ _k can be determined as a 4N cyclic convolution in the frequency domain: X ~ _k = R ~ _K · W ~ _K. (6)

mit

Periodicities of odd-timBy considering the symmetries and e DFT basis functions, the formula for X ~ _{k is} :

With

In matrix notation this can be expressed as follows: A vector R ~ contains the first N SDFT coefficients of r ~ _p . An N × N matrix Q contains the elements q _{k, l} and thus describes the influence of w ~ _n . This can be expressed as X ~ = 1 / 2NQR ~. (8th)

The covariance matrix can now be determined as follows:

sind.For uncorrelated spectral _coefficients R ~ _k , C _{RR is} a diagonal matrix with the elements d _k = E {R ~ 2 / k} (10) while the elements of the covariance matrix

are.

In order to obtain the covariance matrix from the output data X _{k of} the DCT, the normalization has to be considered, which leads to the following form of equation (11):

Using the error distribution w, the coefficients are now predicted by values within the same block, in contrast to prior art prediction methods that use adjacent blocks. It can thereby be achieved a redundancy reduction by a backward prediction, d. H. a prediction of already quantized values, resulting in an uncorrelated white quantization noise. For the special case of a symmetric weighting function, in addition, due to the special features of the DCT, the correlations for all odd index differences are zero.

Such a symmetric weighting function is z. In the aforementioned publication by Klomp, Munderloh and Ostermann. In this case, for the sake of simplicity, it is permissible to consider only the coefficients with even index differences for the prediction. An example with three prediction coefficients is in the 4 represented, wherein here a two-dimensional arrangement of the coefficients is assumed. The prediction for a coefficient X _{k, l} , which in the 4 is called O is X ^ _{k, l} = a _{k, l} x _{k-2, l} + b _{k, l} X _{k, l-2} + c _{k, l} X _{k-2, l-2} (13) for k, l ≥ 2. Here, a, b and c are the prediction coefficients determined by solving the following system of equations.

Here, φ _{(k, l), (i, j) describes} the covariance E {X _{k, l} X _{i, j} }. For borderline cases with k <2 and l <2, the corresponding cross terms are neglected, for example. As can be seen, the coefficient O is thus determined taking into account the already determined coefficients X _a , X _b , X _c . Should a different number of prediction coefficients be required to predict the transformation coefficient, the calculation should be adjusted accordingly, paying attention to how many of the surrounding values are already available for prediction.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• WO 2007/079964 A1 [0003]
• EP 0831660 B1 [0003]

Cited non-patent literature

• S. Klomp, M. Munderloh, J. Ostermann, "Block size dependent error model for motion compensation", Proceedings ICIP 2010 [0050]

Claims (16)

Method for coding a signal which is subdivided into signal sections, wherein each signal section is divided into blocks of defined size and each block is transformed by means of a discrete signal transformation, wherein transformation coefficients of the discrete signal transformation are determined, characterized in that a prediction of at least one transformation coefficient of a block depending on one or more for the same block already determined and possibly quantized transformation coefficients is performed. A method according to claim 1, characterized in that the prediction is carried out in dependence on covariances between the transformation coefficients at the corresponding positions in the block. A method according to claim 1 or 2, characterized in that the prediction comprises a determination of prediction coefficients, wherein in each case a prediction coefficient is associated with a pair of transformation coefficients. Method according to one of the preceding claims, characterized in that a determination of prediction coefficients in the process of quantization of the transformation coefficients, wherein for the prediction already quantized transformation coefficients are used. Method according to one of the preceding claims, characterized in that the prediction and a quantization of the transformation coefficients takes place before an ordered read-in and is thus sequence-independent. Method according to one of Claims 1 to 4, characterized in that the prediction and a quantization of the transformation coefficients takes place after a scan of the transformation coefficients. Method according to one of the preceding claims, characterized in that more than one prediction scheme and / or a variable prediction scheme with variable parameters is provided. Method according to Claim 7, characterized in that the prediction scheme to be used or the parameters of the prediction scheme to be used are transmitted from the encoder to the decoder via the transmission channel. Method according to Claim 7, characterized in that in the decoder the prediction scheme to be used or the parameters of the prediction scheme to be used are automatically determined on the basis of already decoded data. Method according to one of the preceding claims, characterized in that information about which temporal and / or local prediction was applied before the transformation is transmitted from the encoder to the decoder via the transmission channel or is automatically obtained in the decoder based on already decoded data. Method according to one of the preceding claims, characterized in that fixed coefficients which are dependent on the position in the block are used for the prediction. Method according to one of the preceding claims, characterized in that prediction takes place with coefficients which are calculated in the encoder in each case for a signal section and transmitted as side information to the decoder. Method for decoding a binary data stream which has been generated by a method according to one of the preceding claims, characterized in that a reconstruction of the transformation coefficients takes place as a function of the prediction performed during the coding, in particular by means of already decoded transform coefficients of respective assigned prediction coefficients. Encoder for generating a binary data stream, which is set up to carry out a method according to one of Claims 1 to 12. A decoder for decoding a binary data stream, which is set up to carry out a method according to claim 13. A binary data stream generated by a method according to any one of claims 1 to 12.

Images