CN108769696A - A kind of DVC-HEVC video transcoding methods based on Fisher discriminates - Google Patents

Abstract

The present invention provides a DVC-HEVC transcoding method and a transcoder based on a prediction residual, and mainly relates to the division of HEVC encoding methods in the transcoder. In the encoding method division, the prediction residual obtained by subtracting the side information of the WZ frame generated by the DVC decoder from the current HEVC coded frame (that is, the reconstructed frame of the WZ frame) is used to quickly determine the CU Split mode, thereby skipping the complex layer-by-layer rate-distortion optimization process in the HEVC encoding module. The present invention performs conditional judgment according to the relationship between the dispersion of the current prediction residual and the threshold, determines whether to continue CU division, and quickly determines the CU block mode, thereby effectively reducing the computational complexity of the encoding end. Experimental results show that the present invention greatly reduces the encoding time compared with the HM16.5 standard method under the condition that the encoding efficiency and the peak signal-to-noise ratio (PSNR) loss are small.

Description

A DVC-HEVC video transcoding method based on Fisher discriminant

technical field

The present invention relates to the technical problem of video transcoding in the field of image communication, in particular to a video transcoding technology between Fisher discriminant-based Distributed Video Coding (DVC) and HEVC standard.

Background technique

With the rapid development of digital multimedia technology, the popularization of high-definition video and the emergence of ultra-high-definition format (4K×2K or 8K×4K resolution), new video communication has been widely used, such as wireless surveillance video, mobile video call, Video conferencing, mobile remote on-site command, etc., the growth rate of video data volume has far exceeded the development speed of network and storage technology. At present, the widely used H.264/AVC standard in the video field can hardly meet the further video application requirements.

In 2010, MPEG and VCEG established the Joint Video Coding Collaboration Team (JCT-VC), and successfully developed a new generation of video coding standard - High Efficiency Video Coding (HEVC). HEVC adopts the method of prediction and transformation at the encoding end, which improves the compression efficiency by about one time compared with H.264, and at the same time brings a sharp increase in the complexity of the encoder. Distributed Video Coding (DVC) technology transfers the process of mining video correlation from the encoding end to the decoding end, realizing low-complexity and low-power video communication at the encoding end, and has been increasingly recognized by scholars at home and abroad. more extensive attention. However, the complexity of the decoding end is relatively high, and there is still a certain gap between the encoding performance and the traditional video encoding standards. Therefore, neither of the above two encoding schemes can well meet the requirements of low-complexity and low-power video communication between mobile terminal devices. The solution based on DVC-HEVC transcoding can effectively combine the advantages of both, and provide an effective solution to the above existing problems.

At present, the academia has carried out a lot of work on transcoding DVC to traditional video standards, and most of them have made relatively good progress. Alberto Corrales-García et al. realized the acceleration of the mode selection algorithm based on the fact that the block size of the macroblock during H.264 encoding has a high similarity with the residual distribution of the side information of the DVC decoding end. J.L.Martinez et al. use the motion vector generated during DVC decoding to reduce the computational complexity in the motion estimation process during H.264 encoding. Based on the advantages of cloud computing, Rong Song et al. proposed a low-complexity end-to-end video communication system based on cloud transcoding. The above methods can accelerate the DVC-H.264 transcoding process to a certain extent, but the research on DVC-HEVC transcoding is still in its preliminary stage.

Contents of the invention

The purpose of the present invention is to speed up the HEVC encoding process in a DVC-HEVC transcoder. The present invention utilizes the relationship between feature information generated during the DVC decoding process and the CU partition mode to establish a CU fast partition model, and proposes a DVC-HEVC video transcoding method based on Fisher discriminant formula. Compared with the traditional cascaded transcoding scheme, the method of the present invention can greatly reduce the computational complexity of HEVC encoding in the transcoder with little loss in coding efficiency and peak signal-to-noise ratio.

The basic idea of the present invention is to use the side information generated by the DVC decoder as the predicted frame of the current HEVC encoded frame (that is, the reconstructed frame of the WZ frame), and the residual of the two is the predicted residual, combined with the texture complexity of the frame and The average motion vector, the relationship between the three features and the CU partition mode is obtained through machine learning, and a classifier model based on Fisher discriminant analysis is established, skipping the complex layer-by-layer rate-distortion optimization process in the HEVC encoding module , so as to quickly determine the CU partition mode and achieve the purpose of reducing the computational complexity.

Generally speaking, the texture complexity and motion intensity of the image sequence have a great correlation with the CU partition mode in HEVC coding. In the HEVC encoding process, the division of coding units in areas with more complex textures and areas with more obvious motion is smaller, while the division of coding units in areas with simpler textures and smoother motions is larger. In addition, due to the very high correlation between the images of adjacent frames in the video sequence, the predicted image of the current frame to be decoded can be obtained through motion estimation and motion compensation on the basis of adjacent decoded frames, and the prediction residual is The difference between the predicted image and the current frame to be encoded. For the static area or the gentle motion area in the video image, the prediction residual coefficient is small and the fluctuation range is small, while for the severe motion area, the prediction residual coefficient tends to fluctuate greatly. In general, the size and distribution of prediction residual values are also highly correlated with the CU partition mode in HEVC coding. The CU blocks in the image area with small prediction residuals are larger, and the CU blocks with large prediction residuals and uneven distribution are smaller. Therefore, the prediction residual obtained during the DVC decoding process can be combined with the three characteristics of texture complexity and motion vector as the basis for the CU partition mode.

The distribution of the predicted residual image can be measured by the degree of difference (that is, the degree of dispersion) of the residual data, and the present invention selects the standard deviation of the data as the measure of the degree of dispersion. In addition, the present invention selects the absolute error sum of the data as the measure of the image texture complexity, and uses the variance of the data as the measure of the motion vector information.

Fisher discriminant analysis is a classic algorithm in pattern recognition. Its basic idea is to project high-dimensional pattern samples into the best discriminant vector space, and the projected pattern samples have the largest inter-class distance and the smallest class distance in the new subspace. The intra-class distance achieves the function of extracting classification information and compressing the feature dimension. In the present invention, the residual information dispersion, average motion vector and texture complexity are used as the three-dimensional input pattern samples of the discriminant method, and the method of machine learning is used to classify and train it to obtain the best classification model function. When HEVC encoding, only three-dimensional The feature information can be used to obtain the label of the CU partition mode, and determine whether the CU continues to be partitioned.

In the design of DVC-HEVC transcoder, the key step to improve the real-time transcoding is how to efficiently use the relevant information generated in the DVC decoding process to accelerate the HEVC encoding process. The present invention utilizes the residual information dispersion degree, average motion vector and texture complexity available in the DVC decoding process as the parameters of the CU partition model, sets the time period T, and adopts Fisher to the parameters of the previous k frames and WZ frames within one T. The discriminant analysis method obtains the best classifier model, and the remaining WZ frames in the cycle use the classification model for fast CU partitioning. In order to ensure the accuracy of the classification model, an online learning strategy is set up, which can update the threshold and weight vector of the training set in time. Through the method of the present invention, the layer-by-layer rate-distortion optimization process with high computational complexity in the HEVC encoding module can be skipped, thereby achieving the purpose of reducing HEVC encoding complexity.

Specifically, it mainly includes the following process steps:

(1) Set the time period T, divide the reconstructed frame sequence after decoding the DVC code stream into T ₁ , T ₂ , T ₃ ..., the video frame in the period is divided into training frame, verification frame and test frame, wherein the verification frame is fixed Select as a frame;

(2) Perform HEVC encoding on the training frame, extract its features at the same time, obtain the residual information dispersion, average motion vector and texture complexity, respectively establish 64×64, 32×32, 16×16 training sets, and obtain the CU partition model;

(3) Verify that the frame is still HEVC encoded, and use the model in (2) to perform fast CU mode division, and calculate the initial accuracy rate ACC _init ;

(4) The test frame sequence uses the classification model in (2) to perform fast CU mode division;

(5) The steps of T ₂ , T ₃ ... are similar to T ₁ , and the weighted average accuracy rate ACC _weight is calculated at the same time, and the classification vector and threshold of the model are updated.

What the present invention improves is the part with the highest computational complexity in HEVC video coding in the entire DVC-HEVC transcoder. In the entire HEVC video encoding process, the recursive division of CTU accounts for more than 90% of the computational complexity. The most critical step of the present invention is to establish the optimal The classifier model enables the test frame to quickly select the CU partition mode. Therefore, in terms of computational complexity, the method of the present invention focuses on the most critical point of improving computational complexity in HEVC video coding in a DVC-HEVC transcoder.

Description of drawings

Fig. 1 is a system block diagram of the DVC-HEVC video transcoding method based on the Fisher discriminant analysis method of the present invention;

Fig. 2 is a schematic diagram of periodic division and intra-period frame division of DVC decoded and reconstructed frames before encoding in the present invention;

Fig. 3 is the flowchart of the DVC-HEVC video transcoding method based on the Fisher discriminant analysis method of the present invention;

Figures 4 to 7 are the rate-distortion curves of the method of the present invention and the traditional cascade algorithm, wherein Figure 4 is the rate-distortion curve of BasketballDrill; Figure 5 is the rate-distortion curve of BQMall; Figure 6 is the rate-distortion curve of Johnny; Figure 7 is the rate-distortion curve of FourPeople.

Detailed ways

Below in conjunction with accompanying drawing and embodiment the present invention is described in further detail, it is necessary to point out that following embodiment is only used for further description of the present invention, can not be interpreted as the restriction to protection scope of the present invention, those skilled in the art According to the content of the invention above, making some non-essential improvements and adjustments to the present invention for specific implementation shall still belong to the protection scope of the present invention.

In Fig. 3, the DVC-HEVC video transcoding method based on Fisher's discriminant analysis method comprises the following steps:

(1) Decode the DVC code stream to obtain the reconstructed frame. Set the time period T, divide the reconstructed video sequence into T ₁ , T ₂ , T ₃ ......;

(2) Carry out frame grouping to non _- key frame in T1, be divided into training frame, verification frame and test frame, wherein the verification frame is fixedly selected as a frame;

(3) Perform HEVC encoding on the training frame, extract its features at the same time, obtain the residual information dispersion, average motion vector and texture complexity, respectively establish 64×64, 32×32, 16×16 training sets, and obtain the best classification Vectors w(1) ₆₄ , w(1) ₃₂ , w(1) ₁₆ and their thresholds Th(1) ₆₄ , Th(1) ₃₂ , Th(1) ₁₆ ;

(4) Verify that the frame is still HEVC encoded, and use the model in (3) to perform fast CU mode division, and calculate the initial accuracy rate ACC _init ;

(5) Divide the test frame image into LCUs (64×64), perform feature extraction, and establish a test set as the input of the classification model. First multiplied with the optimal classification vector w (1) ₆₄ , the result is compared with the threshold Th (1) ₆₄ , if greater than the threshold, then enter step (6), otherwise, enter step (9);

(6) Continue to divide the LCU into 32×32 blocks, extract its features, multiply it with the best classification vector w(1) ₃₂ and compare it with the threshold Th(1) ₃₂ , if it is greater than the threshold, then enter step (7 ), otherwise, go to step (9);

(7) Continue to be divided into 16×16 blocks, extract its features, multiply it with the best classification vector w(1) ₁₆ and then compare it with the threshold Th(1) ₁₆ , if it is greater than the threshold, then enter step (8), Otherwise, go to step (9);

(8) Continue to divide into 8×8 CU blocks;

(9) Determine the current CU size as the final CU block mode, complete the CU division and enter the selection of the PU mode, and continue the next step of the encoding process.

(10) Carry out frame grouping to non-key frame in T ₂ , be divided into training frame, verification frame and test frame equally, wherein training frame frame number can be reduced appropriately compared with T ₁ ;

(11) Perform HEVC encoding on the training frame, extract its features at the same time, obtain the dispersion of residual information, the variance of the motion vector and the mean value of the texture complexity, establish 64×64, 32×32, 16×16 training sets, and obtain the new most The optimal classification vectors w(2) ₆₄ , w(2) ₃₂ , w(2) ₁₆ and their thresholds Th(2) ₆₄ , Th(2) ₃₂ , Th(2) ₁₆ . _In addition, using the model in T1 to calculate the accuracy rate of each frame in the training frame, a weighted average accuracy rate ACC _weight is obtained;

(12) Verify that the frame is still HEVC encoded, and at the same time use the newly trained model to perform fast CU mode division, and calculate its accuracy rate ACC _update ;

(13) If the ACC _update is greater than the ACC _weight , the newly trained classification model vector and threshold are used for subsequent test frames, otherwise the classification model vector and threshold in the previous cycle are still used;

(14) The test frame adopts _a division step similar to the T1 period;

(15) The steps of T ₃ , T ₄ ... are the same as those of T ₂ .

Specifically, in the step (1), combined with the transcoding process, it can be known that the current HEVC coded frame is a reconstructed frame of the WZ frame after DVC decoding. Since HEVC encoding is adopted for key frames during DVC encoding, there is no need to encode key frames during transcoding.

In the step (3), the sum of absolute errors between each pixel value of the current CU coding unit and the pixel mean value of the coding unit is used to represent the texture complexity of the block, specifically as shown in formula (1). Wherein, W is the width of the current CU block, and P(x, y) represents the pixel value at the coordinate position (x, y). In addition, in order to reduce the complexity of texture calculation, this paper performs horizontal and vertical 1/2 downsampling processing on CU coding units with sizes of 64×64 and 32×32.

In this paper, the average value of the motion vectors of all 8×8 blocks in the current CU coding unit is used to measure the motion intensity of the image block, as shown in formula (2). Among them, N represents the number of 8×8 non-overlapping blocks contained in the current CU coding unit, and MV _i (x) and MV _i (y) represent the horizontal and vertical motions of the ith 8×8 block motion vector respectively offset size

In this paper, the side information obtained by DVC decoding and the difference between its reconstructed frame are used to replace the prediction residual corresponding to HEVC encoding of the current frame. The degree of dispersion of coding units is shown in formula (4). Among them, R(x,y) represents the pixel value of the currently decoded and reconstructed frame, SI(x,y) represents the side information generated during the decoding process of the current frame, and P _res (x,y) represents the residual value of the side information , n represents the number of pixels in the current CU coding unit, and p represents the average value of all pixels.

P _res (x,y)=R(x,y)-SI(x,y) (3)

This paper chooses a learning model based on Fisher's discriminant analysis, and the cut-off threshold Th is obtained by formula (5), where Represents the mean value of various samples after vector w projection. When Th is determined, for the CU unit division of the test set, use formula (6) to make a decision, where the Sgn function is a symbolic function. When f(X) is equal to -1 When it means "divided", otherwise it means "not divided".

f(X)=Sgn(w ^T XT) (6)

In the step (4), the calculation formula of the initial accuracy rate ACC _init is shown in formula (7), wherein, CountCU represents different sizes after transcoding (including "64×64", "32×32", "16×16 ", "8×8") the number of CUs, CountCU _HEVC indicates the number of different CUs after adopting the cascade transcoding algorithm, and CountCU in _{this paper} indicates the number of different CUs after adopting the fast transcoding algorithm proposed in this paper.

In the step (11), the weighted average accuracy rate ACC _weight can be calculated by formula (8).

In order to prove the effectiveness of the algorithm of the present invention, we have carried out experimental verification on it, and the results are shown in Figures 4-7. Figures 4 to 7 are the comparison results of the rate-distortion curves between the DVC-HEVC video transcoding method based on the Fisher discriminant analysis method of the present invention and the traditional cascade transcoding algorithm. The specific process of the comparison is as follows:

(1) DVC codec is performed on the video sequence, and the standard HEVC test video is selected for the video sequence. Their names and resolutions are: BasketballDrill (832×480), FourPeople (1280×720), and the frame rate is 30 frames per second . Wherein, the quantization step size (QP) values are 22, 26, 30, 34 respectively.

(2) Open the programs of the two methods at the same time and set the same configuration file. The HEVC encoding parameter version is selected as HM16.5, and the quantization step (QP) value is 22, 26, 30, and 34 respectively. The present invention will be compared with traditional cascaded transcoding algorithms. This paper selects three kinds of video coding performance: peak signal-to-noise ratio (PSNR), bit rate and coding time (PSNR reflects the objective video quality of the video, and video coding time reflects the computational complexity of coding) for comparative analysis. In order to express this paper more intuitively For the proposed transcoding acceleration, the following three indicators are used to evaluate the performance gap:

ΔPSNR = PSNR _paper - PSNR _cascade

Among them, ΔPSNR represents the difference between the peak signal-to-noise ratio of the method of the present invention and the traditional cascaded transcoding algorithm, ΔBR represents the percentage of the bit rate difference between the method of the present invention and the traditional cascaded transcoding algorithm, and ΔT represents the difference between the method of the present invention and the traditional cascaded transcoding algorithm. The percentage of the time difference of the traditional cascade transcoding algorithm.

(3) import the DVC reconstructed video sequence that obtains in 2 identical steps 1;

(4) carry out video encoding to it respectively;

(5) Using the traditional cascade transcoding algorithm to encode the video sequence in the HEVC mode;

(6) Utilize the method of the present invention to carry out video coding to video sequence under HEVC mode;

(7) two programs respectively output video sequences after video encoding and respective bit rate, PSNR value and total video encoding time, the results of the above three indicators are as shown in table 1-3, statistics show that the method of the present invention and traditional The cascade transcoding method changes the bit rate by 0.6%-3.8%, reduces the PSNR by 0.01dB-0.16dB, and reduces the encoding calculation complexity by 41.4%-68.9%. Experimental verification results show that compared with the traditional cascade transcoding algorithm, the method of the present invention has very little loss in video compression rate (reflected by the degree of bit rate decrease) and video quality (reflected by the degree of decrease in PSNR value) , which greatly reduces the computational complexity of video coding (reflected by the degree of reduction in coding time, as shown in Tables 1-3).

Table 1 Comparison of the bit rate between the algorithm of the present invention and the traditional cascade transcoding algorithm

Table 2 Comparison of PSNR values between the algorithm of the present invention and the traditional cascade transcoding algorithm

Table 3 Comparison of video encoding time between the algorithm of the present invention and the traditional cascade transcoding algorithm

.

Claims (4)

1. a kind of DVC-HEVC video transcoding methods based on Fisher techniquess of discriminant analysis, it is characterised in that include mainly following mistake Journey step：

(1) period of time T is set, the reconstruction frame sequence after DVC code stream decodings is divided into T₁、T₂、T₃..., video in the period Frame is divided into training frames, verification frame and test frame, wherein verification frame fixation is chosen for a frame；

(2) HEVC codings are carried out to training frames, while extracting its feature, obtain residual information dispersion, average motion vector with And Texture complication, 64 × 64,32 × 32,16 × 16 training sets are established respectively, obtain the CU partitioning models of each size；

(3) verification frame still provides for HEVC codings, while carrying out quick CU mode divisions using model in (2), and it is initial accurate to calculate Rate ACC_init；

(4) test frame sequence carries out quick CU mode divisions using disaggregated model in (2)；

(5)T₂、T₃... the step of same T₁It is similar, while calculating weighted average accuracy rate ACC_weight, to the class vector of model It is updated with threshold value.

2. the DVC-HEVC video transcoding methods based on Fisher techniquess of discriminant analysis as described in claim 1, it is characterised in that Need not transcoding be carried out to key frame again in step (1) transit code.

3. the DVC-HEVC video transcoding methods based on Fisher techniquess of discriminant analysis as described in claim 1, it is characterised in that Described in step (3) according to three kinds of residual information dispersion, average motion vector and Texture complication characteristic values, establish Optimal classification device model so that test frame quickly carries out the selection of CU partition modes, and skipped in former algorithm it is complicated successively Rate-distortion optimization process, three kinds of specific measures of feature are as follows：

Using each pixel value of current CU coding units and the absolute error of the coding unit pixel mean value and to indicate the block Texture complication, specifically as shown in formula (1).Wherein, W is the width of current CU blocks, at P (x, y) denotation coordination position (x, y) Pixel value.

It is acute come the movement for weighing image block using the average value of the motion vector of all 8 × 8 size blocks in current CU coding units Strong degree, as shown in formula (2).Wherein, N represents the number that current CU coding units include 8 × 8 non-overlapping blocks, MV_i(x) with MV_i(y) respectively represent i-th 8 × 8 block motion vectors both horizontally and vertically on motion excursion size

The difference of the side information and its reconstructed frame that are obtained using DVC decodings is residual to replace present frame HEVC to encode corresponding prediction Difference, by the dispersion of each pixel value and each CU coding units of pixel mean value computation in current CU, as shown in formula (3).Its In, what n was represented is number of pixels in current CU coding units, and what p was represented is the average value of all pixels.

Based on Fisher discriminant analysis formula learning models, demarcation threshold Th is obtained by formula (4), whereinWhat is represented is all kinds of samples This mean value after vectorial w projections, after dividing threshold value Th and determining, for the CU dividing elements of test set, using formula (5) into Row decision, wherein Sgn functions are sign function, and " division " is represented when f (X) is equal to -1, on the contrary then represent " not dividing ".

F (X)=Sgn (w^TX-T) (5) 。

4. the DVC-HEVC video transcoding methods based on Fisher techniquess of discriminant analysis as described in claim 1, it is characterised in that Weighted average accuracy rate ACC is utilized in step (5)_weightThe class vector w and threshold value Th of model are updated, calculated public Shown in formula such as formula (6), wherein ACC_initFor initial accuracy rate.

。