CN108322740B

CN108322740B - Encoding method with controllable encoding complexity

Info

Publication number: CN108322740B
Application number: CN201810020841.7A
Authority: CN
Inventors: 彭宗举; 文鹏; 陈芬; 蒋刚毅; 郁梅
Original assignee: Ningbo University
Current assignee: Beijing Hongda Yijia Education Technology Co ltd; Shenzhen Lizhuan Technology Transfer Center Co ltd
Priority date: 2018-01-10
Filing date: 2018-01-10
Publication date: 2020-04-17
Anticipated expiration: 2038-01-10
Also published as: CN108322740A

Abstract

The invention discloses a coding method with controllable coding complexity, which intermittently updates the coding reference time of a video, so that the coding reference time of the video is closer to the original coding time, the target coding time distribution of each image group in the video is more reasonable, and the coding complexity control precision of the coded video is greatly improved; the characteristics of a coding frame in a video are reflected by setting flatness, and different coding schemes are selected according to the flatness, so that the quality of the coded video is greatly ensured; aiming at a coding tree unit, reasonable reduction of coding complexity is realized by distributing weight, residual error information, motion information and bit information through complexity, normal coding is selected or coding complexity control is further carried out by comparing the coding time-saving time of all coded coding frames prior to a processed coding frame with the target time-saving time of a video, and rate distortion performance is guaranteed.

Description

Encoding method with controllable encoding complexity

Technical Field

The present invention relates to a method for processing video signals, and more particularly, to a coding method with controllable coding complexity.

Background

With the rapid development of optical fiber technology, 4G communication technology and information processing technology, the industries such as video live broadcast, video monitoring and news live broadcast are developed vigorously. These industries have high requirements on the real-time performance and quality of video coding, but the devices for video coding may be in various environments, which causes the computing power of the devices to be unstable, and the devices cannot code video in real time. By adopting the coding method with controllable coding complexity, the coding complexity can be matched with the computing capability of equipment, and meanwhile, the rate distortion performance and the coding real-time performance can be ensured.

Research aiming at the problem is carried out on High Efficiency Video Coding (HEVC), which is a new generation international Video coding standard determined in 1 month of 2013 by Video coding joint Experts Group JCT-VC (Joint Video coding Experts Group) consisting of ITU-T VCEG (Video coding Experts Group) and ISO/IEC MPEG (Moving Pictures Experts Group). HEVC divides a video sequence into several groups of pictures (GOPs), each GOP containing 4 frames, and each frame is divided into several Coding Tree Units (CTUs) of equal size, and if the video sequence is sampled according to a 4:2:0 sampling format, each CTU contains one luma Tree Block and two chroma Tree blocks, which are the roots of a quad-Tree Coding structure. The CTU may be divided into several equal-sized Coding Units (CUs) according to a quadtree structure, and the size of the Coding units may vary from 64 × 64 to 8 × 8. A CU is a unit that decides whether to perform intra prediction or inter prediction, and the entire CU shares one prediction mode. Each CU can be partitioned into 1,2, or 4 Prediction Units (PU), where a PU is a block of regions using the same Prediction, and a quadtree is partitioned into a number of Transform Units (TU) within the CU, where TU is a block of regions sharing Transform and quantization. Each PU contains a luma, chroma Prediction Block (PB), which may range in size from 4 × 4 up to 64 × 64, and corresponding syntax elements. Each TU contains a luma, chroma Transform Block (TB) and corresponding syntax elements, the size of the TB ranging from 4 × 4 to 32 × 32.

The coding time consumed by continuously traversing the CU and PU prediction modes by the CTU according to the quadtree structure occupies most of the overall coding time of HEVC. Therefore, how to effectively reduce redundant traversal of CU and PU prediction modes and add a reasonable control method is the key of the encoding complexity controllable technology. In recent years, new encoding methods with controllable encoding complexity are proposed internationally, such as: the method reasonably solves the problem of coding complexity distribution of a CTU layer by using the rate-distortion cost corresponding to each CTU of a previous frame, but does not fully consider the characteristics of images; for another example: amaya proposes an HEVC complexity controllable method based on fast coding unit decision, which obtains good rate-distortion performance by updating the early termination threshold in real time, but the control accuracy of the method needs to be further improved. In China, Dunxin of Beijing aerospace university proposes an HEVC hierarchical complexity control method for real-time video coding, which distributes the coding complexity of CTUs through region-of-interest weights to improve the subjective quality, but in the method, coding reference time prediction cannot be updated in real time along with the change of coding characteristics.

Disclosure of Invention

The technical problem to be solved by the invention is to provide an encoding method with controllable encoding complexity, which can discontinuously update the encoding reference time, so that the encoding reference time is closer to the original encoding time, and the control precision is improved by combining an effective encoding complexity control process, and the encoding scheme can be adjusted according to the characteristics of an encoding frame, so that the rate-distortion performance is ensured to the maximum extent.

The technical scheme adopted by the invention for solving the technical problems is as follows: a coding method with controllable coding complexity is characterized in that the coding method adopts JVT recommended low-delay coding configuration to code a video on a video coding check model HM based on an HEVC video coding standard, and the coding method comprises the following steps:

step 1): setting a video to be coded to contain an F frame coding frame; splitting video to be encoded into

The 1 st group of pictures comprises 1 frame of coding frame, and each of the rest groups of pictures comprises 4 frames of coding frames; setting each frame in each image group in video to be coded to contain N_CTUA coding tree unit; wherein F > 9, symbols

Is a sign of an upward rounding operation;

step 2): low-latency coding configuration recommended by JVT is adopted for the first 3 image groups in video to be codedThe method comprises the steps of coding each frame of coded frames, storing bits corresponding to an optimal mode and information of optimal segmentation or non-optimal segmentation of the bits under different depth values of each coding unit in each coding tree unit in each frame of a 2 nd image group and a 3 rd image group in a video to be coded in a coding process, obtaining the actual coding time of each frame of coded frames in the first 3 image groups in the video to be coded and the actual coding time of each coding tree unit in each frame of coded frames after the coding is finished, and recording the actual coding time of the coded frames with the sequence number f' as t_f'The actual coding time of the coding tree unit with sequence number i in the coding frame with sequence number f' is recorded as t_f',i(ii) a Then calculating the proportion of the actual coding time of each frame in the 3 rd image group in the video to be coded in the total actual coding time of all the coding frames in the 3 rd image group in the video to be coded, and marking the proportion of the actual coding time of the k-th frame in the 3 rd image group in the video to be coded in the total actual coding time of all the coding frames in the 3 rd image group in the video to be coded as P_k(ii) a And calculating the coding reference time of the video to be coded, which is recorded as T_o，

According to bits corresponding to an optimal mode and information of optimal segmentation or non-optimal segmentation obtained by each coding unit in each coding tree unit in each frame of coding frames in a 2 nd image group and a 3 rd image group in a video to be coded under different depth values, when the bits corresponding to the optimal mode obtained by all the coding units under different depth values are less than or equal to a set Bit, probability functions of optimal segmentation and non-optimal segmentation of all the coding units under different depth values are obtained, and bits Bit corresponding to the optimal mode obtained by all the coding units under a depth value d are used_dWhen the Bit is less than or equal to the set Bit, the probability function that all the coding units under the depth value d are optimally segmented and the probability function that all the coding units are not optimally segmented are correspondingly marked as

And

and

and will be

Ninety-seven percent of the maximum value of (a) is noted as Y_dWill be

Ninety-seven percent of the maximum value of (a) is noted as N_d(ii) a Then determining an early termination threshold and a continuous traversal threshold, and correspondingly marking as L_dAnd H_d，

H_d＝0.7×Y_d；

Calculating the proportion of the total number of pixel points in all coding units with the optimal depth value of 0 in a coding frame with the sequence number of 8 in the video to be coded to the total number of pixel points in the coding frame with the sequence number of 8 in the video to be coded, and taking the proportion as the flatness of the video to be coded and recording the flatness as delta;

wherein f' is more than or equal to 0 and less than or equal to 8, i is more than or equal to 0 and less than or equal to N_CTU-1, 1 ≦ k ≦ 4, d ≦ 0,1,2, setting Bit to any one Bit,

representing all coding units under depth value d as optimal partitions and satisfying Bit_dThe probability value of not more than Bit,

representing all coding units at depth value d not being optimally partitioned andsatisfy Bit_dProbability value less than or equal to Bit, e is natural base number, T_cRepresenting the target complexity, T, of the video to be encoded_c∈[0.5,1]；

Step 3): defining a current to-be-processed coded frame with the sequence number F in a video to be coded as a current coded frame, wherein F is more than or equal to 9 and less than or equal to F-1;

step 4): judging whether the current coding frame is the 1 st frame in the group of pictures to which the current coding frame belongs, if so, calculating the target coding time of the group of pictures to which the current coding frame belongs, and recording the target coding time as T_GOP，

Then step 5) is executed; otherwise, directly executing the step 5); wherein, t_gRepresenting the actual coding time of a coding frame with the sequence number g in the video to be coded;

step 5): judging whether the current coding frame is separated from the coding frame with the serial number of 8 by a multiple of 16 frames, if so, taking the current coding frame as the coding frame needing to be updated, coding the current coding frame by adopting the low-delay coding configuration recommended by JVT (JVT) and updating T_oAnd delta, T_oPush type

Updating, wherein the value after delta updating is the proportion of the total number of pixel points in all coding units with the optimal depth value of 0 in the current coding frame to the total number of all pixel points in the current coding frame, and then executing the step 15); otherwise, directly executing the step 6); wherein, t_(g-1)×16+8Represents the actual coding time, P, of the coded frame with sequence number (g-1). times.16 +8 in the video to be coded₄Representing the proportion of the actual coding time of the 4 th frame of the coding frame in the 3 rd image group in the video to be coded to the sum of the actual coding times of all the coding frames in the 3 rd image group in the video to be coded;

step 6): calculating the encoded time savings, denoted T, for all encoded frames preceding the current encoded frame_s，

Then, T is judged_sTime saving T if it is less than the target of the video to be encoded_o×(1-T_c) If yes, directly executing step 7); otherwise, encoding the current encoding frame by adopting the low-delay encoding configuration recommended by the JVT, and then executing the step 15);

step 7): calculating the target coding time, denoted T, assigned to the current coding frame according to the position of the current coding frame in the group of pictures to which it belongs_fIf the current coding frame is the k-th frame in the image group to which the current coding frame belongs, T_f＝T_GOP×P_k(ii) a Then judging whether the current coding frame is the 1 st frame in the image group to which the current coding frame belongs or whether the coding frame of the previous frame of the current coding frame adopts the low-delay coding configuration recommended by JVT for coding, if so, directly executing the step 9); otherwise, directly executing step 8);

step 8): judging the difference T between the target coding time and the actual coding time of the previous frame of the current coding frame_f-1-t_f-1Saving time if it is larger than the set frame target

If yes, executing step 13); otherwise, directly executing step 9); wherein, t_f-1Represents the actual coding time of the coded frame preceding the current coded frame, and also represents the actual coding time of the coded frame with sequence number f-1, T_f-1Representing a target encoding time of an encoded frame previous to the current encoded frame;

step 9): calculating target coding time allocated to each coding tree unit in the current coding frame, and recording the target coding time allocated to the coding tree unit with the sequence number i in the current coding frame as T_CTU,i，

Then, the target coding time allocated to each coding tree unit in the current coding frame is normalized, and T is calculated_CTU,iThe value obtained after normalization is recorded as

Obtaining

Taking the actual coding time of a coding tree unit with the sequence number i in the kth frame coding frame in the 3 rd image group in the video to be coded as a normalized denominator; then step 10) is executed; wherein R is_codedRepresents the sum of the actual coding times, ω, of all coded coding tree units in the current coding frame_iIndicates the complexity assignment weight, ω, of the coding tree unit with sequence number i in the current coding frame_pre,mThe complexity of a coding tree unit with the sequence number of m in a coding frame of the kth frame in a group of pictures before the current coding frame is assigned with a weight, wherein m is more than or equal to 0 and less than or equal to N_CTU-1；

Step 10): judging whether delta is smaller than 0.4, if so, skipping all prediction modes behind the 2 Nx 2N mode between frames when the depth value of each coding tree unit in the current coding frame is 0, allocating the maximum depth value to each coding tree unit in the current coding frame, and recording the maximum depth value allocated to the coding tree unit with the sequence number of i in the current coding frame as d_max,i，

Then step 11) is executed; otherwise, directly allocating the maximum depth value to each coding tree unit in the current coding frame, and recording the maximum depth value allocated to the coding tree unit with the sequence number i in the current coding frame as d_max,i，

Then step 11) is executed;

step 11): traversing each coding unit in each coding tree unit in the current coding frame in an inter-frame 2 Nx 2N mode under the condition that the depth value is not larger than the maximum depth value of the coding tree unit to which the coding unit belongs; then, for each coding unit in each coding tree unit in the current coding frame, a coding unit to which the depth value is not greater than that of the coding unit is calculatedThen, judging whether the absolute values of the differences corresponding to all the pixel points in each coding unit in the coding unit are all smaller than a set threshold β and the motion vector of the coding unit is 0 or not under the depth value d ' of each coding unit in each coding tree unit in the current coding frame, if so, enabling the coding unit to skip all the prediction modes behind the 2 Nx 2N mode between frames when the depth value d ' is larger, skipping the traversal process when the depth value is larger than d ', and then executing the step 15), otherwise, directly executing the step 12), wherein the value range of the depth value is the range of the original pixel value and the predicted pixel value of each pixel point in the coding unit under the maximum depth value of the coding tree unit

d_maxRepresenting a maximum depth value of a coding tree unit to which the coding unit belongs;

step 12): after the traversal process of each coding unit in each coding tree unit in the current coding frame under the condition that the depth value is not larger than the maximum depth value of the coding tree unit to which the coding unit belongs is completed, whether a bit corresponding to the optimal mode of the coding unit under the depth value d' is smaller than L or not is judged for each coding unit in each coding tree unit in the current coding frame_dIf it is less than L_dThen the encoding unit skips the traversal process with depth value larger than d', and then executes step 15); if greater than or equal to L_dThen, it is determined whether the bit corresponding to the optimal mode of the coding unit under the depth value d' is larger than H_dAnd whether d' is greater than or equal to the maximum depth value of the coding tree unit to which the coding unit belongs, if so, the coding unit is subjected to a traversal process in which the depth value is greater than the maximum depth value of the coding tree unit to which the coding unit belongs, and then step 15) is executed, otherwise, step 15) is directly executed;

step 13): calculating the complexity distribution weight of each coding tree unit in the current coding frame, and recording the complexity distribution weight of the coding tree unit with the sequence number i in the current coding frame as omega_i(ii) a Then step 14) is executed;

step 14), making each coding unit in each coding tree unit in the current coding frame traverse an inter-frame 2 Nx 2N mode under each depth value, then calculating the difference value between the original pixel value and the predicted pixel value of each pixel point in the coding unit under each depth value for each coding unit in each coding tree unit in the current coding frame, then judging whether the absolute value of the difference value corresponding to all the pixel points in the coding unit is less than a set threshold value β and the motion vector of the coding unit is 0 under each depth value d ' for each coding unit in each coding tree unit in the current coding frame, if so, making the coding unit skip all the predicted modes behind the inter-frame 2 Nx 2N mode when the depth value d ' is reached, and skip the traversal process when the depth value is greater than d ', and then executing step 15), otherwise, executing step 15 after the traversal process under each depth value of each coding unit in each coding tree unit in the current coding frame is completed, wherein the value range of the depth value is [0,2 ];

step 15): taking the next frame to be processed in the video to be coded as a current coding frame, and then returning to the step 4) to continue execution until all coding frames in the video to be coded are processed; wherein, f is the assigned symbol, and f +1 is the assigned symbol.

In said step 9) and said step 13), ω_iThe calculation process of (2) is as follows: traversing the inter-frame 2 Nx 2N mode when the depth value of the coding tree unit with the serial number i in the current coding frame is 0 to obtain the difference value between the original pixel value and the predicted pixel value of each pixel point in the coding tree unit with the serial number i in the current coding frame; then calculating the average value of all the difference values; then accumulating all the difference values larger than the average value, and taking the obtained sum value as omega_iA specific value of (a);

in the step 9), if a group of pictures preceding a group of pictures to which the current encoding frame belongs is a group of pictures 3 in the video to be encoded, ω is a group of pictures_pre,mThe calculation process of (2) is as follows: when the depth value of a coding tree unit with the sequence number of m in the kth frame coding frame in the 3 rd image group in the video to be coded is 0, traversing the 2 Nx 2N mode between frames to obtain the difference between the original pixel value and the predicted pixel value of each pixel point in the coding tree unitA value; then calculating the average value of all the difference values; then accumulating all the difference values larger than the average value, and taking the obtained sum value as omega_pre,mSpecific values of (a).

In the step 11) described above, the step,

wherein QP represents an encoding quantization parameter of the video to be encoded.

Compared with the prior art, the invention has the advantages that:

1) the method of the invention continuously updates the coding reference time of the video according to a certain interruption, so that the coding reference time of the video is closer to the original coding time, the target coding time distribution of each image group in the video is more reasonable, and the coding complexity control precision of the coded video is greatly improved.

2) The method reflects the characteristics of the coding frame in the video by setting the flatness, and selects different coding schemes according to the flatness, thereby greatly ensuring the quality of the coded video.

3) The method of the invention realizes reasonable reduction of coding complexity by distributing weight, residual error information, motion information and bit information for coding tree units through complexity, and selects normal coding or further performs coding complexity control by comparing the coding time-saving time of all coded coding frames prior to the current coding frame with the target time-saving time of the video to be coded, thereby ensuring rate distortion performance.

4) The method of the invention allocates the target coding time of the image group to each frame coding frame in the image group according to the proportion, so that the rate distortion performance loss of each frame coding frame in the image group is similar, the condition that the distortion performance loss difference of each frame rate is larger due to the fact that the frames are allocated according to the average in the past is avoided, and the rate distortion performance loss of the coded video is greatly reduced.

5) The method is realized by distributing the weight through the complexity of each coding tree unit in each frame of coding frame on the target coding time distribution of each coding tree unit in each frame of coding frame, so that the rate distortion performance loss of each coding tree unit in each frame of coding frame is smaller, and the rate distortion performance loss of the coded video is greatly reduced.

Drawings

FIG. 1 is a block diagram of an overall implementation of the method of the present invention;

FIG. 2a is the partitioning of the picture and coding tree unit with sequence number 17 after "BQALL" test video sequence coding reconstruction on HM 13.0 platform;

fig. 2b shows the partitioning of the picture and coding tree unit with sequence number 17 after the "BQMall" test video sequence is coded and reconstructed on HM 13.0 platform incorporating the method of the present invention and the target complexity of the "BQMall" test video sequence is 0.6.

Detailed Description

The invention is described in further detail below with reference to the accompanying examples.

The invention provides a coding method with controllable coding complexity, which is used for coding a video by adopting a low-delay coding configuration recommended by JVT (joint video transform) on a video coding verification model HM based on an HEVC (high efficiency video coding) standard, and a general implementation block diagram of the coding method is shown in figure 1, and the coding method comprises the following steps:

Is the sign of the rounding up operation.

Step 2): coding each frame in the first 3 groups of pictures in a video to be coded by adopting a low-delay coding configuration recommended by JVT (joint video transform), and storing the 2 nd group of pictures and the 3 rd picture in the video to be coded in the coding processObtaining the bit corresponding to the optimal mode and the information of optimal segmentation or non-optimal segmentation of each coding unit in each coding tree unit in each frame of coding frame in the image group under different depth values, obtaining the actual coding time of each frame of coding frame in the first 3 image groups in the video to be coded and the actual coding time of each coding tree unit in each frame of coding frame after the coding is finished, and recording the actual coding time of the coding frame with the sequence number f' as t_f'The actual coding time of the coding tree unit with sequence number i in the coding frame with sequence number f' is recorded as t_f',i(ii) a Then calculating the proportion of the actual coding time of each frame in the 3 rd image group in the video to be coded in the total actual coding time of all the coding frames in the 3 rd image group in the video to be coded, and marking the proportion of the actual coding time of the k-th frame in the 3 rd image group in the video to be coded in the total actual coding time of all the coding frames in the 3 rd image group in the video to be coded as P_k(ii) a And calculating the coding reference time of the video to be coded, which is recorded as T_o，

And

and

and will be

Ninety-seven percent of the maximum value of (a) is noted as Y_dWill be

H_d＝0.7×Y_d。

And calculating the proportion of the total number of the pixel points in all the coding units with the optimal depth value of 0 in the coding frame with the sequence number of 8 in the video to be coded to the total number of the pixel points in the coding frame with the sequence number of 8 in the video to be coded, and taking the proportion as the flatness of the video to be coded and recording the flatness as delta.

Wherein f' is more than or equal to 0 and less than or equal to 8, i is more than or equal to 0 and less than or equal to N_CTU-1, 1 ≦ k ≦ 4, d ≦ 0,1,2, the set Bit may be any Bit since its size has no effect on subsequent operations and therefore may be any Bit,

representing that all coding units under depth value d are not optimally segmented and satisfy Bit_dProbability value less than or equal to Bit, e is natural base number, T_cRepresenting the target complexity, T, of the video to be encoded_c∈[0.5,1]，T_cIs set by itself, e.g. taking T_c＝0.8。

Step 3): defining a current to-be-processed coded frame with the sequence number F in a video to be coded as a current coded frame, wherein F is more than or equal to 9 and less than or equal to F-1.

Then step 5) is executed; otherwise, directly executing the step 5); wherein, t_gWhich represents the actual coding time of the coded frame with sequence number g in the video to be coded.

Updating, wherein the value after delta updating is the proportion of the total number of pixel points in all coding units with the optimal depth value of 0 in the current coding frame to the total number of all pixel points in the current coding frame, and then executing the step 15); otherwise, directly executing the step 6); wherein, t_(g-1)×16+8Represents the actual coding time, P, of the coded frame with sequence number (g-1). times.16 +8 in the video to be coded₄The ratio of the actual coding time of the 4 th frame of the 3 rd image group in the video to be coded to the sum of the actual coding times of all the coding frames in the 3 rd image group in the video to be coded is represented.

Then, T is judged_sTime saving T if it is less than the target of the video to be encoded_o×(1-T_c) If yes, directly executing step 7); otherwise, the current coding frame is coded using the low-latency coding configuration recommended by the JVT, and then step 15) is performed.

Step 7): calculating the target coding time, denoted T, assigned to the current coding frame according to the position of the current coding frame in the group of pictures to which it belongs_fIf the current coding frame is the k-th frame in the image group to which the current coding frame belongs, T_f＝T_GOP×P_k(ii) a Then judging whether the current coding frame is the 1 st frame in the image group to which the current coding frame belongs or whether the coding frame of the previous frame of the current coding frame adopts the low-delay coding configuration recommended by JVT for coding, if so, directly executing the step 9); otherwise, step 8) is performed directly.

If yes, executing step 13); otherwise, directly executing step 9); wherein, t_f-1Represents the actual coding time of the coded frame preceding the current coded frame, and also represents the actual coding time of the coded frame with sequence number f-1, T_f-1Representing the target encoding time of the encoded frame that is previous to the current encoded frame.

Obtaining

Taking the actual coding time of a coding tree unit with the sequence number i in the kth frame coding frame in the 3 rd image group in the video to be coded as a normalized denominator; then step 10) is executed; wherein R is_codedRepresents the sum of the actual coding times, ω, of all coded coding tree units in the current coding frame_iIndicates the complexity assignment weight, ω, of the coding tree unit with sequence number i in the current coding frame_pre,mThe complexity of a coding tree unit with the sequence number of m in a coding frame of the kth frame in a group of pictures before the current coding frame is assigned with a weight, wherein m is more than or equal to 0 and less than or equal to N_CTU-1。

In this embodiment, in step 9), ω_iThe calculation process of (2) is as follows: traversing the inter-frame 2 Nx 2N mode when the depth value of the coding tree unit with the serial number i in the current coding frame is 0 to obtain the difference value between the original pixel value and the predicted pixel value of each pixel point in the coding tree unit with the serial number i in the current coding frame; then calculating the average value of all the difference values; then accumulating all the difference values larger than the average value, and taking the obtained sum value as omega_iSpecific values of (a).

In this embodiment, in step 9), if the group of pictures before the group of pictures to which the current encoding frame belongs is the group of pictures 3 in the video to be encoded, ω is_pre,mThe calculation process of (2) is as follows: when the depth value of a coding tree unit with the sequence number of m in a kth frame coding frame in a 3 rd image group in a video to be coded is 0, traversing an inter-frame 2 Nx 2N mode to obtain a difference value between an original pixel value and a predicted pixel value of each pixel point in the coding tree unit; then calculating the average value of all the difference values; then accumulating all the difference values larger than the average value, and taking the obtained sum value as omega_pre,mSpecific values of (a).

Step 10): judging whether delta is less than 0.4, if so, making each coding tree list in the current coding frameSkipping all the prediction modes behind the 2 Nx 2N mode between frames when the depth value is 0 by each coding unit in the element, allocating the maximum depth value to each coding tree unit in the current coding frame, and recording the maximum depth value allocated to the coding tree unit with the sequence number i in the current coding frame as d_max,i，

Then step 11) is performed.

Step 11) making each coding unit in each coding tree unit in the current coding frame traverse the inter-frame 2 Nx 2N mode under the condition that the depth value is not larger than the maximum depth value of the coding tree unit to which the coding unit belongs, then calculating the difference value between the original pixel value and the predicted pixel value of each pixel point in each coding unit under the condition that the depth value is not larger than the maximum depth value of the coding tree unit to which the coding unit belongs for each coding unit in each coding tree unit in the current coding frame, then judging whether the absolute value of the difference value corresponding to all the pixel points in each coding unit is smaller than the set threshold β and the motion vector of the coding unit is 0 under the condition that each coding unit in each coding tree unit in the current coding frame is under the depth value d ', if so, making the coding unit skip all the prediction modes behind the inter-frame 2 Nx 2N mode when the depth value is d ', skipping the process that the depth value is larger than d ', then executing step 15), otherwise, directly executing step 12), wherein the value range of the depth value here is the value of the traversal range of the depth value

d_maxRepresents the maximum depth value of the coding tree unit to which the coding unit belongs, i.e. if the coding unit to be processed belongs to the coding tree unit with sequence number i in the current coding frame, the value of the depth value is normalizedEnclose as

In this embodiment, in step 11),

Step 12): after the traversal process of each coding unit in each coding tree unit in the current coding frame under the condition that the depth value is not larger than the maximum depth value of the coding tree unit to which the coding unit belongs is completed, whether a bit corresponding to the optimal mode of the coding unit under the depth value d' is smaller than L or not is judged for each coding unit in each coding tree unit in the current coding frame_dIf it is less than L_dThen the encoding unit skips the traversal process with depth value larger than d', and then executes step 15); if greater than or equal to L_dThen, it is determined whether the bit corresponding to the optimal mode of the coding unit under the depth value d' is larger than H_dAnd whether d' is greater than or equal to the maximum depth value of the coding tree unit to which the coding unit belongs, if so, the coding unit is subjected to a traversal process in which the depth value is greater than the maximum depth value of the coding tree unit to which the coding unit belongs, and then step 15) is executed, otherwise, step 15) is directly executed.

Step 13): calculating the complexity distribution weight of each coding tree unit in the current coding frame, and recording the complexity distribution weight of the coding tree unit with the sequence number i in the current coding frame as omega_i(ii) a Then step 14) is performed.

In this embodiment, ω) in step 13)_iThe calculation process of (2) is as follows: traversing the inter-frame 2 Nx 2N mode when the depth value of the coding tree unit with the serial number i in the current coding frame is 0 to obtain the difference value between the original pixel value and the predicted pixel value of each pixel point in the coding tree unit with the serial number i in the current coding frame; then calculating the average value of all the difference values; then accumulating all the difference values larger than the average value, and taking the obtained sum value as omega_iSpecific values of (a).

Step 14), making each coding unit in each coding tree unit in the current coding frame traverse the inter-frame 2 Nx 2N mode under each depth value, then calculating the difference value between the original pixel value and the predicted pixel value of each pixel point in the coding unit under each depth value for each coding unit in each coding tree unit in the current coding frame, then judging whether the absolute value of the difference value corresponding to all pixel points in the coding unit is less than the set threshold value β and the motion vector of the coding unit is 0 under each depth value d ' for each coding unit in each coding tree unit in the current coding frame, if so, making the coding unit skip all the prediction modes behind the inter-frame 2 Nx 2N mode when the depth value d ' is reached, and skip the traversal process when the depth value is greater than d ', and then executing step 15), otherwise, after the traversal process under each depth value of each coding unit in each coding tree unit in the current coding frame is completed, then executing step 15), wherein the value range of the depth value is [0,2], namely, if the coding unit belonging to the current coding frame is the value of the coding unit is less than or than 0, then executing step 15.

FIG. 2a shows the partitioning of the coding tree unit and the picture with sequence number 17 after "BQALL" test video sequence coding reconstruction on HM 13.0 platform; fig. 2b shows the segmentation of the picture and coding tree unit with sequence number 17 after the "BQMall" test video sequence is coded and reconstructed on HM 13.0 platform incorporating the method of the present invention and the target complexity of the "BQMall" test video sequence is 0.6. Analyzing fig. 2a and fig. 2b, from the subjective quality of human eyes, the original reconstructed image (the image shown in fig. 2 a) and the reconstructed image with limited coding complexity (the image shown in fig. 2 b) have almost no difference, and do not affect the subjective feeling of human eyes; from the aspect of coding tree unit segmentation, although the coding tree unit segmentation of the reconstructed image with limited coding complexity is different from the coding tree unit segmentation of the original reconstructed image, the two finely segmented regions are highly overlapped, which shows that the method can better maintain the rate distortion performance under the condition of limited coding complexity.

In order to test the performance of the method, the method is realized on a computer with an Intel (R) core (TM) i5-7500 CPU @3.4GHz and 8GB memory by using an HM 13.0 platform. The configuration mode adopts low-delay coding configuration, the detailed coding configuration is listed as table 1, and the test video sequences in table 2 have different target complexity T_cAnd coding to obtain the corresponding rate distortion performance.

Table 2 shows the coding performance of each video sequence under different target complexity using the method of the present invention, where Δ BR represents the difference between the code rate obtained by the coding using the method of the present invention and the original code rate, and Δ PSNR represents the difference between the peak signal-to-noise ratio obtained by the coding using the method of the present invention and the original peak signal-to-noise ratio, which are used to measure the performance of the method of the present invention. As can be seen from Table 2, the method of the present invention has better adaptability to test video sequences of different resolutions under the condition of the same control degree (i.e. target complexity); the attenuation of the test video sequence under different control degrees (namely the target complexity) is relatively smooth, which shows that the method of the invention can well ensure the rate distortion performance.

Table 3 shows the complexity T of each test video sequence at different target levels by using the method of the present invention_cAnd (3) controlling the video sequence after coding, wherein TS is adopted to reflect the actual time saving, and the Mean Absolute error (MAD) of the actual time saving and the target time saving of all the test video sequences is adopted to reflect the control precision of the method. As can be seen from Table 3, the deviation between the actual time saving and the target time saving of a single test video sequence is only 3.77%, and the maximum MAD which can accurately reflect the control accuracy of the method is 1.22%, which indicates that the method has better control accuracy.

Exemplary configurations of tables 1 HM-13.0

Size of coding tree unit	64×64
		Encoding quantization parameter	22、27、32、37
Maximum depth value	3
		Sample adaptive compensation enabling	1
Fast encoder setup enable	1
		Converged mode fast decision enable	1
Intra frame spacing	-1
		Image group structure	IPPP

Table 2 coding performance of each video sequence at different target complexity using the method of the present invention

Table 3 control deviation of each test video sequence at different target complexity using the method of the present invention

Claims

1. A coding method with controllable coding complexity is characterized in that the coding method adopts JVT recommended low-delay coding configuration to code a video on a video coding check model HM based on an HEVC video coding standard, and the coding method comprises the following steps:

The 1 st group of pictures comprises 1 frame of coding frame, and each of the rest groups of pictures comprises 4 frames of coding frames; setting each frame in each image group in video to be coded to contain N_CTUA coding tree unit; wherein, F>9, symbol

Is a sign of an upward rounding operation;

step 2): coding each frame of the first 3 image groups in the video to be coded by adopting the low-delay coding configuration recommended by JVT, storing the bit corresponding to the optimal mode and the information of optimal segmentation or non-optimal segmentation of each coding unit in each coding tree unit in the 2 nd image group in the video to be coded and each coding unit in each frame of the coding frame in the 3 rd image group in the coding process, and obtaining the actual coding time of each frame of the first 3 image groups in the video to be coded and the actual coding time of each coding tree unit in each frame of the coding frame after the coding is finishedCode time, the actual coding time of the coded frame with the sequence number f' is recorded as t_f'The actual coding time of the coding tree unit with sequence number i in the coding frame with sequence number f' is recorded as t_f',i(ii) a Then calculating the proportion of the actual coding time of each frame in the 3 rd image group in the video to be coded in the total actual coding time of all the coding frames in the 3 rd image group in the video to be coded, and marking the proportion of the actual coding time of the k-th frame in the 3 rd image group in the video to be coded in the total actual coding time of all the coding frames in the 3 rd image group in the video to be coded as P_k(ii) a And calculating the coding reference time of the video to be coded, which is recorded as T_o，

And

and

and will be

Ninety-seven percent of the maximum value of (a) is noted as Y_dWill be

H_d＝0.7×Y_d；

representing that all coding units under depth value d are not optimally segmented and satisfy Bit_dProbability value less than or equal to Bit, e is natural base number, T_cRepresenting the target complexity, T, of the video to be encoded_c∈[0.5,1]；

step 4): judging whether the current coding frame is the 1 st frame in the image group to which the current coding frame belongs, if so, calculating the target coding of the image group to which the current coding frame belongsCode time, noted as T_GOP，

step 7): calculating the target coding time allocated to the current coding frame according to the position of the current coding frame in the image group to which the current coding frame belongsIs marked as T_fIf the current coding frame is the k-th frame in the image group to which the current coding frame belongs, T_f＝T_GOP×P_k(ii) a Then judging whether the current coding frame is the 1 st frame in the image group to which the current coding frame belongs or whether the coding frame of the previous frame of the current coding frame adopts the low-delay coding configuration recommended by JVT for coding, if so, directly executing the step 9); otherwise, directly executing step 8);

Obtaining

Then step 11) is executed;

step 11) making each coding unit in each coding tree unit in the current coding frame traverse the inter-frame 2 Nx 2N mode under the condition that the depth value is not more than the maximum depth value of the coding tree unit to which the coding unit belongs, then calculating the difference value between the original pixel value and the predicted pixel value of each pixel point in each coding unit under the condition that the depth value is not more than the maximum depth value of the coding tree unit to which the coding unit belongs for each coding unit in each coding tree unit in the current coding frame, then judging whether the absolute value of the difference value corresponding to all the pixel points in each coding unit is less than the set threshold β and the motion vector of the coding unit is 0 under the condition that the depth value is d' for each coding unit in each coding tree unit in the current coding frame, and if so, making the coding unit in the current coding frame go through the inter-frame 2 Nx 2N modeSkipping all prediction modes behind the inter-frame 2 Nx 2N mode when the depth value d 'is skipped, skipping the traversal process when the depth value is larger than d', and then executing the step 15), otherwise, directly executing the step 12); wherein the depth value has a value range of

2. A coding method with controllable coding complexity according to claim 1, wherein in step 9) and step 13), ω is_iThe calculation process of (2) is as follows: traversing the inter-frame 2 Nx 2N mode when the depth value of the coding tree unit with the serial number i in the current coding frame is 0 to obtain the difference value between the original pixel value and the predicted pixel value of each pixel point in the coding tree unit with the serial number i in the current coding frame; then calculating the average value of all the difference values; then accumulating all the difference values larger than the average value, and taking the obtained sum value as omega_iA specific value of (a);

in the step 9), if a group of pictures preceding a group of pictures to which the current encoding frame belongs is a group of pictures 3 in the video to be encoded, ω is a group of pictures_pre,mThe calculation process of (2) is as follows: when the depth value of a coding tree unit with the sequence number of m in a kth frame coding frame in a 3 rd image group in a video to be coded is 0, traversing an inter-frame 2 Nx 2N mode to obtain a difference value between an original pixel value and a predicted pixel value of each pixel point in the coding tree unit; then calculating the average value of all the difference values; then accumulating all the difference values larger than the average value, and taking the obtained sum value as omega_pre,mSpecific values of (a).

3. A coding complexity controlled coding method according to claim 1 or 2, characterized in that in step 11),