CN101198042A - Code rate control method for video coding and video data emission device - Google Patents

Abstract

The invention provides a code rate control method for video coding, wherein, control packages are sent at fixed period by a sender through a real-time transmission control feedback link; when feedback control packages are received, code rate is obtained according to package loss rate in the feedback control packages and video data is sent through the real-time transmission link according to the obtained code rate. The code rate control method for video coding and a transmitting device control output code rate by consideration of factors of package loss rate and encoded data can be suitable for transmission of current network conditions better, thereby better video display effect is reached.

Description

Code rate control method for video coding and transmitting equipment for video data

Technical Field

The invention relates to a video coding technology, in particular to a code rate control method for video coding.

Background

Video coding standards generally adopt intra-frame inter-frame prediction, discrete cosine transform quantization and entropy coding techniques to remove temporal-spatial correlation, so as to achieve higher coding efficiency. For video communication, due to limited communication channel bandwidth, the video coding rate needs to be controlled to ensure smooth transmission of the coded code stream and full utilization of the channel bandwidth. For different application occasions, there are various code rate control (RateControl) strategies. The real-time coding code rate control method mainly comprises two methods: the number of bits generated by the current macroblock coding is predicted by the number of bits generated by the previous macroblock coding or by a video coding rate distortion function.

However, both of the above-mentioned real-time coding rate control methods fail to fully consider the network bandwidth condition, and simply generate the current coding rate according to the previous coding rate, so that the predictive coding cannot be well adapted to the transmission of the current network condition.

Disclosure of Invention

The invention provides a code rate control method of video coding, which can fully consider the network condition, improve the control effect on code streams and improve the image quality of video transmission.

The invention provides a code rate control method of video coding.A sender periodically sends a control packet through a real-time transmission control feedback link, obtains an output code rate according to the packet loss rate in the feedback control packet when receiving the feedback control packet, and sends video data through the real-time transmission link according to the obtained code rate.

The invention also provides a video data transmitting device, which comprises:

the data acquisition module is used for acquiring video data;

the encoder is used for encoding the video data according to a target code rate;

a real-time transmission link interface for transmitting the video data; and

a real-time transmission control feedback link interface receives a feedback control packet by periodically sending the control packet;

and the first control module is used for obtaining the target code rate according to the packet loss rate in the feedback control packet.

The present invention also provides another video data transmitting apparatus, comprising:

the data acquisition module is used for acquiring video data;

the encoder is used for obtaining the output code rate of the video data according to the target code rate and encoding the video data according to the output code rate;

a real-time transmission link interface for transmitting the video data; and

a real-time transmission control feedback link interface receives a feedback control packet by periodically sending the control packet;

and the quantization parameter adjuster is used for obtaining the quantization parameter of the I frame according to the packet loss rate.

Because the packet loss rate will affect the transmission quality of the video data, the bit rate control method and the transmitting device of the video coding control the output bit rate by considering the factor of the packet loss rate, so that the coded data can better adapt to the transmission of the current network condition, thereby achieving better video display effect.

Drawings

FIG. 1 is a flow chart of a rate control method for video coding according to the present invention;

fig. 2 is a block diagram of a system for processing video data according to the present invention.

Detailed Description

The packet loss rate refers to the loss condition of network transmission data, when a data packet generated by video coding is relatively large, and under the condition of good network condition, the data packet transmission is good, the packet loss rate is relatively small, even no packet is lost, but when the network condition is not good, the packet loss rate is possibly serious, so that the video image received by a receiver is not smooth, even mosaic appears, therefore, the packet loss rate can influence the transmission quality of the video data.

The sending of the data packet bearing the video data is based on RTP (real time transport Protocol) Protocol transmission, wherein RTP is mainly used for transmitting real-time data streams (such as video streams and the like), in addition, the invention utilizes RTCP (real time transport control Protocol) feedback link to send a control packet and a feedback control packet, and the packet loss rate is obtained through the feedback control packet. The RTCP protocol is the supplement and control of RTP protocol, the packet loss rate of RTP protocol transmission can be obtained by feedback control link of RTCP protocol, control packet can be sent periodically by RTCP link, for example, every N can be sent_sThe video data packet is sent to a forward RTCP control packet, and the RTCP control packet is used for analyzing and comparing the difference between the data sent by the sender and the data received by the receiver, so as to calculate the packet loss rate of network transmission. The control packet describes information of the data packet sent this time, such as data packet sequence, number, length, and the like, and after receiving the data packet, the receiving side performs analysis and comparison with the received control packet to detect whether the data packet received this time is consistent with the data packet sent by the sending side, and then sends a feedback control packet to the sending side, and the sending side can know the packet loss rate of this data transmission according to the feedback control packet of the receiving side.

The following describes the implementation of the present invention.

The invention provides a code rate control method of video coding, a sender collects video data from a video acquisition device such as a hardware camera and the like, sends the video data through an RTP link (step 101), regularly sends a control packet through a real-time transmission control feedback link (step 102), obtains an output code rate according to the packet loss rate in the feedback control packet (step 103) when receiving the feedback control packet, and sends the video data through a real-time transmission link according to the obtained code rate (step 104).

For the process of obtaining the output code rate according to the packet loss rate in step 103, the present invention provides various implementation manners, for example: firstly, controlling a target frame rate according to a packet loss rate, and controlling video data acquired by a video acquisition device according to the target frame rate so as to achieve the purpose of controlling an output code rate; secondly, controlling a target rate according to the packet loss rate, obtaining a quantization parameter of the P frame according to the corresponding relation between the target rate and the quantization parameter of the P frame, and controlling an output code rate through the quantization parameter of the P frame; and thirdly, controlling the quantization parameter of the I frame according to the packet loss rate, and controlling the output code rate through the quantization parameter of the I frame. It should be noted that the three implementation manners listed above may only use one manner to implement the control of the output code rate, or may combine any two manners to control the output code rate, or may use the three manners to implement the control of the output code rate simultaneously. The implementation processes of the above three modes are described below, and details of specific combination modes are not described, and those skilled in the art can implement the three modes by conventional technical means.

The following describes a process of controlling an output bitrate according to a target frame rate.

The frame rate is a physical quantity used for describing the speed of the acquired video data, and the larger the frame rate is, the larger the acquired data in unit time is, so that the larger the video data output in unit time is, that is, the larger the output code rate is, and therefore, the purpose of controlling the output code rate can be achieved by controlling the target frame rate.

As an embodiment of the present invention, there is provided an adaptive adjustment formula for obtaining a target frame rate by a packet loss rate,

if(P_th1＜P_loss＜P_th2)

formula (1)

F＝max{(F_i-Δ_f*P_loss)，F_min}

Wherein, F_iTo an initial frame rate, F_minAt minimum frame rate, Δ_fTo increment the frame rate multiplier, P_thIs the packet loss rate threshold.

Two types of frames are included in x.264 coding: the invention relates to a method for coding a key frame and a supplementary frame, wherein the ratio of the key frame to the supplementary frame can be obtained by setting the frequency of the key frame in the coding process. The change of the quantization parameter of the P frame or the I frame causes the change of the coded video data volume, thereby causing the change of the output code rate.

The process of obtaining the quantization parameter of the P frame by the target rate is described below.

In the x.264 coding standard, the macroblock is the smallest unit, a frame is composed of macroblocks, and the frame is further composed of a group of pictures, GOP, the size of data of one macroblock determines the size of data of one frame, and the size of data of one frame also determines the size of data of the GOP layer, which finally affects the output code rate. At the macroblock layer, the key parameter that affects the data size is the quantization coefficient QP_PThus, this disclosure controls QP_PThe size of the coded data is controlled, so that the data quantity of the frame is influenced, the data quantity of the GOP layer is further influenced, and the output code rate R is finally determined.

The invention obtains the target rate through the packet loss rate, and the P frame quantization parameter QP which can be adaptively adjusted through the target rate_P。

As an embodiment of the present invention, an adaptive adjustment formula for obtaining a target code rate according to a packet loss rate is provided herein:

if(P_th1＜P_loss＜P_th2)

formula (2)

R＝max{(R_i-Δ_r*P_loss)，R_down}

In the formula, R_iFor the initial rate, the value 100kb/s, d is taken in one embodiment_ownFor minimum rate, which represents the minimum bandwidth requirement for normal transmission, in one embodiment 50kb/s, R is taken_upFor peak rate, values of 200kb/s, Δ in one embodiment_rTo increment the rate multiplier, in one embodiment the value 200, P, is taken_thThe packet loss rate threshold can be 0-50%. These values can be determined according to the actual situation, such as frame rate, image resolution, etc.

The sender receives the feedback RTCP packet (which carries the packet loss rate P of this time detected by the receiver)_loss) From the above formula, it can be seen that when the packet loss rate P is exceeded_lossWhen the network is in bad condition, the target rate is always adjusted not to be lower than R_downSo as to ensure the minimum bandwidth requirement of the normal transmission of the video data, and the quantization parameter QP of the P frame is adjusted according to the target rate_PAnd obtaining the actual output rate.

The following is a process of obtaining a quantization parameter of a P frame by a target rate.

First, the effective bandwidth u of the network is obtained at the GOP level according to the above formula (2), and the virtual buffer occupancy B before the encoding of the nth frame of the kth GOP is obtained according to the following formula_c(k，n)：

B_c(k，n)＝min(max(0，B_c(k，n-1)+R(k，n-1)-u(k，n-1)/F)，B_s)

In the formula, B_sIs the size of the whole buffer area, B_c(1，1)＝B_s/8, corresponding to B_c(K, n-1) represents the virtual buffer occupancy before the encoding of the (n-1) th frame in the Kth GOP, R (K, n-1) represents the bit size after the encoding of the (n-1) th frame, and u (K, n-1) represents the effective bandwidth at that time.

Secondly, at the frame layer, the data amount available for encoding the current frame is obtained by a complexity comparison technology and the virtual buffer occupancy obtained by combining the GOP layer.

After the (n-1) frame data is encoded, the remaining allocation amount of GOP is T_r(k，n)

$T_r(k,n)=T_r(k,n-1)+\frac{u(k,n)-u(k,n-1)}{F}(N_{\mathrm{gop}}-n)-R(k,n-1)$

The initial values are:

$T_r(k,1)=\frac{u(k,1)}{F}*N_{\mathrm{gop}}-\frac{B_s}{8}+B_c(k-1,N_{\mathrm{gop}})$

determining the bit usage T according to the upper layer_r(k, n) to determine how much data volume T should be used for the frame (i.e., the nth frame)_f(k，n)：

<math><mrow> <msub> <mi>T</mi> <mi>f</mi> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>,</mo> <mi>n</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mrow> <msub> <msup> <mi>MAD</mi> <mi>f</mi> </msup> <mrow> <mi>ratio</mi> <mo>,</mo> <mi>n</mi> </mrow> </msub> <mo>*</mo> <msub> <mi>T</mi> <mi>r</mi> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>,</mo> <mi>n</mi> <mo>)</mo> </mrow> </mrow> <mrow> <mi>N</mi> <mo>-</mo> <mi>n</mi> <mo>+</mo> <mn>1</mn> </mrow> </mfrac> <mo>*</mo> <mrow> <mo>(</mo> <mn>1</mn> <mo>-</mo> <mi>&delta;</mi> <mo>)</mo> </mrow> <mo>+</mo> <mi>&delta;</mi> <mo>*</mo> <mo>[</mo> <msub> <mi>T</mi> <mi>f</mi> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>,</mo> <mi>n</mi> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> <mo>-</mo> <mi>R</mi> <mrow> <mo>(</mo> <mi>k</mi> <mo>,</mo> <mi>n</mi> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> <mo>]</mo> </mrow></math> Formula (5)

In the formula: t is_r(k, n) is the amount of bits used, <math><mrow> <msub> <msup> <mi>MAD</mi> <mi>f</mi> </msup> <mrow> <mi>ratio</mi> <mo>,</mo> <mi>n</mi> </mrow> </msub> <mo>=</mo> <mfrac> <msub> <msup> <mi>MAD</mi> <mi>f</mi> </msup> <mrow> <mi>n</mi> <mo>-</mo> <mn>1</mn> </mrow> </msub> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mrow> <mi>n</mi> <mo>-</mo> <mn>1</mn> </mrow> </munderover> <msub> <msup> <mi>MAD</mi> <mi>f</mi> </msup> <mi>i</mi> </msub> <mo>/</mo> <mrow> <mo>(</mo> <mi>n</mi> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow></math> MAD^f _n-1which represents the complexity of the previous frame, <math><mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mrow> <mi>n</mi> <mo>-</mo> <mn>1</mn> </mrow> </munderover> <msub> <mi>MAD</mi> <mi>i</mi> </msub> <mo>/</mo> <mrow> <mo>(</mo> <mi>n</mi> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> </mrow></math> representing the average complexity of already encoded frames.

Finally, in the macroblock layer, since the usable data amount is already determined in the frame layer, the usable data amount of each macroblock can also be determined, when the usable data amount of the macroblock is certain, the quantization parameter needs to be adjusted to adapt to the usable data amount provided by the frame layer, so far, the quantization parameter QP of the P frame_PTherefore, adaptive adjustment is realized, and a second-order rate distortion model can be adopted to determine the current quantization step size q (m), and the formula is as follows.

$T_{\mathrm{mb}}(k,n,m)=\frac{a*{\mathrm{MAD}}_n^{\mathrm{pred}}\left(m\right)}{Q\left(m\right)}+\frac{b*{\left[{\mathrm{MAD}}_n^{\mathrm{pred}}\left(m\right)\right]}^2}{Q{\left(m\right)}^2}$

In the formula, T_mb(k, n, m) denotes a bit allocation size, MAD, of the mth macroblock (mb)_n ^pred(m) represents the complexity prediction of the current frame, the quantization parameter QP being derived from the above equation_P(m)。

Q(m)＝0.67*2Q^Pp(m)/6

In addition, for image smoothing, some defining work of the quantization parameters can be done.

Through the process, when the packet loss rate is increased, the target rate is reduced, the effective bandwidth is reduced, so that the quantization parameter of the P frame is reduced, the output code rate is reduced, the network transmission is better adapted, and the effect of smaller packet loss rate is achieved.

The process of controlling the output code rate by the quantization parameter of the I frame is described below.

Since the encoded data of the I frame accounts for a large proportion of the video image, an adaptive measure is adopted for the quantization parameter of the I frame according to the packet loss rate. As one embodiment, the present invention provides that the following formula obtains the quantization parameter for the I frame:

${\mathrm{QP}}_I=\{\frac{{\mathrm{Sum}}_{\mathrm{PQP}}}{N_p}-1-\frac{8*T_r(k-1,N_{\mathrm{gop}})}{T_r(k,1)}-\frac{N_{\mathrm{gop}}}{15}\}*\frac{P_{\mathrm{loss}}}{P_{\mathrm{loss}}^{\mathrm{avg}}}$ formula (3)

Wherein,

representing the mean value of the quantization parameter, T, of the P frame_r(k-1，N_gop) Is the bit usage, N, of the k-1 GOP_gopIndicating the number of frames, P, in a group of pictures_loss ^avgRepresents the average packet loss rate, P, in the last group of pictures_lossIndicating the current packet loss rate.

It can be seen from formula (3) that the larger the current packet loss rate is, the smaller the quantization parameter of the I frame is, so that the output code rate is reduced, thereby achieving better adaptation to network transmission and achieving the effect of smaller packet loss rate.

It should be noted that the process of obtaining the output code rate from the quantization parameter of the I frame can be obtained by the means disclosed in the prior art, and is not described herein again. In addition, the user can design different formulas according to actual conditions to obtain the quantization parameter of the I frame.

For the above method for controlling the bit rate of video data, the present invention improves the transmitting device of video data in the prior art, as shown in fig. 2, fig. 2 shows a schematic diagram of a system composed of the transmitting device and the receiving device, and as shown in the figure, the system includes a camera for transmitting video data, the transmitting device for encoding video data and then transmitting the encoded video data, and the receiving device.

The transmitting device 202 includes a data acquisition module, an encoder, an RTP interface, an RTCP interface, and further includes a first control module or a quantization parameter adjuster (QP adjuster) for controlling an output code rate. The receiving apparatus includes an RTP interface, an RTCP interface, and a second control module.

After the data acquisition module acquires video data from the camera, the video data is encoded according to a target code rate through an encoder; then sending the video data packet to the receiving device through the RTP interface, in addition, the transmitting device periodically sends a control packet to the receiving device through the RTCP interface, the receiving device feeds back a feedback message to the second control module according to information of the control packet, the second control module constructs the feedback control packet and sends the feedback control packet to the transmitting device through the RTCP interface, after receiving the feedback control packet, the transmitting device adjusts the output bit rate according to the packet loss rate information therein, as described above, the adjustment mode can be adjusted through the target frame rate, the target bit rate and the quantization parameter. Accordingly, the first control module may include a frame rate control module connected to the data acquisition module, a rate control module connected to the encoder, and a quantization parameter adjuster connected to the encoder. Also as described above, the frame rate control module, the rate control module, and the quantization parameter adjuster may be present individually or in any combination. Fig. 2 shows a schematic diagram of the coexistence of three.

The above-described embodiments of the present invention do not limit the scope of the present invention. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims (12)

1. A method for controlling code rate of video coding, comprising the steps of:

and the sender regularly sends a control packet through a real-time transmission control feedback link, obtains an output code rate according to the packet loss rate in the feedback control packet when receiving the feedback control packet, and sends the video data through the real-time transmission link according to the obtained code rate.

2. The method of claim 1, wherein obtaining the output code rate comprises:

and obtaining a target rate according to the packet loss rate, obtaining a quantization parameter of a P frame according to the target rate, and obtaining the output code rate according to the quantization parameter of the P frame.

3. The method of claim 2, wherein the target rate is obtained according to the following equation:

if(P_th1＜P_loss＜P_th2)

R＝max{(R_i-Δ_r*P_loss)，R_down}

wherein R is_iAs an initial rate, R_downTo minimum rate, Δ_rTo increment the rate multiplier, P_thIs the packet loss rate threshold.

4. The method according to claim 1, 2 or 3, wherein the step of obtaining the output code rate according to the packet loss rate comprises:

and acquiring a target frame rate according to the packet loss rate, acquiring video data according to the target frame rate, and acquiring the output code rate according to the acquired video data.

5. The method of claim 4, wherein the target frame rate is adjusted according to the following formula:

if(P_th1＜P_loss＜P_th2)

F＝max{(F_i-Δ_f*P_loss)，F_min}

wherein, F_iTo an initial frame rate, F_minAt minimum frame rate, Δ_fTo increment the frame rate multiplier, P_thIs the packet loss rate threshold.

6. The method of claim 1 or 2 or 3 or 5, wherein the step of obtaining the output code rate comprises:

obtaining the quantization parameter of the I frame according to the following formula, and obtaining the output code rate according to the quantization parameter of the I frame:

${\mathrm{QP}}_I=\{\frac{{\mathrm{Sum}}_{\mathrm{PQP}}}{N_p}-1-\frac{8*T_r(k-1,N_{\mathrm{gop}})}{T_r(k,1)}-\frac{N_{\mathrm{gop}}}{15}\}*\frac{P_{\mathrm{loss}}}{P_{\mathrm{loss}}^{\mathrm{avg}}}$

wherein, QP_IIs the quantization parameter for the I-frame,

representing the mean value of the quantization parameter, T, of the P frame_r(k-1，N_gop) Bit usage amount, N, for the k-1 st image group_gopRepresenting the number of frames in a group of pictures, P_loss ^avgRepresents the average packet loss rate, P, in the last group of pictures_lossIndicating the current packet loss rate.

7. A transmitting device of video data, comprising:

the data acquisition module is used for acquiring video data;

the encoder is used for obtaining the output code rate of the video data according to the target code rate and encoding the video data according to the output code rate;

a real-time transmission link interface for transmitting the video data; and

a real-time transmission control feedback link interface receives a feedback control packet by periodically sending the control packet;

it is characterized by also comprising:

and the first control module is used for obtaining the target code rate according to the packet loss rate in the feedback control packet.

8. The apparatus for transmitting video data according to claim 7, wherein the control module includes a rate control module configured to obtain a target rate according to the packet loss rate; the target rate is used for obtaining the quantization parameter of the P frame, and the output code rate is obtained according to the quantization parameter of the P frame.

9. The apparatus according to claim 7 or 8, wherein the control module comprises a frame rate control module configured to obtain a target frame rate according to the packet loss rate, and the data acquisition module acquires video data according to the target frame rate.

10. The apparatus for transmitting video data according to claim 9, further comprising a quantization parameter adjuster configured to obtain a quantization parameter of an I frame according to the packet loss rate, wherein the output code rate is obtained according to the quantization parameter of the I frame.

11. A transmitting device of video data, comprising:

the data acquisition module is used for acquiring video data;

the encoder is used for encoding the video data according to an output code rate;

a real-time transmission link interface for transmitting the video data; and

a real-time transmission control feedback link interface receives a feedback control packet by periodically sending the control packet;

it is characterized by also comprising:

and the quantization parameter adjuster is used for obtaining the quantization parameter of the I frame according to the packet loss rate, and the output code rate is obtained according to the quantization parameter of the I frame.

12. The apparatus for transmitting video data according to claim 11, further comprising:

and the first control module is used for obtaining the target code rate according to the packet loss rate in the feedback control packet.

Images