GB2353654A - Processing GOPs to be stored as all I-frames - Google Patents

Abstract

A signal processing system has an encoder 20 which produces an MPEG 2 long Group of Pictures (GOP) IBBPBBPBBPBB. The long GOP is fed to a studio 11 having an I-frame store 12 such as a digital I-frame VTR, via a recoder 22. The recoder 22 recodes the long GOP as all I-frames which match the characteristics of the I-frame store 12. Thus the stored signal may be decoded using any frame as a start frame, such as would be required in editing and other common processes. The encoder 20 is arranged so that the I-frames, <U>ab initio</U>, match the characteristics of the I-frame store 12. The recoder 22 reuses the MPEG transcoding parameters of the original I-frames. In this way picture quality is maintained and may be maintained through several generations of I-frame processing in the studio. A recoder 4 recodes the processed I-frames from store 12 as a long GOP. Such encoding also allows recoding from a long GOP to I-frames and back to a long GOP through several generations whilst maintaining picture quality.

Description

1 2353654 SIGNAL PROCESSING The present invention relates to signal

processing. One aspect of the present invention relates to an encoder for producing a compressed bit stream. Another aspect of the present invention relates to a signal processing system comprising the encoder and a store.

The invention and its background will be discussed by way of example with reference to MPEG 2 video bit streams. However, the invention is not limited to MPEG 2.

MPEG 2 is well known from ISO/IEC/13818-2 and this will not be described here in detail. MPEG 2 compressed bit streams comprise at least I frames and typically also P and/or B frames. The 1, P and/or B frames are arranged in groups called Groups of Pictures (GOP's). An I or intra encoded frame contains all the information of the frame independently of any other frame. A P frame in a GOP is a predicted frame and ultimately depends on an I-frame and may depend on another P frame to be decoded. A B frame is a predicted frame and, whilst ultimately it depends on an I frame, may depend on a preceding and/or succeeding frame (I or P) to be decoded. A B frame must not depend on another B frame. Using P and/or B frames allows greater compression to be achieved than can be achieved with I-frames alone, for the same subjective picture quality. A GOP may comprise typically a sequence of 12 or 15 frames known as "long GOP" which are inter-dependent if they are to be decoded. Common video signal processes such as. editing are relatively straight C forward with conventional analogue frames because all the picture information is contained in each frame. In MPEG 2, if a GOP is edited, information needed to correctly decode the GOP will be lost because information needed to decode frames each side of the edit point in the edited sequence may be lost.

Thus to allow editing and other common processes to be carried out on compressed bit streams in for example television studios it is proposed to store in the studios only I frames. However it is desirable to transmit compressed bit streams as long GOPs to reduce the band width needed in the transmission links. That requires Iona, GOPs to be encoded, decoded and recoded to all I frames, stored as I frames and 2 possibly edited, and re-encoded as long GOPs. Such encoding, decoding to I frames and re-encoding may occur several times over on the same bit stream, which can severely reduce picture quality because encoding and decoding are not loss-less processes. It is desired to maintain picture quality as much as possible.

According to one aspect of the present invention, there is provided a signal processing system comprising an I-frame store for storing I-frames, a recoder for receiving a first generation compression encoded video bit stream comprising I-frames together with P and/or B frames and for recording the bitstream as second generation I-frames, having characteristics which match the I-frame store, for storage in the I frame store, and an encoder for producing the said first generation compression encoded video bit stream comprising I-frames and P and/or B frames, the encoder encoding the first generation I-frames such that they match the characteristics of the I frame store.

Whilst reference is made to the "first generation" video bit stream produced by the said encoder, that bit stream may be a recoding of a previous generation.

Thus the invention controls the encoding of the I frames of the first generation GOPs so that they can be stored in the I frame store, and reproduced from the I-frame store substantially unchanged. Thus at least for the I- frames, picture quality is maintainable despite decoding, storage and recoding possibly several times over.

It will be appreciated that in some circumstances P and B frames may be less than optimally encoded because of the modified encoding of the I-frame. However it is considered that the benefit of modifying the I-frame encoding exceeds any deterioration in the P and/or B frames, especially where several generations of encoding and decoding take place.

According to another aspect of the present invention, there is provided an encoder for compression encoding a video bit stream, the encoded bit stream comprising 1-frames together with P and/or B frames, the encoder including reference means for setting a desired reference target bit allocation for the I- frames, control means for controlling the size of the I-frames, the control means producing an initial targ get bit allocation for of the 1-frames; means for comparing the initial target bit 3 allocation with the reference; and means for modifying the target bit allocation in dependence upon the comparison.

For a better understanding of the present invention reference will now be made by way of example to the accompanying drawings in which:

Figure I is a schematic block diagram of one example of a signal processing system incorporating aspects of the present invention; Figure 2 is schematic block diagram of another example of a signal processing system incorporating the present invention; Figure 3 is a timing diagram illustrating an encoding process; and 10 Figure 4 is a simplified schematic block diagram of an MPEG encoder Figure 1 shows an example of a signal processing system comprising an MPEG-2 encoder 20 having an input for receiving baseband digital video.. The encoder 20 encodes thedigital video bit stream according to MPEG 2. In the example, the encoder 20 produces a long GOP of 12 or 15 frames, e. g.

15.... IBBPBBPBBPBB for a 12 frame GOP in known manner.

This long GOP is referred to herein as Generation 1 (Gen 1).

The compressed video bit stream is transmitted to a studio or other signal processing plant 11. For ease of processing compressed bit streams in plant 11, the plant I I comprises a store 12 which stores only I-frames. Examples of such a store include an I - frame video server, an I- frame digital disc recorder, and an I-frame video tape recorder.

Thus the long GOP of Gen 1 is decoded to baseband and recoded as all Iframes in a recoder 22. The I- frames are then stored in the store 12, as Generation 2 (Gen 2).

The I- frames may be simply reproduced from the store 12 for transmission onwards. The I- frames may be subject to other processing: for example, the store 12 may be part of an editing suite 1.

The I- frames of Gen 2 are decoded to baseband and recoded in a recoder 4 as Generation 3 (Gen3).

4 Figure 2 shows an example of signal processing in the plant 11. In this example a first generation bit stream A produced by an encoder such as 20 in Figure I is supplied to a first input A.,and another first generation bit stream B produced by another encoder such as 20 in Figure I is supplied to a second input B. The bit streams are supplied to recoders 22A and 22B corresponding to recoder 22 of Figure 1, in which the bit streams A and B are decoded to baseband and recoded to I- frame bit streams.

The I frame bit streams IA and IB are stored in respective l- frame stores 121 and 122, each corresponding to store 12 of Figure 1. An editing operation, e.g.

splicing of the bit streams IA and IB, is carried out in the editing suite 1, as schematically indicated by switch S1. The spliced bit stream IA /IB is stored in another 1-frame store 123 again corresponding to store 12 of Figure 1 as Generation 2.

The spliced bitstream is then recoded in a recoder 4 as generation 3 (Gen 3).

The I frame stores, 12, 121, 122, 123, are for example I- frame digital VTRs which have for example a maximum input/output rate of 50 Mbit/sec. Thus Gen 2 has a bit rate of 50 Mbit/sec and a relatively low compression ratio. Gen I has a relatively high compression ratio and a lower bit rate. Examples of the lower bit rate are 30 Mbit/sec, and 15Mbit/sec.

The recoder 22, which may be part of the I- frame store 12, produces Iframes which match the characteristics of the I- frame store 12.

Embodiments of the invention can operate at a bit rate for Generation I of 15 Mbit/sec. However it is preferred that the bit rate for Gen I is 30 Mbit/sec because the I frames at Gen I would then be similar in size to the I-frames of Gen 2 (in the absence of the present invention). That is:

a) the bit rate of the I- frames produced by the recoder 22 has a maximum rate which matches the maximum input rate of the store 12: and b) all the I-frames produced by the recoder 22 have the same maximum size, or in other words the bit rate is constant.

The I-frames of Generation 2 produced by the recoder 22 are derived from the 1, P and/or B frames of the long -GOP of Generation 1. Assuming no splicing or other processing which disrupts the GOPs, the recoder 4 regenerates from the I- frames of generation 2 the original I, P and /or B frames of the GOPs of generation 1. For that purpose, the transcoding parameters of the 1, P and/or B frames of generation 1 are retained in association with the corresponding I-frames of generation 2 as user data in the MPEG bit stream. The transcoding parameters are used in recoder 4 to regenerate approxiamately the original 1, P and/or B frames of generation 1. The decoding and recoding: processes cause errors which result in the frames of Gen 3 having different niumbers. of bits than the frames of Gen 1. Reuse of transcoding parameters minimises any loss of picture quality.

Referring to Figure 3, if a splicing operation takes place which splices a bit stream A to a bit stream B there is a transitional region, including the splice point, in which the frames are fully recoded by the recoder 4, without use of the original transcoding parameters. Before the splice point the transcoding parameters of bit stream A are reused and after the splice point the transcoding parameters of bit stream B are reused.

In the recoder 22, the transcoding parameters of the I frames of the GOPs of generation I are used in the recoding of these I frames so that the recoded I frames produced by the recoder 22 are identical to the corresponding 1-frames of generation 1.

The decoding and recoding of such I-frames is (virtually) loss-less.

The transcoding parameters of the 1-frames of generation 1, that is a) Q b) DCT-type and c) Q-Matrix are controlled in the encoder 20 so that the I frames as originally produced have the appropriate transcoding parameters which allow the 1-frames as recoded by recoder 22 to match the characteristics of the I-frame store without needing to modify those transcoding parameters.

In an MPEG encoder, separate target bit rates are set for 1, P and B frames.

The target bit rates for I, P and B frames are controlled by varying the quantisation of 0 4n the frames.

The control may be performed by any bit rate control system known in the art.

One example is 6 "Test Model 5" published by "International Organisation for Standardisation, Organisation International De Normalisation, Coded Representation of Picture and Audio Information, ISO/EEC/JTCI/SC29/WGI LN0400."

Test Model 5 is not the only rate control system which may be used in the present invention but it is the currently preferred example of a rate control system for implementing the present invention. In Test Model 5 a target bit allocation occurs which is a step of estimating the number of bits available to code the next frame. It is performed before encoding the frame. In embodiments of the present invention, Test Model 5 determines in its normal manner the target bit allocation. In the embodiments however this is an initial allocation. The initial allocation is compared with desired conditions as set out in Examples I to 4 below for example and if the initial allocation does not comply with the desired conditions it is modified so as to comply.

In "Test Model 5" picture target bit rates, called "Target bit Allocations" Ti, Tp, Th are set for 1, P and B frames before coding those frames. As discussed in "Test Model 5", the targets depend on: "universal constants Kp and Kb (which in turn are dependent on quantisation matrices); the number of bits remaining in a GOP; Np and Nb which are the numbers of P and B frames remaining in a GOP; and complexity measures X!, Xp, Xb. The target bit allocations of a GOP are dependent on the parameters set for a previous GOP. The actual bit rate is monitored macro-block by macro-block of e.g. a I-frame, by means of a virtual buffer.

The quantisation parameter Q is controlled in dependence upon the actual bit rate to achieve the target bit allocation and thus the target bit rate. In practice the target bit rate tends not to be exactly achieved. Thus the targets are approxiamations to what is actually achieved.

Reference will be made, by way of example, to Test Model 5 in the following discussion. In embodiments of the present invention, the target bit allocation Ti for Iframes initially determined by Test Model 5 operating normal but the embodiment smodify the initial target bit allocation according to one of the following 4 examples.

Exmple I A desired maximum target allocation is set 7 The initial target bit allocation for the I frames, as set by Test Model 5, is adjusted so that it does not exceed a maximum reference allocation which is a preset percentage of the maximum frame size in bits of the I-frames in Generation 2:

i.e. (Gen 2 bit rate)/(Gen 2 frame rate).

Initial target bit allocations greater than the refernee allocation are reduced to the refernce allocation. Initial target allocations less than or equal to the reference are notchanged.

Example 2 The target allocation for all I frames is forced to the reference allocation.

Theinitial target bit allocation for the I-frames as set by Test Model 5 is adjusted so that all 1-frames have the reference target bit allocation which is the same as in Example 1. Initial target bit allocations less than or greater than the reference are modified to comply with the reference.

This may reduce the bits available for the P and B frames of a long GOP, if the resulting I frame of the GOP has a greater number of bits than normal encoding would produce.

ExMple 3 The target allocation for I frames is conditionally forced to the reference allocation A reference allocation (which is the same as in Example 1) and a threshold allocation (which is less than the reference) are set for I frames. Test Model 5 produces an initial target bit allocation as normal. If the initial allocation is less than the threshold, then theallocattion for I-frame is not changed. If the initial allocation exceeds the threshold, but is less than the reference then the allcation is changed to comply with the reference. This is applied to all I-frames.

ExMple 4 The target allocation for I frames is conditionally forced to the reference allocation A reference allocation (which is the same as in Example 1) is set for I- frames.

Test Model 5 sets an initial target bit allocation Ti. If an I-frame target is capable of complying with the target provided it is not increased in size by more than a predetermined percentage of the initial target bit allocation set by Test Model 5, then 8 the initial target is increased to the reference target. Otherwise the initial target allocation is increased in size by the predetermined percentage.

In examples 1 to 4, initial targets indicated to be too small are controlled to be bigger, and initial targets indicated to be too big are controlled to be smaller, subject to the conditions set out in the examples.

In examples 1 to 4 the reference target is a preset percentage of the maximum size in bits of the I-frames of Generation 2. By way of example it may in the range of 90% to 98% preferably 97% of the maximum size.

In example 3 the threshold target, by way of example, may be in the range 70% to 90% of the reference target, preferably about 80% of the reference target.

In example 4 the predetermined percentage, by way of example, may be in the range 20% to 50%. Preferably it may be about 25%.

I-frame size and bit rate For an I-frame store 12 having a maximum data transfer rate of 50 Mbits/sec (which is typical of current digital I-frame VTRs), and for I-frames produced at e.g. 30 frames/sec, an I-frame has a maximum of 1.666 Mbits.

Thus to produce I-frames at a maximum of e.g. 97% of the maximum bit rate of 50 Mbits/sec, the I-frames are controlled to have a maximum size of 97 1.666 Mbits i.e. about 1.62 Mbits, for the purposes of examples 1 to 4.

For example 3, the threshold target then corresponds to an I-frame size of 80% of 1.62 Mbits, i.e. about 1.3 Mbits.

For example 4, the number of bits corresponding to the predetermined percentage depends on the size of the initial target set by Test Model 5 Bit Rate The bit rate produced by encoder 20 is less than the bit rate produced by the I frame recoder 22 because the large GOP produced by encoder 20 has P and B frames providing a higher degree of compression than the I-frames produced by recoder 22.

Example of Encoder 20 9 Figure 4 shows in a conventional manner, a highly simplified block diagram of an MPEG 2 encoder.

Figure 4 is a schematic diagram of a video data compression apparatus comprising a frame reorderer 40, a motion estimator 30, a motion predictor 201, a subtractor 50, an adder 60, a DCT transform unit 80, a quantiser 90, an entropy encoder 100, an inverse quantiser 120 and an inverse DCT transform unit 130.

The apparatus of Figure 4 operates in accordance with MPEG 2 and so will not be described in detail here.

P-pictures are encoded with respect to the nearest previous 1-picture or P picture, so that only the differences between a P-picture and the previous P- or I picture need to be transmitted. Also, motion compensation is used to encode the differences, so a much higher degree of compression is obtained than with I-pictures.

"B-pictures" or bi-directional pictures are encoded with respect to two other pictures, namely the nearest previous I- or P-picture and the nearest following I- or P picture. B-pictures are not used as references for encoding other pictures, so a still higher degree of compression can be used for B-pictures because any coding errors caused by high compression will not be propagated to other pictures.

Therefore... in each GOP there are (up to) three classes of picture, 1-, P- and Bpictures, which tend to achieve different degrees of compression and so tend to require different shares of the overall available encoded bit stream. Generally, I-pictures require a large share of the available transmission or storage capacity, followed by Ppictures, and followed by B-pictures.

Briefly, therefore, the frame reorderer 40 receives input video data and acts on successive groups of pictures (GOP) to reorder the pictures so that each picture within the GOP is compressed after those pictures on which it depends. For example, if a Bpicture (bi-directionally predicted picture) depends on a following I-or P-picture, it is reordered to be compressed after that I- or P-picture.

For example, if a GOP comprises the following four initial frames (in the order in which they are displayed), LBS2P3..., where the P-picture uses the 1-picture as a reference and the two B-pictures use the surrounding Iand P-pictures as references, then the frame reorderer 10 will reorder the GOP to be compressed in the following order; loP,B,B2 I- pictures are intra-picture encoded, that is to say the encoding is not based on any other reference pictures. An I -picture in a GOP is therefore passed from the frame reorderer 40 to the DCT transform unit 80, the auto quantiser 90 and the entropy encoder 100 to generate output compressed data representing that I- picture.

The compressed I-picture data is also passed through a decompression chain formed by the inverse quantiser 120, and the inverse DCT transform unit 130. This reconstructs a version of the I-picture present in the decoder which is passed to the motion predictor 20.

The next picture of the GOP to be compressed, which will generally be a P picture which depends on the I-picture as a reference, is passed from the frame reorderer 40 to the motion estimator 30 which generates motion vectors indicative of imacre motion between the I-and P-pictures. The motion predictor 201 then generates a predicted version of the P picture using the motion vectors and the decoded version of the I-picture. This predicted version of the P-picture is subtracted from the actual P-picture by the subtractor 50 and the difference between the 2 frames is passed to the DCT transform unit 80 for compression. As before, the encoded (compressed) difference data is output by the entropy encoder 100 and is decoded by the decompression chain 120, 130 to regenerate a version of the difference data.

In the adder 60 the difference data is then added to the previously decompressed version of the I-picture to generate a decompressed version of the P - picture which is then stored in the motion predictor 40 for use in the compression of the next picture.

This process continues, so that each picture which uses other pictures as a reference is in fact compressed by encoding difference data between the input picture and a version of the input picture formed by motion prediction from a previously compressed and then decompressed version of the reference picture. This means that the compression is performed with the respect to the pictures which will be available at the decompressor.

The quantiser 90 quantise the DCT encoded data to achieve the target bit rates.

Inpractice the target is not exactly achieved. The target bit rates are set separately for I, P and B frames. They are set by a bit rate control 140. In conventional MPEG encoders the bit rate control is dependent on the content of a buffer 150. The control 140 controls the quantiser 90 to achieve the target bit rate and to keep the buffer 150 usually half full preventing over - and under-flow. The control 140 operates in accordance with Test Model 5 for example.

In accordance with the present invention, in the illustrative embodiment of Figure 4 the bit rate control 140 also receives from a Target Reference unit 160 the reference target bit allocation (equivalent to 97% of the maximum transfer rate of the I-frame store 12), for the purpose of the examples 1 to 4 above.

For the purposes of examples 3 and 4, the reference unit 160 also sets the threshold level and the predetermined percentage respectively.

The buffer 150 provides measures of the size of I-frames (and other frames).

In accordance with the example of the preset invention, the target bit allocation produced by Test Model 5, in dependence on the size measures provided by the buffer, is modified in that the target bit allocation produced by Test Model 5 is compared in a comparison unit 141 with the reference from reference unit 160 to adjust the taregt bit allocation. The bit rate control then supplies to the quantiser 90 quantisation parameters toenable it to produce I-frames of the sizelrate required in accordance with Example 1, 2, 3 or 4.

It will be appreciated that the encoder 20 operates independently of the I-frame store in the systems of Figures 1 and 2. It is assumed that it is known in advance that the encoder 20 is to be used with the I-frame store 12 and the characteristics of the P frame store 12 are known. The encoding parameters of the encoder 20 are set to match these of the store 12.

The encoder of Figure 4 may be implemented as a hardware system or by software in a special purpose data processor.

Whilst the invention has been described by way of example with reference to Test Model 5, it may be used with any other rate control system for controlling the sizes/bit rates of the frames.

12

Claims (12)

1. A signal processing system comprising an I-frame store for storing Iframes, a recoder for receiving a first generation compression encoded video bit stream comprising I-frames together with P and/or B frames and for recoding the bit stream as second generation I-frames, having characteristics which match the I-frame store, for storage in the I-frame store, and an encoder for producing the said first generation compression encoded video bit stream comprising I-frames and P and/or B frames, the encoder encoding the first generation 1-frames such that they match the characteristics of the I-frame store.

2. A system according to claim 1, wherein the recoder reuses the transcoding parameters of the first generation I-frames to recode them as second generation I-frames for storage in the I-frame store.

3. A system according to claim I or 2, wherein the encoder comprises means (160) for setting a reference target bit allocation for the first generation Iframes, and means (140, 90) for controlling the production of the I-frames in dependence on the reference allocation so that they do not substantially exceed the reference allocation.

4. A system according to claim 3, wherein the controlling means (140, 141, 90) is arranged to control the production of the I-frarnes so that all the first generation I-framessubstantially have the reference allocation..

5. A system according to claim 3, wherein the encoder comprises means (150, 140, 90) for controlling the size of the first generation I-frames, the controlling means producing an initial targetbit allocation for the first generation I-frames, means (160) for setting the reference allocation and a threshold allocation less than the reference, means (141) for comparing the initial target with the reference and threshold allocations, and means (140, 90) for modifying the target for those first generation I- 13 frames that have an initial allocation greater than the threshold but less than the reference to the reference allocation.

6. A s ystern according to claim 5, wherein the allocation of I-frames having an initial target target less than the threshold is increased by a preset percentage.

7. A system according to any preceeding claim, comprising a further encoder for recoding the stored I-frames as a third generation video bit stream comprising I-frames together with P and/or B frames.

8. An encoder for compression encoding a video bit stream, the encoded bit stream comprising I-frames together with P and/or B frames, the encoder including reference means (160) for setting a desired reference target bit allocation for the I- frames, control means (150) for controlling the size of the 1-frames, the control means producing an initial target bit allocation for of the 1- frames; means (140) for comparing the initial target bit allocation with the reference; and means (140,90) for modifying the target bit allocation in dependence upon the comparison.

9. An encoder according to claim 8, wherein the modifying means (140, 90) is arranged to modify the target allocations for the I-frames in dependence on the comparison so that the target allocation does not substantially exceed the reference allocation

10. An encoder according to claim 9, wherein the modifying means (140, 141, 90) is arranged to modify the target bit allocation of all the I- frames so that all the first generation 1-frames have the reference allocation

11. A system according to claim 9, wherein the reference means (160) is 0 arranged to set the reference allocation and a threshold allocation less than the reference, and the comparing means (141) is armged to compare the initial target with 14 the reference and threshold allocations, and the modifying means (140, 90)is arranged to modify the target for those first generation I-frames that have an initial allocation greater than the threshold but less than the reference to the reference allocation.

12. An encoder according to claim 11, wherein the target allocation of P frames having an initial target less than the threshold is increased by a preset percentage.