RU2463728C2 - Enhancing layer improved coding for scalable video coding - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: each nonzero coefficient of the enhancing layer coefficient vector is encoded without knowing any other subsequent coefficients. Encoding the enhancing layer in a single pass can avoid the need to perform a first pass for analysing the coefficient vector and a second pass for encoding the coefficient vector based on the analysis.

EFFECT: providing statistical encoding of the bit stream of an enhancing layer in one encoding pass, thereby simplifying encoding, reducing encoding delay and memory requirements.

66 cl, 15 dwg

Description

This application claims priority by provisional patent application US No. 60/979919, filed October 15, 2007, and provisional patent application US No. 60/940214, filed October 16, 2007. The contents of each of these applications are hereby incorporated by reference.

FIELD OF THE INVENTION

This disclosure relates to digital video encoding and, in particular, to scalable video encoding.

BACKGROUND

Digital video capabilities can be enclosed in a wide range of devices, including digital TVs, direct digital broadcasting systems, wireless devices, wireless broadcasting systems, personal digital assistants (PDAs), laptops or desktop computers, digital cameras, digital recording devices , video game devices, game consoles, cellular or satellite radiotelephones, etc. Devices for working with digital video implement video compression techniques, such as the standards of the Moving Picture Expert Group (MPEG) -2, MPEG-4, or the standard of the standardization sector of the International Telecommunication Union (ITU-T) H.264 / MPEG-4, Part 10, Advanced Video Encoding (AVC) (hereinafter “MPEG-4 / H.264 Standard, Part 10, AVC”) for more efficient transmission and reception of digital video. Video compression techniques perform spatial and temporal prediction to reduce or remove the redundancy inherent in video sequences.

In video encoding, video compression typically includes spatial prediction and / or motion estimation and motion compensation to form a video prediction block. Internal coding relies on spatial prediction to reduce or remove spatial redundancy among video blocks within a given coding element, such as a frame or tile. In other words, the video encoder performs spatial prediction to compress data based on other data within the same encoding element. In contrast, external coding relies on temporal prediction to reduce or remove temporal redundancy among video blocks of sequential video frames of a video sequence. Thus, for external encoding, the video encoder performs motion estimation and motion compensation to track the movement of matching video blocks from two or more adjacent encoding elements.

After spatial or temporal prediction, a residual coefficient block (called a residual block or residual information) is generated by subtracting the prediction video block from the original encoded video block. The residual block may be a two-dimensional matrix of coefficient values that determine the number of differences between the prediction video block and the original block. The video encoder may apply transform, quantization, and statistical coding processes to the residual block to further reduce the bit rate associated with transmitting the residual block. Transformation techniques may include discrete cosine transforms (DCT), wavelet transforms, integer transforms, or other types of transforms.

In a DCT transform, for example, the transform process converts a set of coefficients in the pixel region into transform coefficients, which are the energy of the coefficients in the pixel region in the frequency or transform region. Quantization is applied to transform coefficients to form quantized transform coefficients. Quantization usually limits the number of bits associated with any given coefficient. The video encoder statistically encodes the quantized transform coefficients to further compress the quantized transform coefficients. A video encoder can statistically encode coefficients using variable length coding (VLC), arithmetic coding, fixed length coding, or a combination thereof. The video decoder can perform inverse operations to recreate the video sequence.

Some video encoding standards, such as MPEG-2, encode videos with relatively consistent quality, bit rate, or spatial resolution. Such a technique may be sufficient to provide video applications for devices having similar decoder capabilities (e.g., memory or processing resources) and / or connection quality. However, more modern video transmission systems typically include devices with variable decoder capabilities and / or connection qualities. In such systems, the transmission of video encoded with a relatively constant quality, bit rate or spatial resolution leads to the fact that video applications work for devices that have suitable decoder capabilities and / or connection quality and do not work for devices that do not have suitable decoder capabilities and / or connection quality. In the context of wireless communications, for example, devices located closer to the video transmission source may have better connection quality than devices located further from the source. Thus, devices located farther from the source may not be able to receive encoded video transmitted with constant quality, bit rate or spatial resolution.

Other video coding standards use scalable coding techniques to overcome these problems. Scalable video encoding (SVC), for example, in accordance with the extension of the ITU-T H.264 / MPEG-4 standard, part 10, AVC is called video encoding, in which a video sequence is encoded as a base layer and one or more scalable enhancement layers. For SVC, the base layer usually carries video with a basic spatial, temporal and / or quality level. One or more enhancement layers carry an incremental video to support higher spatial, temporal and / or quality levels. Enhancement layers can, for example, add spatial resolution to the frames of the base layer or can add additional frames to increase the overall frame rate. In some cases, the base layer may be transmitted in a more reliable manner than the transfer of enhancement layers. Thus, devices located further from the encoded video source or with lower decoder capabilities may be able to accept the base layer and thus the video sequence, albeit with the lowest spatial, temporal and / or quality level.

SUMMARY OF THE INVENTION

This disclosure describes scalable video coding techniques that enable statistical coding of an enhancement layer bitstream in a single coding pass. Traditionally, the enhancement layer bitstream is encoded using multiple coding passes. For each video block of the enhancement layer, for example, the first coding pass can collect statistics for the block to use when selecting coding tables (or codebooks) for statistical coding of the block, and the second coding pass can statistically encode the block using the selected coding tables. However, in accordance with the techniques in this disclosure, video blocks of the enhancement layer bitstream are statistically encoded without performing a first encoding pass to collect statistics for use in selecting a video encoding table.

Instead, the enhancement layer is encoded using an encoding technique that encodes the coefficients of the enhancement layer one by one in a single encoding pass. In one case, a video encoder for each of the non-zero coefficients of the video block of the enhancement layer may encode a block end symbol (EOB), a series length, and a character. The video encoder can use only one coding table to encode the video block of the enhancement layer, thereby eliminating the need to perform the first coding pass to accumulate statistics that should be used when selecting coding tables.

In addition, the video encoder may not encode the value of non-zero coefficients in the enhancement layer. In this method, the value of all non-zero coefficients of the enhancement layer can be limited to a value of unity. The lack of coding of the magnitude of the coefficients of the enhancement layer may result in some loss of the peak signal to noise ratio (PSNR), but reduces the number of bits used to encode the enhancement layer. The techniques of this disclosure may provide several advantages. For example, techniques can reduce coding complexity, coding delay, and memory requirements for coding an enhancement layer bitstream while maintaining coding efficiency.

In one aspect, a method of encoding video using scalable video encoding comprises the step of encoding a video block with a first quality as part of a base layer bitstream. The method also includes a stage where, as part of at least one bit stream of the enhancement layer, video block refinements are encoded, which, when combined with the video block encoded with the first quality, result in a video block having a second quality that is greater than the first quality. The method also includes refinement of the video block, which must be encoded in one coding pass.

In yet another aspect, a device for encoding video using scalable video encoding comprises at least one encoder that encodes a video block with a first quality as part of a base layer bitstream and as part of at least one enhancement layer bit stream encodes video block refinements that, when combining with a video block encoded with the first quality, the result is a video block having a second quality that is greater than the first quality. Video block refinements are encoded in a single coding pass.

In yet another aspect, a computer-readable medium comprises instructions to cause one or more processors to encode a video block with a first quality as part of a base layer bitstream and as part of at least one enhancement layer bit stream to encode video block refinements that, when combined with a video block encoded with the first quality, result in a video block having a second quality that is larger than the first quality. Video block refinements are encoded in a single coding pass.

In yet another aspect, an apparatus for encoding video data using scalable video encoding comprises first means for encoding a video block with a first quality as part of a base layer bitstream and second means for encoding as part of at least one bit stream of an enhancement layer of video block refinements that, when combined with the video block encoded with the first quality, the result is a video block having a second quality that is greater than the first quality. Video block refinements are encoded in a single coding pass.

In yet another aspect, a method for decoding video data using scalable video encoding comprises the steps of decoding a base layer bitstream to obtain a video block with a first quality and decode an enhancement layer bitstream to obtain video block refinements that, when combined with a video block decoded with first quality result in a video block having a second quality. Decoding the enhancement layer includes the step of decoding a symbol for each non-zero refinement coefficient of the video block indicating the presence of at least one remaining non-zero coefficient, a series length indicating the number of zero coefficients preceding the non-zero coefficient, and a non-zero sign from zero coefficient.

In yet another aspect, a device for decoding video data using scalable video encoding comprises at least one decoder that decodes the base layer bitstream to obtain a video block with a first quality and decodes the enhancement layer bitstream to obtain refinements to the video block, which when combined with the video block, decoded with the first quality, the result is a video block having a second quality. At least one decoder for each non-zero refinement coefficient of the video block decodes a symbol indicating the presence of at least one remaining non-zero coefficient, a series length indicating the number of zero coefficients preceding the non-zero coefficient, and the sign of the non-zero coefficient.

In yet another aspect, a computer-readable medium comprises instructions to cause one or more processors to decode a base layer bitstream to obtain a video block with a first quality and decode an enhancement layer bitstream to obtain video block refinements that, when combined with a video block decoded with the first quality, give as a result, a video block having a second quality. The commands cause one or more processors to decode a symbol for each non-zero refinement coefficient of the video block indicating the presence of at least one remaining non-zero coefficient, a series length indicating the number of zero coefficients preceding the non-zero coefficient, and the sign of a non-zero coefficient.

In yet another aspect, a device for decoding video data using scalable video encoding comprises first means for decoding a base layer bitstream to obtain a video block with a first quality and second means for decoding an enhancement layer bitstream to obtain refinements to the video block, which when combined with the video block decoded with the first quality, result in a video block having a second quality. The second decoding means for each non-zero refinement coefficient of the video block decodes a symbol indicating the presence of at least one remaining non-zero coefficient, a series length indicating the number of zero coefficients preceding the non-zero coefficient, and the sign of the non-zero coefficient.

The techniques described in this disclosure may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the software may execute on a processor, which may refer to one or more processors, such as a microprocessor, application specific integrated circuit (ASIC), programmable gate array (FPGA), digital signal processor (DSP), or other equivalent integrated or discrete logic circuit. Software containing instructions for executing the techniques may first be stored on a computer-readable medium and downloaded and executed by a processor.

Accordingly, this disclosure also contemplates computer-readable media containing instructions to cause a processor to execute any variety of techniques described in this disclosure. In some cases, the computer-readable medium may form part of a computer program product that can be sold to manufacturers and / or used in a device. The computer program product may include computer-readable media, and in some cases may also include packaging materials.

Details of one or more aspects of the disclosure are set forth below in the accompanying drawings and in the description. Other features, objectives, and advantages of the techniques described in this disclosure will be apparent from the description and drawings, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

1 is a block diagram illustrating a video transmission system that supports video scalability.

FIG. 2 is a block diagram illustrating a source device and a destination device of the encoding system shown in FIG. 1 in further details.

FIG. 3 is a block diagram illustrating an example base layer encoder and an enhancement layer encoder in further detail. FIG.

4 is a block diagram illustrating an example statistical base layer encoder and an enhancement layer statistical encoder in further detail.

5 is a block diagram illustrating an example of a base layer decoder and an enhancement layer decoder in further detail.

6 is a block diagram illustrating an exemplary statistical base layer decoder and an enhancement layer statistical decoder in further detail.

7 is a conceptual diagram illustrating a zigzag scan of a block of 4 × 4 coefficients.

8 is a conceptual diagram illustrating a hypothetical example of a coefficient block of a video block of an enhancement layer.

9 is a flowchart illustrating an example operation of a video encoder performing scalable video coding techniques of this disclosure.

10 is a flowchart illustrating an exemplary operation of an enhancement layer encoder encoding the residual coefficients of a video block of an enhancement layer in accordance with one aspect of this disclosure.

11 is a flowchart illustrating an exemplary operation of an enhancement layer decoder decoding an enhancement layer bitstream to obtain a vector of residual transform coefficients.

12-15 are block diagrams illustrating various configurations of encoders and / or decoders for use in scalable video encoding in accordance with this disclosure.

DETAILED DESCRIPTION

1 is a block diagram illustrating a video transmission system 10 that supports video scalability. In the example of FIG. 1, the video transmission system 10 includes a source device 12 and a plurality of destination devices 14A, 14B (collectively, “destination devices 14”). Source device 12 receives digital video content from one or more sources and encodes video content for transmission to destination devices 14. Video content, for example, can be captured in real time or in close to real time mode, archived (for example, pre-captured), generated on a computer, or a combination of the above. In each case, the video content can be encoded by the source device 12 for transmission to the destination device 14 via a communication channel. The source device 12 may include a transmitter or be connected to a transmitter that includes suitable components for modulating radio frequency (RF), filtering, and an amplifier to drive one or more antennas for delivering encoded video over a communication channel.

To support scalable video, source device 12 encodes the source video as a base layer bitstream (or base layer) and one or more scalable enhancement layer (or enhancement layer) bit streams. The base layer bitstream typically carries video data with a basic level of quality. One or more enhancement layers carry incremental video data, referred to herein as refinements, to support higher quality levels. Refinements encoded in enhancement layers, for example, can gradually increase fidelity (for example, visual quality) by providing additional higher frequency coefficients or further refinement of existing coefficients. In some cases, the base layer may be transmitted in a more reliable manner than the transmission of enhancement layers, for example with a lower probability of packet distortion (PER).

In the example illustrated in FIG. 1, for simplicity, a base layer and a single enhancement layer of one channel are shown. However, source device 12 may encode more than one enhancement layer carrying additional channel video data. In some cases, source device 12 may encode the source video in separate bitstreams to support different channels for users to associate with destination devices 14. The channels are usually transmitted simultaneously so that the destination devices 14 can select different channels for viewing at any time. Thus, the destination devices 14 under the control of the user can select one channel for watching sports programs and then select another channel for watching news or some other event planned for the program, which is very similar to the practice of watching TV shows. In general, each channel can be encoded as a base layer and one or more enhancement layers.

In addition, the techniques of this disclosure are described for illustrative purposes in the context of quality scalability (also called signal-to-noise ratio scalability (SNR)). However, the techniques can be extended to spatial scalability. In spatial scalability applications, the base layer carries video with basic spatial resolution, and the enhancement layers carry additional video data to support higher spatial resolution. In some cases, system 10 may utilize video scalability, which combines signal-to-noise ratio (SNR) scalability, spatial and / or temporal scalability.

Source device 12, for example, can encode the source video as a base layer in accordance with the H.264 / MPEG-4 SVC extension, Part 10, AVC and encode the source video as an enhancement layer in accordance with the techniques described in this disclosure. Thus, the techniques described in this disclosure may be applied in some aspects to implement video scalability extensions for devices that are otherwise consistent with the H.264 standard. In fact, the techniques of this disclosure may represent potential modifications for future versions or extensions of the H.264 standard or other standards. However, the techniques can be used in conjunction with any variety of other video compression standards such as MPEG-1 and MPEG-2, ITU-T H.263, 421M Video Codec Standard of the Society of Film and Television Engineers (SMPTE) (commonly referred to as "VC-1" ), a standard defined by the China Audio and Video Coding Standards Working Group (commonly referred to as “AVS”), as well as any other video coding standard defined by the standardization body or developed by the organization as a private standard.

Destination devices 14 may support wired and / or wireless reception of encoded video. Destination devices 14 may comprise any device capable of receiving and decoding digital video data, such as wireless communication devices, such as cellular or satellite radiotelephones, wireless broadcasting systems, personal digital assistants (PDAs), laptops or desktop computers, digital cameras, digital recording devices, video game devices, game consoles, digital TVs, direct digital broadcasting systems, etc. In the example of FIG. 1, two destination devices 14A, 14B are shown. However, the system 10 may include any number of destination devices 14. Destination devices 14 may also operate in accordance with any of the variety of video compression standards described above.

Figure 1 represents the positioning of destination devices 14 relative to a source device 12 that transmits encoded video. In particular, the destination device 14A is closer to the transmission source, that is, the source device 12 in FIG. 1, and the destination device 14B is further away from the transmission source. In cases in which the base layer is encoded with a lower packet distortion probability (PER), both destination devices 14A and 14B can reliably receive and decode the base layer. The destination device 14A, which is located closer to the source device 12, can also reliably receive the enhancement layer. However, the destination device 14B, which is further located further from the source device 12, cannot reliably receive the enhancement layer, for example, due to network or channel conditions.

Thus, the closer destination device 14A is capable of working with higher quality video because it has both base layer and enhancement layer data available, while destination device 14B is capable of representing only the minimum quality level provided by the base layer data. Thus, the video received by destination devices 14 is scalable in the sense that additional bits of the enhancement layer can be decoded and added to the bitstream of the base layer to increase the signal-to-noise ratio (SNR) of the decoded video. However, scalability is only possible when the data of the enhancement layer is presented. Thus, the term “quality” as used in this disclosure may refer to objective and / or subjective visual quality. In other words, refinement of the enhancement layer results in video data with higher quality reproduction of the original data. In this method, video fidelity is enhanced by an enhancement layer.

In other cases, the network or channel conditions may be sufficient for both destination devices 14A and 14B to receive the base layer and the enhancement layer. However, destination devices 14A and 14B may have different decoder capabilities that prevent one of destination devices 14A and 14B from using additional enhancement layer video data to play higher quality video. If one of the destination devices 14 is such a client device, such as a mobile handset or other small portable device, there may be limitations due to computational complexity and memory requirements. Accordingly, scalable video coding can be created in such a way that destination devices 14 with limited computing or memory resources can decode only the base layer. In this method, destination devices 14 with better network or channel conditions and / or better decoder capabilities will be able to recreate higher quality video using additional video layer enhancement data.

The techniques described in this disclosure employ statistical coding techniques that facilitate efficient coding of enhancement layer bit streams. The statistical encoding techniques of this disclosure may enable the encoding of additional video data, for example, in the form of refinements in the bitstream of the enhancement layer in one encoding pass, thereby reducing encoding complexity, encoding delay, and memory requirements. As will be described in further detail, the source device 12 may, in some cases, encode each nonzero coefficient vector of the coefficients of the enhancement layer without knowing any subsequent coefficients, that is, any coefficients after the non-zero coefficient currently encoded. Encoding a one-pass enhancement layer can eliminate the need for a first pass to analyze the coefficient vector and a second pass to encode the coefficient vector based on this analysis.

For example, some conventional statistical encoders may perform a first coding pass to generate symbols for representing a coefficient vector, with at least some of the symbols representing more than one non-zero coefficient. In other words, knowledge of the following coefficients is necessary for coding non-zero coefficients of the coefficient vector. In addition or alternatively, some conventional statistical encoders may also select during the first or subsequent pass of the coding of the VLC coding table for use in character coding. In one aspect, VLC coding tables may be selected based on the generated symbols. Alternatively, during the first coding pass through analysis of the coefficient vector, statistics can be collected, and VLC coding tables can be selected based on the accumulated statistics.

Then, a second coding pass is performed by the conventional statistical encoder for statistical coding of the coefficient vector based on the analysis performed during the first coding pass. As one example, some conventional statistical encoders during the second encoding pass may encode the symbols generated during the first pass using VLC coding tables selected based on the generated symbols or other statistics. The generation of symbols that represent more than one non-zero coefficient and / or the selection of VLC coding tables based on the generated symbols or other statistics may allow more efficient coding of the coefficient vector.

The techniques of this disclosure not only eliminate the need for more than one pass to encode the enhancement layer, but the statistical encoding techniques of this disclosure can also lead to the encoding of the enhancement layer without storing and accessing video coefficient coefficients of the base layer, which further reduces computational complexity and memory requirements .

The source device 12, the destination device 14, or both, can be a wireless or wired communication device, as described above. Also, the source device 12, the destination device 14, or both can be implemented as an integrated circuit device, such as an integrated circuit or a chipset, which can be enclosed in a wireless or wired communication device or in another type of device that supports digital applications videos such as digital media player, personal digital assistant (PDA), digital television, etc.

2 is a block diagram illustrating a source device 12 and a destination device 14 of an encoding system 10 in further detail. The destination device 14, for example, can be any of the destination devices 14A or 14B shown in FIG. As shown in FIG. 2, the source device 12 may include a video source 18, a video encoder 20, and a transmitter 22. The video source 18 of the source device 12 may include a video capture device, such as a video camera, a video archive containing previously captured video , or video supplied by a video content provider. As an additional alternative, the video source 18 may generate computer-based data as a video source, or a combination of live video and computer-generated video. In some cases, the source device 12 may be a so-called “camera phone” or a video phone, in which case the video source 18 may be a video camera. In each case, captured, pre-captured or computer-generated video can be encoded by video encoder 20 for transmission from source device 12 to destination device 14 via transmitter 22 and communication channel 16.

Video encoder 20 receives video data from video source 18 and encodes the video data as a base layer bitstream and one or more enhancement layer bit streams. In the example illustrated in FIG. 2, video encoder 20 includes a base layer encoder 30 and an enhancement layer encoder 32. The base layer encoder 30 and the enhancement layer encoder 32 receive ordinary video data from the video source 18. The base layer encoder 30 encodes video data at a first bit rate to form a bitstream of the video base layer at a first quality level. Enhancement layer encoder 32 encodes incremental bits to form one or more enhancement layers, which, when added to the bit level of the base layer, enhance the video to a second, higher quality level. In other words, the enhancement layer when added to the base layer provides a second, higher bit rate, which provides a higher level of quality. Thus, the enhancement layer can be considered as a refinement of the video data encoded in the base layer. The refinements may be, for example, additional coefficients and / or refinements to existing coefficients. The refinements encoded in the enhancement layers can be hierarchical in the sense that the refinements in the enhancement layers gradually increase the quality of the video data as they are decoded. Thus, decoding refinements of all enhancement layers, for example, will result in the highest bit rate and maximum quality, while decoding refinements of only the first enhancement layer will increase bit rate and quality relative to decoding only the base layer.

The video data received from the video source 18 may be a series of video frames. The base layer encoder 30 and the enhancement layer encoder 32 divide the series of frames into encoded elements and process the encoded elements to encode a series of video frames. The encoded elements may, for example, be solid frames or sections of frames, such as mosaic fragments of frames. The base layer encoder 30 and the enhancement layer encoder 32 divide each encoded element into pixel blocks (referred to herein as video blocks or blocks) and work on the video blocks within the individual encoded elements for encoding the video data. Thus, video data may include a plurality of frames, a frame may include a plurality of tile fragments, and a tile fragment may include a plurality of video blocks.

Video blocks may have fixed or variable sizes and may vary in size in accordance with a specified coding standard. As an example, the ITU-T H.264 / MPEG-4 standard, part 10, AVC supports intra prediction with a variety of block sizes, such as 16 × 16, 8 × 8 or 4 × 4 for luminance components and 8 × 8 for components chroma, as well as inter prediction with various block sizes such as 16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8 and 4 × 4 for luminance components and corresponding scaled sizes for chroma components . In the H.264 / MPEG-4 standard, Part 10, AVC, each video block, often called a macroblock (MB), can be divided into sub-blocks of the same size or different sizes. Thus, the encoded element may contain subunits of the same or different sizes. In the general case, macroblocks and various subblocks can be considered as video blocks. Thus, macroblocks can be considered as video blocks, and if they are divided or subdivided, it can be considered that the macroblocks themselves define a plurality of video blocks.

Encoders 30, 32 perform internal and external encoding of video blocks of frames. Intra-coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given encoded element, such as a frame or tile. For intra-coding, encoders 30, 32 form a spatial prediction block based on one or more previously encoded blocks within the same frame as the current encoded block. The prediction block may be a predicted version of the current encoded video block. The base layer encoder 30 may generate a prediction block based on one or more previously encoded blocks within the frame, for example, by performing interpolation (in accordance with the internal coding mode associated with the block) using pixel values of one or more previously encoded blocks within the base layer current frame. Enhancement layer encoder 32 may generate a prediction block based on one or more previously encoded blocks within a frame. Enhancement layer encoder 32, for example, may generate a prediction block based on one or more previously encoded video blocks from a base layer and an enhancement layer within a frame. For example, enhancement layer encoder 32 may generate a prediction block using a weighted sum of pixel values from at least one previously encoded video block from the base layer and at least one previously encoded video block from the enhancement layer.

External coding relies on temporal prediction to reduce or remove temporal redundancy within adjacent frames of a video sequence. For external coding, encoders 30, 32 perform motion estimation to track the movement of closely matching video blocks between two or more adjacent frames within the encoded element. In the case of prediction with external encoding, encoders 30, 32 may generate a temporal prediction block based on one or more previously encoded blocks from other frames within the encoded element. Encoders 30, 32 can, for example, compare the current video block with blocks in one or more adjacent video frames to identify the block in an adjacent frame that most closely matches the current video block, for example, a block in one or more adjacent frames that has the lowest mean square error ( MSE), sum of squared differences (SSD), sum of absolute differences (SAD), or another measure of difference. Encoders 30, 32 select an identified block in an adjacent frame as a prediction block. The base layer encoder 30 compares the current video block with the blocks in still adjacent frames of the base layer. Enhancement layer encoder 32 may compare the current video block with blocks in one or more adjacent frames in the base layer and / or enhancement layer.

After internal or external prediction of the video blocks, encoders 30, 32 form a residual block by subtracting the generated prediction block from the original encoded video block. Thus, the residual block shows the differences between the prediction block and the current encoded block. Encoders 30, 32 may apply transform, quantization, and statistical encoding processes to further reduce the bit rate associated with transmitting the residual block. Conversion techniques, which may include discrete cosine transform (DCT), integer transform, wavelet transform, directional transform, or other transform operation, replace a plurality of pixel difference values with residual transform coefficients that represent the energy of pixel difference values in the frequency domain. Encoders 30, 32 apply quantization to the residual transform coefficients, which usually entails a process that limits the number of bits associated with any given coefficient. Encoders 30, 32 scan a two-dimensional residual block to form a one-dimensional coefficient vector, and statistically encode the coefficient vector to further compress the residual coefficients. Statistical coding may, for example, include variable length coding (VLC), arithmetic coding, fixed-length coding, VLC context adaptive coding (CAVLC), context adaptive binary arithmetic coding (CABAC) and / or other statistical coding techniques .

Signal to noise ratio scalability (SNR) can be achieved through residual quantization. In particular, the base layer encoder 30 may quantize the residual transform coefficients using the first quantization parameter (QP), and the enhancement layer encoder 32 may quantize the residual transform coefficients using the second QP parameter. In ITU-T H.264 / MPEG-4 Part 10, AVC, large QP parameters typically result in lower quality video encoded with fewer bits, while lower QP parameters result in lower QP video data is encoded with higher quality using more bits. Thus, the base layer encoder 30, which encodes the video data at a minimum quality level, can quantize the base layer coefficients using a larger QP parameter value than the QP parameter value used by the enhancement layer encoder 32 to quantize the enhancement layer coefficients. As a result, the quantized residual transform coefficients of the base layer encoder 30 represent the video sequence with the first quality, and the quantized residual transform coefficient of the enhancement layer encoder represents additional coefficients or refinements to the existing video sequence coefficients, which, when combined with the base layer, increase the quality of the video sequence to a second, higher quality .

Each of the encoders 30, 32 receives a one-dimensional coefficient vector that represents the quantized residual transform coefficients of the base layer and enhancement layer, respectively. In other words, the base layer encoder 30 receives the coefficient vector of the base layer, and the enhancement layer encoder 32 receives the coefficient vector of the corresponding enhancement layer. Although encoders 30, 32 receive the same initial video data, coefficient vectors may be different. This may be due to the fact that the base layer encoder 30 and the enhancement layer encoder 32 form different prediction blocks, for example, the base layer encoder 30 generates a prediction block from one or more previously encoded base layer blocks, and the enhancement layer encoder 32 forms a prediction block from one or more previously encoded blocks of the base layer and blocks of the enhancement layer.

Each of the base layer encoder 30 and the enhancement layer encoder 32 encodes corresponding coefficient vectors to form a base layer bitstream and at least one enhancement layer bitstream, respectively. In accordance with the techniques of this disclosure, the base layer encoder 30 and the enhancement layer encoder 32 encode the respective coefficient vectors using different encoding techniques. The base layer encoder 30 may encode the coefficient vector using a multi-coding pass process in which the base layer encoder 30 analyzes the coefficient vector at least during one coding pass and encodes the coefficient vector during at least one subsequent coding pass based on the analysis. In one case, the base layer encoder 30 may encode the quantized residual transform coefficients of the base layer coefficient vector in accordance with CAVLC coding, as defined in H.264 / MPEG-4, Part 10, AVC. CAVLC coding, as defined in H.264 / MPEG-4, Part 10, AVC, can encode the base layer coefficient vector using multiple coding passes.

During the first encoding pass, the base layer encoder 30 may generate symbols to represent a coefficient vector, at least some of which represent more than one non-zero coefficient, and in some cases all coefficients of the coefficient vector. The base layer encoder 30 may, for example, in accordance with CAVLC coding, as defined in H.264 / MPEG-4, part 10, AVC, generate symbols that represent the total number of coefficients in the coefficient vector ("TotalCoeffs"), the number of trailing units in the coefficient vector ("T1"), signs of each of the trailing units, values (or levels) of non-zero coefficients, except for trailing units, the sum of all series ("sumRuns") and the series before each non-zero coefficient. To form some of the symbols, such as TotalCoeff and sumRuns, the base layer encoder 30 may analyze the entire coefficient vector.

During the first coding pass, the base layer encoder 30 may also select VLC coding tables for use during the subsequent coding pass based on the analysis of the coefficient vector. In some cases, the base layer encoder 30 may select VLC coding tables for use during a subsequent (eg, second) coding pass based on the symbols generated during the first coding pass. For example, the base layer encoder 30 may select a VLC coding table to use when encoding the sumRuns symbol based on the total number of coefficients in the block (TotalCoeffs), since there is some relationship between the two values. In particular, as the value of TotalCoeffs increases, the value of sumRuns decreases, and as the value of TotalCoeffs decreases, the value of sumRuns increases. Recall that selecting a VLC coding table to use when encoding a sumRuns symbol based on the total number of coefficients in a block (TotalCoeffs) may allow the base layer encoder 30 to select a VLC coding table that encodes the sumRuns symbol more efficiently. A similar selection of the VLC coding table can be made for other characters to be encoded, or using other collected statistics.

The base layer encoder 30 during the second or other subsequent coding pass encodes a character that represents the total number of non-zero coefficients in the coefficient vector (TotalCoeff), and a character that represents the number of trailing units (denoted as T1). The number of trailing units is the number of coefficients with a value equal to one that appear in the coefficient vector before a coefficient with a value greater than one appears when the coefficient vector is read in the reverse order, that is, starting from the end of the coefficient vector. The base layer encoder 30 may select a VLC coding table for use in encoding the TotalCoeff and T1 symbols based on the predicted number of non-zero coefficients and encode the TotalCoeff and T1 symbols using the selected VLC coding table. Selecting a VLC coding table for use in encoding the TotalCoeff and T1 symbols based on the predicted number of non-zero coefficients may allow the base layer encoder 30 to select a VLC coding table that more efficiently encodes the TotalCoeff and T1 symbols. Thus, different VLC coding tables may be more efficient for different predicted numbers of non-zero coefficients. In one example, the base layer encoder 30 may predict the number of non-zero coefficients in the current block based on the number of non-zero coefficients of one or more previously encoded video blocks, for example, an upper neighboring video block and a left neighboring video block.

The base layer encoder 30 may encode the characters of each of the trailing units. For example, the base layer encoder 30 for each of the trailing units may encode '1' if the character of the trailing unit is positive, and encode '0' if the character of the trailing unit is negative. Thus, the base layer encoder 30 may not need to select a VLC coding table for characters. The base layer encoder 30 may encode values of non-zero coefficients other than trailing units. The base layer encoder 30 may encode levels of non-zero coefficients using a VLC coding table, fixed-length coding, or another type of statistical coding. For example, base layer encoder 30 may encode levels of non-zero coefficients using binary coding.

The base layer encoder 30 may encode a symbol that represents the number of zero coefficients that are in the coefficient vector before the last nonzero coefficient (sumRuns). As described above, the base layer encoder 30 may select a VLC coding table for use in encoding the sumRuns symbol based on the total number of coefficients in the block (TotalCoeffs), since there is some relationship between the two values.

The base layer encoder 30 may encode the series (or series lengths) that are in front of each non-zero coefficient, starting from the last non-zero coefficient vector coefficient. Series lengths are the number of zero coefficients that precede a non-zero coefficient. Thus, the base layer encoder 30 may first encode the series length (i.e., the number of zero coefficients) before the last nonzero coefficient of the coefficient vector, then the series length before the previous nonzero coefficient, and so on, until the series length is encoded before the first excellent from zero by the coefficient of the coefficient vector.

The base layer encoder 30 may select a VLC coding table for use in encoding each of the series lengths. The base layer encoder 30 may select a VLC coding table to use when encoding the current series value based on the series sum symbol (sumRuns) and the series sum encoded up to the present moment. As an example, if the coefficient vector has a sum of series (sumRuns) equal to eight, and the number of series encoded before the last encoded nonzero coefficient was six, then all other series should be zero, one, or two. As the possible series gradually becomes shorter as each additional series is encoded, the base layer encoder 30 may select more efficient VLC encoding tables to reduce the number of bits used to represent the series.

In this method, the base layer encoder 30 performs multi-pass coding for encoding the coefficients of the base layer, including the first pass for analyzing the coefficient vector of the residual block of the base layer, for example, to generate characters and / or select VLC coding tables and a second coding pass for coding coefficient vectors based on this analysis. Although the base layer encoder 30 is described above as encoding the quantized residual transform coefficients using CAVLC coding, as defined in H.264 / MPEG-4, Part 10, AVC, the base layer encoder 30 may use other encoding methodologies to encode the quantized residual transform coefficients .

Enhancement layer encoder 32 encodes the quantized residual enhancement layer transform coefficients, which may be in the form of a coefficient vector. Enhancement layer encoder 32 may generate quantized residual coefficients that differ from the quantized residual coefficients of the base layer. The quantized residual coefficients of the enhancement layer may differ from the quantized residual coefficients of the base layer due to the use of another QP parameter during quantization. In addition, the quantized residual transform coefficients may differ from the quantized residual transform coefficients of the base layer, since the residual block represents the difference between the original video block and the prediction block formed using previously encoded blocks that form the base layer and the enhancement layer. The residual block of the base layer is the difference between the original video block and the prediction block formed using only previously encoded blocks from the base layer. Thus, the enhancement layer may include additional coefficients and / or refinements to existing coefficients. In this sense, the quantized residual transform coefficients of the video block in the enhancement layer are refinements to the video block encoded with the first quality in the base layer and, when added to the base layer, provide higher quality video data.

Enhancement layer encoder 32 may discard one or more quantized residual coefficients of the coefficient vector during encoding, depending on the available bit rate. For example, enhancement layer encoder 32 may discard coefficients corresponding to the base functions of the high frequency conversion, for example, coefficients located at the end of the coefficient vector when coefficient scanning is performed using a zigzag scan, as illustrated in FIG. 3. The encoding of the quantized residual coefficients in accordance with the CAVLC encoding, as defined in the H.264 / MPEG-4 standard, part 10, AVC, may not allow the enhancement layer encoder 32 to reject the coefficients, since at least some of the symbols to be encoded , for example, the symbols TotalCoeffs and sumRuns, refer to all the coefficients in the block. If the enhancement layer encoder 32 discarded one or more coefficients of the coefficient vector, the received information would be redundant, which would lead to lower coding efficiency. In addition, since the decoder must accept the series for all non-zero coefficients in the block, in order to be able to properly decode the position of each coefficient in a zigzag scan when encoding using CAVLC encoding as defined in the H.264 / MPEG-4 standard, part 10, AVC, enhancement layer encoder 32 may not be able to discard coefficients from the coefficient vector of the enhancement layer.

Thus, the enhancement layer encoder 32 encodes the coefficients of the enhancement layer or layers in accordance with the coding techniques of this disclosure. Enhancement layer encoder 32 encodes the quantized residual transform coefficients of the coefficient vector in a single coding pass. In other words, the enhancement layer encoder 32 does not perform the first pass to analyze the coefficient vector and then encode the symbols during the second pass based on this analysis. Instead, the enhancement layer encoder 32 starts at the beginning of the coefficient vector and encodes each of the non-zero coefficients one after the other in one coding pass. In this method, the enhancement layer encoder 32 may encode each of the non-zero coefficients without analyzing subsequent coefficients in the coefficient vector, that is, not knowing any subsequent coefficients of the coefficient vector.

In one aspect, the enhancement layer encoder 32 for each of the non-zero coefficients may encode a symbol indicating that there is at least one remaining non-zero coefficient in the coefficient vector. For example, a character may be a block end character (EOB). Enhancement layer encoder 32 may encode a symbol using a single bit. For example, enhancement layer encoder 32 may encode zero when there is at least one remaining non-zero coefficient, for example at least the current non-zero coefficient, and encode one when there are no more non-zero coefficients remaining.

After the EOB symbol of each coefficient, the enhancement layer encoder 32 encodes a series before the current non-zero coefficient. As described above, the series represents the number of zero coefficients that are between the previous non-zero coefficient of the coefficient vector or the beginning of the coefficient vector in the case of the first non-zero coefficient and the current non-zero coefficient. Enhancement layer encoder 32 may encode series using a single VLC coding table. In one case, enhancement layer encoder 32 may encode series using the VLC coding table used in CAVLC encoding as defined in H.264 / MPEG-4, part 10, AVC to encode the sumRuns symbol when the TotalCoeffs symbol is one. In other words, enhancement layer encoder 32 may reuse one of the VLC coding tables already supported by video encoder 20. In other cases, enhancement layer encoder 32 may use one of the other VLC coding tables already supported by video encoder 20 to encode series. Alternatively, enhancement layer encoder 32 may support a separate VLC coding table specifically designed to encode series of coefficient layer coefficient vector vectors. In either case, the enhancement layer encoder 32 may not need to adaptively select a VLC coding table to use for series coding. Instead, the enhancement layer encoder 32 may use a single VLC coding table, thereby eliminating the need for a first pass to accumulate statistics used to select the VLC coding table.

After encoding the series for each coefficient, the enhancement layer encoder 32 encodes the sign of a non-zero coefficient. Enhancement layer encoder 32, for example, may encode '1' if the sign of a non-zero coefficient is positive, and encode '0' if the sign of a non-zero coefficient is negative. Enhancement layer encoder 32 may correct non-zero coefficients by setting non-zero coefficients to unity. In some cases, the enhancement layer encoder 32 may not encode non-zero coefficients. In this method, an enhancement layer encoder 32 may limit the value of non-zero coefficients to unity. Then the destination device 14 is configured to decode all non-zero coefficients identified in the refinements as having a value equal to one. The lack of coding of the coefficient values of the enhancement layer may lead to some loss of the peak signal to noise ratio (PSNR), but reduces the number of bits used to code the coefficients.

In this method, the enhancement layer encoder 32 may encode the coefficients of the enhancement layer bitstream in a single pass, for example, without knowing any subsequent coefficients in the coefficient vector. Since the enhancement layer encoder 32 does not have to analyze the coefficient vector, for example, generate symbols representing more than one non-zero vector coefficient or select VLC coding tables for character encoding, only one coding pass is performed. Traditional encoders typically perform at least two passes; (1) a first pass for analyzing the coefficient vector; and (2) a second pass for encoding the coefficient vector based on the analysis. In addition, the enhancement layer encoder 32 can encode the enhancement layer coefficients using a single VLC coding table, thereby eliminating the need to perform an encoding pass to form characters for use in adaptive table selection. Thus, enhancement layer encoder 32 can reduce encoding complexity, encoding delay, and memory requirements. In this method, the statistical coding techniques of this disclosure may in addition lead to coding the coefficients of the enhancement layer without storing and gaining access to the information of the coefficients of the base layer, which further reduces the computational complexity and memory requirements.

The source device 12 transmits the encoded video data to the destination device 14 through the transmitter 22. The destination device 14 may include a receiver 24, a video decoder 26, and a display device 28. The receiver 24 receives the encoded video bitstream from the source device 12 through channel 16. As described above, the encoded video bitstream includes a base layer bitstream and one or more enhancement layer bit streams. Video decoder 26 decodes the base layer and, if available, one or more enhancement layers to obtain video data.

In particular, video decoder 26 includes a base layer decoder 34 and an enhancement layer decoder 36. The base layer decoder 34 decodes the base layer bitstream received via channel 16 to obtain video data with first quality for presentation on the display device 28. Enhancement layer decoder 36 decodes the bit streams of one or more enhancement layers to obtain additional video data, such as refinements, which increase the quality of the decoded video data to a second, higher quality. Recall that the number of enhancement layers, for example, one, two, three or more, received by destination device 14 may depend on channel conditions or other restrictions. In addition, the number of received enhancement layers processed by the enhancement layer decoder 36 may depend on the limitations of the decoder. In general, encoding and decoding a base layer in combination with a selected number of enhancement layers enables an incremental improvement in signal quality to noise ratio (SNR) of the decoded video.

The base layer decoder 34 decodes the base layer to obtain symbols representing a vector of quantized residual coefficients of the base layer. The base layer decoder 34 may decode the base layer to obtain the total number of non-zero coefficients in the block, the number of trailing units of the block, the characters of trailing units, the values of the coefficients except trailing units, the sum of all series and series before each non-zero coefficient. The base layer decoder 34 may further further decode the base layer bitstream to identify VLC coding tables for use in decoding base layer symbols. In other cases, the base layer decoder 34 may select VLC coding tables for use based on previously decoded symbols. Using decoded symbols, the base layer decoder 34 can recreate the coefficient vector of the base layer.

Enhancement layer decoder 36 decodes the enhancement layer bitstream to obtain enhancement layer refinements, for example in the form of a vector of incremental residual coefficients or refinements to existing residual coefficients. In particular, the enhancement layer decoder 36 decodes using the same VLC coding table that is used by the enhancement layer encoder 32, the series and signs of the enhancement layer coefficients until the EOB symbol indicates that there are no non-zero coefficients left. Using the decoded symbols, the enhancement layer decoder 36 recreates the coefficient vector of the enhancement layer block.

Decoders 34, 36 recreate each of the blocks of the encoded element using decoded quantized residual coefficients. After the formation of the coefficient vectors, the decoders 34, 36 perform an inverse scan of the coefficient vectors to form a two-dimensional block of quantized residual coefficients. The decoders 34, 36 inverse quantize the quantized residual coefficients and apply the inverse transform, e.g., the inverse DCT transform, the inverse integer transform, the inverse wavelet transform, or the inverse directional transform to the residual coefficients inverse quantization to obtain a residual block of pixel values.

Decoders 34, 36 summarize the prediction block generated by the decoders 34, 36 with a residual block of pixel values to form a recreated video block of the base layer and a video block of the enhancement layer, respectively. The video blocks of the base and enhancement layers are combined to form a video block with a higher resolution. Decoders 34, 36 form a prediction block in the same manner as described above with respect to encoders 30, 32. Destination device 14 may display recreated video blocks to a user through display device 28. The display device 28 may include any of a variety of display devices, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, LED display (LRD), organic LED display, or another type of display unit.

In some cases, video encoder 20 and video decoder 26 are configured to provide scalable enhancement bitstreams that can be arbitrarily truncated. Thus, system 10 can avoid the use of discrete enhancement layers, which must be encoded in their entirety, to achieve scalability. However, in some embodiments, system 10 may be configured to support scalability using either a generalized granularity of scale (FGS) scalability approach or discrete enhancement layers, for example, based on selection.

Source device 12 and destination device 14 can operate substantially symmetrically. For example, each of source device 12 and destination device 14 may include video encoding and decoding components. Thus, the system 10 can support one-way or two-way video transmission between devices 12, 14, for example, for transmitting video streams, video broadcasting, or video telephony.

In some aspects for video broadcasting, the techniques described in this disclosure can be applied to advanced H.264 video coding for providing real-time video services in terrestrial mobile multimedia multimedia (TM3) systems using the forward link only (FLO) radio interface specification, "The specification of the air interface only direct line of communication for terrestrial mobile multimedia broadcasting", published in July 2007 as the technical standard TIA-1099 ("Specification FLO"). That is, the communication channel 16 may include a wireless data channel used for broadcast wireless transmission of video data in accordance with the FLO specification and the like. The FLO specification includes examples defining the syntax and semantics of the bitstream and decoding processes suitable for the FLO radio interface.

Alternatively, video can be broadcast in accordance with other standards such as DVB-H (Digital Video Broadcasting for Handheld Devices), ISDB-T (Integrated Digital Broadcasting Service - Terrestrial) or DMB (Digital Multimedia Broadcasting). Thus, the source device 12 may be a mobile wireless terminal, a video stream server, or a video broadcast server. However, the techniques described in this disclosure are not limited to any particular type of broadcast, multicast, or point-to-point broadcast system. In the case of broadcasting, source device 12 may transmit multiple video channels to multiple destination devices, each of which may be similar to destination device 14 in FIG. Thus, although FIG. 1 shows one destination device 14, for video broadcasting, source device 12 typically broadcasts video content simultaneously with many destination devices.

In other examples, transmitter 22, communication channel 16, and receiver 24 may be configured to communicate in accordance with any wired or wireless communication system, including one or more Ethernet systems, telephone systems (eg, POTS), cable systems, power lines and fiber optic systems and / or a wireless system comprising one or more code division multiple access (CDMA or CDMA2000) communication systems, frequency division multiple access (FDMA) communication systems, multi-communication systems Orthogonal Frequency Division Multiple Access (OFDM), Time Division Multiple Access (TDMA) communication systems such as Global System for Mobile Communications (GSM), General Packet Radio Communication Systems (GPRS) or GSM environment with enhanced data transmission ( EDGE), terrestrial radio communication systems (TETRA), wideband code division multiple access (WCDMA) systems, 1xEV-DO high-speed data systems (the first generation of data-only systems) or systems 1xEV-DO Gold Multicast, IEEE 402.18 systems, MediaFLO ™ technology systems, DMB systems, DVB-H systems or other data transfer schemes between two or more devices.

Both video encoder 20 and video decoder 26 can be implemented as one or more microprocessors, digital signal processors (DSP), specialized integrated circuits (ASIC), programmable gate arrays (FPGA), discrete logic circuits, such as software, hardware, software hardware or any combination thereof. Both video encoder 20 and video decoder 26 may be included in one or more encoders or decoders, each of which may be integrated as part of a combined encoder / decoder (CODEC) into a respective mobile device, subscriber unit, broadcast device, server, and the like. In addition, both the source device 12 and the destination device 14 may include suitable modulation, demodulation, frequency conversion, filtering, and amplification components for transmitting and receiving encoded video, and if applicable, including radio frequency (RF) wireless components and antennas, sufficient to support wireless communications. However, for simplicity of illustration, such components are generally represented as a transmitter 22 of the source device 12 and a receiver 24 of the destination device 14 in FIG.

3 is a block diagram illustrating an exemplary base layer encoder 30 and an enhancement layer encoder 32 in further detail. In the example of FIG. 3, the base layer encoder 30 includes a prediction module 33A, a frame store 35A, a transform module 38A, a quantization module 40A, a coefficient scanning module 41A, an inverse quantization module 42A, an inverse transform module 44A, a base layer statistic encoder 46, and adders 48A and 48B ("adders 48"). The reflection of the various distinguishing features in FIG. 3 as modules is intended to highlight the various functional aspects of the illustrated devices and does not necessarily imply that such modules must be implemented using separate hardware or software components. In contrast, the functionality associated with one or more elements may be integrated into ordinary or separate hardware or software components.

Prediction unit 33A generates a prediction block using intra or inter prediction. The prediction block may be a predicted version of the current encoded video block. As described above, prediction unit 33A may generate a prediction block using intra prediction based on one or more previously encoded blocks of the base layer within the same frame as the current encoded block. Alternatively, the prediction module may generate a prediction block using inter prediction based on one or more previously encoded blocks within one or more adjacent frames of the base layer. Prediction unit 33A may retrieve a previously encoded block from frame storage 35A.

After internal or external prediction of the video blocks, the base layer encoder 30 forms a residual block by subtracting the prediction block produced by the prediction unit 33A from the current video block in the adder 48A. The residual block includes a plurality of pixel difference values that determine the number of differences between the pixel values of the current video block and the pixel values of the prediction block. The residual block can be represented in the format of a two-dimensional block (for example, a two-dimensional matrix or an array of pixel values). In other words, the residual block is a two-dimensional representation of pixel values.

The transform unit 38A applies the transform to the residual block to obtain residual transform coefficients. Transform unit 38A, for example, may apply a DCT transform, integer transform, directional transform, wavelet transform, or a combination thereof. After applying the transform to the residual block of pixel values, the quantization unit 40A quantizes the transform coefficients to further reduce the bit rate. After quantization, the inverse quantization unit 42A and the inverse transform unit 44A may apply inverse quantization and inverse transform, respectively, to recreate the residual block. The adder 48B adds the recreated residual block to the prediction block produced by the prediction unit 33 to obtain a recreated video block for storage in the frame store 35A. The reconstructed video block stored in the frame storage 34 may be used by the prediction module 32 from the base layer encoder 30 for internal or external encoding of the subsequent video block. In addition, as will be described in more detail below, the reconstructed video block stored in the frame store 35A may be used by the prediction unit 33B of the enhancement layer encoder 32 for internally or externally encoding the refinements of the video block in the enhancement layer.

After quantization, coefficient scanning module 41A scans the coefficients from a two-dimensional block format into a one-dimensional vector format, a process often referred to as coefficient scanning. The coefficient scanning module 41A, for example, can scan a two-dimensional block of coefficients using a zigzag scan order, as described in further detail in FIG. 7. After scanning, the statistical encoder 46 of the base layer statistically encodes the coefficients of the one-dimensional vector. The base layer encoder 46, for example, can statistically encode coefficient vector coefficients using CAVLC coding, as defined in H.264 / MPEG-4, Part 10, AVC, and described in detail above with respect to FIG. 2.

Enhancement layer encoder 32 includes prediction module 33B, frame storage 35B, transform module 38B, quantization module 40B, coefficient scanning module 41B, inverse quantization module 42B, inverse transform module 44B, enhancement layer statistical encoder 49, and adders 48C and 48D (" 48 "adders). The modules of the enhancement layer encoder 32 are substantially similar to the modules of the base layer encoder 30. Therefore, only differences will be described.

Prediction unit 33B of enhancement layer encoder 32 generates a prediction block, which is a predicted version of the current video block. Unlike the prediction module 33 of the base layer encoder 30, which uses only previously encoded base layer blocks to form the prediction block, the prediction module 33B of the enhancement layer encoder 32 can generate a prediction block based on one or more previously encoded base layer blocks and the enhancement layer. In other words, the prediction unit 33B may generate a prediction block using the recreated video block from the base layer and the recreated video block of the enhancement layer. For example, prediction module 33B may combine the recreated video block of the base layer with the recreated enhancement layer block to form a prediction block with a second, higher quality.

Since the prediction block generated by the prediction unit 33B is formed based on recreated video blocks and the base and enhancement layers, the residual block formed in the adder 48C represents the differences between the current video block and the previously encoded block created from the base and enhancement layers, i.e., the second higher visual quality.

Although the quantization module 40B of the encoder 32 of the enhancement layer 32 is functionally similar to the quantization module 40A of the base layer encoder 30, it can use another QP parameter to quantize the transform coefficients. As described above with respect to FIG. 2, signal-to-noise ratio (SNR) scalability can be achieved by using different quantization parameters. For example, when the base layer encoder 30 and the enhancement layer encoder 32 operate in accordance with the H.264 / MPEG-4 standard, part 10, AVC, the quantization module 40A may encode video using a larger QP parameter value than the QP parameter value used by the module 40B quantization. As a result, the quantized residual transform coefficients from the base layer encoder 30 represent the video sequence with the first quality, and the quantized residual transform coefficients from the enhancement layer encoder 32 represent additional coefficients or refinements of the existing video sequence coefficients, which, when combined with the base layer, increase the quality of the video sequence to a second, more high visual quality.

In addition, as described in detail with respect to FIG. 2, the enhancement layer statistical encoder 49 encodes the quantized residual transform coefficients in a single coding pass. In other words, the enhancement layer statistic encoder 49 can encode each nonzero coefficient vector of the coefficient of the enhancement layer without knowing any subsequent coefficients of the coefficient vector. Encoding the enhancement layer in one pass can eliminate the need for the first pass to analyze the coefficient vector and the second pass to encode the coefficient vector based on this analysis. Instead, the statistical encoder 49 of the enhancement layer starts at the beginning of the coefficient vector and encodes each of the coefficients one after the other in one coding pass. In more detail, statistical coding of the enhancement layer is described below with respect to FIG. 4.

4 is a block diagram illustrating an exemplary statistical base layer encoder 46 and statistical encoder 49 of enhancement layer 49 with further details. The base layer statistic encoder 46 may include an analysis module 50, a plurality of VLC coding tables 52A-52N ("VLC coding tables 52"), a total coefficient encoder 54, trailing unit encoder 56 (T1), 58 character encoder, 60 value encoder 60 coefficients, encoder 62 sums of series and encoder 64 lengths of series. The enhancement layer statistical encoder 49 may include an EOB symbol encoder 66, a series length encoder 68, a character encoder 70, and a VLC coding table 69.

The base layer statistic encoder 46 encodes a coefficient vector representing a video block with a first quality by performing a plurality of coding passes. According to CAVLC coding, as defined in H.264 / MPEG-4 standard, part 10, AVC, for example, base layer statistic encoder 46 may perform a first coding pass to analyze the coefficient vector, for example, to generate symbols representing the coefficient vector, and / or selecting a VLC coding table, and a second coding pass for coding the coefficient vector based on the analysis.

As an example, the analysis module 50 from the statistical base layer encoder 46 may analyze a coefficient vector to generate one or more symbols that represent a block of coefficients. Analysis module 50, for example, in accordance with the H.264 / MPEG-4 standard, part 10, AVC can determine the total number of coefficients in a block (TotalCoeff), the number of trailing units (T1), the characters of trailing units, the value of each non-zero coefficient , the total amount of series (sumRuns) and the length of the series preceding each non-zero coefficient. At least some of the symbols, for example, the TotalCoeff and sumRuns symbols generated by the analysis module 50, can represent all coefficients of the coefficient vector. The analysis module 50 in other examples may generate more or less characters.

In addition or alternatively, the analysis module 50 may select, during a first or subsequent coding pass, a subset of the VLC coding tables 52 for use in character coding. In one aspect, the analysis module 50 may select a subset of the VLC coding tables 52 based on the generated symbols. Alternatively, the analysis module 50 may collect statistics during the analysis of the coefficient vector to select a subset of VLC coding tables 52 based on the accumulated statistics. For example, the base layer encoder 30 may select a VLC coding table to use when encoding the sumRuns symbol based on the total number of coefficients in the block (TotalCoeffs), since there is some relationship between the two values. As will be described in detail below, the selection of a subset of VLC coding tables 52 based on the generated symbols or other statistics may enable more efficient coding of symbols representing a coefficient vector.

The base layer statistic encoder 46 encodes a coefficient vector during a second or other subsequent coding pass. In particular, encoder 54 of the total number of coefficients encodes the total number of non-zero coefficients (TotalCoeff) in the coefficient vector. The total number of coefficients encoder 54 may encode the TotalCoeff symbol using one of the VLC coding tables 52 selected based on predicting the number of non-zero coefficients of the current coefficient vector. In one example, predicting the number of non-zero coefficients of the current coefficient vector can be made based on the number of non-zero coefficients of one or more previously encoded video blocks, for example, an upper neighboring video block and a left neighboring video block. In this method, a statistical base layer decoder may select the same VLC coding table based on a previously decoded block.

After the encoder 54 of the total number of coefficients encodes the total number of non-zero coefficients, the encoder 56 of the closing units encodes the symbol of the closing units. The closure unit encoder 56 may encode the closure unit symbol using the same VLC coding table 52 selected based on the predicted number of non-zero coefficients, for example, by the same method described above with respect to the total number of coefficients encoder 54.

The 58 character encoder encodes the characters of the trailing units. For example, an encoder 58 characters for each of the trailing units may encode '1' if the character of the trailing unit is positive, and encode '0' if the character of the trailing unit is negative. Coder 60 of coefficient values encodes levels (e.g., values) of non-zero coefficients, except for trailing units. Coder 60 of coefficient values may encode levels of non-zero coefficients using a fixed-length VLC coding table or other type of statistical coding.

The series sum encoder 62 may encode a symbol that represents the number of zero coefficients that are in the coefficient vector before the last nonzero coefficient, that is, the sumRuns symbol. The series sum encoder 62 encodes the sumRuns symbol using one of the VLC coding tables 52 selected based on the total number of coefficients in the block (TotalCoeffs). Recall that selecting a VLC coding table to use when encoding a sumRuns symbol based on the total number of coefficients in a block (TotalCoeffs) can enable the series sum encoder 62 to select a VLC coding table that encodes the sumRuns symbol more efficiently.

A series length encoder 64 encodes a series length of a coefficient vector. The series length encoder 64 may first encode the series length of the last non-zero coefficient vector coefficient, then the series length of the previous non-zero coefficient vector, and so on, until the series length of the first non-zero coefficient vector coefficient is encoded. In other words, a series length encoder can start encoding from the last series length. A series length encoder 64 may encode each of the series lengths using the VLC coding table 52 selected based on the total sum of the series of zeros of the coefficient vector (sumRuns) and the sum of the series encoded to date. As an example, if the coefficient vectors have a sum of series (sumRuns) equal to eight, and the number of series encoded to the last encoded nonzero coefficient was six, then all the remaining series should be zero, one, or two. As the possible run length gradually becomes shorter as each additional run is encoded, the run length encoder 64 may select more efficient VLC coding tables to reduce the number of bits used to represent the run. In this method, the VLC coding table 52 used by the series length encoder 64 may vary for each of the series lengths.

Enhancement layer statistical encoder 49 encodes a coefficient vector that represents refinements, for example, in the form of incremental coefficients, or refinements to existing coefficients for a video block in one coding pass to form an enhancement layer. As will be described in further detail, the source device 12 in some cases may encode each nonzero coefficient vector of the coefficients of the enhancement layer without knowing any subsequent coefficients. The enhancement layer statistical encoder 49 may start from the beginning of the coefficient vector and encode each of the coefficients one after another in one coding pass. In this method, the enhancement layer encoder 49 encodes the coefficients of the coefficient vector one after another without analyzing the coefficients that occur later in the coefficient vector. Encoding the enhancement layer in one pass can eliminate the need for the first pass to analyze the coefficient vector and the second pass to encode the coefficient vector based on this analysis.

For each of the non-zero coefficients, the EOB symbol encoder 66 encodes an EOB symbol indicating that there is at least one remaining non-zero coefficient in the coefficient vector. For example, the EOB symbol encoder 66 may encode zero when there is at least one remaining non-zero coefficient, for example at least the current non-zero coefficient, and encode one when there are no more remaining non-zero coefficients.

After encoding the EOB symbol of each coefficient, a series length encoder 68 encodes a series length preceding a non-zero coefficient. As described above, the series length represents the number of zero coefficients that precede the current non-zero coefficient. Series length encoder 68 may encode series lengths using a single VLC coding table 69. In one case, the VLC coding table 69 may be the same as one of the VLC coding tables 52 from the statistical base layer encoder 46. Alternatively, the series length encoder 68 may support a separate VLC coding table specifically designed to encode a series of coefficient vector coefficients of the enhancement layer. In any case, the series-length encoder 68 may not need to adaptively select a VLC coding table to use for series coding. Instead, the series-length encoder 68 may use a single VLC coding table, thereby eliminating the need for a first pass that accumulates statistics used to select the VLC table.

After encoding the series length of each coefficient, the character encoder 70 encodes the sign of a non-zero coefficient. The sign encoder 70, for example, can encode '1' if the sign of a non-zero coefficient is positive, and encode '0' if the sign of a non-zero coefficient is negative. The enhancement layer statistical encoder 49 may not encode the nonzero coefficients of the enhancement layer, which may result in some loss of the peak signal to noise ratio (PSNR), but reduce the number of bits used to encode the coefficients.

The statistical coding techniques of this disclosure may enable the enhancement layer statistic encoder 49 to encode the bitstream coefficients of the enhancement layer in one pass. Since the statistical encoder 49 of the enhancement layer does not analyze the coefficient vector, for example, to generate symbols and / or select VLC coding tables, only one coding pass is needed. Conventional encoders typically perform at least two passes: (1) a first pass for analyzing the coefficient vector and (2) a second pass for encoding the coefficient vector based on the analysis. In addition, the enhancement layer statistic encoder 49 can encode the enhancement layer coefficients using a single VLC coding table, thereby eliminating the need to perform a coding pass to select a variety of VLC coding tables. In this method, the statistical enhancement layer encoder 49 can reduce coding complexity, coding delay, and memory requirements. In addition, the statistical coding techniques of this disclosure may in addition lead to coding the coefficients of the enhancement layer without storing and gaining access to the information of the coefficients of the base layer, which further reduces the computational complexity and memory requirements.

5 is a block diagram illustrating an example of a base layer decoder 34 and an enhancement layer decoder 36 in further detail. The base layer decoder 34 includes a statistical base layer decoder 72, coefficient scanning module 74A, inverse quantization module 76A, inverse transform module 78A, prediction module 80A, frame storage 82A, and adder 84A. Enhancement layer decoder 34 includes a statistical enhancement layer decoder 86, coefficient scanning unit 74B, inverse quantization unit 76B, inverse transform unit 78B, prediction unit 80B, frame storage 82B, and adder 84B.

The base layer statistic decoder 72 decodes the received base layer bitstream to obtain video data with first quality for presentation on a display device. The base layer statistic decoder 72 receives the base layer bitstream and decodes the base layer bitstream to obtain residual information (for example, in the form of a one-dimensional vector of quantized residual coefficients) and header information (for example, in the form of one or more header syntax elements). The base layer statistic decoder 72 performs a decoding function that is reciprocal to the encoding performed by the base layer statistic encoder 46 in FIGS. 3 and 4.

In particular, the base layer statistic decoder 72 decodes the base layer to obtain symbols representing a vector of quantized residual coefficients of the base layer. When encoding using CAVLC, as defined in H.264 / MPEG-4 standard, part 10, AVC, for example, statistical base layer decoder 72 can decode the base layer to obtain the total number of non-zero coefficients in the block (TotalCoeff), the number of trailing unit units (T1), signs of trailing units, coefficient values other than trailing units, the sum of all series (sumRuns) and series before each non-zero coefficient. In some cases, VLC coding tables selected for decoding may be selected based on previously decoded blocks or previously decoded symbols of the current block. In other cases, the base layer statistic decoder 34 may decode the base layer bitstream to identify VLC coding tables for use in decoding base layer symbols. Using decoded symbols, the base layer decoder 34 can recreate the coefficient vector of the base layer.

After generating the coefficient vector, coefficient scanning module 74A performs an inverse scan of the coefficient vector to form a two-dimensional block of quantized residual coefficients. The inverse quantization unit 76A inversely quantizes the quantized residual coefficients, and the inverse transform unit 78A applies an inverse transform, such as an inverse DCT transform, an inverse integer transform, an inverse wavelet transform, or an inverse directional transform, to the residual coefficients inverse quantization to obtain a residual block of values pixels.

Prediction unit 80A generates a prediction block using one or more adjacent blocks within an ordinary frame in the case of intra prediction or one or more blocks within adjacent frames in the case of inter prediction. The prediction module generates a prediction block using only previously encoded blocks from the base layer. Adder 84A sums the prediction block generated by the prediction unit 80A with a residual block of pixel values to form a recreated video block of the base layer. The base layer video block is stored in frame storage 82A for use in the formation of subsequent prediction blocks.

Enhancement layer decoder 36 decodes the enhancement layer bitstream to obtain refinements to video data, for example in the form of a vector of incremental residual coefficients or refinements to existing residual coefficients. The enhancement layer statistical decoder 86 decodes using the same VLC coding table that is used by the enhancement layer statistical encoder 49, the series and signs of the enhancement layer coefficients until the EOB symbol indicates that there are no non-zero coefficients left. Using the decoded symbols, the statistical enhancement layer decoder 86 recreates the coefficient vector of the enhancement layer block. The decoded coefficient vector represents incremental bits, which are refinements that, when combined with the bits of the base layer, increase the quality of the decoded video data to a second, higher quality.

After generating the coefficient vector, coefficient scanning module 74B performs an inverse scan of the coefficient vector to form a two-dimensional block of quantized residual coefficients. The inverse quantization unit 76B inversely quantizes the quantized residual coefficients, and the inverse transform unit 78B applies the inverse transform, for example, the DCT inverse transform, the inverse integer transform, the inverse wavelet transform, or the inverse directional transform to the inverse quantized residual coefficients to obtain a residual block of pixel values .

Prediction unit 80B generates a prediction block using one or more adjacent blocks within an ordinary frame in the case of intra prediction or one or more blocks within adjacent frames in the case of inter prediction. The prediction module generates a prediction block using previously encoded blocks from both the base layer and the enhancement layer. Adder 84B sums the prediction block generated by the prediction unit 80B with a residual block of pixel values to form a recreated video block of the enhancement layer. The video block of the enhancement layer is stored in the frame storage 82B for use by the prediction unit 80B in the formation of subsequent prediction blocks. The recreated video block of the base layer and the recreated video block of the enhancement layer are combined in the adder 84C to form a video block with higher quality.

6 is a block diagram illustrating exemplary statistical base layer decoder 72 and statistical enhancement layer decoder 86 in further detail. The base layer statistic decoder 72 may include a plurality of VLC coding tables 52A-52N ("VLC coding tables 52"), a total coefficient decoder 90, a trailing unit decoder (T1) 92, a character decoder 94, a vector magnitude decoder 96, a decoder 98 series totals and decoder 100 series lengths. The enhancement layer statistical decoder 86 may include an EOB symbol decoder 102, a series length decoder 104, a character decoder 106, and a VLC coding table 69.

The base layer statistic decoder 72 decodes the base layer bitstream to obtain symbols representing the coefficient vector of the video block at a basic quality level. The total coefficient decoder 90 decodes the bitstream using one of the VLC coding tables 52 to obtain the total number of non-zero coefficients (TotalCoeff) in the coefficient vector. The total coefficient decoder 90 may select a VLC coding table 52 for decoding the TotalCoeff symbol based on predicting the number of non-zero coefficients of the current coefficient vector, for example, based on the number of non-zero coefficients of one or more previously decoded video blocks. In this method, the total coefficient decoder 90 may select the same VLC coding table 52 that is used by the total coefficient encoder 54 to encode the TotalCoeff symbol.

After the total number of coefficients decoder 90 decodes the total number of non-zero coefficients, the closure unit decoder 92 decodes the closure unit symbol. The symbol of the closing units represents the number of coefficients with a value equal to one that occur before the coefficient with a value greater than one when the coefficient vector is read in the reverse order. The closure unit decoder 92 may decode the closure unit symbol using one of the VLC decoding tables 52 selected based on the predicted number of non-zero coefficients.

A character decoder 94 decodes the characters of all trailing units. For example, a character decoder 94 for each of the trailing units may determine that the sign of the coefficient is positive when '1' is received, and to determine that the sign of the coefficient is negative when '0' is received. A decoder of 96 coefficient values decodes non-zero coefficient values, except for trailing units. The coefficient value decoder 96 may decode non-zero coefficient levels using a VLC coding table, fixed-length coding, or other type of statistical coding.

The series sum decoder 98 may decode a symbol that represents the number of zero coefficients that occur in the coefficient vector before the last nonzero coefficient, that is, the sumRuns symbol. The series sum decoder 98 decodes the sumRuns symbol using one of the VLC coding tables 52 selected based on the total number of coefficients in the block (TotalCoeffs), which was previously decoded by the total number of coefficients decoder 90. Recall that selecting a VLC coding table to use when decoding the sumRuns symbol based on the total number of coefficients in a block (TotalCoeffs) may allow the series sum decoder 98 to select a VLC coding table that decodes the sumRuns symbol more efficiently.

The series length decoder 100 decodes the series lengths of the coefficient vector. The series length decoder 100 may first decode the series length of the last non-zero coefficient vector coefficient, then the series length of the previous non-zero coefficient vector, and so on, until the series length before the first non-zero coefficient vector coefficient is decoded. In other words, the series length decoder 100 may begin to decode the last series length. A series length decoder 64 may decode each of the series lengths using the VLC coding table 52 selected based on the total sum of the series of coefficient vector (sumRuns) and the sum of the series decoded to date. The sumRuns symbol was previously decoded by series 98 decoder. However, the series length decoder 100 may accumulate statistics regarding the sum of the series decoded to date. As the possible series length gradually becomes shorter as each additional series is decoded, the series length decoder 100 may select more efficient VLC coding tables to reduce the number of bits used to represent the series. In this method, the VLC coding table 52 used by the series length decoder 100 may vary for each of the series lengths.

Enhancement layer statistical decoder 86 decodes the enhancement layer bitstream to obtain refinements for the video block, for example in the form of incremental coefficients or refinements to existing coefficients. The EOB symbol decoder 102 determines whether the EOB symbol indicates whether there is at least one remaining non-zero coefficient. When there is at least one remaining non-zero coefficient, the series length decoder 104 decodes the series length preceding the next non-zero coefficient. The series length decoder 104 may decode the series length of the next non-zero coefficient using the VLC coding table 96, which is the same VLC table used by the series length encoder 68. The character decoder 106 decodes the sign of a non-zero coefficient. For example, the character decoder 106 may determine that the character is positive when '1' is received and negative when '0' is received. The enhancement layer statistical decoder 86 continues to decode non-zero coefficients until the EOB symbol decoder 102 indicates that there are no more non-zero coefficients remaining.

7 is a conceptual diagram illustrating a zigzag scan of a block 40 of 4 × 4 dimensional coefficients. The zigzag scan shown in FIG. 7 can be performed by encoders 30, 32 in FIG. 2. The scan order for such a zigzag scan shown in FIG. 7 follows the arrow through the video blocks 110, and the coefficients c1-c16 are marked in the scan order. In particular, the numerical values shown in FIG. 7 indicate the positions of the coefficients within a sequential one-dimensional vector and do not represent the actual values of the coefficients. The result of the zigzag scan illustrated in FIG. 7 is a one-dimensional coefficient vector X, where

X = [c1, c2, c3, c4, c5, c6, c7, c8, c9, c10, c11, c12, c13, c14, c15, c16]

where c1-c16 represent the positions of the coefficients within a two-dimensional array of coefficients.

The techniques of this disclosure are not limited to any particular scan order or technique. For example, the scan order used in this disclosure may be the zigzag scan order shown in FIG. 7. Alternatively, the scan orders used in this disclosure may be other scan orders, such as horizontal scanning, vertical scanning, or any other scanning technique.

FIG. 8 is a conceptual diagram illustrating a hypothetical example of an enhancement layer coefficient block 120. In this example, the numerical values shown in FIG. 8 indicate the actual values of the coefficients in the positions. Actual coefficient values of coefficient block 120 may represent quantized residual coefficients, non-quantization transform coefficients, or other type of video block coefficients in the enhancement layer. In the example illustrated in FIG. 8, the coefficient block 120 has a 4x4 dimension. However, the techniques of this disclosure may be extended to apply to blocks of any size. After scanning the coefficient block 120 in accordance with the zigzag scanning illustrated in FIG. 7, the resulting coefficient vector V is as follows:

V = [4, 0, 0, -2, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0].

Enhancement layer encoder 32 encodes each of the coefficients of the coefficient vector V in accordance with the techniques described in this disclosure. As an example, for each of the non-zero coefficients of the vector V of the coefficients, the enhancement layer encoder 32 encodes an EOB symbol, a series length, and a sign. As described in detail above, the EOB symbol indicates whether there are any remaining non-zero coefficients in the coefficient vector, the length of the series represents the number of zero coefficients that are in front of the current non-zero coefficient of the coefficient vector, and the sign indicates whether the coefficient value is positive or negative.

In accordance with one aspect of this disclosure, enhancement layer encoder 32 may not code the magnitude of the coefficients. Instead, the enhancement layer encoder 32 may encode each of the non-zero coefficients, as if the value of all non-zero coefficients was equal to one. In this method, it can be seen that the enhancement layer encoder 32 encodes the next coefficient vector V ′ instead of the vector V.

V '= [1, 0, 0, -1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0].

Enhancement layer encoder 32, for example, can encode the first coefficient (i.e., 4 in the coefficient vector V or 1 in the coefficient vector V ') using the zero symbol EOB, the code word for the series equal to zero, and the symbol equal to one, encode a second coefficient (i.e., -2 in the vector V of coefficients or -1 in the vector V 'of coefficients) using the EOB symbol equal to zero, the code word for the series equal to two, and the sign equal to zero, and encode the third non-zero coefficient ( i.e. 1 in a vector of V coefficients or per century Ore V 'coefficients) using the EOB symbol being zero, the code word for the series of unity and the mark, equal to one, followed by EOB symbol equal to one. As described above, the codewords used for encoding the series can be obtained from the VLC encoding table defined in the H.264 / MPEG-4 standard, part 10, AVC.

An exemplary encoded bitstream is described for purposes of illustration. Enhancement layer encoder 32 may encode coefficient vector V, V ′ by other methods without departing from the context of this disclosure. For example, the EOB symbol may be encoded as a unit to represent additional non-zero coefficients in a block and as zero to represent the absence of remaining non-zero coefficients. Similarly, a character symbol can be encoded as zero to represent a positive non-zero coefficient and as a unit to represent a negative non-zero coefficient. As another example, the EOB symbol encoded for each non-zero coefficient may represent whether the current coefficient is the last non-zero coefficient of the vector. Thus, there may not be an EOB symbol at the end of the encoded bitstream. Instead, when the EOB symbol indicates that the current coefficient is the last non-zero coefficient, the video decoder knows that after decoding the series and the symbol of the current coefficient, there are no more block block coefficients.

FIG. 9 is a flowchart illustrating an example operation of a video encoder, such as video encoder 20 of FIG. 2, performing scalable video encoding techniques of this disclosure. The base layer encoder 30 and the enhancement layer encoder 32 from video encoder 20 receive video data from video source 18 (step 130). As described above, the base layer encoder 30 and the enhancement layer encoder 32 receive the same initial video data. The video data obtained from the video source 18, for example, may be a series of video frames.

For each video block, the base layer encoder 30 encodes the base layer using an encoding technique that performs multiple encoding passes (step 132). The base layer encodes the video block at the first quality level. The base layer encoder 30 may generate a coefficient vector that represents the video block in first quality and encode the residual transform coefficients of the block to form the base layer. The base layer encoder 30 may encode a coefficient vector to form the base layer in accordance with CAVLC coding, as defined in H.264 / MPEG-4, Part 10, AVC. As described in detail above with respect to FIG. 2, the base layer encoder 30 may perform a first coding pass for analyzing the coefficient vector and a second pass for coding the coefficient vector based on the analysis.

For each video block, the enhancement layer encoder 32 encodes the incremental bits as an enhancement layer using an encoding technique that performs one encoding pass (step 134). Additional bits of the enhancement layer bitstream encode refinements that, when added to the base layer bitstream, enhance the video to a second, higher quality level. Although in this example, the enhancement layer encoder 32 is described as encoding only one enhancement layer, the enhancement layer encoder 32 can encode more than one enhancement layer bitstream. In this case, the enhancement layers can be hierarchical in the sense that the enhancement layers offer ever higher quality as they are decoded.

The second statistical coding technique used by the enhancement layer encoder 32 can encode for each of the non-zero coefficients of the enhancement layer coefficient vector coefficients, an EOB symbol, a series, and a sign. As described in detail above, the EOB symbol can indicate whether there are any remaining non-zero coefficients, the series length represents the number of zero coefficients that precede the non-zero coefficient, and the sign indicates whether the coefficient value is positive or negative. After the sign of the last non-zero coefficient, the enhancement layer encoder 32 may encode the EOB symbol to indicate that there are no more non-zero coefficients remaining.

The base layer encoder 30 and the enhancement layer encoder 32 output the encoded base layer bitstreams and the enhancement layer bitstreams, respectively (step 136). The statistical coding technique used by the enhancement layer encoder 32 may enable encoding the residual coefficients of the enhancement layer with lower computational complexity and implementation complexity without a large loss in coding efficiency. The statistical encoding techniques of this disclosure may enable the encoding of additional video data, for example, in the form of refinements, in the enhancement layer bitstream in a single encoding pass, thereby reducing encoding complexity, encoding delay, and memory requirements. For example, enhancement layer encoder 32 may encode each nonzero coefficient of the coefficient vector of the enhancement layer without knowing any subsequent coefficients, thereby enabling the coefficient vector to be encoded in one pass and eliminating the need to perform a first pass to analyze the coefficient vector and a second pass for coding a coefficient vector based on analysis.

FIG. 10 is a flowchart illustrating an example operation of an enhancement layer encoder, such as enhancement layer encoder 32 of FIG. 2, encoding the residual coefficients of a enhancement layer video block in accordance with one aspect of this disclosure. Enhancement layer encoder 32 identifies the first non-zero coefficient in the coefficient vector of the enhancement layer block (step 140). Enhancement layer encoder 32 encodes an EOB symbol indicating that there is at least one remaining nonzero coefficient in the coefficient vector of the enhancement layer block (step 142). Enhancement layer encoder 32 may encode the EOB symbol using one bit, for example, encode zero when there is at least one remaining nonzero coefficient, and encode one when there are no more nonzero coefficients remaining.

Enhancement layer encoder 32 encodes a series that indicates the number of zero coefficients that precede a non-zero coefficient (step 144). Enhancement layer encoder 32 may, in some cases, encode a series using a VLC coding table that is already stored for CAVLC coding, as defined in H.264 / MPEG-4, Part 10, AVC. For example, enhancement layer encoder 32 may encode a series using the VLC coding table used to encode the total series (sumRuns) when the total number of coefficients (TotalCoeffs) is one. Alternatively, enhancement layer encoder 32 may support a separate VLC coding table specifically designed to encode series of coefficient layer coefficient vector vectors.

Enhancement layer encoder 32 may encode the sign of a non-zero coefficient (block 146). Enhancement layer encoder 32, for example, may encode '1' if the sign of a non-zero coefficient is positive, and encode '0' if the sign of a non-zero coefficient is negative. In some cases, the enhancement layer encoder 32 may not encode non-zero coefficients. In this method, an enhancement layer encoder 32 may limit the value of non-zero coefficients to unity. Thus, any non-zero coefficient with a value greater than one is set equal to one. The lack of coding of the coefficient values of the enhancement layer may lead to some loss of the peak signal to noise ratio (PSNR), but reduces the number of bits used to code the coefficients.

Enhancement layer encoder 32 determines whether there are any remaining non-zero coefficients in the enhancement layer unit (step 148). When there is at least one non-zero coefficient remaining in the enhancement layer unit, the enhancement layer encoder 32 continues to encode the EOB symbol, series, and sign for each of the remaining non-zero coefficients. When there are no more non-zero coefficients remaining in the enhancement layer block, the enhancement layer encoder 32 encodes an EOB symbol to indicate that there are no more non-zero coefficients remaining in the enhancement layer block vector (block 149). As described above, the enhancement layer is transmitted along with the base layer.

Since the enhancement layer coding technique described in FIG. 10 does not encode characters that refer to more than one coefficient, the enhancement layer coding technique may allow the enhancement layer encoder 32 to reject one or more quantized residual coefficients of the coefficient vector during coding, depending on available bit rate. In addition, the coding technique of the enhancement layer reduces coding complexity and simplifies implementation.

11 is a flowchart illustrating an example operation of an enhancement layer decoder, such as enhancement layer decoder 36 of FIG. 2, which decodes an enhancement layer bitstream to obtain a residual transform coefficient vector. Enhancement layer decoder 36 receives an enhancement layer bitstream (step 150). Enhancement layer decoder 36 analyzes the EOB symbol to determine if there are any remaining non-zero coefficients (block 152). The enhancement layer decoder 36, for example, can determine that there is at least one remaining non-zero coefficient when the EOB symbol is zero, and determine that there are no more remaining non-zero coefficients when the EOB symbol is one.

When the enhancement layer decoder 36 determines that there is at least one remaining non-zero coefficient, for example, the EOB symbol is zero, the enhancement layer decoder 36 decodes the series associated with the next non-zero coefficient (block 154). The series associated with the next non-zero coefficient represents the number of zero coefficients preceding the non-zero coefficient. Enhancement layer decoder 36 decodes a series using the same VLC coding table that is used by enhancement layer encoder 32. In one case, the enhancement layer decoder 36 may decode a series using the VLC coding table used in CAVLC coding, as defined in H.264 / MPEG-4, part 10, AVC, for use in coding the total series (sumRuns) when the total number of odds (TotalCoeffs) is one. However, other VLC coding tables may be used as long as this is the same table that is used by the enhancement layer encoder 32. Enhancement layer decoder 36 sets the coefficients to zero in an amount equal to the length of the series preceding the nonzero coefficient (block 156). If the series length, for example, is two, the enhancement layer decoder 36 may set two coefficients preceding the non-zero coefficient to zero.

Enhancement layer decoder 36 decodes the sign of a non-zero coefficient (block 158). The sign of a non-zero coefficient can be decoded as positive when the sign symbol is equal to one, and as negative when the sign symbol is zero. After decoding the sign of the non-zero coefficient, the enhancement layer decoder 36 may set the non-zero coefficient to plus one or minus one based on the decoded sign (step 160). As described above, the enhancement layer may not code the magnitude of the coefficients of the enhancement layer. Thus, the enhancement layer decoder 36 can be configured to set the value of all non-zero coefficients to unity.

Enhancement layer decoder 36 continues to decode the series and signs of the non-zero coefficient until the enhancement layer decoder 36 determines that there are no more non-zero coefficients remaining, for example, the EOB symbol is one. At this point, enhancement layer decoder 36 sets the remaining vector coefficients to zero if any coefficients remain (step 162). As described in detail with respect to FIG. 2, the enhancement layer decoder 36 uses a coefficient vector in addition to the prediction block and other data to recreate the video block for display on the display 28.

12-15 are block diagrams illustrating various configurations of encoders and / or decoders for use in scalable video encoding. These exemplary encoders and decoders are presented to illustrate the types of encoders in which the techniques of this disclosure may be used. However, exemplary configurations should in no way limit the described techniques. Techniques can be used in any scalable video encoder.

Each of the exemplary video encoders and decoders illustrated in FIGS. 12-15 may use the statistical coding techniques described in this disclosure to facilitate efficient coding of enhancement layer bit streams. The statistical encoding techniques of this disclosure may enable the encoding of additional video data, for example, in the form of refinements, in the enhancement layer bitstream in a single encoding pass, thereby reducing encoding complexity, encoding delay, and memory requirements. As will be described in further detail, each non-zero coefficient coefficient vector of the enhancement layer can be encoded without information about any subsequent coefficients, that is, any coefficients after the non-zero coefficient currently encoded. Encoding the enhancement layer in one pass can eliminate the need for the first pass to analyze the coefficient vector and the second pass to encode the coefficient vector based on the analysis.

12 is a block diagram illustrating an example scalable video encoder 170. Scalable video encoder 170, for example, may correspond to video encoder 20 in FIG. In the example of FIG. 12, the scalable video encoder 170 includes a base layer encoder 30 that includes a prediction module 172, a frame store 173, a transform module 174, quantization modules 175 A and 175B, inverse quantization units 176A and 176B, and inverse quantization unit 177 conversions, multiplexing module 178, and adders 179A-179C. The reflection of the various distinguishing features in FIG. 12 as modules is intended to highlight the various functional aspects of the illustrated devices and does not necessarily imply that such modules must be implemented using separate hardware or software components. In contrast, the functionality associated with one or more modules can be integrated into ordinary or separate hardware or software components.

Prediction module 712 generates a prediction block using intra or inter prediction. The prediction block may be a predicted version of the current encoded video block. As described above, prediction module 172 can generate a prediction block using intra prediction based on one or more previously encoded blocks of the base layer within the same frame as the current encoded block. Alternatively, the prediction module may generate a prediction block using inter prediction based on one or more previously encoded blocks within one or more adjacent frames of the base layer. Prediction module 172 may retrieve a previously encoded block from frame storage 173.

After internal or external prediction of the video blocks, the base layer encoder 30 forms a residual block by subtracting the prediction block created by the prediction unit 172 from the current video block in adder 179A. The residual block includes a plurality of pixel difference values that determine the number of differences between the pixel values of the current video block and the pixel values of the prediction block. The residual block may be represented in the format of a two-dimensional block (for example, a two-dimensional matrix or an array of pixel values). In other words, the residual block is a two-dimensional representation of pixel values.

The transform unit 174 applies the transform to the residual block to obtain residual transform coefficients. Transform module 174, for example, can apply a DCT transform, integer transform, directional transform, wavelet transform, or a combination thereof. After applying the transform to the residual block of pixel values, the quantization unit 175A quantizes the transform coefficients to further reduce the bit rate. The output of quantization module 175A, which correspond to the quantized coefficients associated with the base layer, is provided to multiplexing module 178.

After quantization, the inverse quantization module 176A applies inverse quantization to generate a recreated version of the residual block of transform coefficients. An adder 179B subtracts a recreated version of the residual transform coefficient block derived from the inverse quantization module 176A from the original residual transform coefficient block derived by the transform module 174. This block, which is here referred to as a transform difference block, is provided to the quantization module 175B. Quantization module 175B quantizes the transform coefficients to further reduce the bit rate. The output of quantization module 175B, which correspond to the quantized coefficients associated with the enhancement layer, is provided to multiplexing module 178. In one example, quantization module 175A may quantize residual coefficients using the first QP parameter, and quantization module 175B may quantize residual coefficient differences using the second QP parameter. The second parameter QP, for example, can be half the value of the first parameter QP, that is, equal to QP / 2.

After quantization by the quantization module 175B, the inverse quantization module 176B applies inverse quantization to generate a recreated version of the transform difference block. Adder 179C adds a recreated version of the residual transform coefficient block derived from the inverse quantization module 176A, with a recreated version of the transform difference block derived by the inverse quantization module 176B to form a recreated residual block.

The inverse transform module 177 applies the inverse transform to the recreated version of the video block. The reconstructed version of the video block is stored in the storage 173 frames and can be used by the prediction module 172 for internal or external encoding of the subsequent video block. Prediction unit 172 may provide control data, such as motion vectors, split sizes, intra-coding modes, and the like, to multiplexing unit 178. Multiplexing module 178 may combine base and enhancement layer data. In some cases, multiplexing module 178 may include statistical encoders for statistical encoding of base and enhancement layer data. In other cases, the encoders of the base and enhancement layer may be separate from the multiplexing module.

13 is a block diagram illustrating an exemplary scalable video decoder 180. Scalable video decoder 180, for example, may correspond to video decoder 26 in FIG. The scalable video decoder 180 in FIG. 13 includes a demultiplexing unit 181, inverse quantization units 182A and 182B, inverse transform unit 183, prediction unit 184, frame storage 185, and adders 186A and 186B.

The demultiplexing unit 181 receives scalable encoded video and demultiplexes the signals. In some cases, demultiplexing unit 181 may include statistical decoders for statistical decoding of base and enhancement layer data. In other cases, the base and enhancement layer decoders may be separate from the demultiplexing module.

The inverse quantization unit 182A inverse quantizes the quantized residual coefficients associated with the base layer, and the inverse quantization unit 182B inverse quantizes the quantized residual coefficients associated with the enhancement layer. In one example, the inverse quantization unit 182A may quantize the residual coefficients using the first QP parameter, and the inverse quantization unit 182B may quantize the residual coefficient differences using the second QP parameter. The second parameter QP, for example, can be half the value of the first parameter QP, that is, equal to QP / 2. The respective sets of inverse quantized transform coefficients output by the inverse quantization units 182A and 182B are added to the adder 186A to form a recreated residual transform unit. As described above, the inverse quantized transform coefficients output by the inverse quantization unit 182A may correspond to a basic quality level, and the inverse quantized transform coefficients output by the inverse quantization unit 182B, when added to the output of the inverse quantization unit 182A, result in an increased level quality.

The inverse transform unit 183 applies the inverse transform, such as the inverse DCT transform, the inverse integer transform, the inverse wavelet transform, or the inverse directional transform, to the sum of the residual blocks of coefficients subjected to inverse quantization to obtain a residual block of pixel values. Adder 186B adds the prediction block generated by the prediction unit 184 with a residual block of pixel values to form a recreated video block of the base layer. As described in detail above, prediction module 184 may generate a prediction block using one or more adjacent blocks within a normal frame in the case of intra prediction or one or more blocks within adjacent frames in the case of inter prediction that can be stored in a frame store 185.

14 is a block diagram illustrating another exemplary scalable video decoder 190. Scalable video decoder 190, for example, may correspond to video decoder 26 in FIG. The scalable video decoder 190 of FIG. 14 includes a demultiplexing unit 191, inverse quantization units 192A and 192B, inverse transform units 193A and 193B, a prediction unit 194, frame storage 195, and adders 196A and 196B.

The demultiplexing unit 191 receives scalable encoded video and demultiplexes the signals. In some cases, demultiplexing unit 181 may include statistical decoders for statistical decoding of base and enhancement layer data. In other cases, the base and enhancement layer decoders may be separate from the demultiplexing module.

The inverse quantization module 192A and the inverse transform module 193A apply the inverse quantization and inverse transform operations to the decoded residual coefficients associated with the base layer to obtain a recreated version of the residual block of the base layer. The inverse quantization module 192B and the inverse transform module 193B apply the inverse quantization and inverse transform operations to the decoded residual coefficients associated with the enhancement layer to obtain a recreated version of the residual enhancement layer block. In one example, the inverse quantization module 192A may quantize the residual coefficients using the first QP parameter, and the inverse quantization module 192B may quantize the residual coefficient differences using the second QP parameter. The second parameter QP, for example, can be half the value of the first parameter QP, that is, equal to QP / 2.

Prediction unit 194 may generate a prediction block using one or more adjacent blocks within an ordinary frame in the case of intra prediction or one or more blocks within adjacent frames in the case of inter prediction, which can be stored in a frame store 195. Adder 196A adds the prediction block generated by the prediction unit 194 to the recreated residual block derived from the inverse transform module 193 to generate decoded video data at a basic quality level. Decoded video data having a basic level of quality is output from scalable video encoder 190.

Decoded video data with a basic level of quality is also provided on the adder 196B. Adder 196B adds the output of adder 196 with a recreated version of the residual block of the enhancement layer derived from the inverse transform module 193B to generate decoded video data at a second, higher quality level. Decoded video data having a basic level of quality is output from scalable video encoder 190.

15 is a block diagram illustrating another exemplary video encoder 200. In the example of FIG. 15, the base layer encoder 30 includes a prediction unit 33A, a frame store 35A, a transform module 38A, a quantization module 40A, a coefficient scanning module 41A, a module Inverse quantization 42A, inverse transform module 44A, base layer statistic encoder 46, adders 48A-48C, and intra prediction module 40A. The reflection of the various distinguishing features in FIG. 15 as modules is intended to highlight various functional aspects of the illustrated devices and does not necessarily imply that such modules must be implemented through separate hardware or software components. In contrast, the functionality associated with one or more modules can be integrated into ordinary or separate hardware or software components.

Prediction unit 33A generates a prediction block using inter prediction, for example motion compensation prediction. The prediction block may be a predicted version of the current encoded video block. As described above, prediction unit 33A may generate a prediction block using inter prediction based on one or more previously encoded blocks within one or more adjacent frames of the base layer. Prediction unit 33A may retrieve a previously encoded block from frame storage 35A.

After inter prediction of the video blocks, the base layer encoder 30 generates a residual block by subtracting the prediction block created by the prediction unit 33A from the current video block in the adder 48A. The residual block includes a plurality of pixel difference values that determine the number of differences between the pixel values of the current video block and the pixel values of the prediction block. The residual block can be represented in the format of a two-dimensional block (for example, a two-dimensional matrix or an array of pixel values). In other words, the residual block is a two-dimensional representation of pixel values.

The transform unit 38A applies the transform to the residual block to obtain residual transform coefficients. Transform unit 38A, for example, may apply a DCT transform, integer transform, directional transform, wavelet transform, or a combination thereof. After applying the transform to the residual block of pixel values, the quantization unit 40A quantizes the transform coefficients to further reduce the bit rate. After quantization, the inverse quantization unit 42A and the inverse transform unit 44A may apply inverse quantization and inverse transform, respectively, to recreate the residual block. Adder 48B adds the recreated residual block to the prediction block created by the prediction unit 33A to obtain a recreated video block for storage in the frame store 35A. The reconstructed video block stored in the frame storage 34 may be used by the prediction module 32 from the base layer encoder 30 for internal or external encoding of the subsequent video block. In addition, as will be described in more detail below, the reconstructed video block stored in the frame store 35A may be used by the prediction unit 33B of the enhancement layer encoder 32 for internally or externally encoding the refinements of the video block in the enhancement layer.

After quantization, the adder 48C subtracts from the quantized residual coefficients the intra-predicted block generated by the intra-prediction unit 40A. The intra prediction unit 40A may generate a prediction block using intra prediction based on one or more previously encoded blocks within the same frame as the current encoded block. The base layer statistic encoder 46 statistically encodes coefficients derived from the adder 48C, for example, using CAVLC encoding as defined in the H.264 / MPEG-4 standard, AVC part 10 and described in detail above with respect to FIG. 2.

Enhancement layer encoder 72 includes prediction module 33B, frame storage 35B, transform module 38B, quantization module 40B, coefficient scanning module 41B, inverse quantization module 42B, inverse transform module 44B, enhancement layer statistical encoder 49, and adders 48D-48F. The modules of the enhancement layer encoder 32 are substantially similar to the similarly numbered modules of the base layer encoder 30. Thus, only differences will be described.

Prediction unit 33B of enhancement layer encoder 32 generates a prediction block, which is a predicted version of the current video block. Unlike the prediction unit 33A of the base layer encoder 30, which uses the previously encoded base layer blocks to form the prediction blocks, the prediction module 33B of the enhancement layer encoder 32 may generate a prediction block based on one or more previously encoded enhancement layer blocks. The recreated video block of the enhancement layer may have a second, higher quality level than the prediction block of the base layer.

An additional difference between the enhancement layer encoder 32 and the base layer encoder 30 is that the output of the inverse quantization module 42B of the enhancement layer encoder 32 is combined with the output of the inverse quantization module 42A of enhancement layer encoder 30 in the adder 48F. Adding the output of the inverse quantization modules 42A and 42B forms a recreated video block of higher quality, thereby enabling better prediction of the prediction module described above.

The techniques described in this disclosure may be implemented in hardware, software, firmware, or any combination thereof. Any distinguishing features described as modules or components can be implemented together in an integrated logic device or separately as separate but interacting logic devices. When implemented in software, the techniques can be implemented at least in part by a computer-readable medium containing such instructions that, when executed, execute one or more of the methods described above. A computer-readable medium may form part of a computer program product, which may include packaging materials. The computer-readable medium may comprise random access memory (RAM), such as synchronous dynamic random access memory (SDRAM), read-only memory (ROM; ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM; EEPROM), flash memory, magnetic or optical storage media, etc. The techniques, in addition or alternatively, can be implemented at least in part by a computer-readable transmission medium that carries or transmits code in the form of instructions or data structures that can be accessed, which can be read and / or executed by a computer.

The code may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, specialized integrated circuits (ASICs), programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Accordingly, the term “processor” as used herein may mean any preceding structure or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functionality described herein may be provided within specialized software modules or hardware modules configured to encode and decode, or integrated into a combined video codec (CODEC). The description of the various distinguishing features as modules is intended to highlight the various functional aspects of the illustrated devices and does not necessarily imply that such modules must be implemented through separate hardware or software components. In contrast, the functionality associated with one or more modules can be integrated into ordinary or separate hardware or software components.

A variety of embodiments have been described. These and other embodiments are within the scope of the following claims.

Claims (66)

1. A method of encoding video data using scalable video encoding, the method comprises the steps of:
encode the video block with the first quality as part of the bitstream of the base layer; and
encode as part of at least one bit stream of the enhancement layer, refinement of the video block, which when combined with the video block encoded with the first quality, results in a video block having a second quality that is greater than the first quality, and the refinement of the video block is encoded without performing a first encoding pass for collecting statistics for use when choosing a video encoding table, the refinement encoding contains, for each non-zero refinement coefficient of the video block, an encoding the use of a symbol to indicate the presence of at least one remaining non-zero coefficient, a series length to indicate the number of zero coefficients preceding the non-zero coefficient, and the sign of the non-zero coefficient; and correct the values of non-zero refinement coefficients of the video block, making them equal to unity. 2. The method according to claim 1, wherein encoding the refinement of the video block comprises the step of encoding each non-zero refinement coefficient without analyzing any subsequent coefficients. 3. The method according to claim 1, further comprising encoding a symbol to indicate the absence of the remaining non-zero coefficients in the refinement of the video block after encoding the last non-zero coefficient. 4. The method according to claim 1, wherein adjusting the magnitude of each of the non-zero refinement coefficients of the video block, making them equal to one, comprises the step of encoding non-zero coefficients without encoding the coefficient values. 5. The method according to claim 1, in which the encoding of the refinement of the video block as part of the bitstream of the enhancement layer comprises the step of encoding the refinement of the video block in such a way that the additional coefficients of the refinement of the video block are decoded without access to the coefficient information of the video block encoded with the first quality as part base layer bitstream. 6. The method according to claim 1, further comprising the step of encoding the refinement of the video block using only one variable length coding table (VLC). 7. The method according to claim 1, in which the encoding of the video block with the first quality as part of the base layer comprises the step of encoding the video block with the first quality using an encoding technique in which the vector of coefficients of the video block in the first encoding pass is analyzed and the coefficient vector in the second pass coding based analysis. 8. The method according to claim 7, in which: encoding a video block with a first quality comprises encoding a video block with a first quality using a variable length context adaptive coding (CAVLC) process; and encoding the refinement of the video block comprises the step of encoding the refinement of the video block using one of the VLC coding tables defined in the CAVLC process. 9. The method according to claim 1, in which the first and second quality comprise one of: the first and second signal to noise ratios (SNR) and the first and second spatial resolution. 10. A device for encoding video data using scalable video encoding, the device contains at least one encoder, which:
encodes the video block with the first quality as part of the bitstream of the base layer,
encodes, as part of at least one bit stream of the enhancement layer, the refinement of the video block, which when combined with the video block encoded with the first quality, results in a video block having a second quality that is greater than the first quality, wherein the refinement of the video block is encoded in one encoding pass, and wherein refinements include at least one of additional coefficients or refinements to the coefficients;
encodes for each non-zero coefficient of refinement of the video block a symbol to indicate the presence of at least one remaining non-zero coefficient, a series length to indicate the number of zero coefficients preceding the non-zero coefficient, and the sign of the non-zero coefficient; and
corrects the values of non-zero refinement coefficients of the video block, making them equal to one. 11. The device according to claim 10, in which at least one encoder encodes each nonzero refinement coefficient without analyzing subsequent coefficients. 12. The device of claim 10, in which at least one encoder encodes a symbol to indicate the absence of the remaining non-zero coefficients in the refinement of the video block after encoding the last non-zero coefficient. 13. The device of claim 10, in which at least one encoder encodes non-zero coefficients without encoding the values of the coefficients. 14. The device of claim 10, in which at least one encoder encodes the refinement of the video block so that the refinement coefficients of the video block are decoded without access to the coefficient information of the video block encoded with the first quality as part of the base layer bitstream. 15. The device according to claim 10, in which at least one encoder encodes the refinement of the video block using only one variable length coding table (VLC). 16. The device of claim 10, in which at least one encoder encodes the video block with a first quality using an encoding technique in which the vector of coefficients of the video block in the first encoding pass is analyzed and the coefficient vector in the second encoding pass is encoded based on the analysis. 17. The device according to clause 16, in which at least one encoder: encodes a video block with a first quality using a context-adaptive variable-length coding (CAVLC) process; and encodes the video block refinements using one of the VLC coding tables defined in the CAVLC process. 18. The device according to claim 10, in which the first and second quality contain one of: the first and second signal to noise ratios (SNR) and the first and second spatial resolution. 19. The device of claim 10, wherein the at least one encoder comprises: a base layer encoder that encodes a video block with a first quality as part of a base layer bitstream, and an enhancement layer encoder that encodes as part of at least one enhancement bitstream layer, refinement of the video block, which, when combined with the video block encoded with the first quality, results in a video block having a second quality that is greater than the first quality. 20. The device according to claim 10, wherein the device is a wireless communication device. 21. The device according to claim 10, wherein the device is an integrated circuit device. 22. Machine-readable medium containing instructions to force one or more processors:
encode the video block with the first quality as part of the bitstream of the base layer; and
encode, as part of at least one bit stream of the enhancement layer, additional refinement coefficients of the video block, which, when combined with the video block encoded with the first quality, result in a video block having a second quality that is greater than the first quality, and the refinement of the video block is encoded in one a coding pass, and wherein refinements include at least one of additional coefficients or refinements to the coefficients;
encode, for each non-zero refinement coefficient of the video block, a symbol to indicate the presence of at least one remaining non-zero coefficient, a series length to indicate the number of zero coefficients preceding the non-zero coefficient, and the sign of the non-zero coefficient; and
Correct the values of non-zero refinement coefficients of the video block, making them equal to unity. 23. The computer-readable medium of claim 22, wherein the instructions cause one or more processors to encode each non-zero refinement coefficient without analyzing subsequent coefficients. 24. The computer-readable medium of claim 22, wherein the instructions cause one or more processors to encode a character to indicate that there are no remaining non-zero coefficients in the refinement of the video block after encoding the last non-zero coefficient. 25. The computer-readable medium of claim 22, wherein the instructions cause one or more processors to encode non-zero coefficients without encoding coefficient values. 26. The computer-readable medium of claim 22, wherein the instructions cause one or more processors to encode the video block refinements in such a way that the video block refinement coefficients are decoded without access to the video block coefficient information encoded with the first quality as part of the base layer bitstream. 27. The computer-readable medium of claim 22, wherein the instructions cause one or more processors to encode the refinement of the video block using only one variable length coding table (VLC). 28. The computer-readable medium of claim 22, wherein the instructions cause one or more processors to encode the video block with a first quality using an encoding technique in which the coefficient vector of the video block in the first encoding pass is analyzed and the coefficient vector in the second encoding pass is encoded based on the analysis. 29. The computer-readable medium of claim 28, wherein the instructions force one or more processors: encode a video block with a first quality using a variable length context adaptive coding (CAVLC) process; and encode the video block refinements using one of the VLC coding tables defined in the CAVLC process. 30. The computer readable medium of claim 22, wherein the first and second quality comprise one of: a first and second signal to noise ratio (SNR) and a first and second spatial resolution. 31. A device for encoding video using scalable video encoding, the device comprises:
first means for encoding a video block with a first quality as part of the base layer bitstream; and
second means for encoding, as part of at least one bit stream of the enhancement layer, the refinement of the video block, which when combined with the video block encoded with the first quality, results in a video block having a second quality that is greater than the first quality, and the refinement of the video block is encoded in one pass coding, and refinements include at least one of additional coefficients or refinements to the coefficients;
wherein the second encoding means encodes, for each non-zero refinement coefficient of the video block, a symbol for indicating the presence of at least one remaining non-zero coefficient, a series length for indicating the number of zero coefficients preceding the non-zero coefficient, and the sign of the non-zero coefficient , and
wherein the second encoding means corrects the values of non-zero refinement coefficients of the video block, making them equal to one. 32. The device according to p, in which the second encoding means encodes each nonzero refinement coefficient without analyzing subsequent coefficients. 33. The device according to p, in which the second encoding means encodes a symbol to indicate the absence of the remaining non-zero coefficients in the refinement of the video block after encoding the last non-zero coefficient. 34. The device according to p, in which the second encoding means encodes non-zero coefficients without encoding the values of the coefficients. 35. The device according to p, in which the second encoding means encodes the refinement of the video block so that the coefficients of the refinement of the video block are decoded without access to the coefficient information of the video block encoded with the first quality as part of the base layer bitstream. 36. The device according to p, in which the second encoding means encodes the refinement of the video block using only one variable length coding table (VLC). 37. The device according to p, in which the first encoding means encoding the video block with the first quality as part of the base layer, comprises encoding the video block with the first quality using the encoding technique, in which the vector of coefficients of the video block in the first encoding pass is analyzed and the coefficient vector second pass analysis-based coding. 38. The device according to clause 37, in which the first encoding means: encodes a video block with a first quality using a context-adaptive variable-length coding (CAVLC) process; and encodes the video block refinements using one of the VLC coding tables defined in the CAVLC process. 39. The device according to p, in which the first and second quality contain one of: the first and second signal to noise ratios (SNR) and the first and second spatial resolution. 40. A method for decoding video data using scalable video encoding, the method comprises the steps of:
decode the bitstream of the base layer to obtain a video block with the first quality;
the enhancement layer bitstream is decoded to obtain video block refinements which, when combined with the video block decoded with the first quality, result in a video block having a second quality, and the refinements include at least one of additional coefficients or refinements to the coefficients, and decoding the enhancement layer includes the step of decoding, for each non-zero refinement coefficient of the video block, a symbol indicating the presence of at least one remaining I have a non-zero coefficient, the length of the series, indicating the number of zero coefficients preceding the nonzero coefficient and a sign of non-zero coefficient, and
set the value of each non-zero coefficient of the coefficient vector equal to unity. 41. The method of claim 40, further comprising decoding a character following the last non-zero coefficient, which indicates the absence of the remaining non-zero coefficients in the refinement of the video block. 42. The method according to paragraph 41, further comprising forming a vector of coefficients for refinement of the video block using a decoded series of each coefficient, the sign of each coefficient, and a symbol indicating the absence of remaining non-zero coefficients. 43. The method of claim 40, wherein decoding the refinement of the video block comprises decoding the refinement of the video block without accessing the coefficient information of the video block encoded with the first quality. 44. The method according to clause 40, further comprising the step of decoding the refinement of the video block using only one variable length coding table (VLC). 45. The method according to item 44, in which one VLC coding table contains one of the VLC coding tables specified in the CAVLC process. 46. A device for decoding video data using scalable video encoding, the device comprises at least one decoder, which:
decodes the base layer bitstream to obtain a video block with a first quality; and
decodes the bitstream of the enhancement layer to obtain refinements of the video block, which, when combined with the video block decoded with the first quality, results in a video block having a second quality,
wherein refinements include at least one of additional coefficients or refinements to the coefficients,
wherein at least one decoder decodes, for each non-zero refinement coefficient of the video block, a symbol indicating the presence of at least one remaining non-zero coefficient, a series length indicating the number of zero coefficients preceding the non-zero coefficient, and a non-zero sign coefficient, and
wherein at least one decoder sets the values of each non-zero coefficient of the coefficient vector to unity. 47. The device according to item 46, in which at least one decoder decodes the character following the last non-zero coefficient, which indicates the absence of the remaining non-zero coefficients in the refinement of the video block. 48. The device according to clause 47, in which at least one decoder generates a vector of coefficients for refinement of the video block using a decoded series of each coefficient, the sign of each coefficient and a symbol indicating the absence of remaining non-zero coefficients. 49. The device according to item 46, in which at least one decoder decodes the refinement of the video block without access to the coefficient information of the video block encoded with the first quality. 50. The device according to item 46, in which at least one decoder decodes the refinement of the video block using only one variable length coding table (VLC). 51. The device according to item 50, in which one VLC coding table contains one of the VLC coding tables specified in the CAVLC process. 52. The device according to item 46, in which at least one decoder comprises:
a base layer decoder that decodes the base layer bitstream to obtain a video block with a first quality; and an enhancement layer decoder that decodes the enhancement layer bitstream to obtain refinements to the video block. 53. The device according to item 46, and the device is a wireless communication device. 54. The device according to item 46, and the device is a device on integrated circuits. 55. A computer-readable medium containing instructions to force one or more processors:
decode the base layer bitstream to obtain a video block with a first quality; and
decode the enhancement layer bitstream to obtain video block refinements which, when combined with the video block decoded with the first quality, result in a video block having a second quality, wherein the refinements include at least one of additional coefficients or refinements to the coefficients;
decode, for each non-zero refinement coefficient of the video block, a symbol indicating the presence of at least one remaining non-zero coefficient, a series length indicating the number of zero coefficients preceding the non-zero coefficient, and the sign of the non-zero coefficient; and
set the value of each non-zero coefficient of the coefficient vector equal to unity. 56. The computer-readable medium of claim 55, wherein the instructions cause one or more processors to decode the character following the last non-zero coefficient, which indicates the absence of the remaining non-zero coefficients in the refinement of the video block. 57. The computer-readable medium of claim 56, wherein the instructions cause one or more processors to form a coefficient vector to refine the video block using a decoded series of each coefficient, the sign of each coefficient, and a symbol indicating the absence of remaining non-zero coefficients. 58. The computer-readable medium of claim 55, wherein the instructions cause one or more processors to decode the refinement of the video block without accessing the coefficient information of the video block encoded with the first quality. 59. The computer-readable medium of claim 55, wherein the instructions cause one or more processors to decode the video block refinements using only one variable length coding table (VLC). 60. The computer-readable medium of claim 59, wherein one VLC coding table comprises one of the VLC coding tables specified in the CAVLC process. 61. A device for decoding video data using scalable video encoding, the device comprises:
first means for decoding the base layer bitstream to obtain a video block with a first quality;
second means for decoding the enhancement layer bitstream to obtain refinements of the video block, which when combined with the video block decoded with the first quality, results in a video block having a second quality, and
third means for establishing the magnitude of each non-zero coefficient of the coefficient vector equal to one,
wherein refinements include at least one of additional coefficients or refinements to the coefficients,
wherein the second decoding means decodes, for each non-zero refinement coefficient of the video block, a symbol indicating the presence of at least one remaining non-zero coefficient, a series length indicating the number of zero coefficients preceding the non-zero coefficient, and the sign of the non-zero coefficient. 62. The device according to 61, in which the second decoding means decodes the symbol following the last non-zero coefficient, which indicates the absence of the remaining non-zero coefficients in the refinement of the video block. 63. The device according to item 62, further comprising means for generating a vector of coefficients for refinement of the video block using a decoded series of each coefficient, the sign of each coefficient and a symbol indicating the absence of remaining non-zero coefficients. 64. The device according to p, in which the second decoding means decodes the refinement of the video block without access to the coefficient information of the video block encoded with the first quality. 65. The device according to 61, in which the second decoding means decodes the refinement of the video block using only one variable length coding table (VLC). 66. The device according to item 65, in which one VLC coding table contains one of the VLC coding tables specified in the CAVLC process.

Images