JP7033182B2 - Audio decoder for interleaving signals - Google Patents

Description

The disclosure herein generally relates to multi-channel audio coding. In particular, the present disclosure relates to encoders and decoders for hybrid coding, including parametric coding and discrete multi-channel coding.

"Cross-reference to related applications"
This application also claims priority to US Provisional Patent Application No. 61 / 808,680 filed April 5, 2013 PCT Application No. PCT / EP2014 filed April 4, 2014. It is a continuation of US Patent Application No. 14 / 772,001 filed on September 1, 2015, which is the national stage application of Article 371 of / 056852, and as a result, each of these applications is a continuation of that. It is included herein by reference in its entirety.

In conventional multi-channel audio coding, possible coding schemes include discrete multi-channel coding or parametric coding such as MPEG Surround®. The scheme used depends on the bandwidth of the audio system. Parametric coding methods are known to be scalable and efficient with respect to listening quality, which makes parametric coding methods particularly attractive in low bit rate applications. Discrete multi-channel coding is often used in high bit rate applications. Existing distribution or processing formats, and associated coding techniques, may be improved in terms of their bandwidth efficiency, especially in applications with bit rates between low and high bit rates.

U.S. Pat. No. 7,292,901 (US7292901) (according to "Kroon" et al.) Is a hybrid audio signal formed from at least one downmixed spectral component and at least one unmixed spectral component. It is related to the hybrid coding method. The methods exposed in that application can increase the capacity of applications with a particular bit rate, but further improvements may be needed to further increase the efficiency of the audio processing system.

Examples of examples will be described herein with reference to the accompanying drawings.

It is a generalized block diagram of the decoding system by an example embodiment. It is a figure which illustrates the 1st part of the decoding system in FIG. It is a figure which illustrates the 2nd part of the decoding system in FIG. It is a figure which illustrates the 3rd part of the decoding system in FIG. It is a generalized block diagram of the coding system by an example embodiment. It is a generalized block diagram of the decoding system by an example embodiment. It is a figure which illustrates the 3rd part of the decoding system in FIG. It is a generalized block diagram of the coding system by an example embodiment.

All drawings are schematic and generally show only the elements necessary to illustrate the present disclosure, while other elements may be omitted or simply suggested. Unless otherwise indicated, equivalent reference numerals refer to equivalent elements in different drawings.

"Overview of decoder"
As used herein, the audio signal can be either a pure audio signal, an audio portion of an audiovisual signal or multimedia signal, or any of these combined with metadata.

As used herein, downmixing of a plurality of signals means combining the plurality of signals, for example, by forming a linear combination so that a smaller number of signals are acquired. The reverse operation for down-mixing is called up-mixing, that is, it refers to manipulating a smaller number of signals so as to acquire a larger number of signals.

According to the first aspect, the embodiment of the example proposes a method, an apparatus, and a computer program product for restoring a multi-channel audio signal based on an input signal. The proposed methods, devices, and computer program products may generally have the same features and advantages.

According to an embodiment of the example, a decoder suitable for a multi-channel audio processing system for restoring M (M> 2) encoded channels is provided. The decoder is to receive N (1 <N <M) waveform-coded downmix signals containing spectral coefficients corresponding to frequencies between the first crossover frequency and the second crossover frequency. Includes a first receiving stage configured.

The decoder is a second receive stage configured to receive M waveform coded signals including spectral coefficients corresponding to frequencies up to the first crossover frequency, with M waveform coding. It further comprises a second receive stage, each of which corresponds to each of the M coded channels.

The decoder is configured to downmix the M waveform coded signals into N downmix signals containing spectral coefficients corresponding to frequencies up to the first crossover frequency of the second receive stage. Further includes the downstream downmix stage.

The decoder combines each of the N waveform-coded downmix signals received by the first receive stage with the corresponding one of the N downmix signals from the downmix stage to N. Further includes a first receiving stage and a first coupling stage downstream of the downmix stage configured to be a coupled downmix signal of.

The decoder is configured to extend each of the N coupled downmix signals from the first coupling stage to a frequency range above the second crossover frequency by performing a high frequency restoration. , Further includes a high frequency restoration stage downstream of the first coupling stage.

The decoder is a parametric of N combined downmix signals frequency-extended from the high frequency restoration stage to M upmix signals containing spectral coefficients corresponding to frequencies above the first crossover frequency. An upmix stage downstream of the high frequency restore stage configured to perform the upmix, where each of the M upmix signals corresponds to one of the M coded channels. Includes more upmix stages.

The decoder is configured to combine the M upmix signals from the upmix stage with the M waveform coded signals received by the second receive stage, the upmix stage and the second receive. Further includes a second coupling stage downstream of the stage.

The M waveform-encoded signals are purely waveform-encoded signals without mixing parametric signals, i.e. they are an unmixed discrete representation of the processed multi-channel audio signal. be. The advantage of having a lower frequency represented by these waveform coded signals may be that the human ear is more sensitive to the portion of the audio signal that has the lower frequency. Encoding this portion with even better quality can enhance the overall impression of the decoded audio.

The advantage of having at least two downmix signals is that this embodiment provides an increased dimensionality of the downmix signal when compared to a system with only one downmix channel. be. According to this embodiment, better decoded audio quality, which may outweigh the gain at the bit rate provided by one downmix signaling system, may therefore be provided.

The advantage of using hybrid coding, including parametric downmixing and discrete multi-channel coding, is that this decodes for a particular bit rate when compared to the traditional parametric coding approach, MPEG Surround with HE-AAC. It means that the quality of the audio signal can be improved. At bit rates of about 72 kilobits per second (kbps), traditional parametric coding models can saturate, i.e. because the quality of the decoded audio signal is due to a lack of bits for coding. Not limited by the shortcomings of the parametric model. Therefore, for bit rates from about 72 kbps, it may be more useful to use the bits for discrete waveform coding of lower frequencies. At the same time, the hybrid approach, which uses parametric downmix and discrete multi-channel coding, uses the approach in which all bits are used to waveform-code lower frequencies. Improves the quality of decoded audio signals for specific bit rates, eg, bit rates below 128 kbps, compared to doing and using spectral band replication (SBR) for the remaining frequencies. It is possible.

The advantage of having N waveform-coded downmix signals containing only spectral data corresponding to frequencies between the first crossover frequency and the second crossover frequency is a requirement for audio signal processing systems. It means that the bit communication speed can be reduced. Instead, the bits saved by having a bandpass filtered downmix signal can be used to waveform encode lower frequencies, eg sample frequencies for those frequencies. Can be made higher, or the first crossover frequency can be increased.

As mentioned above, high frequencies as parts of the audio signal with frequencies above the second crossover frequency are more sensitive to the parts of the audio signal that have low frequencies, as the human ear is more sensitive. It can be reproduced by high frequency restoration without reducing the perceived audio quality of the decoded audio signal.

A further advantage with respect to this embodiment is that the parametric upmix performed in the upmix stage processes only the spectral coefficients corresponding to frequencies above the first crossover frequency, thus reducing the complexity of the upmix. It can be that.

According to another embodiment, each of the N waveform-coded downmix signals containing the spectral coefficients corresponding to the frequencies between the first crossover frequency and the second crossover frequency is the first crossover. The coupling performed in the first coupling stage is combined with the corresponding one of the N downmix signals containing the spectral coefficients corresponding to the frequencies up to the frequency to form the N combined downmix signals. , Performed in the frequency domain.

The advantage of this embodiment is that the M waveform coded signals and the N waveform coded downmix signals are independent of the M waveform coded signals and the N waveform coded downmix signals, respectively. It could be that it could be encoded by a waveform coder using an overlapping windowed transform by window processing and still be decodable by a decoder.

According to another embodiment, extending each of the N coupled downmix signals in the high frequency restoration stage to a frequency range above the second crossover frequency is performed in the frequency domain.

According to a further embodiment, the coupling performed in the second coupling stage, i.e., the first cross of M upmix signals containing spectral coefficients corresponding to frequencies above the first crossover frequency. Coupling with M waveform coded signals containing spectral coefficients corresponding to frequencies up to the overfrequency is performed in the frequency domain. As mentioned above, the advantage of coupling the signal in the QMF region is that the independent windowing of the overlap windowing transform used to encode the signal in the MDCT region can be used. Is.

According to another embodiment, the parametric upmix of the frequency-extended N coupled downmix signals to the M upmix signals performed in the upmix stage is performed in the frequency domain. ..

According to yet another embodiment, downmixing M waveform coded signals to N downmix signals containing spectral coefficients corresponding to frequencies up to the first crossover frequency is in the frequency domain. Will be executed.

According to one embodiment, the frequency domain is a Quadrature Mirror Filter (QMF) domain.

According to another embodiment, it is performed in a downmixing stage where M waveform coded signals are downmixed to N downmix signals containing spectral coefficients corresponding to frequencies up to the first crossover frequency. Down mixing is performed in the time domain.

According to yet another embodiment, the first crossover frequency is determined by the bit transmission rate of the multi-channel audio processing system. This is utilized to improve the quality of the decoded audio signal with the available bandwidth simply because the portion of the audio signal having a frequency below the first crossover frequency is simply waveform coded. Can bring about that.

According to another embodiment, extending each of the N coupled downmix signals to a frequency range above the second crossover frequency by performing high frequency restoration in the high frequency restoration stage is high frequency. Performed using restore parameters. The high frequency restoration parameters can be received by the decoder, for example, in the receiving stage and then transmitted to the high frequency restoring stage. Radio frequency restoration may include performing, for example, spectral band replication (SBR).

According to another embodiment, the parametric upmix in the upmixing stage is done with the use of upmix parameters. The upmix parameters are received by the encoder, for example, in the receiving stage and transmitted to the upmixing stage. An uncorrelated version of the frequency-extended N coupled downmix signals is generated, along with the frequency-extended N coupled downmix signals and the frequency-extended N coupled. Matrix operations are performed on the uncorrelated version of the downmix signal. The parameters of the matrix operation are given by the upmix parameters.

According to another embodiment, the received N waveform-coded downmix signals in the first receiving stage and the received M waveform-coded signals in the second receiving stage are N, respectively. Waveform-coded downmix signals, and overlap-windowization conversion by independent window processing for M waveform-encoded signals.

The advantage of this can be that it allows for improved coding quality and, therefore, improved quality of the decoded multi-channel audio signal. For example, if at some point in time a transient signal is detected in the higher frequency band, the default window sequence may be retained for the lower frequency band, while the waveform encoder may use this shorter window sequence. Special timeframes can be encoded.

According to an embodiment, the decoder includes a third receive stage configured to receive additional waveform coded signals including spectral coefficients corresponding to a subset of frequencies above the first crossover frequency. obtain. The decoder may further include an interleaving stage downstream of the upmix stage. The interleaving stage may be configured to interleave the additional waveform coded signal with one of the M upmix signals. A third receive stage may be further configured to receive a plurality of further waveform coded signals, and an interleaving stage may interleave the plurality of further waveform coded signals with a plurality of M upmix signals. It may be further configured to do so.

This is because certain parts of the frequency range above the first crossover frequency, which are difficult to parameterically restore from the downmix signal, are interleaved with the parameterically restored upmix signal. It is advantageous in that it can be provided in a waveform coding format.

In one exemplary embodiment, interleaving is performed by adding a further waveform coded signal to one of the M upmix signals. According to another representative embodiment, the step of interleaving the additional waveform coded signal with one of the M upmix signals is one of the M upmix signals with the additional waveform code. It comprises the step of replacing with a further waveform coded signal in a subset of frequencies above the first crossover frequency corresponding to the spectral coefficient of the signal.

According to a representative embodiment, the decoder may be further configured to receive a control signal, for example by a third receive stage. The control signal can indicate how to interleave the additional waveform coded signal with one of the M upmix signals, and the additional waveform coded signal of the M upmix signals. The step of interleaving with one is based on the control signal. Specifically, the control signal, such as one or more time / frequency tiles in the QMF region, where the additional waveform coded signal should be interleaved with one of the M upmix signals. It may indicate a frequency range and a time range. Therefore, interleaving can occur at time and frequency within one channel.

The advantage of this is that the overlap windowed transform aliasing used to encode the waveform coded signal, or the time and frequency ranges that are not plagued by startup / fade-out issues can be selected. be.

Some embodiments disclose methods for decoding an encoded audio bitstream in an audio processing system. The method involves extracting a first waveform coded signal containing spectral coefficients corresponding to frequencies from the encoded audio bitstream to the first crossover frequency, and parametric decoding at the second crossover frequency. Includes steps to perform and generate the restored signal. The second crossover frequency is above the first crossover frequency, and parametric decoding produces a restored signal using the restoration parameters obtained from the encoded audio bitstream. The method comprises extracting a second waveform coded signal from the encoded audio bitstream containing a spectral coefficient corresponding to a subset of frequencies above the first crossover frequency and a second waveform coding. It further comprises interleaving the signal with the restored signal and generating an interleaved signal. The interleaved signal is then combined with the first waveform coded signal.

There are many variants as well. For example, the first crossover frequency can be determined by the bit transmission rate of the audio processing system, and the interleaving steps are (i) adding the second waveform coded signal to the restored signal, (ii). It may include combining the second waveform coded signal with the restored signal, or (iii) replacing the restored signal with the second waveform coded signal. The step of combining the interleaved signal with the first waveform coded signal can be performed in the frequency domain, or the step of performing parametric decoding at the second crossover frequency to generate the restored signal. Can be performed in the frequency domain. Parametric decoding can include either (i) parametric upmixing using upmix parameters or (ii) high frequency restoration using high frequency restoration parameters such as spectral band replication (SBR). The method may further include receiving a control signal used during the interleaving step to generate the interleaved signal. The control signal may indicate how to interleave the second waveform coded signal with the restored signal by specifying either a frequency range or a time range for the interleaving step. The first value of the control signal may indicate that the interleaving steps are performed for each frequency range. The interleaving steps can also be performed prior to the joining step. The interleaving step and the joining step can be similarly combined into a single stage or action. The first waveform coded signal and the second waveform coded signal may include signals that represent the waveform of the audio signal in the frequency or time domain.

"Overview of the encoder"
According to the second aspect, the embodiment of the example proposes a method, an apparatus, and a computer program product for encoding a multi-channel audio signal based on an input signal.

The proposed methods, devices, and computer program products may generally have the same features and advantages.

The features and configuration advantages presented in the decoder overview above may generally be valid for the corresponding features and configurations for the encoder.

According to an embodiment of the example, a encoder suitable for a multi-channel audio processing system for encoding M (M> 2) channels is provided.

The encoder includes a receive stage configured to receive M signals corresponding to the M channels to be encoded.

The encoder receives the M signals from the receiving stage and waveform-codes the M signals individually with respect to the frequency range corresponding to the frequency up to the first crossover frequency, so that the first crossover is performed. It further comprises a first waveform coding stage configured to generate M waveform coded signals including spectral coefficients corresponding to frequencies up to frequency.

The encoder further includes a downmixing stage configured to receive M signals from the receiving stage and downmix the M signals to N (1 <N <M) downmix signals.

The encoder is a high frequency restoration coding stage configured to receive N downmix signals from the downmixing stage and perform high frequency restoration coding on the N downmix signals, and is a second cross. It also includes a high frequency restoration coding stage configured to extract high frequency restoration parameters that allow high frequency restoration of N downmix signals above the overfrequency.

The encoder receives M signals from the receiving stage and N downmix signals from the downmixing stage, and M signals with respect to the frequency range corresponding to frequencies above the first crossover frequency. A parametric coding stage configured to perform parametric coding to N to M restored signals corresponding to M channels over a frequency range above the first crossover frequency. It further includes a parametric coding stage configured to extract upmix parameters that allow upmixing of the downmix signal.

The encoder receives N downmix signals from the downmixing stage and receives N downmix signals with respect to the frequency range corresponding to the frequency between the first crossover frequency and the second crossover frequency. A second waveform coding stage configured to generate N waveform-coded downmix signals by waveform coding, wherein the N waveform-coded downmix signals have a first crossover frequency. Further includes a second waveform coding stage, including a spectral coefficient corresponding to the frequency between and the second crossover frequency.

According to one embodiment, performing high frequency restoration coding on N downmix signals in the high frequency restoration coding stage is performed in the frequency domain, preferably in the quadrature mirror filter (QMF) domain.

According to a further embodiment, parametric coding of the M signals in the parametric coding stage is performed in the frequency domain, preferably in the quadrature mirror filter (QMF) domain.

According to yet another embodiment, generating M waveform coded signals by individually waveform coding M signals in the first waveform coding stage overlaps with M signals. A different overlapping window sequence is used for at least two of the M signals, including applying a windowing transformation.

According to an embodiment, the encoder waveform encodes one of the M signals with respect to a frequency range corresponding to a subset of the frequency range above the first crossover frequency, thereby further waveform coding. It further includes a third waveform coding stage configured to generate the conversion signal.

According to the embodiment, the encoder may include a control signal generation stage. The control signal generation stage is configured to generate a control signal indicating how the further waveform coded signal is interleaved with one parametric reconstruction of the M signals in the decoder. To. For example, the control signal may indicate a frequency range and a time range in which the additional waveform coded signal should be interleaved with one of the M upmix signals.

"Examples of examples"
FIG. 1 is a generalized configuration diagram of a decoder 100 in a multi-channel audio processing system for restoring M coded channels. The decoder 100 comprises three conceptual elements 200, 300, 400 that will be described in more detail in connection with FIGS. 2-4. In the first conceptual element 200, the decoder receives N waveform-coded downmix signals and M waveform-coded signals representing the multi-channel audio signal to be decoded, where 1 <N <M. In the illustrated example, N is set to 2. In the second conceptual element 300, the M waveform coded signals are downmixed and combined with the N waveform coded downmix signals. Radio frequency restoration (HFR) is then performed for the coupled downmix signal. In the third conceptual element 400, the high frequency restored signal is upmixed and the M waveform coded signals are combined with the upmix signal to restore the M coded channels. Will be done.

Representative embodiments described in connection with FIGS. 2 to 4 illustrate the restoration of encoded 5.1 surround sound. It may be noted that the low frequency effect signal is not mentioned in the embodiments or drawings described. This does not mean that any low frequency effects will be ignored. The low frequency effect (Lfe) is applied to the restored 5 channels by any suitable method well known to those of skill in the art. It may also be noted that the decoders described are equally well suited for other types of encoded surround sound, such as 7.1 or 9.1 surround sound.

FIG. 2 illustrates the first conceptual element 200 of the decoder 100 in FIG. The decoder includes two receive stages 212, 241. In the first reception stage 212, the bitstream 202 is decoded and dequantized into two waveform coded downmix signals 208a-b. Each of the two waveform-coded downmix signals 208a-b contains a spectral coefficient corresponding to a frequency between the first crossover frequency _{ky and the second crossover frequency k x .}

In the second reception stage 214, the bitstream 202 is decoded into five waveform coded signals 210a to e and dequantized. Each of the five waveform coded signals 210a-e includes spectral coefficients corresponding to frequencies up to the first crossover frequency _ky .

As an example, the signals 210a-e include two channel pair components and one single channel component for the center. The channel pair component can be, for example, a combination of a left front signal and a left surround signal, and a combination of a right front signal and a right surround signal. Further examples are a combination of a left front signal and a right front signal, and a combination of a left surround signal and a right surround signal. These channel pair components can be encoded, for example, in a sum-and-difference format. All of the five signals 210a-e can be encoded using the overlap windowing transformation by independent window processing and are still decodable by the decoder. This may allow for improved coding quality and, therefore, improved quality of the decoded signal.

As an example, the first crossover frequency _ky is 1.1 kHz. As an example, the second crossover frequency k _x is in the range of 5.6 to 8 kHz. The first crossover frequency _ky can vary, even if it is based on individual signals, i.e. the encoder is more faithful to the stereo downmix signals 208a-b in the signal component at a particular output signal. During that particular time instance, the bandwidth, ie, the associated waveform-encoded signal, can be detected to be unreproducible and to perform proper waveform coding of the signal component. That is, it should be noted that the first crossover frequency _ky of 210a to e can be increased.

As will be described later in this description, the remaining stages of the decoder 100 generally operate in the Quadrature Mirror Filter (QMF) region. For this reason, the signals 208a-b and 210a-e received in the modified discrete cosine transform (MDCT) format by the first and second receiving stages 212, 214 are reversed, respectively. It is converted into a time domain by applying the MDCT216. Each signal is then converted to the original frequency domain by applying the QMF conversion 218.

In FIG. 3, the five waveform coded signals 210 are downmixed in the downmix stage 308 to two downmix signals 310 and 312 containing spectral coefficients corresponding to frequencies up to the first crossover frequency _ky . .. These downmix signals 310 and 312 use the same downmixing scheme used in the encoder to create the two downmix signals 208a-b shown in FIG. 2, and the lowpass multichannel signal 210a. It can be formed by performing a downmix on ~ e.

The two new downmix signals 310 and 312 are then combined with the corresponding downmix signals 208a-b in the first coupling stages 320 and 322 so as to form the combined downmix signals 302a-b. To. Therefore, each of the combined downmix signals 302a-b has a spectral coefficient corresponding to a frequency up to the first crossover frequency _ky originating from the downmix signals 310 and 312, and a first reception stage 212 ( Corresponds to the frequency between the first crossover frequency _ky and the second crossover frequency _kx originating from the two waveform coded downmix signals 208a-b received in (shown in FIG. 2). Includes spectral coefficients.

The decoder further includes a radio frequency restoration (HFR) stage 314. The HFR stage is configured to extend each of the two coupled downmix signals 302a-b from the coupling stage to a frequency range above the second crossover frequency _kx by performing high frequency restoration. To. According to some embodiments, the radio frequency restoration performed involves performing spectral band replication (SBR). High frequency restoration can be performed by using high frequency restoration parameters that can be received by the HFR stage 314 in any suitable way.

The output from the high frequency restoration stage 314 is two signals 304a-b including the downmix signals 208a-b with the applied HFR extension portions 316 and 318. As described above, the HFR stage 314 is at frequencies present in the input signals 210a-e from the second receive stage 214 (shown in FIG. 2) coupled with the two downmix signals 208a-b. Based on that, high frequency restoration will be performed. Somewhat simplified, the HFR range 316, 318 includes a portion of the spectral coefficients from the downmix signals 310, 312 copied up to the HFR range 316, 318. Therefore, the portions of the five waveform coded signals 210a to e will appear in the HFR range 316, 318 of the output 304 from the HFR stage 314.

The downmixing in the downmixing stage 308 prior to the radio frequency restoration stage 314 and the coupling in the first coupling stage 320, 322 are in the time domain, i.e., the inverse modified discrete cosine transform (MDCT) 216 (shown in FIG. 2). It should be noted that each signal can be executed after it has been transformed into the time domain by applying. However, if the waveform coded signals 210a-e and the waveform coded downmix signals 208a-b are encoded by the waveform encoder using the overlap windowing transformation with independent window processing. If so, the signals 210a-e and 208a-b may not be seamlessly coupled in the time domain. Therefore, a better controlled scenario is realized if the binding at least in the first binding stage 320, 322 is performed in the QMF region.

FIG. 4 illustrates the third and final conceptual elements 400 of the decoder 100. The output 304 from the HFR stage 314 constitutes an input to the upmix stage 402. The upmix stage 402 creates five signal outputs 404a to e by performing a parametric upmix on the frequency-extended signals 304a to b. Each of the five upmix signals 404a-e corresponds to one of the five encoded channels in the encoded 5.1 surround sound for frequencies above the first crossover frequency _ky . do. According to a typical parametric upmix procedure, the upmix stage 402 first receives the parametric mixing parameters. The upmix stage 402 further produces an uncorrelated version of the two frequency-extended combined downmix signals 304a-b. The upmix stage 402 further performs matrix operations on two uncorrelated versions of the two frequency-extended combined downmix signals 304a-b and the two frequency-extended combined downmix signals 304a-b. And here the parameters of the matrix operation are given by the upmix parameters. Instead, any other parametric upmix procedure known in the art may be applied. Applicable parametric up-mixing procedures include, for example, “MPEG Surround-The ISO / MPEG Standard for Efficient and Compatible Multichannel Audio Coding” (“Herre”, etc., Journal of the Audio Engineering Society, Vol. 56, No. 11, 2008. (November, 2011).

Therefore, the outputs 404a to e from the upmix stage 402 do not include frequencies below the first crossover frequency _ky . _Spectral coefficients corresponding to the remaining frequencies up to the first crossover frequency ky are present in the five waveform coded signals 210a-e delayed by the delay stage 412 to match the timing of the upmix signal 404. ..

The decoder 100 further includes a second coupling stage 416, 418. The second coupling stages 416 and 418 combine the five upmix signals 404a-e with the five waveform coded signals 210a-e received by the second receiving stage 214 (shown in FIG. 2). It is composed of.

It may be noted that any current Lfe signal can be added as a separate signal to the resulting combined signal 422. Each of the signals 422 is then converted into the time domain by applying the inverse QMF conversion 414. Therefore, the output from the inverse QMF conversion 414 is a fully decoded 5.1 channel audio signal.

FIG. 6 illustrates a decoding system 100', which is an improved version of the decoding system 100. The decoding system 100'has conceptual elements 200', 300', and 400' corresponding to the conceptual elements 200, 300, and 400 of FIG. The difference between the decoding system 100'of FIG. 6 and the decoding system of FIG. 1 is that a third receive stage 616 is present at the conceptual element 200'and interleaving at the third conceptual element 400'. It means that stage 714 exists.

The third receive stage 616 is configured to receive additional waveform coded signals. Further waveform coded signals include spectral coefficients corresponding to a subset of frequencies above the first crossover frequency. Further waveform coded signals can be converted into the time domain by applying the inverse MDCT216. In that case, it can be converted to the original frequency domain by applying the QMF conversion 218.

It should be understood that additional waveform coded signals can be received as separate signals. However, additional waveform-encoded signals may likewise form one or more portions of the five waveform-encoded signals 210a-e. In other words, the additional waveform coded signal may be encoded together with one or more of the five waveform coded signals 210a-e, eg using the same MDCT transform. If so, the third receive stage 616 corresponds to the second receive stage, i.e., the additional waveform coded signals are the five waveform coded signals 210a-e by the second receive stage 214. Received together.

FIG. 7 illustrates in more detail the third conceptual element 300'of the decoder 100' in FIG. In addition to the high frequency extended downmix signals 304a-b and the five waveform coding signals 210a-e, a further waveform coding signal 710 is input to the third conceptual element 400'. In the illustrated embodiment, the additional waveform coded signal 710 corresponds to the third channel of the five channels. The additional waveform coded signal 710 further includes a spectral coefficient corresponding to a frequency interval starting from the first crossover frequency _ky . However, the format of the subset of frequency ranges above the first crossover frequency covered by the additional waveform coded signal 710 can, of course, vary in different embodiments. It should also be noted that multiple waveform coded signals 710a-e can be received and different waveform coded signals can correspond to different output channels. A subset of the frequency range covered by the plurality of further waveform coded signals 710a-e can vary between different signals of the plurality of further waveform coded signals 710a-e.

The additional waveform coded signal 710 can be delayed by the delay stage 712 to match the timing of the upmix signal 404 output from the upmix stage 402. The upmix signal 404 and the further waveform coded signal 710 are then input to the interleaved stage 714. The interleaved stage 714 interleaves or couples the upmix signal 404 with a further waveform coded signal 710 in order to generate the interleaved signal 704. In this embodiment, the interleaving stage 714 therefore interleaves the third upmix signal 404c with the further waveform coded signal 710. Interleaving can be performed by adding two signals together. However, in general, interleaving is performed by exchanging the upmix signal 404 with a further waveform coded signal 710 in the frequency and time ranges where the signals overlap.

The interleaved signal 704 was then input to a second coupling stage 416, 418, where the interleaved signal 704 was described with reference to FIG. 4 to generate an output signal 722. It is combined with the waveform coded signals 201a to e in the same manner as in. It should be noted that the order of the interleaving stages 714 and the second binding stages 416, 418 may be reversed so that the binding occurs before the interleaving.

Further, in a situation where the further waveform coded signal 710 forms one or more portions of the five waveform coded signals 210a-e, the second coupling stage 416, 418, and the interleaved stage 714 are simple. Can be combined into one stage. Specifically, such coupled stages will use the spectral components of the five waveform coded signals 210a-e for frequencies up to the first crossover frequency _ky . For frequencies above the first crossover frequency, the coupled stage will use an additional waveform coded signal 710 and an interleaved upmix signal 404.

The interleaved stage 714 may operate under the control of a control signal. For this purpose, the decoder 100'provides a control signal indicating how to interleave the additional waveform coded signal with one of the M upmix signals, eg, through a third receive stage 616. Can be received. For example, the control signal may indicate a frequency range and a time range in which the additional waveform coded signal 710 should be interleaved with one of the upmix signals 404. For example, a frequency range and a time range may be represented with respect to the time / frequency tile in which the interleaving should be performed. The time / frequency tile can be a time / frequency tile with respect to the time / frequency grid in the QMF region where interleaving is performed.

The control signal may use a vector, such as a binary vector, to indicate the time / frequency tile on which the interleaving should be performed. Specifically, there may be a first vector for frequency indication indicating the frequency at which interleaving should be performed. The instruction may be given, for example, by indicating logic 1 for the corresponding frequency interval in the first vector. There can be a second vector with respect to the time indication as well, indicating the time interval in which the interleaving should be performed. The instruction may be given, for example, by showing logic 1 for the corresponding time interval in the second vector. For this purpose, time frames are generally divided into multiple time slots so that time indications can be made on a subframe basis. By intersecting the first and second vectors, a time / frequency matrix can be constructed. For example, the time / frequency matrix can be a binary matrix containing logic 1 for each time / frequency tile in which the first and second vectors represent logic 1. The interleaved stage 714 is then such that one or more of the upmix signals 404 are further waveform coded signals 710 with respect to the time / frequency tiles indicated, for example, by logic 1 in the time / frequency matrix. A time / frequency matrix may be used with respect to performing interleaving so that it can be replaced.

It is noted that the vector may use other schemes rather than the binary scheme to indicate the time / frequency tile in which the interleaving should be performed. For example, a vector indicates that interleaving should not be performed using a first value, such as zero, and interleaving using a second value is by a second value. It will indicate that it should be performed for the particular channel identified.

FIG. 5 shows, as an example, a generalized configuration diagram of a coding system 500 suitable for a multi-channel audio processing system for coding M channels according to an embodiment.

In the representative embodiment described in FIG. 5, the coding of 5.1 surround sound is described. Therefore, in the illustrated example, M is set to 5. It may be noted that low frequency effect signals are not mentioned in the embodiments described or in the drawings. This does not mean that any low frequency effects will be ignored. The low frequency effect (Lfe) is added to the bitstream 552 by any suitable method well known to those of skill in the art. It may also be noted that the encoder described is equally well suited to encode other types of surround sound, such as 7.1 or 9.1 surround sound. In the encoder 500, the five signals 502, 504 are received in a receiving stage (not shown). The encoder 500 is configured to receive five signals 502, 504 from the receiving stage and generate five waveform-encoded signals 518 by individually waveform-coding the five signals 502, 504. Includes 1 waveform coding stage 506. The waveform coding stage 506 may perform MDCT conversion on each of the five received signals 502, 504, for example. As discussed with respect to the decoder, the encoder may choose to encode each of the five received signals 502, 504 using an M DCT transform with independent window processing. This may allow for improved coding quality and, therefore, improved quality of the decoded signal.

The five waveform-encoded signals 518 are waveform-encoded with respect to the frequency range corresponding to the frequency up to the first crossover frequency. Therefore, the five waveform coded signals 518 include spectral coefficients corresponding to frequencies up to the first crossover frequency. This can be obtained by low-pass filtering each of the five waveform coded signals 518. The five waveform coded signals 518 are in that case quantized 520 according to a psychoacoustic model. The psychoacoustic model considers the bit rates available in a multi-channel audio processing system as accurately as possible and reproduces the encoded signal perceived by the listener when decoded on the decoder side of the system. It is composed of.

As discussed above, the encoder 500 performs hybrid coding, including discrete multi-channel coding and parametric coding. Discrete multi-channel coding is performed in the waveform coding stage 506 for each of the input signals 502, 504 with respect to the frequency up to the first crossover frequency, as described above. Parametric coding is performed so that the decoder can recover the five input signals 502, 504 from the N downmix signals for frequencies above the first crossover frequency. In the illustrated example in FIG. 5, N is set to 2. The downmixing of the five input signals 502 and 504 is performed at the downmixing stage 534. The down mixing stage 534 operates advantageously in the QMF region. Therefore, the five signals 502, 504 are converted into a QMF region by the QMF analysis stage 526 before being input to the downmixing stage 534. The downmixing stage performs a linear downmixing operation on the five signals 502 and 504 and outputs two downmix signals 544 and 546.

These two downmix signals 544, 546 are received by the second waveform coding stage 508 after they have been converted to the original time domain by performing the inverse QMF conversion 554. The second waveform coding stage 508 waveform encodes the two downmix signals 544 and 546 with respect to the frequency range corresponding to the frequency between the first crossover frequency and the second crossover frequency. Two waveform coded downmix signals will be generated. The waveform coding stage 508 may perform MDCT conversion on each of the two downmix signals, for example. Therefore, the two waveform-coded downmix signals include spectral coefficients corresponding to frequencies between the first crossover frequency and the second crossover frequency. The two waveform-coded downmix signals are then quantized 522 according to the psychoacoustic model.

A radio frequency restore (HFR) parameter 538 is extracted from the two downmix signals 544 and 546 to allow the decoder to restore frequencies above the second crossover frequency. These parameters are extracted at HFR coding stage 532.

Five input signals 502, 504 are received by the parametric coding stage 530 to allow the decoder side to recover five signals from the two downmix signals 544, 546. The five signals 502, 504 are parametrically coded for a frequency range corresponding to frequencies above the first crossover frequency. The parametric coding stage 530, in that case, corresponds to five input signals 502, 504 (ie, five channels in the encoded 5.1 surround sound) for a frequency range above the first crossover frequency. It is configured to extract an upmix parameter 536 that allows upmixing of two downmix signals 544, 546 to the five restored signals. It may be noted that the upmix parameter 536 is extracted only for frequencies above the first crossover frequency. This can reduce the complexity of the parametric coding stage 530 and the bit rate of the corresponding parametric data.

It may be noted that down mixing 534 can be achieved in the time domain. In such cases, the HFR coding stage 532 generally operates in the QMF region, so the QMF analysis stage 526 should be located downstream of the downmixing stage 534 and before the HFR coding stage 532. .. In this case, the reverse QMF stage 554 can be omitted.

The encoder 500 further includes a bitstream generation stage, i.e., a bitstream multiplexer 524. According to a typical embodiment of the encoder 500, the bitstream generation stage has five coded and quantized signals 548, two parameter signals 536, 538, and two coded and quantized downs. It is configured to receive the mix signal 550. These are converted to bitstream 552 by the bitstream generation stage 524 for further distribution in a multi-channel audio system.

In the multi-channel audio systems described, there is often the maximum available bit rate, for example when streaming audio over the Internet. Due to the different characteristics of each time frame of the input signals 502, 504, the exact same bit allocation between the five waveform coded signals 548 and the two downmix waveform coded signals 550 may not be used. In addition, each individual signal 548 and 550 may require more or less allocated bits so that the signal can be restored according to the psychoacoustic model. According to a representative embodiment, the first and second waveform coding stages 506, 508 share a common bit reservoir. The available bits per encoded frame are first between the first and second waveform coding stages 506, 508, depending on the characteristics of the signal to be encoded and the current psychoacoustic model. Will be distributed. As described above, the bits are then distributed among the individual signals 548,550. The number of bits used for the high frequency restore parameter 538 and the upmix parameter 536 is, of course, taken into account when distributing the available bits. Psychoacoustics for the first and second waveform coding stages 506, 508 with respect to the number of bits allocated in a particular time frame for a perceptually smooth transition around the first crossover frequency. Attention is paid to adjusting the model.

FIG. 8 illustrates an alternative embodiment of the coding system 800. The difference between the coding system 800 of FIG. 8 and the coding system 500 of FIG. 5 is that the encoder 800 has an input signal 502 with respect to a frequency range corresponding to a subset of the frequency range above the first crossover frequency. , 504, by waveform-encoding one or more, is prepared to generate further waveform-encoded signals.

For this purpose, the encoder 800 includes an interleaved detection stage 802. The interleaved detection stage 802 is configured to identify parts of the input signals 502, 504 that are not successfully restored by the parametric reconstruction encoded by the parametric coding stage 530 and the high frequency restoration coding stage 532. For example, the interleave detection stage 802 may compare the input signals 502, 504 to the parametric restores of the input signals 502, 504 defined by the parametric coding stage 530 and the high frequency restoration coding stage 532. Based on the comparison, the interleaved detection stage 802 may identify a subset 804 of the frequency range above the first crossover frequency to be waveform encoded. The interleaved detection stage 802 can also identify the time range in which the identified subset 804 of the frequency range above the first crossover frequency should be waveform-encoded. The identified frequency and time subsets 804, 806 can be input to the first waveform coding stage 506. Based on the received frequency and time subsets 804 and 806, the first waveform coding stage 506 may be one of the input signals 502, 504 with respect to the time range and frequency range identified by the subsets 804, 806. Further waveform-encoded signals 808 are generated by waveform-encoding the plurality. The additional waveform coded signal 808 can then be encoded and quantized by stage 520 and added to the bitstream 846.

The interleaved detection stage 802 may further include a control signal generation stage. The control signal generation stage is configured to generate a control signal 810 indicating how the further waveform coded signal is interleaved with the parametric restore of one of the input signals 502, 504 in the decoder. As described with reference to FIG. 7, for example, the control signal may indicate a frequency range and a time range in which the further waveform coded signal should be interleaved with the parametric restore. The control signal can be applied to the bitstream 846.

"Equivalents, extensions, alternatives, and other things"
Further embodiments of the present disclosure will be apparent to those of skill in the art after reviewing the above description. While the description and drawings disclose examples and examples, the present disclosure is not limited to these particular examples. Many amendments and changes may be made without departing from the scope of the present disclosure as defined in the appended claims. The quotation marks appearing in the claims should not be understood as limiting their scope.

Further, variants to the disclosed examples can be understood and achieved by one of ordinary skill in the art in practicing the present disclosure from the drawings, the present disclosure, and the examination of the accompanying claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude more than one. The mere fact that certain means are recited in different dependent claims does not indicate that a combination of these means cannot be used effectively.

The systems and methods disclosed above may be implemented as software, firmware, hardware, or a combination thereof. In a hardware implementation, the division of tasks between functional units mentioned in the above description does not necessarily correspond to the division into physical units, and conversely, one physical component has multiple functions. And may be performed by several physical components in which one task collaborates. The particular component or all components may be implemented as software executed by a digital signal processor or microprocessor, or as hardware or as an integrated circuit for a particular application. Such software may be distributed on computer-readable media that may include computer storage media (or non-temporary media) and communication media (or temporary media). As is well known to those of skill in the art, the term computer storage medium is practiced by any method or technique for storing information such as computer readable instructions, data structures, program modules, or other data. It also includes both volatile and non-volatile media, removable and non-removable media. Computer storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage device, magnetic cassette, magnetic tape, magnetic disk storage device or other. It includes, but is not limited to, a magnetic storage device, or any other medium that can be used to store desired information and is accessible by a computer. Further, to those skilled in the art, the communication medium generally embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transfer means, and is optional. It is well known to include the information distribution medium of.

Claims (2)

A decoding method in a multi-channel audio processing system for restoring M (M> 2) encoded channels.
A step of receiving N (N <M) waveform-coded downmix signals containing spectral coefficients corresponding to frequencies between the first crossover frequency and the second crossover frequency.
A step of receiving M waveform-coded signals including spectral coefficients corresponding to each channel of the M coded channels.
A step of determining N restored downmix signals by performing frequency restoration, wherein the frequency restoration is based on the N waveform-coded downmix signals and the second crossover frequency. A step and a step comprising determining the N restored downmix signals corresponding to the higher frequency range.
A step of performing a parametric upmix of the N frequency-restored downmix signals to M upmix signals,
A decoding method comprising interleaving the M upmix signals with the M waveform coded signals. A decoder in a multi-channel audio processing system for restoring M (M> 2) encoded channels.
First reception configured to receive N (N <M) waveform-coded downmix signals containing spectral coefficients corresponding to frequencies between the first crossover frequency and the second crossover frequency. Step and
A second receiving stage configured to receive M waveform coded signals including spectral coefficients corresponding to each of the M coded channels.
A frequency restoration stage configured to determine N restored downmix signals by performing frequency restoration, wherein the frequency restoration is based on the N waveform-coded downmix signals. A frequency restoration stage comprising determining the N restored downmix signals corresponding to a frequency range above the second crossover frequency.
An upmix stage downstream of the frequency restore stage configured to perform a parametric upmix of the N frequency restored downmix signals to M upmix signals.
The upmix stage and the second receiving stage configured to interleave the M upmix signals from the upmix stage with the M waveform coded signals from the second receiving stage. Decoder with an interleaver stage downstream of.

Images