RU2305870C2 - Alternating frame length encoding optimized for precision - Google Patents

Abstract

FIELD: encoding of audio-signals, in particular, encoding of multi-channel audio signals.

SUBSTANCE: in accordance to the invention, polyphonic signals are used for creation of main signal, typically, a signal and a collateral signal. A row of encoding schemes of collateral signal (x_side) is implemented, each encoding scheme is characterized by a set of sub-frames of varying length, while total length of sub-frames corresponds to encoding frame length of encoding scheme. Encoding scheme for collateral signal (x_side) is selected on basis of current content of polyphonic signals, and collateral remainder signal is created as a difference between collateral signal and main signal, scaled with usage of balancing coefficient, which is selected for minimization of collateral remainder signal. Optimized collateral remainder signal and balancing coefficient are encoded and implemented as encoding parameters, representing the collateral signal.

EFFECT: increased quality of perception of multi-channel sound signals.

5 cl, 15 dwg

Description

FIELD OF THE INVENTION

The present invention relates to the encoding of audio signals, in particular to the encoding of multi-channel audio signals.

State of the art

There is a great need on the market to transmit and store audio signals at a low bit rate, while maintaining high sound quality. In particular, in cases where transmission resources or memory is limited, operation with a low bit rate is an important cost factor. Typically, this is the case, for example, in streaming and messaging applications in mobile communication systems, for example, GSM, UMTS or CDMA.

Currently, there are no standardized codecs that provide high stereo sound quality at bit rates that are of economic interest for use in mobile communication systems. Using the available codecs, monophonic audio transmission is possible. Stereophonic transmission is also available to some extent. However, bit rate limits usually require a fairly radical restriction on stereo performance.

The easiest way to stereo or multichannel encoding audio signals is to encode the signals of the various channels individually as separate and independent signals. Another basic method used for FM stereo radio transmission, which ensures compatibility with traditional monophonic radios, is to transmit the sum and difference signal of the two channels used.

Modern audio codecs, for example, MPEG-1/2 Layer III and MPEG-2/4 AAC use the so-called combined stereo coding. According to this method, the signals of various channels are processed together, and not separately and one after another. The two most commonly used combined stereo coding techniques are known as Mid / Side (M / S) stereo coding and intensity stereo coding, which are typically used on the subbands of stereo or multi-channel signals to be encoded.

M / S stereo coding is similar to the described procedure in FM stereo radio transmission in the sense that it encodes and transmits the sum and difference signals of the channel subbands and thereby uses the redundancy between the channel subbands. The structure and operation of an encoder based on M / S stereo coding is described, for example, in US Pat. No. 5,285,498 to J.D. Johnston.

Intensity stereo coding, on the other hand, can use stereo irrelevance. It transmits the combined intensity of the channels (various subbands) along with certain location information showing how the intensity is distributed between the channels. Intensity stereo coding not only provides information on the spectral amplitude of the channels. Phase information is not transmitted. For this reason, and since temporal inter-channel information (more specifically, inter-channel time difference) is of paramount psychoacoustic significance, especially at lower frequencies, intensity stereo coding can only be used at high frequencies around, for example, 2 KHz. A stereo intensity coding method is described, for example, in European Patent No. 0497413 to R, Veldhuis et al.

A recently developed stereo coding method is described, for example, in a conference call C. C. Faller et al. "Binaural cue coding applied to stereo and multi-channel audio compression", 112th AES convention, May 2002, Munich, Germany, S. Faller et al. This method is a method for parametric coding of a multi-channel audio signal. The basic principle is that on the encoding side, the input signals from N channels C1, C2, ..., CN are combined into a single mono signal m. A mono signal is an audio signal encoded using any traditional mono audio audio codec. In parallel, parameters are extracted from channel signals that describe a multi-channel image. The parameters are encoded and transmitted to the decoder together with the bitstream of the audio signal. The decoder first decodes the mono signal m 'and then regenerates the channel signals C ₁ ', C ₂ ', ..., C _N ' based on the parametric description of the multi-channel image.

The principle of the stereo label coding (BCC) method is that it transmits a coded mono signal and so-called BCC parameters. The BCC parameters contain the encoded differences of the inter-channel levels and the difference of the inter-channel times for the subbands of the original multi-channel input signal. The decoder reconstructs the various channel signals by applying control over the sub-bands of the mono signal phase level based on the BCC parameters. An advantage over M / S or stereo intensity coding is that stereo information containing temporal inter-channel information is transmitted at much lower bit rates. However, this method requires time-frequency transformations, requiring large amounts of computation, for each channel, both in the encoder and in the decoder.

Moreover, BCC does not use the fact that a significant part of the stereo information, especially at low frequencies, is scattered, i.e. she does not come from any particular direction. Diffuse acoustic fields exist in both stereo channels, but they do not substantially coincide in phase with respect to each other. If an algorithm such as BCC is used for recordings with a large number of diffuse acoustic fields, the reproduced stereo image will be distorted due to jumps from left to right, since the BCC algorithm can select a signal to be displayed only in specific frequency bands from left or right.

A possible means of encoding a stereo signal and ensuring good reproduction of diffuse acoustic fields is to use a coding scheme similar to the method used in FM stereo broadcasting, namely, encoding mono (left + right) and difference (left-right) signals separately.

The method described in US patent 5434948, in the name of C. E. Holt et al., Uses a similar method, as in BCC, to encode a mono signal and additional information. In this case, the additional information consists of predictor filters and optionally a residual signal. Predictor filters evaluated using the least squares algorithm, when applied to a mono signal, make it possible to predict multi-channel audio signals. Using this method, it is possible to implement coding at very low bit rates of multichannel sound sources, however, at the cost of lower quality, as further described below.

Finally, for completeness, mention should be made of the method used in the three-dimensional audio signal. This method synthesizes the right and left channel signals by filtering the sound source signals using so-called head-based filters. However, this method requires that the signals of different sound sources be separated and, therefore, cannot be generally used for stereo or multichannel coding.

SUMMARY OF THE INVENTION

A problem in existing coding schemes based on encoding frames of signals, in particular the main signal and one or more side signals, is that dividing audio information into frames can introduce unattractive perceptual defects. Dividing information into frames of relatively long duration generally reduces the average required bit rate. This may be useful, for example, for music containing a large proportion of diffuse sound. However, for rich music with many transitions or speech, fast temporal variations are smeared over the duration of the frame, resulting in spurious sounds or even problems with pre-emptive echoes. Encoding short frames, on the other hand, provides a more accurate representation of sound, minimizing energy, but requires higher bit rates and greater computing resources. Essentially, coding efficiency may also drop with frames of very short lengths. The introduction of a larger number of frame boundaries can also lead to heterogeneous coding parameters, which can manifest as perceptual defects.

An additional problem in schemes based on the coding of the main and one or more side signals is that they require relatively significant computational resources. In particular, when short frames are used, processing parameter inhomogeneities from one frame to another is a complex task.

When long frames are used, audio estimation errors with transitions can cause very significant side signals, which in turn increase the speed requirement.

The purpose of the present invention, therefore, is to provide a coding method and apparatus that improves the perception quality of multi-channel audio signals, in particular, to avoid defects such as pre-emptive echoes, spurious sounds or frame heterogeneity defects. An additional objective of the present invention is to provide a coding method and apparatus requiring less processing power and having more constant bit rate requirements.

The above objectives are achieved by methods and devices according to the claims. In general, polyphonic signals are used to create the main signal, typically a mono signal, and a side signal. The main signal is encoded according to known coding principles. A number of side signal coding schemes have been laid. Each coding scheme is characterized by a set of subframes of different lengths. The total length of the subframes corresponds to the coding frame length of the coding scheme. Sets of subframes comprise at least one subframe. The coding scheme to be used for the side signal is selected, at least in part, based on the current contents of the polyphonic signals.

In one embodiment, the choice is made (or before coding) based on an analysis of the characteristics of the signal. In another embodiment, the side signal is encoded by each of the coding schemes, and based on the coding quality measurements, the best coding scheme is selected.

In a preferred embodiment, a side residual signal is created as the difference between the side signal and the main signal, scaled with a symmetry coefficient. A balancing factor is selected to minimize the incidental residual signal. The optimized side residual signal and the balance factor are encoded and provided as parameters representing the side signal. On the decoder side, a balance factor, a side residual signal, and a main signal are used to reconstruct the side signal.

In a further preferred embodiment, the encoding of the side signal comprises scaling the energy loop to avoid pre-emptive echo effects. Moreover, different coding schemes comprise different coding procedures in separate subframes.

The main advantage of the present invention is that the reliability of perception of audio signals is increased. Moreover, the present invention still enables the transmission of multi-channel signals at very low bit rates.

Brief Description of the Drawings

The invention together with its additional objectives and advantages are explained in the following description, with reference to the drawings, of which:

Figure 1 - block diagram of a system for transmitting polyphonic signals;

Fig. 2a is a block diagram of an encoder in a transmitter;

Fig.2b is a block diagram of a decoder in a transmitting device;

Figa is a diagram illustrating the encoding of frames of various lengths;

3b and 3c are block diagrams of embodiments of side signal encoding devices according to the present invention;

FIG. 4 is a block diagram of an embodiment of an encoder using side-signal coding, taking into account a balance factor; FIG.

5 is a block diagram of an embodiment of an encoder in multi-signal systems;

6 is a block diagram of an embodiment of a decoder suitable for decoding signals from the device shown in FIG. 5;

7a and 7b are diagrams illustrating a preemptive echo defect;

Fig. 8 is a block diagram of an embodiment of a side signal encoding apparatus according to the present invention using different encoding principles in different subframes;

9 illustrates the application of various coding principles in different frequency subbands;

10 is a flowchart of the main steps of an embodiment of an encoding method according to the present invention; and

11 is a flowchart of the main steps of an embodiment of a decoding method according to the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Figure 1 illustrates a typical system 1 in which the present invention can be advantageously used. The transmitting device 10 comprises an antenna 12 including hardware and software enabling radio signals 5 to be transmitted to the receiving device 20. The transmitting device 10 includes, in addition to other components, a multi-channel encoder 14 that converts the signals of a number of input channels 16 into output signals suitable for radio transmission . Examples of suitable multi-channel encoders 14 are described in more detail below. The signals of the input channels 16 can be provided, for example, from the memory 18 of the audio signals, for example, from a data file of a digital representation of audio recordings, audio recording of sound on a magnetic tape or vinyl disc, etc. The signals of the input channels 16 can also be provided live, for example, from a set of microphones 19. The audio signals are digitized, if not already in digital form, before being input to the multi-channel encoder 14.

On the receiving device 20 side, an antenna 22 with associated hardware and software provides processing for the actual reception of radio signals 5 representing polyphonic audio signals. Typical functionalities, such as error correction, are implemented. The decoder 24 decodes the received radio signals 5 and converts the audio data thus transmitted to the signals of a number of output channels 26. The output signals can be output, for example, to the speakers 29 for immediate presentation or can be stored in the memory 28 of any type of audio signal.

System 1 may be, for example, a telephone conferencing system, a system for providing audio services or other audio applications. In some systems, such as, for example, a telephone conference system, the data exchange must be of a duplex type, whereas, for example, the distribution of music from a service provider to a subscriber must be essentially one-way type. The transmission of signals from the transmitting device 10 to the receiving device 20 can also be performed by any other means, for example, by means of other types of electromagnetic waves, cables or optical fiber, as well as combinations thereof.

Fig. 2a illustrates an embodiment of an encoder according to the present invention. In this embodiment, the polyphonic signal is a stereo signal comprising two channels a and b received at input 16A and 16B, respectively. The signals of channel a and b are supplied to pre-processing device 32, where various procedures for preliminary signal conditioning can be performed. The signals (possibly modified) from the output of the pre-processing device 32 are summed in the summing device 34. The summing device 34 also divides the sum into two. The x _mono signal generated in this way is the main signal from stereo signals, since it essentially contains all the data from both channels. In this embodiment, the main signal thus represents a pure "mono" signal. The main signal x _mono is supplied to the main signal encoding device 38, which encodes the main signal according to any appropriate encoding principles. These principles are known from the prior art and therefore are not further described. The main signal encoding device 38 provides an output signal p _mono representing encoding parameters characterizing the main signal.

In the subtractor 36, the difference (divided by two) of the channel signals is provided as a side signal x _side . In this embodiment, the side signal represents the difference between the two channels in the stereo signal. The side signal x _side is supplied to the side signal encoder 30. Preferred embodiments of the side signal encoding apparatus 30 are further described below. According to the side signal coding procedure, which is described in more detail below, the side signal x _{side is} converted to the c _side encoding parameters p _side representing the side signal x _side . In some embodiments, this encoding is also carried out using the main signal information x _mono . Arrow 42 shows this possibility when the original uncoded main signal x _{mono is used} . In other embodiments, the primary signal information that is used in the side signal encoding device 30 may be derived from the encoding parameters p _mono representing the main signal, as shown by dashed line 44.

The encoding parameters p _mono representing the main signal x _mono are the first output signal, and the encoding parameters p _side representing the side signal x _side are the second output signal. Typically, these two output signals, p _mono and p _side , together representing fully stereo sound, are multiplexed into a single transmission signal 52 in the multiplexing device 40. In other embodiments, the transmission of the first and second output signals p _mono , p _side may be performed separately.

2b, an embodiment of a decoder 24 according to the present invention is illustrated as a block diagram. A received signal 54 containing encoding parameters representing the main and side signal information is supplied to the demultiplexing device 56, which separates the first and second input signal, respectively. The first input signal corresponding to the encoding parameters p _{mono of the} main signal is supplied to the main signal decoding apparatus 64. Traditionally, the p _mono encoding parameters representing the main signal are used to generate a decoded main signal x ″ _mono , which is as similar as possible to the main signal x _mono (FIG. 2 a) of encoder 14 (FIG. 2 a).

Similarly, the second input signal corresponding to the side signal is supplied to the side signal decoding apparatus 60. Here, the p _side coding parameters representing the side signal are used to reconstruct the decoded side signal x ″ _side . In some embodiments, the decoding procedure uses information about the main signal x ″ _mono , as shown by arrow 65.

The decoded main and side signals x ″ _mono , x ″ _side are supplied to an adder 70 that provides an output signal that is a representation of the original channel signal a. Similarly, the difference provided by the subtractor 68 is an output signal that is a representation of the original channel signal b. These channel signals may be processed in the post processor 74 according to prior art signal processing procedures. Finally, channel signals a and b are provided to outputs 26A and 26B of the decoder.

As mentioned in the description of the invention, coding is typically performed on a frame-by-frame basis. A frame contains audio samples for a predetermined period of time. At the bottom of FIG. 3a, an SF2 frame of length L is illustrated. Audio samples in the unshaded area should be encoded together. Previous samples and subsequent samples are encoded in other frames. Dividing the samples into frames will in any case lead to some heterogeneity within the boundaries of the frames. The shift of sounds will lead to a shift in the encoding parameters, significantly changing at the border of each frame. This will lead to perceived errors. A possible way to compensate for this to some extent is to encode not only the samples to be encoded, but also the samples in the absolute vicinity of the frame, as indicated by the shaded parts. Thus, a softer transition between different frames is achieved. As an alternative or addition, interpolation techniques are sometimes also used to reduce perceptual defects caused by frame boundaries. However, all of these procedures require significant additional computing resources, and for some special encoding methods it can also be difficult to provide them with specific resources.

From this point of view, it is advantageous to use frames as long as possible, since the number of frame boundaries will be small. Also, the coding efficiency in a typical case becomes high, and the required bit rate is minimized. However, long shots lead to problems with preemptive echo defects and spurious sounds.

By using shorter frames, such as SF1 or even SF0, having a duration of L / 2 and L / 4, respectively, as those skilled in the art understand, coding efficiency can be reduced, bit rate may be higher and problems with defects in frame boundaries will increase. However, shorter frames are less susceptible, for example, to other perceptual defects, such as spurious sounds and pre-emptive echoes. To be able to minimize coding errors to the maximum extent, you should use a frame of the shortest possible length.

According to the present invention, the perception of audio signals is improved by using a frame length to encode a similar signal, which depends on the current contents of the signal. Since the effect of different frame lengths on the perception of the audio signal varies depending on the nature of the sound to be encoded, improvement can be achieved by allowing the very nature of the signal to affect the used frame length. The encoding of the main signal is not the purpose of the present invention and therefore is not described in detail. However, the frame lengths used for the main signal may or may not equal the lengths of the frames used for the side signal.

Due to slight temporal variations, it may be advantageous, for example, in some cases to encode a side signal using relatively long frames. This may be the case with recordings with a large amount of diffuse acoustic field, for example, concert recordings. In other cases, such as conversations in stereo, short frames are probably preferable. The decision about which frame length to prefer can be made in two main ways.

One embodiment of a side signal encoding apparatus 30 according to the present invention is illustrated in FIG. 3b, which utilizes a feedback solution. Here, a base coding frame of length L is used. A series of coding schemes 81 are created, characterized by a separate set of 80 subframes 90. Each set of 80 subframes 90 contains one or more subframes 90 of equal or different lengths. The total length of the set of 80 subframes 90, however, is always equal to the length of the encoding base frame L. 3b, the upper coding scheme is characterized by a set of subframes, contains only one subframe of length L. The next set of subframes contains two frames of length L / 2. The third set contains two L / 4 frames followed by an L / 2 frame.

The x _side signal supplied to the side signal encoding device 30 is encoded by all encoding schemes 81. In the upper coding scheme, the entire basic coding frame is encoded in a full frame. However, in other coding schemes, the x _side signal is encoded independently in each subframe. The result from each coding scheme is supplied to a selector 85. The accuracy measuring means 83 determines a measured accuracy value for each of the encoded signals. The measured accuracy value is an objective quality value, preferably a measure of the signal-to-noise ratio or a weighted signal-to-noise ratio. The measured accuracy values associated with each coding scheme are compared, and the result controls the switching means 87 for selecting coding parameters representing a side signal from the coding scheme providing the best measured accuracy value as an output signal p _side from the side signal encoding device 30.

Preferably, all possible combinations of frame lengths are tested and a set of subframes is selected that provides the best objective quality, for example, a signal-to-noise ratio.

In the present embodiment, the length of the used subframes is selected according to:

where l _sf is the length of the subframes, l _f is the length of the encoding frame, and n is an integer. In the present embodiment, n is selected from 0 to 3. However, it is possible to use any frame length if the total set length is kept constant.

FIG. 3c illustrates another embodiment of a side signal encoding apparatus 30. Here, the frame length decision is a non-feedback decision based on signal statistics. In other words, the spectral characteristics of the side signal are used as the basis for deciding which coding scheme will be used. As before, there are various coding schemes characterized by different sets of subframes. However, in this embodiment, the selector 85 precedes the actual encoding. The input side signal x _{side is} supplied to the selector 85 and the signal analysis unit 84. The result of the analysis is the input of the switch 86, in which only one of the coding schemes 81 is used. The output of this coding scheme is also the p _side output of the side signal coding apparatus 30.

The advantage of a non-feedback solution is that only one actual coding should be performed. The disadvantage is that the analysis of signal characteristics can be very complicated, and it can be difficult to predict possible behaviors in advance in order to be able to provide an appropriate choice in switch 86. A large amount of statistical sound analysis must be performed and included in the signal analysis device 84. Any minor change in coding schemes can lead to a fundamental change in the statistical line of behavior.

By selecting with feedback (FIG. 3b), coding schemes can be replaced without making any changes to the rest of the device. On the other hand, if multiple coding schemes are to be analyzed, the computational requirements will be high.

The advantage of such variable frame length coding for the side signal is that it is possible to choose between precise time resolution and coarse frequency resolution, on the one hand, and coarse time resolution and exact frequency resolution, on the other hand. The above embodiments preserve the stereo image in the best possible way.

There are also some requirements for the actual coding used in various coding schemes. In particular, when a closed-loop selection is used, the computing resources in order to perform a series of more or less simultaneous coding operations must be significant. The more complex the coding process, the more processing power is required. Moreover, a low bit rate is also preferred.

The method presented in US patent No. 5434948, uses a filtered version of the monosignal (main), to recreate a secondary or differential signal. Filtering options are optimized and may change over time. Then, filtering parameters representing coding of the side signal are transmitted. In one embodiment, a residual side signal is also transmitted. In many cases, this approach may be possible to use as a method of encoding a side signal within the scope of the present invention. However, this approach has some disadvantages. The quantization of the filter coefficients and any residual side signal often require relatively high bit rates, since the filter order must be high in order to provide an accurate estimate of the side signal. Evaluating the filtering itself can be problematic, especially in cases of rich music with many transitions. Evaluation errors provide a modified side signal, which is sometimes larger in amplitude than an unmodified signal. This leads to higher bit rate requirements. Moreover, if a new set of filter coefficients is calculated every N samples, then the filter coefficients should be interpolated to achieve a smooth transition from one set of filter coefficients to another, as described above. Interpolating filter coefficients is a complex task, and interpolation errors will manifest themselves in large side-signal errors, resulting in higher bit rates necessary for the differential error encoder.

To avoid the need for interpolation, it is necessary to update the filter coefficients for each sample and use backward compatibility analysis. To obtain a good result, it is necessary that the bit rate of the residual signal encoder be high enough. Therefore, this is not an optimal alternative to stereo coding with a low bit rate.

There are cases, for example, quite often in music, when monosignals and difference signals are practically uncorrelated. The estimation of filtering in this case becomes very complicated with the additional risk of worsening the situation in the encoder of differential error signals.

The solution according to US patent 5434948 can work quite well in cases where the filter coefficients change very slowly over time, for example, in telephone conference systems. In the case of music signals, this approach does not work optimally, since the filters must change very quickly in order to track the stereo image. This means that the length of the subframes with a significantly different value should be used, i.e. the number of tested combinations is increasing rapidly. This, in turn, means that the computational requirements for all possible coding schemes become impossibly high.

Therefore, in a preferred embodiment, the coding of the side signal is based on the idea of reducing redundancy between mono signals and side signals by using a simple symmetry coefficient instead of a complex predictor filter with high bit rate requirements. Then, the residual signal resulting from this operation is encoded. The magnitude of this residual signal is relatively small and does not require a very high bit rate. This idea is very suitable for combining with the approach described above, based on a set of variable frames, since the computational complexity is low.

Using a symmetry factor in combination with a variable-length frame approach eliminates the need for complex interpolation and the associated problems that interpolation can cause. Moreover, using a simple symmetric coefficient instead of a complex filter causes fewer estimation problems, since possible estimation errors for the symmetric coefficient have less impact. The preferred solution allows you to reproduce both panned signals and diffuse acoustic fields with good quality and with limited requirements for bit rate and computational resources.

4 illustrates a preferred embodiment of a stereo encoder according to the present invention. This option is in many ways similar to that shown in FIG. 2a, but with a more detailed representation of the side signal encoding device 30. The encoder 14 of this embodiment does not have a preprocessing device, and input signals are supplied directly to the summing and subtracting devices 34, 36. The _mono signal x _mono is multiplied by a specific balancing factor g _sm in the multiplier 33. In the subtractor 35, the multiplied mono signal is subtracted from the side signal x _side , i.e. in fact, the difference between the two channels to form a residual side signal. The balancing factor g _sm is determined based on the content of mono and side signals by the optimizer 37 in order to minimize the side residual signal according to the quality criterion. The quality criterion is preferably the least squares criterion. The residual side signal is encoded in the encoder 39 residual side signals according to any encoding procedures. Preferably, the residual side signal encoder 39 is a low bit rate transform encoder or a linear code prediction encoder (CELP). The p _side encoding parameters representing the side signal, then comprise the p _{side residual} encoding parameters representing the residual side signal and the optimized balance factor 49.

In the embodiment of FIG. 4, the mono signal 42 used to synthesize the side signals is the target signal x _mono for the _mono encoder 38. As mentioned above (in connection with FIG. 2a), a local synthesized signal of the mono signal encoder 38 can also be used. In the latter case, the overall delay in the encoder may be increased and the computational complexity of the side signal may increase. On the other hand, the quality may be better, since in this case it is possible to eliminate coding errors due to the encoder monosignals.

From a mathematical point of view, the basic coding scheme can be described as follows. We denote the two channel signals as a and b, which can be the left and right channels of the stereo pair. Channel signals are combined into a mono signal by summing and into a side signal by subtraction. In the form of equations, operations are described as follows:

It is useful to halve the x _mono and x _side signals. This implies that there are other ways to create x _mono and x _side . You can, for example, use:

In blocks of input signals, a modified, or residual, side signal is calculated according to:

where f (x _mono , x _side ) is a function of the symmetry coefficient, which, based on a block of N samples, i.e. subframe, side and mono signals seeks to increase the proportion of the signal removed from the side signal. In other words, a balancing factor is used to minimize the residual side signal. In the special case, when it is minimized in the sense of least squares, this is equivalent to minimizing the energy of the residual side signal x _{side residual} .

In the above special case, f (x _mono , x _side ) is described as:

where x _side is a side signal, x _mono is a _mono signal. Note that the function is based on a block starting at the “beginning of the frame” and ending at the “end of the frame”.

You can add weighting in the frequency domain to calculate the coefficient of symmetry. This is done by convolution of the x _side and x _mono signals using the impulse response of the weighing filter. Then you can transfer the estimation error to the frequency range, where they are easier to hear. This is defined as perceptual weighing.

A quantized version of the value of the symmetry coefficient specified by the function f (x _mono , x _side ) is transmitted to the decoder. It is preferable to consider quantization after the formation of the modified side signal. Then we get the expression below:

where Q _g (..) is the quantization function that is applied to the symmetry coefficient given by the function f (x _mono , x _side ). The balancing coefficient is transmitted over the transmission channel. In ordinary signals panned from left to right, the symmetry coefficient is limited to the interval [-1.0 ... 1.0]. If, on the other hand, the channels are not in phase with each other, then the symmetrization coefficient may go beyond these limits.

As an additional means to stabilize the stereo image, it is possible to limit the symmetrization coefficient if the normalized cross-correlation between mono and side signals is given by the following equation:

Where

These situations arise quite often in the case of, for example, classical music or studio music with a large number of diffuse sounds, where in some situations channels a and b can almost cancel each other out when creating a mono signal. The effect on the coefficient of symmetry is that it can change quickly, causing interference in the stereo image. The above limitation mitigates this problem.

A filtering-based approach according to US Pat. No. 5,434,948 has similar problems, but in this case, the solution is not so simple.

If E _s is the encoding function (for example, a conversion encoder) of the residual side signal, and E _m is the mono signal encoding function, then the decoded signals a "and b" in the decoder can be described as follows (provided that γ = 0.5 ):

An important advantage of calculating the symmetry coefficient for each frame is that interpolation can be avoided. Instead, typically, as described above, frame processing is performed using overlapping frames.

The principle of coding with symmetrical coefficients works particularly well in the case of music signals, where typically fast changes are needed to track a stereo image.

Recently, multi-channel coding has become popular. An example is 5.1 channel surround sound in DVD movies. The channels are placed as follows: left front, central front, right front, left rear, right rear and a separate low-frequency speaker (subwoofer). 5 shows an embodiment of an encoder that encodes three front channels in this configuration using inter-channel redundancies according to the present invention.

Three channel signals L, C, R are provided at three inputs 16A-C, and a _mono signal x _mono is created by summing these three signals. A central signal coding apparatus 130 has been added that receives a central signal x _center . The mono signal 42 in this embodiment is the encoded and decoded mono signal x ″ _mono , multiplied by a certain balancing factor g _Q in the multiplier 133. In the subtractor 135, the multiplied mono signal is subtracted from the central signal x _center to obtain a central residual signal. The balancing factor g _Q is determined based on the content of the mono and central signals by the optimizer 137 to minimize the central residual signal according to the quality criterion. The central residual signal is encoded in the central residual encoder 139 according to any encoding procedures. Preferably, the central residual encoder 139 is a low bit rate encoder or CELP encoder. The p _center encoding parameters representing the center signal, in this case, contain the p _{center residual} encoding parameters representing the center residual signal, and the optimized balancing factor 149. The central residual signal and the scaled mono signal are summed in the summing device 235, creating a modified central signal 142 compensated with taking into account coding errors.

Side signal x _side , i.e. the difference between the left L and right R channels is supplied to the side signal coding device 30, as in previous embodiments. However, here the optimizer 37 also depends on the modified central signal 142 provided by the central signal encoding device 130. Therefore, the side residual signal is created as an optimal linear combination of the mono signal 42, the modified central signal 142 and the side signal in the subtractor 35.

The above concept of variable length frames can be applied to any of the secondary and central signals, or both.

FIG. 6 illustrates a decoding apparatus suitable for receiving encoded audio signals from the encoding apparatus shown in FIG. The received signal 54 is divided into encoding parameters p _mono representing the main signal, encoding parameters p _center representing the central signal, and encoding parameters p _side representing the side signal. At decoder 64, the p _mono encoding parameters representing the main signal are used to form the main x " _mono signal. At decoder 160, the p _mono encoding parameters representing the central signal are used to generate the central x" _center signal based on the main x " _mono signal. to the decoder 60, p _side encoding parameters representing the side signal are decoded to obtain the side signal x " _side based on the main signal x" _mono and the center signal x " _center .

This procedure can be mathematically expressed as follows:

The input signals x _left , x _rignt and x _center are combined into a mono channel according to:

The α, β, and χ in the remaining section are set to 1.0 for simplicity, but they can be set to arbitrary values. The values of α, β, and χ can be either constants or dependent on the signal content to select one or two channels in order to achieve optimal quality.

The normalized cross-correlation between the mono and central signal is calculated as follows:

Where

where x _center is the central signal, x _mono is a _mono signal. The mono signal is output from the target mono signal, but local synthesis of the mono signal encoder can also be used.

The central residual signal to be encoded is expressed as follows:

where Q _g (..) is the quantization function that is applied to the symmetry coefficient. The balancing coefficient is transmitted over the transmission channel.

If E _c is the encoding function (for example, an encoder with conversion) of the central signal, E _m is the mono signal encoding function, then the decoded signal x " _center in the decoder can be described as follows:

The side residual signal to be encoded is expressed as follows:

where g _Qsm and g _Qsc are the quantized values of the parameters g _sm and g _sc , which minimize the expression:

may be, for example, equal to 2 to minimize the least squares error. The parameters g _sm and g _sc can be quantized together or separately.

If E _s is the coding function of the secondary residual signal, then the decoded channel signals x " _left and x" _right are specified as follows:

One of the most annoying perceptual defects is the proactive echo effect. 7a-b, diagrams illustrate this defect. Let the signal component change in time, as shown by curve 100. At the beginning, from the moment t0, the signal component is not present in the audio sample. At time t between t1 and t2, the signal component suddenly appears. When the signal component is encoded using the frame length t2-t1, the appearance of the signal component is smeared over the entire frame, as shown in curve 101. If curve 101 is decoded, the signal component appears Δt earlier than the planned occurrence of the signal component, and a "preemptive" echo".

Proactive echo defects become more emphasized if long coding frames are used. By using shorter frames, the defect is suppressed to some extent. Another way to solve the proactive echo problems described above is to exploit the fact that the mono signal is present in both the encoder and the decoder. This makes it possible to scale the side signal according to the mono signal energy circuit. The decoder performs inverse scaling and, thus, some of the proactive echo problems can be mitigated.

The mono signal energy loop is calculated per frame as follows:

where w (n) is the window function. The simplest window function is a rectangular window, but other window types, such as the Hamming weighting function, may be more desirable.

The side residual signal is then scaled as follows:

,

,

In a more general form, the above equation can be written as follows:

,

,

where f (..) is a monotonic continuous function. In the decoder, the energy loop is calculated on the decoded mono signal and applied to the decoded side signal as follows:

,

,

Since this energy-scale scaling is in some ways an alternative to using shorter frames, this concept is particularly suitable for combining with the concept of variable-length frames, which is further described above. If there are several coding schemes that apply energy-scaling, of which some do not apply, while others use energy-scaling only for certain subframes, a more flexible set of decoding schemes can be provided. FIG. 8 illustrates an embodiment of a signal encoding apparatus 30 according to the present invention. Here, various coding schemes 81 comprise shaded subframes 91 representing coding employing energy-scaling and unshaded subframes 92 representing coding procedures not applying energy-scaling. Thus, there are combinations of not only subframes of different lengths, but also subframes with different coding principles. In the present illustrative example, the application of energy-scale scaling is different in different coding schemes. More generally, coding principles can be combined with a variable-length concept in a similar way.

The coding scheme set shown in FIG. 8 contains circuits that process, for example, proactive echo defects in various ways. In some cases, longer subframes are used while minimizing the pre-emptive echo according to the principle of the energy loop. Other schemes use shorter subframes without scaling along the energy path. Depending on the contents of the signal, one of the alternatives may be more advantageous. For significant pre-echo cases, coding schemes using short energy-scaling subframes may be required.

The proposed solution can be used in the full frequency range or in one or more different subbands. The use of a subband can be applied either to both the main and secondary signal, or to one of them separately. A preferred embodiment comprises dividing the side signal into several frequency ranges. The reason is simply that it is easier to remove redundancy in the isolated frequency range than in the entire frequency range. This is especially important when encoding music signals with rich spectral content.

A possible use case is to encode the frequency range below a predetermined threshold using the above method. The predetermined threshold may preferably be 2 KHz, or even more preferably 1 KHz. For the remainder of the useful frequency range, you can either encode another additional frequency range using the above method, or use a completely different method.

The motivation for using the above method, preferably for low frequencies, is that diffuse acoustic fields usually have a small energy reserve at high frequencies. The natural reason is that sound absorption typically increases with frequency. In addition, the components of diffuse acoustic fields are believed to play a less important role in the human auditory system at higher frequencies. Therefore, it is useful to use this solution at low frequencies (below 1 or 2 KHz) and be based on other, even more efficient in terms of bits, coding schemes at higher frequencies. The fact that the circuit is used only at low frequencies provides significant savings in bit rate, since the required bit rate in the proposed method is proportional to the required bandwidth. In most cases, a mono signal encoder can encode the entire frequency range, while the proposed coding of side signals is assumed to be performed only in the lower part of the frequency range, as schematically illustrated in FIG. 9. Reference numeral 301 denotes a side signal coding scheme according to the present invention, reference numeral 302 denotes any other side signal coding scheme, and reference numeral 303 denotes a side signal coding scheme.

It is also possible to use the proposed method for several different frequency ranges.

10, the main steps of an embodiment of an encoding method according to the present invention are illustrated in a flowchart. The procedure starts at step 200. At step 210, a main signal derived from polyphonic signals is encoded. At step 212, coding schemes are provided that comprise subframes with different lengths and / or order. The side signal output from polyphonic signals in step 214 is encoded by a coding scheme selected, depending at least in part, on the actual contents of the current polyphonic signals. The procedure ends at step 299.

11, the main steps of an embodiment of a decoding method according to the present invention are illustrated in a flowchart. The procedure begins at step 200. At step 220, the received encoded base signal is decoded. At step 222, coding schemes are provided that comprise subframes with different lengths and / or order. The received side signal is decoded in step 224 by the selected coding scheme. At step 226, the decoded main and side signals are combined into a polyphonic signal. The procedure ends at step 299.

The above embodiments should be considered as illustrative examples of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, and changes can be made in the embodiments without departing from the scope of the present invention. In particular, decisions regarding various parts in different embodiments may be combined in other configurations, if technically feasible. The scope of the present invention is defined by the claims.

References

EP 0497413

U.S. Patent 5,285,498

US Patent 5,434,948

C. Faller et al., "Binaural cue coding applied to stereo and multi-channel audio compression", 112th AES convention, May 2002, Munich, Germany.

Claims (26)

1. A method for encoding polyphonic signals, comprising the steps of: generating a first output signal representing encoding parameters characterizing a main signal based on signals of at least the first and second channel; and generating a second output signal representing coding parameters characterizing a side signal based on the signals of at least the first and second channel in the coding frame, characterized in that it further comprises the step of providing at least two coding schemes, each of at least two coding schemes is characterized by a corresponding set of subframes together constituting a coding frame, wherein the sum of the lengths of the subframes in each coding scheme is equal to the length of the coding frame; each set of subframes contains at least one subframe; moreover, the step of generating the second output signal comprises the step of selecting a coding scheme, at least in part, depending on the contents of the current side signal; wherein the second output signal is encoded in each of the subframes of the selected set of subframes separately. 2. The method according to claim 1, characterized in that the step of generating the second output signal comprises the steps of generating coding parameters representing a side signal, which is the first linear combination of signals of at least the first and second channel, within all subframes of each from at least two sets of subframes separately; calculating a general measure of accuracy for each of the at least two coding schemes, said measure of accuracy being an objective value of signal coding quality; and selecting the encoded signal from the encoding scheme having the best measure of accuracy as encoding parameters representing a side signal. 3. The method according to claim 2, characterized in that the measure of accuracy is based on measuring the signal-to-noise ratio. 4. The method according to claim 1, characterized in that the subframes have a length l _sf according to

, where l _f is the length of the encoding frame, n is an integer. 5. The method according to claim 4, characterized in that n is less than a predetermined value. 6. The method according to claim 5, characterized in that at least two coding schemes contain all permutations of the lengths of subframes. 7. The method according to any one of claims 1 to 6, characterized in that the step of generating encoding parameters representing the main signal comprises the steps of creating the main signal as a second linear combination of signals of at least the first and second channel; and encode the main signal into encoding parameters representing the main signal, the step of encoding the side signals comprises the steps of creating a side residual signal as the difference between the side signal and the main signal scaled by a balancing coefficient; wherein the coefficient of balancing is defined as a coefficient minimizing the side residual signal according to the quality criterion; encode the side residual signal and the balancing coefficient into coding parameters representing the side signal. 8. The method according to claim 7, characterized in that the quality criterion is based on the measured value according to the least squares method. 9. The method according to any one of claims 1 to 6, characterized in that the step of encoding the side signal further comprises the step of scaling the side signal to the energy circuit of the main signal. 10. The method according to claim 9, characterized in that the scaling of the side signal is a division by a coefficient, which is a monotonic continuous function of the energy circuit of the main signal. 11. The method according to claim 10, characterized in that the monotonic continuous function is the square root function. 12. The method according to claim 10, characterized in that the energy circuit E _{from the} main signal X _{mono is} calculated from a subframe according to

start of window≤m≤ end of window, where L is an arbitrary coefficient, n is the summation index, m is the sample within the subframe, and w (n) is the window function. 13. The method according to p. 12, characterized in that the function of the window is the function of a rectangular window. 14. The method of claim 12, wherein the window function is a Hamming window function. 15. The method according to any one of claims 1 to 6, characterized in that at least two coding schemes comprise different coding principles of the side signal. 16. The method according to clause 15, wherein the at least first coding scheme of the at least two coding schemes comprises a first coding principle of a side signal for all subframes, and at least a second coding scheme of, at least two coding schemes contains the second principle of encoding a side signal for all subframes. 17. The method according to clause 15, wherein the at least one encoding scheme of the at least two encoding schemes comprises a first principle of encoding a side signal for one subframe and a second principle of encoding a side signal for another subframe. 18. The method according to claim 1, characterized in that the step of generating a second output signal, in turn, comprises the steps of analyzing the spectral characteristics of the side signal, which is the first linear combination of signals of at least the first and second channel; selecting a set of subframes based on the analyzed spectral characteristics; and encode the side signal within all subframes of the selected set of subframes separately. 19. The method according to any one of claims 1 to 6, characterized in that the step of generating a second output signal is used in a limited frequency range. 20. The method according to claim 19, characterized in that the step of generating a second output signal is applied only for frequencies below 2 kHz. 21. The method according to claim 20, characterized in that the step of generating a second output signal is applied only for frequencies below 1 kHz. 22. The method according to any one of claims 1 to 6, characterized in that the polyphonic signals represent musical signals. 23. A method for decoding polyphonic signals, comprising the steps of decoding encoding parameters representing the main signal; decoding encoding parameters representing a side signal within the encoding frame; and combining at least a decoded main signal and a decoded side signal into signals of at least the first and second channel, characterized in that it comprises the steps of providing at least two coding schemes, each of the two schemes encoding is characterized by a set of subframes together constituting an encoding frame, wherein the sum of the lengths of the subframes in each encoding scheme is equal to the length of the encoding frame; wherein each set of subframes comprises at least one subframe; wherein the step of decoding the encoding parameters representing the side signal comprises the step of decoding the encoding parameters representing the side signal independently in subframes of one of the at least two encoding schemes. 24. A device for encoding polyphonic signals, comprising means for inputting polyphonic signals, comprising at least a first and second channel, means for generating a first output signal representing encoding parameters characterizing a main signal based on signals of at least the first and second channel means for generating a second output signal representing encoding parameters characterizing a side signal based on the signals of at least the first and second channel in the encoding frame; and means for output; characterized in that it comprises means for providing at least two coding schemes, each of the at least two coding schemes being characterized by a corresponding set of subframes together constituting a coding frame, wherein the sum of the lengths of the subframes in each coding scheme is equal to the frame length coding; each set of subframes contains at least one subframe; moreover, the means for generating a second output signal comprises means for selecting a coding scheme, at least in part, depending on the contents of the current side signal; means for encoding a side signal in each of the subframes of the selected coding scheme separately. 25. A device for decoding polyphonic signals, comprising: means for inputting encoding parameters representing a main signal and encoding parameters representing a side signal; means for decoding encoding parameters representing a main signal; means for decoding encoding parameters representing a side signal in the encoding frame; means for combining at least the decoded main signal and the decoded side signal into signals of at least the first and second channel; and means for output, characterized in that the means for decoding encoding parameters representing a side signal, comprises means for providing at least two encoding schemes, each of at least two encoding schemes characterized by a corresponding set of subframes together constituting a coding frame, wherein the sum of the lengths of the subframes in each coding scheme is equal to the length of the coding frame; each set of subframes contains at least one subframe; and means for decoding encoding parameters representing the side signal separately in subframes of one of the at least two encoding schemes. 26. An audio system comprising at least one of: a polyphonic signal encoding device according to claim 24 and a polyphonic signal decoding device according to claim 25.

Images