CN111669697B - A method and system for extracting coherent sound and ambient sound from multi-channel signals - Google Patents

Abstract

The invention discloses a method and system for extracting coherent sound and ambient sound of multi-channel signals. The method comprises: calculating the weight expression of the coherent sound of N channel signals, estimating the coherent sound according to the weight expression, and calculating each channel accordingly. Among them, the ambient sound energy of each channel is the same; calculate the ambient sound of each channel according to the coherent sound of each channel; perform inverse Fourier transform on the coherent sound of the N channels and the ambient sound of the N channels to obtain the time domain Represents coherent and ambient sound. The method of the invention explores the weight expression for estimating coherent sound for signals with any number of channels under the condition that the ambient sound energy of each channel is the same, and uses the signal energy of each channel and the correlation value between channels to obtain each of the weight expressions. Unknown parameters can realize the extraction of coherent sound and ambient sound of multi-channel signals, and the extraction accuracy is high.

Description

A method and system for extracting coherent sound and ambient sound from multi-channel signals

technical field

The invention relates to the field of spatial sound reproduction, in particular to a method and system for extracting coherent sound and ambient sound of multi-channel signals.

Background technique

Spatial sound playback technology has been widely used in entertainment media, such as movie theaters, home theaters, and portable electronic devices. When playing movies, the spatial sound with a certain sound image width and good immersion is played back through headphones or speakers, which can be used for consumption. bring a better audio-visual experience. In recent years, space acoustic reproduction has gradually shown important application prospects in cutting-edge scientific research and practical engineering fields, such as virtual reality, aviation, aerospace and other fields.

Spatial sound mainly contains two components with different properties, one is the sound component with directionality, called coherent sound; the other is the sound component with diffusivity and cannot distinguish the direction, called ambient sound. In order to achieve better sound reproduction effect, coherent sound and ambient sound extraction (Primary-Ambient Extraction, PAE) need to be performed on spatial sound and different processing is performed. For example, in an audio coding and decoding system, using PAE as the front end of audio coding or decoding can achieve effective and immersive spatial sound playback.

The PAE method for two-channel signals is relatively mature, and the principal component analysis method and the least squares method are widely used. For multi-channel signals, the pairwise correlation method can be used for PAE. However, the accuracy of extracting components by pairwise correlation method is not high. Therefore, it is of great significance to extend the PAE method suitable for stereo to multi-channel signals. The principal component analysis method performs PAE on the stereo signal by calculating the eigenvalues of the covariance matrix of the input signal under the premise that coherent sound occupies the main energy. This method can also extract components from multi-channel signals, but when the number of channels is large, the computational complexity increases, and the principal component analysis method is only effective when coherent sound occupies the main energy. The least squares method realizes the PAE of the stereo signal by calculating and estimating the weight of the coherent sound under the premise that the ambient sound energy of each channel is equal. However, when the least squares method is directly applied to multi-channel signals, the estimated weights are not easy to solve. The ambient sound component mainly plays the role of setting off the atmosphere in the spatial sound. In order to achieve a better surround feeling, the energy distribution difference of the ambient sound in each channel is small. Therefore, under the premise that the ambient sound energy of each channel is equal, it is of great significance to extract coherent sound and ambient sound for multi-channel signals.

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the above-mentioned technical defects. Under the premise that the energy of the ambient sound in each channel is equal, the least squares method is used to estimate the weight of the coherent sound when the number of channels is small. According to the regularity of the weight change with the number of channels, the The weight expression for coherent sound estimation for multi-channel signals with arbitrary number of channels is obtained.

In order to achieve the above purpose, the present invention proposes a method for extracting coherent sound and ambient sound from multi-channel signals, the method comprising:

Calculate the weight expression of the coherent sound of the N channel signals, estimate the coherent sound according to the weight expression, and thereby calculate the coherent sound of each channel; wherein, the ambient sound energy of each channel is the same;

Calculate the ambient sound of each channel according to the coherent sound of each channel;

Inverse Fourier transform is performed on the N-channel coherent sound and the N-channel ambient sound to obtain the coherent sound and the ambient sound represented in the time domain.

As an improvement of the above method, the weight expression of the coherent sound of the N channel signals is calculated, and the coherent sound is estimated according to the weight expression, thereby calculating the coherent sound of each channel; wherein, the ambient sound energy of each channel is the same; Specifically include:

Fourier transform is performed on the time-domain multi-channel signal, and the input signal X _n of the nth channel is expressed as:

X _n =β _n S+A _n

Among them, S represents the spectrum of the coherent sound, β _n represents the amplitude difference factor between the coherent sound of the nth channel and the coherent sound of the first channel, 1≤n≤N, β ₁ =1, An represents the _nth channel The spectrum of the ambient sound of the channel;

Calculate the short-term energy of the _nth channel input signal Xn

Calculate the correlation value Φ ₁₂ for the first channel and the second channel:

第二个通道的短时能量

以及两个通道间相关值Φ₁₂，计算中间参数C和D：According to the short-term energy of the first channel

Short-term energy of the second channel

and the correlation value Φ ₁₂ between the two channels, calculate the intermediate parameters C and D:

From this, the short-term energy P _S of the coherent sound, the short _- term energy PA and β ₂ of the ambient sound are calculated.

Calculate β _n :

The weight value of the nth channel is:

为：Then the estimated value of coherent sound

for:

Then the _nth channel coherent sound Sn:

As an improvement of the above method, the ambient sound of each channel is calculated according to the coherent sound of each channel; specifically:

The ambient sound A _n of the nth channel is:

An = _{Xn -Sn .}

Embodiment 2 of the present invention proposes a multi-channel signal coherent sound and ambient sound extraction system, the system includes:

The coherent sound extraction module is used to calculate the weight expression of the coherent sound of the N channel signals, and estimates the coherent sound according to the weight expression, thereby calculating the coherent sound of each channel; wherein, the ambient sound energy of each channel is the same;

The ambient sound extraction module is used to calculate the ambient sound of each channel according to the coherent sound of each channel;

The frequency-domain to time-domain module is used to perform inverse Fourier transform on the N-channel coherent sound and the N-channel ambient sound to obtain the coherent sound and ambient sound represented in the time domain.

As an improvement of the above system, the specific implementation process of the coherent sound extraction module includes:

Fourier transform is performed on the time-domain multi-channel signal, and the input signal X _n of the nth channel is expressed as:

X _n =β _n S+A _n

Among them, S represents the spectrum of the coherent sound, β _n represents the amplitude difference factor between the coherent sound of the nth channel and the coherent sound of the first channel, 1≤n≤N, β ₁ =1, An represents the _nth channel The spectrum of the ambient sound of the channel;

Calculate the short-term energy of the _nth channel input signal Xn

Calculate the correlation value Φ ₁₂ for the first channel and the second channel:

第二个通道的短时能量

以及两个通道间相关值Φ₁₂，计算中间参数C和D：According to the short-term energy of the first channel

Short-term energy of the second channel

and the correlation value Φ ₁₂ between the two channels, calculate the intermediate parameters C and D:

From this, the short-term energy P _S of the coherent sound, the short _- term energy PA and β ₂ of the ambient sound are calculated.

Calculate β _n :

The weight value of the nth channel is:

为：Then the estimated value of coherent sound

for:

Then the _nth channel coherent sound Sn:

As an improvement of the above system, the specific implementation process of the ambient sound calculation module includes:

The ambient sound A _n of the nth channel is:

An = _{Xn -Sn .}

The advantages of the present invention are:

The method of the invention explores the weight expression for estimating coherent sound for signals with any number of channels under the condition that the ambient sound energy of each channel is the same, and uses the signal energy of each channel and the correlation value between channels to obtain each of the weight expressions. Unknown parameters can realize the extraction of coherent sound and ambient sound of multi-channel signals, and the extraction accuracy is high.

Description of drawings

Fig. 1 is the flow chart of the coherent sound and ambient sound extraction method of multi-channel signal of the present invention;

Fig. 2 (a) is the error diagram that uses the method of the present invention and pairwise correlation method to carry out coherent sound component extraction to mixed five-channel signal 1;

Fig. 2 (b) is the error diagram that uses the method of the present invention and pairwise correlation method to carry out ambient sound component extraction to mixed five-channel signal 1;

Fig. 3 (a) is the error diagram that uses the method of the present invention and pairwise correlation method to carry out coherent sound component extraction to mixed five-channel signal 2;

FIG. 3( b ) is an error diagram of extracting ambient sound components from the mixed five-channel signal 2 using the method of the present invention and the pairwise correlation method.

Detailed ways

The technical solutions of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

Example 1

As shown in FIG. 1 , Embodiment 1 of the present invention proposes a method for extracting coherent sound and ambient sound when the ambient sound energy of each channel of a multi-channel signal is equal, including the following steps:

Step 1) After dividing the multi-channel signal into frames, perform Fourier transform to obtain the frequency spectrum, and represent the short-term energy of each channel and the correlation value between any two channels according to the multi-channel signal model, specifically including:

In the multi-channel signal model, the input signal is represented as the superposition of coherent sound and ambient sound. Due to the different characteristics of coherent sound and ambient sound, it is assumed that the coherent sound of each channel is completely correlated, that is, there is a linear relationship; it is assumed that the coherent sound is uncorrelated with the ambient sound of each channel and the ambient sound between channels .

Step 1-1) Fourier transform the time-domain multi-channel signal to obtain the spectrum:

X _n =β _n S+A _n ,n=1,2,...,N

Among them, N is the number of channels, S represents the spectrum of the coherent sound, β _n represents the amplitude difference factor between the coherent sound of the nth channel and the coherent sound of the first channel, and β ₁ =1, An represents the _nth channel. the spectrum of ambient sound;

Step 1-2) The signal energy of each channel can be expressed as:

Among them, E{} represents the short-term average.

Step 1-3) The correlation value between each channel can be expressed as:

为第n₁个通道和第n₂个通道间的相关值，n₁＝1,2,…,N,n₂＝1,2,…,N,n₁≠n₂；in,

is the correlation value between the _{n1th channel and the n2th channel, n 1 =} 1,2,...,N,n ₂ =1,2,...,N,n ₁ ≠n _{2 ;}

Step 2) Calculate the weight value of coherent sound using the least squares method when the number of channels is small, and explore its regularity, including:

Step 2-1) For the two-channel signal, calculate the weight values of the input signals X ₁ and X ₂ to estimate the coherent sound S:

Step 2-1-1) Estimate coherent sound S:

Among them, w ₁ and w ₂ represent the estimated weights to be sought.

Step 2-1-2) The estimation error σ _S of S is expressed as:

Step 2-1-3) Use the least squares algorithm to solve, that is, when the estimation error is completely uncorrelated with the input stereo signal, the obtained weight is the optimal estimation:

E{σ _S X ₁ }=0

E{σ _S X ₂ }=0.

At this time, the weight of the optimal estimate is expressed as:

Among them, _PS represents the short _- term energy of coherent sound, and PA represents the short-term energy of ambient sound.

Step 2-2) For the three-channel signal, calculate the weight values of the input signals X ₁ , X ₂ and X ₃ to estimate the coherent sound S:

Step 2-2-1) Estimate coherent sound S:

Among them, w ₁ , w ₂ and w ₃ represent the estimated weights to be sought.

Step 2-2-2) Similar to step 2-1), the weight value of the estimated coherent sound of the three-channel signal can be obtained:

Step 2-3) By calculating the estimated weight of the coherent sound when the number of channels is more, it is found that the weight value can be expressed uniformly. For a multi-channel signal with N channels, the estimated coherent sound is expressed as:

Among them, the weight value can be expressed as:

Step 3) Calculate and estimate each unknown parameter in the weight of the coherent sound, and complete the extraction of the coherent sound and the ambient sound of the multi-channel signal, specifically including:

Step 3-1) It is known that β _{1 =} 1, so the unknown parameters P _S , PA and β ₂ can be obtained according to the signal energy and inter-channel correlation value of the first two channels in step 1):

in,

Step 3-2) According to the energy values of other channels except the first channel and the second channel, β _n when 3≤n≤N can be obtained:

Step 3-3) For the multi-channel signal with the number of channels N, the parameters P _S , P _A and β _n (n=1, 2, . . . , N) calculated in step 3-1) and step 3-2) ) into the estimated coherent sound weight value w _n (1, 2, .

Step 4) Perform PAE on multi-channel signals of any number of channels, specifically including:

Step 4-1) Calculate the coherent sound of each channel, specifically including:

Since step 2) calculates the weight expression for estimating coherent sound when performing PAE on a multi-channel signal of any number of channels, and step 3) calculates each unknown parameter in the weight expression, so when the number of channels of the multi-channel signal is determined, The coherent sound S can be estimated directly from the weight expression. This coherent sound is directly the coherent sound of the first channel, and the coherent sound of other channels is obtained by S linear processing, namely β _n S (n=2,...,N).

Step 4-2) Calculate the ambient sound of each channel, including:

The remaining components of each channel are identified as ambient sound, that is, An = _{Xn - βnS} .

Step 4-3) Perform inverse Fourier transform on the obtained N-channel coherent sounds and N-channel ambient sounds to obtain coherent sounds and ambient sounds represented in the time domain.

Below in conjunction with the simulation example, the method performance proposed by the present invention is described:

The fully correlated coherent sound and the completely uncorrelated ambient sound are synthesized and mixed with a five-channel signal according to a certain ratio, and the components are extracted by the multi-channel PAE method and the paired correlation method proposed by the present invention. Two groups of mixed multi-channel signals are synthesized, namely, the mixed five-channel signal 1 with pure voice as coherent sound and ocean wave sound as ambient sound, and the mixed five-channel signal 2 with pure music sound as coherent sound and forest background sound as ambient sound. When mixing, in order to control the distribution of coherent sound energy between each channel, set a certain proportional relationship between the coherent sound amplitude difference factor β _n between each channel and its reference value β ₀ ; set the ambient sound energy of each channel to be equal to P _A0 ; In order to control the proportion of coherent sound components in the mixed signal, set different proportions of coherent sound energy γ. The reference values β ₀ and P _A0 are determined by γ.

In this experiment, the amplitude of coherent sound of each channel is set to have a proportional relationship of β ₁ =β ₂ =β ₀ , β ₃ =2β ₀ , and β ₄ =β ₅ =0.5β ₀ . The coherent sound energy ratio γ is 0.05 to 0.95 (with an interval of 0.1). The extraction error ε _P of coherent sound is expressed as:

The extraction error ε _a of ambient sound is expressed as:

Fig. 2(a) and Fig. 2(b) represent the extraction errors of coherent sound and ambient sound when the algorithm proposed by the present invention and the pairwise correlation method respectively perform PAE on the mixed five-channel signal 1; Fig. 3(a) and Fig. 3(b) represents the extraction error of coherent sound and ambient sound when the algorithm proposed in the present invention and the pairwise correlation method respectively perform PAE on the mixed five-channel signal 2. It can be seen that the extraction error of the method proposed in the present invention is smaller than that of the pairwise correlation method in the entire range of the coherent acoustic energy ratio γ from 0.05 to 0.95 (with an interval of 0.1).

Embodiment 2 of the present invention proposes a multi-channel signal coherent sound and ambient sound extraction system, the system includes:

The coherent sound extraction module is used to calculate the weight expression of the coherent sound of the N channel signals, and estimates the coherent sound according to the weight expression, thereby calculating the coherent sound of each channel; wherein, the ambient sound energy of each channel is the same;

The ambient sound extraction module is used to calculate the ambient sound of each channel according to the coherent sound of each channel;

The frequency-domain to time-domain module is used to perform inverse Fourier transform on the N-channel coherent sound and the N-channel ambient sound to obtain the coherent sound and ambient sound represented in the time domain.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the embodiments, those of ordinary skill in the art should understand that any modification or equivalent replacement of the technical solutions of the present invention will not depart from the spirit and scope of the technical solutions of the present invention, and should be included in the present invention. within the scope of the claims.

Claims (4)

1. A method for extracting coherent sound and ambient sound of a multi-channel signal, the method comprising: Calculate the weight expression of the coherent sound of the N channel signals, estimate the coherent sound according to the weight expression, and thereby calculate the coherent sound of each channel; wherein, the ambient sound energy of each channel is the same; Calculate the ambient sound of each channel according to the coherent sound of each channel; Perform inverse Fourier transform on the N-channel coherent sound and the N-channel ambient sound to obtain the coherent sound and ambient sound represented in the time domain; Described calculating the weight expression of the coherent sound of the N channel signals, estimating the coherent sound according to the weight expression, thereby calculating the coherent sound of each channel; wherein, the ambient sound energy of each channel is the same; specifically including: Fourier transform is performed on the time-domain multi-channel signal, and the input signal X _n of the nth channel is expressed as: X _n =β _n S+A _n Among them, S represents the spectrum of the coherent sound, β _n represents the amplitude difference factor between the coherent sound of the nth channel and the coherent sound of the first channel, 1≤n≤N, β ₁ =1, An represents the _nth channel The spectrum of the ambient sound of the channel; Calculate the short-term energy of the _nth channel input signal Xn

Calculate the correlation value Φ ₁₂ for the first channel and the second channel:

According to the short-term energy of the first channel

Short-term energy of the second channel

and the correlation value Φ ₁₂ between the two channels, calculate the intermediate parameters C and D:

From this, the short-term energy P _S of the coherent sound, the short _- term energy PA and β ₂ of the ambient sound are calculated.

Calculate β _n :

The weight value of the nth channel is:

Then the estimated value of coherent sound

for:

Then the _nth channel coherent sound Sn:

2. The method for extracting coherent sound and ambient sound of multi-channel signals according to claim 1, wherein the ambient sound of each channel is calculated according to the coherent sound of each channel; specifically: The ambient sound A _n of the nth channel is: An = _{Xn -Sn .} 3. A coherent sound and ambient sound extraction system for multi-channel signals, wherein the system comprises: The coherent sound extraction module is used to calculate the weight expression of the coherent sound of the N channel signals, and estimates the coherent sound according to the weight expression, thereby calculating the coherent sound of each channel; wherein, the ambient sound energy of each channel is the same; The ambient sound extraction module is used to calculate the ambient sound of each channel according to the coherent sound of each channel; The frequency-domain to time-domain module is used to inverse Fourier transform the N-channel coherent sound and the N-channel ambient sound to obtain the coherent sound and ambient sound represented in the time domain; The specific implementation process of the coherent sound extraction module includes: Fourier transform is performed on the time-domain multi-channel signal, and the input signal X _n of the nth channel is expressed as: X _n =β _n S+A _n Among them, S represents the spectrum of the coherent sound, β _n represents the amplitude difference factor between the coherent sound of the nth channel and the coherent sound of the first channel, 1≤n≤N, β ₁ =1, An represents the _nth channel The spectrum of the ambient sound of the channel; Calculate the short-term energy of the _nth channel input signal Xn

Calculate the correlation value Φ ₁₂ for the first channel and the second channel:

According to the short-term energy of the first channel

short-term energy of the second channel

and the correlation value Φ ₁₂ between the two channels, calculate the intermediate parameters C and D:

From this, the short-term energy P _S of the coherent sound, the short _- term energy PA and β ₂ of the ambient sound are calculated.

Calculate β _n :

The weight value of the nth channel is:

Then the estimated value of coherent sound

for:

Then the _nth channel coherent sound Sn:

4. The coherent sound and ambient sound extraction system of multi-channel signals according to claim 3, wherein the specific implementation process of the ambient sound extraction module comprises: The ambient sound A _n of the nth channel is: An = _{Xn -Sn .}

Images