TWI559295B - Elimination of non - steady - state noise - Google Patents

Description

Eliminate non-steady state noise methods

The invention relates to a method for eliminating non-steady state noise, in particular to a method for eliminating unsteady noise in combination with background signal detection.

Voice-activated operation has gradually become an indispensable part of daily life. It has been widely used in, for example, smart phones and tablets to replace traditional keyboard or touch input, while voice and noise obfuscation signal processing affects voice control operations. The most critical factor of correctness is also the most important part of the solution, because it is directly related to the practicality and convenience of voice-activated operation.

In recent years, there have been quite a lot of researches on the application of noise in the field of speech, such as noise cancellation, speech separation, and even music-related separation of songs or instrument separation. Most algorithms must estimate the target sound or the background signal to be removed (which may include background speech and background noise) before processing. However, the handheld wireless communication system often encounters a time-varying and non-stationary environment. Since this unsteady noise will change with time, it is easy to cause the original voice signal during the general filtering of noise. Background noise estimation errors lead to speech distortion. Therefore, how to eliminate the non-steady state noise to filter out the correct target speech is the primary purpose of the present invention.

In view of the above problems of the prior art, the object of the present invention is to provide a method for eliminating unsteady noise, which is suitable for solving the original voice signal when performing noise filtering in a time-varying non-steady-state environment. The problem of speech distortion is caused by background noise estimation errors.

According to an object of the present invention, a method for canceling an unsteady noise is provided, which comprises the steps of: inputting a stream of sounds for estimation to distinguish a primary voice from one of a background voice and a background noise; Background signal; using a basal background noise estimation method to obtain a spectral characteristic substrate of the estimated background signal; and performing spectral noise separation on the base of the spectral characteristic substrate to obtain a clean signal for eliminating background noise; The vocal endpoint detection method is used to obtain a clean voice signal that has been eliminated by noise from the clean signal of background noise.

Preferably, the method for canceling the non-steady state noise further comprises the steps of: dividing the sound stream into a plurality of sound boxes, and obtaining a two-dimensional spectrum map of the sound stream in units of the sound box; The dimensional spectrogram is brought into the impulse response filter bank with two-dimensional time and frequency domain to find a harmonic frequency modulation energy value, and the harmonic frequency modulation energy value is compared with the threshold value, when the harmonic The frequency modulation energy value is lower than the threshold, which is regarded as background noise, and the threshold value is regarded as background speech.

Among them, the threshold value is calculated by combining the voice signal and noise signal of the mixed noise and the adjustment coefficient.

Among them, the operation formula of the threshold is: γ = ρ . { M [ FME _{S + N} ( t )]- M [ FME _N ( t )]}+ M [ FME _N ( t )]; where γ is the threshold and FME _{S + N} ( t ) is the whole speech The harmonic frequency modulation energy value of the previous time when the modulation energy is maximum, FME _N ( t ) is the harmonic frequency modulation energy value of the time when the modulation energy is the smallest in the whole speech, and ρ is the adjustment coefficient. M represents an average of the harmonic frequency modulation energy values for the time.

Where ρ can be 0.25.

Preferably, the method for canceling the non-steady state noise further comprises the steps of: separating the spectral characteristic substrate by using the base noise separation method to obtain a speech structure and a speech excitation matrix, thereby obtaining a clean signal for eliminating background noise.

According to the present invention, the method for eliminating non-steady state noise according to the present invention may have one or more of the following characteristics:

1. Conventional techniques Using non-speech segments to estimate noise sometimes fails to make very accurate estimates, especially when estimating unsteady noise is not accurate, which also affects the effect of signal cancellation. In the present invention, the background signal detection combined with the frequency modulation energy of the speech harmonics can be accurately analyzed from the speech structure to find the background signal that is expected to be separated, even in the segment where the target speech and the noise exist simultaneously. Effectively distinguishes noise, and improves the subsequent noise cancellation performance through this accurate noise estimation.

2. Known techniques In order to improve the effect of noise cancellation, noise is estimated in non-speech segments, so unpredictable signal cancellation for unsteady noise is extremely poor (because of unsteady noise only A small part of the non-speech paragraph). The present invention is directed to the analysis of the specific structure of the sound in nature, and the base of the unsteady noise speech structure can be found to be eliminated, so that it can be applied to any sound signal to eliminate unsteady noise interference.

3. The conventional speech endpoint detection cannot distinguish the target speech signal when encountering unsteady noise interference (which may also mix the components of the external background speech), and the present invention eliminates instability by adding The method of state noise estimates the possible unsteady noise and separates it, which can effectively improve the speech endpoint detection performance and eliminate the unsteady background signal in the speech.

S11~S14‧‧‧Steps

1 is a flow chart of a method for eliminating non-steady state noise according to the present invention.

Figure 2 is a schematic diagram of the combined time domain frequency domain impulse response of the present invention.

Figure 3 is a schematic diagram of the noise model training and identification of the present invention.

Fig. 4 is a diagram showing an example of noise cancellation according to an embodiment of the present invention.

The technical features, contents, and advantages of the present invention, as well as the advantages thereof, can be understood by the present inventors, and the present invention will be described in detail with reference to the accompanying drawings. The subject matter is only for the purpose of illustration and description. It is not intended to be a true proportion and precise configuration after the implementation of the present invention. Therefore, the scope and configuration relationship of the attached drawings should not be interpreted or limited. First described.

Please refer to FIG. 1 , which is a flow chart of the method for eliminating non-steady state noise according to the present invention. The method of the present invention comprises the main four steps. First, in step S11, a short-time Fourier transform is used to obtain a time-frequency representation of the sound, input the sound stream and calculate each sound box. The short-time Fourier spectrum is used to obtain a two-dimensional spectrogram of the entire sound signal, and a plurality of specially designed two-dimensional time-frequency domain impulse response bandpass filter banks are used. The concept of this analytical method is derived from an auditory model based on the neural response of the known auditory region of the cerebral cortex, which is the frequency-modulating energy representative of the most robust speech harmonics in speech segment recognition. Method We can parse out the characteristics unique to speech and judge the speech or non-speech fragments with a threshold (calculated as the following formula (4)). Next, in step S12, the background signal segment is input into the training process of the base-type sparse non-negative matrix decomposition, and the base B _noise belonging to the noise is trained. B _noise is a two-dimensional matrix composed of several background noise base vectors. Then, step S13 is performed to separate the trained noise base B _{noise into} speech and noise, and the sparse non-negative matrix decomposition method is finally obtained. Clean the voice, and finally, perform step S14 and perform another voice or non-speech recognition. The equation is in the following order (1)~(3), and in the process of doing the basic sparse non-negative matrix factorization, the original confusing speech signal X can be approximated into B and H, and in order to avoid the data too much. When sparse, the number of bases obtained is relatively large, which will cause the B _noise error to be too large. Therefore, adding the sparse parameter λ as the constraint of H (sparseness constraints) reduces the importance of some large errors (lower weight H). To reduce the B _noise error:

This section contains a number of parameters, including the parameter λ of sparsity, the number of noise floor B _noise , and the base with time domain characteristics. According to the non-negative matrix factorization method, a signal X containing speech and noise can be approximated by equation (2). According to the separation process of the substrate B _noise trained in the previous step by the sparse non-negative matrix decomposition, the base matrix Bspeech and the excitation matrix Hspeech of the speech can be obtained after the iterative operation, and then the speech matrix and the excitation matrix of the speech are used to restore. The cleaner signal is as in equation (3), and the fourth part of the vocal endpoint detection is input. This part is similar to the first part but because it is a cleaner signal, it can cut out more accurate speech segments.

In detail, the present invention obtains a Fourier spectrogram from a sound stream. According to neurophysiological findings, it can be assumed in the auditory model that the auditory region of the cerebral cortex is basically outputting the midbrain. The auditory spectrogram is treated as a two-dimensional image Processing, so we designed two two-dimensional time and frequency domain impulse response filter banks for the traditional Fourier spectrogram. Figure 2 is the two most discriminating filters after the selection of the present invention. The left side is the filter impulse response with the largest response to the downward FM signal (rate=1Hz, scale=5ms). On the right is the filter impulse response that has the greatest response to the upward FM signal (rate = -1 Hz, scale = 5 ms). In principle, the number of filters is not limited. In order to simplify the calculation, at this stage, we actually design a total of two two-dimensional band-pass filter banks, and the rate of change of the packet on the frequency axis and the rate of change of the packet on the time axis are respectively For the combination of <5>ms and <1>Hz and <downward, upward>, the threshold value is finally set for the energy values obtained by the two filters, and the calculation method is as shown in equation (4) to judge each sound box. The signal is a voice signal or a noise. After obtaining some of the noise signals, the base of the noise can be calculated through the base noise estimation (also referred to as the speech structure of the noise), and the present invention is implemented by using a non-negative matrix operation, as shown in FIG. . As for why the noise part is selected for training, the degree of change in speech is generally so large that a higher-dimensional base must be used to represent the speech structure of the speech signal, and the base is estimated from noise. Because the substrate is smaller, the amount of calculation can be reduced a lot. The base Bnoise (Noise bases) of the noise can be obtained through the training process of the basal sparse non-negative matrix factorization. When we train the spectral characteristics of the noise (also referred to as the substrate) in the above manner, the mixed noise can be separated by using the background type background noise separation. The present invention uses the separation process of the non-negative matrix decomposition to eliminate the noise. As in equation (2), only the three parts of Bspeech, Hspeech, and Hnoise need to be updated, and Bspeech, Hspeech, and Hnoise are obtained by the separation process of the base-type sparse non-negative matrix factorization. Finally, the Bspeech, Hspeech substituting (3) finds the separated target speech signal Xspeech.

In addition, as shown in Fig. 4 is an example diagram of a practical case, which is a mixed signal recorded by a mobile phone with a target voice and vocal interference noise in a restaurant environment. That is to say, the situation often encountered under normal use, in addition to the voice has uncertain vocal interference, we found that due to the harmonicity of the voice and the directionality of the frequency modulation (FM), it can be seen from the figure The frequency modulation energy (FME) contour of the four harmonics is used to observe whether there is interference noise part that completely filters out the human voice. Directly observe the 10-second audio file, the target voice signal appears between 0.7 seconds and 2.8 seconds, while the other part is the interference noise of the vocal that must be filtered out. As shown in the frequency modulation energy (FME) contour of the harmonics in Fig. 4, a threshold value is taken to make the speech non-speech recognition of the first step of Fig. 1, and the threshold γ is selected as follows: γ = ρ . { M [ FME _{S + N} ( t )]- M [ FME _N ( t )]} + M [ FME _N ( t )]... Equation (4).

Where M is the average operator, FME _{S + N} ( t ) is the FME value of the first 250 ms with the largest modulation energy in the whole speech, and FME _N ( t ) is the FME value with the minimum modulation energy of 250 ms in the whole speech. ρ is an adjustment factor of 0.25, and M represents an average of the 250 ms energy values. It can be found that after the speech non-speech recognition processing, the upper part of the fourth picture still has some vocal interference noise before the non-negative matrix decomposition processing, because the energy value is higher than the set threshold γ, and is temporarily regarded as the general voice. Signal. Although the spectral modulation energy is higher than the threshold, the spectral characteristics of this part of the vocal noise are similar to those of other background vocal noises. Therefore, the vocal interference noise is trained by inputting the non-speech part into the basic sparse non-negative matrix decomposition training process. The spectrum base Bnoise (Noise bases), and input the feature into the separation part of the base-type sparse non-negative matrix decomposition of step S13 in Fig. 1, and obtain Bspeech and Hspeech, and the pattern (3) can be restored to obtain the background noise elimination. The clean signal Xspeech. Finally, the vocal endpoint detection in step S14 of FIG. 1 is input, and the threshold value of the modulation energy is obtained by combining the separated target speech signal and the threshold update formula (4), when the modulation energy signal is lower than the valve. The value will be filtered out. Through this detection method, the target speech after background noise cancellation can be obtained. The threshold used in this method is calculated in the same way as equation (4), but since FME _{S + N} ( t ) and FME _N ( t ) have been eliminated by noise in these two phases, the final threshold and the previous result It is not the same, and the result is shown in Figure 4. It can be found that the signal modulated by the base-type sparse non-negative matrix factor, the harmonic frequency modulation energy will be eliminated due to noise, and the contour line is more easy to segment the voice accurately.

In summary, the method for eliminating unsteady noise in the present invention is to quickly find the background signal (including background voice and noise) through the background signal detection method, and then mark the background signal and pass through the base background. Estimating to train the spectral base of steady-state or unsteady noise, this method can effectively estimate the spectral characteristics of the noise even in the segment where the target speech and noise exist simultaneously. Then use the trained noise spectrum base to perform background noise separation on the original sound stream to eliminate unsteady noise, and finally input a clean voice signal for vocal endpoint detection to filter out the correct Target speech.

The above is intended to be illustrative only and not limiting. Any equivalent modifications or alterations to the spirit and scope of the invention are intended to be included in the scope of the appended claims.

S11~S14‧‧‧Steps

Claims (7)

A method for eliminating non-steady state noise includes the steps of: inputting a stream of sounds for estimation to distinguish a primary speech from a background signal including a background speech and a background noise; using a base background The noise estimation method obtains a spectrum characteristic base of the background signal after estimating the background signal; and the spectral characteristic base of the background signal performs a base-type noise separation on a sound stream to obtain the elimination One of the background noises is a clean signal; the vocal endpoint detection method is used to obtain a clean voice signal from the clean signal of the background noise; wherein the voice stream is input for estimation to distinguish The main voice and the background signal including the background voice and the background noise further comprise the steps of: dividing the sound stream into a plurality of sound boxes, and determining one of the sound streams in units of the sound box a two-dimensional spectrogram; and the two-dimensional spectrogram is brought into an impulse response filter bank having a two-dimensional time domain and a frequency domain to obtain a harmonic frequency modulation energy value, and the harmonic frequency is modulated Comparing the magnitude with a threshold value, when the modulated harmonic frequency energy value is below the threshold, the background noise that is considered higher than the threshold is considered the background speech. The method for eliminating unsteady noise according to claim 1, wherein the threshold is calculated by combining the voice signal and the noise signal of the mixed noise and the adjustment coefficient. The method for eliminating non-steady state noise according to item 2 of the patent application scope, wherein the operation formula of the threshold is: γ = ρ . { M [ FME _{S + N} ( t )]- M [ FME _N ( t )]}+ M [ FME _N ( t )]; where γ is the threshold and FME _{S + N} ( t ) is the whole speech The harmonic frequency modulation energy value of the previous time when the modulation energy is maximum, FME _N ( t ) is the harmonic frequency modulation energy value of the time when the modulation energy is the smallest in the whole speech, and ρ is the adjustment coefficient. M represents an average of the harmonic frequency modulation energy values for the time. The method for eliminating non-steady state noise as described in claim 3, wherein ρ is 0.25. The method for canceling an unsteady state noise according to claim 1, wherein the background characteristic of the background signal after the background signal is estimated using a background background noise estimation method further comprises the following Step: Decompose the background signal to obtain the spectral characteristic base of the background signal and its excited state matrix. The method for canceling an unsteady state noise according to claim 1, wherein the spectral characteristic of the background signal is subjected to a ground-based noise separation of the sound stream to obtain the background noise. The clean signal further comprises the steps of: separating the spectral characteristic substrate by using a matrix-based noise separation method to obtain a speech structure and a speech excitation matrix, thereby obtaining the clean signal for eliminating the background noise. The method for eliminating an unsteady noise according to the first aspect of the patent application, wherein the clean voice signal obtained by removing the noise from the background noise by using the vocal endpoint detection method further includes The following steps: dividing the sound stream into a plurality of sound boxes, and obtaining the two-dimensional spectrogram of the sound stream in units of the sound box; bringing the two-dimensional spectrogram into a two-dimensional frequency domain The impulse response filter bank to find the The harmonic frequency modulates the energy value, and compares the harmonic frequency modulation energy value with a threshold value. When the harmonic frequency modulation energy value is lower than the threshold value, the background noise is regarded as higher than the valve. The value is regarded as the background speech; the second stage noise cancellation is performed on the clean signal for eliminating the background noise; and the harmonic frequency modulation energy value is compared with the threshold value for each frame, which is higher than the threshold It is regarded as a clean voice signal, below the threshold is regarded as background noise; and a clean voice signal is obtained from a stream of sound.