JPH1152977A - Method and device for voice processing - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately detect a voice interval against an object sound source employing a small number of microphones under the environment in which the S/N ratio is low and the direction to a noise sound source is not specified. SOLUTION: In the device, a voice input section 10 inputs audio signals to terminals 10-1 to 10-n through plural channels ch1 to chn. A beam former processing section 20 conducts a beam former process against the audio signals inputted by the section 10 to suppress the signals arriving from an object sound source. An object sound source direction estimating section 30 obtains the direction to the object sound source from the filter coefficients obtained by the section 20. A voice/non-voice determining section 40 determines the voice interval of the audio signals based on the estimated direction to the object sound source.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an audio processing method / device for detecting an audio section of an input audio signal and for suppressing noise to enhance audio.

[0002]

2. Description of the Related Art As a method of detecting a speech section in a noisy environment, for example, as disclosed in Reference 1: Yasunaga Niimi, "Speech Recognition," Kyoritsu Shuppan, speech and energy are used using the number of zero crossings. There is a method of detecting a section. However, with this method, it is difficult to accurately detect a voice section when the SN ratio is significantly reduced.

In order to enable speech input in an environment with a low SN ratio, noise suppression processing using a microphone array has been studied. For example, Reference 2: "Acoustic system and digital processing" IEICE In this volume, a method for improving an SN ratio by using an adaptive microphone array having a small number of microphones is disclosed. However, in an environment where there are many noise sources and the direction of the noise source cannot be specified, it is difficult to improve the S / N ratio by this method. Therefore, it is necessary to accurately detect a voice section using the output power of the microphone array. It is difficult.

[0004]

As described above, SN using a microphone array with a small number of microphones is used.
According to the method for improving the ratio, it is difficult to accurately detect a voice section using the output power of the microphone array because it is not expected to improve the SN ratio in a noise environment where the direction of the noise source cannot be specified. There was a problem.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and has as its object the purpose of using a small number of microphones for a target sound source in an environment where the SN ratio is low and the direction of the noise source cannot be specified. It is an object of the present invention to provide a voice processing method and apparatus capable of accurately detecting a voice section. Another object of the present invention is to provide an audio processing method and apparatus capable of reliably performing a process of suppressing noise and emphasizing only audio.

[0006]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, the present invention relates to a digital signal for suppressing a signal coming from a target sound source by a beamformer for an audio signal input through a plurality of channels. Calculation processing, that is, beamformer processing, is performed, and the direction of the target sound source is estimated from the filter coefficients obtained by this beamformer processing.
The basic feature is that the voice section of the voice signal is determined based on the direction of the target sound source.

In an environment where the direction of the noise source cannot be specified, it is difficult to improve the SN ratio of the target sound source by the beamformer. However, since the sound from the target sound source has directionality, the target sound source is not included in the voice section. Can be estimated from the filter coefficient of the beamformer, and a voice section can be detected based on the estimated direction of the target sound source.

Also, the present invention suppresses a signal coming from a noise source and outputs a signal from the target sound source, separately from the first beamformer which performs a beamformer process for suppressing a signal coming from the target sound source. A second beamformer for performing beamformer processing for estimating the direction of a noise source from the filter coefficients obtained by the second beamformer, and calculating the direction of the target sound source and the first and second beamformers. The second based on the obtained output power
And controlling the first beamformer based on the direction of the noise source and the output power obtained by the first and second beamformers.

In this way, even when a directional noise source is present, the direction of the target sound source can be estimated with high accuracy by making the input direction of the first beamformer follow the direction of the noise source. This makes it possible to more reliably detect the voice section.

The determination of the voice section may be performed using the power of the voice signal in addition to the estimated direction of the target sound source. Further, the present invention is characterized in that at least one of the output of the first beamformer and the estimated direction of the target sound source is used to suppress noise in the output of the second beamformer to enhance the sound. .

That is, in an environment in which the direction of the noise source cannot be specified because the number of noise sources is so large, the noise suppression performance by the beamformer is reduced, but since the voice signal has directionality, the direction of the noise source is reduced. Since the output of only the noise that suppresses the target signal can be extracted by the first beamformer in which the target direction is set, the speech enhancement processing is performed on the output of the second beamformer by using the spectrum extraction method. It is possible to do.

Here, when the directions of the target sound source and the noise source are fixed and known, it is not necessary to estimate the direction of the target sound source and control the first and second beamformers. The former toward the strongest noise source,
May be directed toward the target sound source.
In this case, speech enhancement processing can be performed on the output of the second beamformer based on the output of the first beamformer.

Further, in the present invention, it is possible to detect a voice section using the target sound source direction estimated as described above and the voice-emphasized signal, thereby further improving the voice section detection performance. Can be done.

[0014]

Embodiments of the present invention will be described below with reference to the drawings. (First Embodiment) In this embodiment, a speech processing apparatus having a function of estimating a direction of a target sound source from speech signals input through a plurality of channels and detecting a speech section will be described.

As shown in FIG. 1, the audio processing apparatus according to this embodiment has a plurality of (n) channels ch1 to chn.
Audio input unit 10 for inputting audio signals from input terminals 10-1 to 10-n via the input terminal, and a beamformer for performing beamformer processing on these audio signals to suppress a signal coming from a target sound source 20, a target sound source direction estimating unit 30 for estimating the direction of the target sound source based on the filter coefficient obtained from the beamformer 20, a time-series value of the estimated direction of the target sound source, Speech / non-speech for determining speech / non-speech of a speech signal based on one or both of a time series value of the power of the obtained signal and a time series value of a correlation value between channels of the signal obtained from the speech input unit 10. And a non-speech determining unit 40.

Here, for simplicity, a case where the number n of channels is 2 will be described as an example. The beamformer 20 performs a filter calculation process called an adaptive beamformer process for suppressing a target sound source on a signal from the audio input unit 10. Various methods are known as a processing method inside the beamformer 20, for example, the above-mentioned document 2 and
Reference 3: Heykin, “Adaptive Filter Theory (Plentice
Hall) ", a generalized sidelobe canceller (GSC), a frost type beamformer, a reference signal method, etc. The present embodiment can be applied to any adaptive beamformer. However, here, a description will be given using a 2-channel GSC as an example.

FIG. 2 shows an example of the beam former 20.
Jim-Griffith type G which is common among two channel GSC
3 shows a configuration example of an SC. This is, for example, a GSC composed of a subtractor 21, an adder 22, a delay unit 23, an adaptive filter 24, and a subtractor 25, as described in Document 2. The adaptive filter 24 is an LMS, RLS, projection type LMS
For example, La = 50 is used as the filter length La. The delay amount of the delay unit 23 is, for example, La / 2.

The adaptive filter 2 of the two-channel Jim-Griffith type GSC shown in FIG.
When the LMS adaptive filter is used for No. 4, the updating of this filter is performed by setting the time to n and setting the coefficient of the adaptive filter 24 to W
(N), the input signal of the i-th channel is xi (n), and the input signal vector of the i-th channel is Xi (n) = (xi
(N), xi (n-1), ..., xi (n-La + 1))
Then, it is expressed by the following equation.

Y (n) = x0 (n) + xl (n) (1) X ′ (n) = X1 (n) −X0 (n) (2) e (n) = y (n) −W (n ) X ′ (n) (3) W (n + 1) = W (n) −μX ′ (n) e (n) (4) The input direction of the GSC in FIG.
For example, it is set to 90 ° with reference to the direction of the target sound source. Here, by delaying the signals of the two channels, signals from the set input directions are equivalently and simultaneously arrive at the array. For this reason, as shown in FIG.
Into the channel 1 side. The delay time of the delay unit 26 is
When the input direction is 90 °, τ = d / c. Here, c is the speed of sound, and d is the distance between the microphones.

When a signal arrives from the direction of the target sound source,
Since the filter in the beamformer 20 has low sensitivity in the direction of the target sound source, the direction of the target sound source can be estimated by examining the directivity, which is the direction dependency of the sensitivity, from the filter coefficient of the filter. .

FIG. 4 shows a procedure for estimating the direction of the target sound source in the target sound source direction estimating section 30. First, as initial settings, a search range θr in the target direction, a filter length L, FFT
The length (number of FFT points) N and the number of channels M are set (step S101). For example, θr = 20 °, L = 5
0, N = 64, and M = 2. Since the beamformer searches only the range of the direction of arrival of the signal from the target sound source, the search angle range is set to the range of θr, for example, based on the direction of the target sound source.

Next, if the beamformer is GSC, the filter coefficients are converted into a form equivalent to a transversal type beamformer (step S102). For example, in the case of a two-channel Jim-Griffith type GSC, if the coefficients of the adaptive filter in the GSC are given by wg = (w ₀ , w ₁ , w ₂ ,..., W _L-2 , w _L-1 ), coefficient of the equivalent filter of the channel ch1 _{is, w e1 = (- W 0} , -W 1, -W 2, ..., -W L / 2 +1,
_{..., -W L-2, -W} L-1) coefficient of the equivalent filter of the second channel ch2 _{is, w e2 = (w 0,} w 1, w 2, ..., w L / 2 -1, ..., w
_L-2 , WL _-1 ).

Next, the FFT of the filter coefficient for each channel
To obtain the frequency component Wei (k) (step S103). Here, k is a frequency component number, and i is a channel number.

Next, one direction in the search range is defined as θ
Then, a direction vector S (k, θ) representing the propagation phase delay of each channel with respect to the signal arriving from the θ direction is generated (step S104). Direction vector S (k, θ)
Is, for example, in the case of the microphone arrangement shown in FIG.
On the basis of the channel ch1, S (k, θ) = (1, exp (−jk / N fs d sin
(Θ))). fs is the sampling frequency, and d is the distance between the microphones.

Next, the square | S · W | ² of the absolute value of the inner product of the frequency component We = (We1 (k), We2 (k)) of the filter obtained by the FFT and the direction vector S (k, θ) Is obtained (step S105).

The processing of steps S103 to S106 is performed for all frequencies, that is, from k = 1 to k = N / 2, and the sum of squares of the obtained inner products is added for the frequency k for each direction θ, and all the bands are summarized. The sensitivity D (θ) = Σ | W (k) · S (k, θ) | ² is obtained for each of the directions (step S106). At this time, the direction is changed, for example, by 1 °, and all the directions in the search range are checked (step S107). Next, a direction θmin at which the sensitivity in each of the obtained directions is minimized is obtained from D (θ), and this is set as an arrival direction of a signal (a signal from a target sound source or a signal from a noise source) (step S108).

Next, the processing of the voice / non-voice determination section 40 will be described. The speech / non-speech determination unit 40 determines the speech / non-speech based on one or both of the time series value of the direction of the target sound source estimated by the target sound source direction estimation unit 30 and the time series value of the power of the input signal. Make a decision. It is also possible to use a time-series value of the correlation value of two channels.

The voice / non-voice determination can be made, for example, by the following two methods. That is,
(1) A method using a time variation in the direction of a target sound source,
(2) This method uses the amount of time variation in the direction of the target sound source and the power of the input signal. Here, the reason for determining speech / non-speech using the time variation amount without using the direction of the target sound source is that when a signal does not arrive from the target sound source, a directional signal is not included in the input signal. However, the estimated value of the direction of the target sound source takes a random value, and when a signal arrives from the target sound source, the estimated value of the direction of the target sound source takes a value within a certain range. This is because if the amount is within a certain range, it can be detected if it is regarded as voice.

First, the procedure for determining voice / non-voice in the method (1) will be described with reference to FIG. FIG. 6 is a state transition diagram showing a flow of processing in the determination of voice / non-voice, with the non-voice state as a starting point. The time variation in the direction of the target sound source at time n is represented by Δθ (n) = | θ (n)
Θ (n−1) |, the maximum temporal variation of θ (n) required to be recognized as a voice fragment is θth (eg, θth = 5 °)
When Δθ (n) ≦ θth, the time is set as a temporary start point of the voice, and a transition is made to a temporary voice state indicating a state where the temporary start point is found.

In the provisional voice state, the minimum time length required to be recognized as a voice fragment is T1 (for example, T1 = 20 ms).
c), and if Δθ (n)> θth is satisfied within this time length T1, the state returns to the non-voice state; otherwise, Δθ (n)> θ
The time when th is reached is set as a temporary end of the sound, and the state transits to an end waiting state indicating a state of waiting for the end of the sound to be determined.

In the end waiting state, the minimum time length required for determining the end of voice is T2 (for example, T2 = 100 msec). If Δθ (n) ≦ θth is satisfied within this time length T2,
The state transits to a temporary sound continuation state indicating a state in which the sound is continuing. Otherwise, the time of the last transition to the end wait state is set as the temporary end of the voice, and the minimum time length T3 required to recognize the time from the temporary start end to the temporary end as the voice (for example, T3 = If it is less than 300 msec), the state returns to the non-speech state.

In the temporary voice continuation state, Δ
If θ (n)> θth, the process returns to the end waiting state, and if not, the state transits to a sound continuation state indicating a state in which sound is continuing.

On the other hand, in the voice continuation state, Δθ (n)> θth
When the state becomes, the state transits to the termination waiting state. Next, the procedure for determining voice / non-voice in the method (2) will be described with reference to FIG. Here, two values of Pth1 and Pth2 are provided as the minimum values of the power of the input signal necessary to recognize the sound (Pth1> Pth2). In FIG. 7, first, a non-speech state is set as a starting point, and the amount of time variation in the direction of the target sound source at time n is Δθ (n), which is necessary to recognize as a speech fragment.
When the maximum time variation of (n) is θth, Δθ (n) ≦ θ
When th or P (n)> Pth1, the time is set as a temporary start point of the voice, and the state transits to a temporary voice state indicating a state where the temporary start point is found.

In the provisional voice state, "ΔT within T1
(N)> θth and P (n) ≦ Pth1 ”or“ Δθ ”
P until (n)> θth and P (n) ≦ Pth1
If the maximum value of (n) is equal to or less than the threshold value Pth, the state returns to the non-speech state. Otherwise, the state transits to the end waiting state indicating a state of waiting for the end of the sound to be determined. Where Pth
Is the minimum value of the power of the input signal required to be received as voice.

In the termination waiting state, Δθ (n) within T2
If .ltoreq..theta.th or P (n)> Pth1, the state transits to a temporary sound continuation state indicating a state in which sound is continuing. Otherwise, the time of the last transition to the end wait state is set as the temporary end of the voice, and the minimum time length T3 required to recognize the time from the temporary start end to the temporary end as the voice (for example, T3 = If it is less than 300 msec), the state returns to the non-speech state.

In the tentative voice continuation state, "ΔT within T1
(N)> θth and P (n) ≦ Pth1 ”or“ Δθ ”
P until (n)> θth and P (n) ≦ Pth1
If the maximum value of (n) is equal to or less than the threshold value Pth, the process returns to the end waiting state, and if not, the state transits to the sound continuation state indicating the state in which the sound continues.

In the voice continuation state, when Δθ (n)> θth and P (n) ≦ Pth1, the state transits to the end waiting state. In the voice / non-voice determination method of (2), in the voice section obtained by the above procedure, P (n)> Pth2
A section that satisfies is defined as a voice section. Here, Pth2 is the second threshold value of P (n) as described above.

In the method (2), when the SN ratio is low, P
If th and Pth2 are set to large values, the voice section may not be detected. Therefore, the values of Pth and Pth2 are set to smaller values than in the case of detection using only power. Even if Pth and Pth2 are set to small values, the value of the target sound source direction obtained is preferentially used, so that the voice detection performance can be reliably improved. For example, Pth,
The values of Pth1 and Pth2 are relative values P to the background noise level.
It is assumed that th = 5 dB, Pth1 = 2 dB, and Pth2 = 5 dB.
It is desirable that the values of Pth, Pth1 and Pth2 be determined experimentally according to the situation of the background noise level.

According to the present embodiment, the direction of the target sound source is obtained from the filter coefficient of the filter inside the beamformer instead of suppressing the noise by the beamformer, so that the direction of the noise source cannot be specified. It is possible to accurately detect the voice section of the target sound source even in a simple environment.

Next, another embodiment of the present invention will be described. In the block diagrams used in the following embodiments, blocks having the same name have basically the same function, and a detailed description thereof will be omitted.

(Second Embodiment) In this embodiment, even if there is a directional noise source, the beamformer for suppressing the signal of the target sound source can be extracted with high accuracy. A case will be described in which the input direction follows the direction of noise.

In order to make the direction of the noise source set by the beamformer follow the direction of the actual noise source, in the present embodiment, a second beamformer is provided separately from the first beamformer for suppressing the signal coming from the target sound source. A beamformer is provided, the direction of the noise source is estimated from the directivity of the filter in the second beamformer, and the first beamformer is controlled based on the estimation result.

FIG. 8 shows a configuration of a voice processing apparatus having a voice section detection function according to the present embodiment. In the present embodiment, processing for the case where the number of channels is 2 will be described as an example for simplicity, but the present invention is not limited to two channels.

The audio signals input from the input terminals 50-1 and 50-2 to the audio input unit 50 via the channels ch1 and ch2 are converted into first and second beamformers 61 and 62.
Respectively. Estimating the direction of the target sound source from the filter coefficients of the filter in the first beamformer 61,
The estimation result is provided to the first control unit 64. The noise source direction estimating unit 65 estimates the direction of the noise source from the filter coefficients of the filters in the second beamformer 62, and supplies the result to the second control unit 66.

The speech / non-speech deciding section 70 includes a time series of the direction of the target sound source estimated by the target sound source direction estimating section 63, a time series value of the power of the signal obtained from the speech input section 50, and a speech input section. Speech / non-speech is determined based on at least one of the time-series values of the correlation value between channels of the signal obtained from 50. Hereinafter, the first and second beamformers 6
The directions of the noise source and target sound source set in 1, 62 will be referred to as input directions.

The first controller 64 controls the second beamformer 62 so that the direction of the target sound source estimated by the target sound source direction estimator 63 is set as the input direction. The second control unit 66 controls the first beamformer 61 such that the direction of the noise source estimated by the noise source direction estimation unit 65 is set as the input direction. The reason why the input direction of the first beamformer 61 is set to the direction of the noise source is to prevent the direction of the noise source from being estimated by the first beamformer 61. Setting the direction to the direction of the target sound source is the second
This is to prevent the beamformer 62 from estimating the direction of the target sound source.

First and second beam formers 61 and 6
2 may be a GSC, a frost type, or a reference signal type as described above. In this case, the first beam former 61
In the filters inside, the sensitivity is lower in the direction of the target sound source and in the filter in the second beamformer 62, the sensitivity is lower in the direction of the noise source. By examining the characteristics, the directions of the target sound source and the noise source can be estimated.

The target sound source direction estimating unit 63 and the noise source direction estimating unit 65 estimate the directions of the target sound source and the noise source from the directivity of the filters in the first and second beamformers 61 and 62 as described above. Therefore, the processing is performed according to the procedure shown in FIG. Here, the search range of the arrival direction of the target sound source of the first beamformer 61 set by the initial setting is 2
The search range of the noise arrival direction of the second beamformer 62 is, for example, 90 °.

The control unit 64 and the control unit 66 update the input direction while averaging the estimated sound source direction with the output power of the beamformer and averaging with the past estimated sound source direction. To do. For example,
The calculation is performed according to the formula disclosed in Japanese Patent Application No. 9-9794. By such an update, it is possible to control so that the update is accelerated when the power of the signal from the target sound source is large and the power of the noise is small, and the update is delayed otherwise.

FIG. 9 shows an overall processing flow of the present embodiment including the above-described estimation processing. First, as an initial setting, a range Φ allowed as the direction of the target sound source is set, the input direction θ1 of the first beamformer 61 is set to, for example, 0 °, and the input direction θ2 of the second beamformer 62 is set to, for example, 90 °.
Then, the search range θr1 of the target sound source direction estimating unit 63 is set to, for example, 20 °, and the search range θr2 of the noise source direction estimating unit 65 is set to, for example, 90 ° (step S201).
Here, an allowable range Φ is provided in the direction of the target sound source so that a signal arriving within a certain angle range is regarded as a signal from the target sound source. The value of Φ is, for example, the same value as the search range θr1 of the first beam former 61, and Φ = θr1 = 20
°. As a direction reference, a direction perpendicular to a straight line connecting the two microphones is set to 0 ° as shown in FIG.

Next, the input direction of the first beam former 61 is set (step S202). Here, by delaying the signals of the two channels, signals from the set input directions are equivalently and simultaneously arrive at the array. Therefore, in the first beamformer 61, the first channel ch1 is output by the delay unit 26 shown in FIG.
Is calculated by τ = dsin (θ1) / c. Here, c is the speed of sound, and d is the distance between microphones.

Next, the processing of the first beamformer 61 is performed (step S203), and the direction of the target sound source is estimated within the search range ± θr1 from the obtained filter coefficients by the method described above (step S204). The direction of the estimated target sound source is defined as θn.

Next, it is determined whether or not the direction θn of the target sound source estimated in step S204 is in the vicinity (0 ° ± Φ) of the direction of the noise source (step S205). Proceed to S207.

On the other hand, if the direction θn of the target sound source estimated in step S204 is not close to the direction of the noise source, the input direction of the second beamformer 62 is set so that the estimated direction of the target sound source is set as the input direction. (Step S
206). That is, the value of θ2 is updated by the averaging described above. As in step S202, the second beamformer 62 delays the signal of the second channel ch2 as shown in FIG. Vessel 2
6, the delay given to the first channel ch1 is τ = ds.
It is calculated by in (θ2) / c.

Next, the processing of the second beamformer 62 is performed (step S207), the direction of the noise source is estimated within the search range ± θr2 (step S208), and the process returns to step S202 again. The input direction of the first beamformer 61 is set so that the direction of the noise source is set as the input direction. Also at this time, the input direction is updated by the averaging described above. Thereafter, the above processing is repeated.

The voice / non-voice determination section 70 determines voice / non-voice according to the processing procedure shown in FIGS. As the specific determination method, the two methods described in the first embodiment can be considered, but the description is omitted because they are duplicated.

As described above, according to this embodiment, two beamformers are provided, one of the beamformers estimates the direction of the target sound source, and the other beamformer estimates the direction of the noise source. Even if there is a noise source with a possibility, the voice section of the target sound source can be accurately detected.

(Third Embodiment) In the present embodiment, the second
In the configuration using two beamformers described in the first embodiment, a method of extracting a target voice with high accuracy by performing voice enhancement instead of detecting a voice section will be described. FIG. 10 shows the configuration of the present embodiment.

The sound processing apparatus shown in FIG. 10 includes a sound input unit 80 for inputting sound via a plurality of channels, a first beamformer 91 for filtering the input sound and suppressing a signal from a target sound source, and an input sound. , A second beamformer 9 for extracting a target voice by suppressing noise
2. A target sound source direction estimating unit 93 for estimating the target sound source direction from the filter coefficients of the first beamformer 91, and sets the target direction of the second beamformer 92 to the target sound source direction estimated by the target sound source direction estimating unit. First control unit 9
4. a noise source direction estimating unit 95 for estimating the noise source direction from the filter 92 of the second beamformer; a second control unit 96 for setting the target direction of the first beamformer 91 to the estimated noise source direction; It comprises a voice emphasizing unit 100 for suppressing noise components in the output signal of the second beamformer 92 and emphasizing voice.

In this configuration, the voice / non-voice determination unit 70 in the configuration of the second embodiment shown in FIG. In the second embodiment, the output signal of the beamformer 91 is not used, but in the present embodiment, the signal is used as a signal for noise reference in voice enhancement to perform voice enhancement processing.

As mentioned earlier, there are many noise sources,
In an environment where the direction of the noise source cannot be specified, the noise suppression performance of the beamformer decreases, but since the input speech has directivity, the signal from the target sound source is suppressed by the beamformer that sets the target direction in the noise direction. The output of only the noise can be extracted. Therefore, the output of the beamformer 91 is a signal containing only noise, and the signal is used to emphasize the sound using a well-known technique of spectral subtraction (SS). For details of the spectral subtraction, see, for example, Reference 4: S. Boll: “Su
ppression of acoustics noise in speech using spect
ral subtraction ", IEEE Trans., ASSP-27, No.2, p
pp. 113-120, 1979 ”.

For the spectral subtraction, 2chSS using two channels of a reference noise signal and a voice signal is used.
And 1chSS using only one-channel audio signal. In the present embodiment, audio enhancement is performed by 2chSS using the output of beamformer 91 as reference noise. Normally, as a noise signal of 2chSS, a signal of a microphone separated from the microphone for collecting the target voice is used so that the target voice is not input. However, the noise signal has a different characteristic from the noise mixed in the microphone for collecting the target voice. As a result, there is a problem that the accuracy of the SS decreases.

On the other hand, in the present embodiment, the microphone dedicated to noise collection is not used, and the noise signal is extracted from the microphone for voice collection. It can be performed. The only difference from the second embodiment is the 2chSS part, and the other parts are the same.
S will be described.

The 2chSS has, for example, a configuration as shown in FIG. 13, and performs the processing shown in FIG. 13 for each block by performing block processing on input data. The 2chSS illustrated in FIG. 13 includes a first FFT 101 that performs a Fourier transform on a noise signal, a first FFT 101,
A first band power converter 102 for converting a frequency component obtained by the FT into a band power, a noise power calculator 103 for averaging the obtained band power in a time direction, and a second FFT 104 for performing a Fourier transform on the audio signal , The second FF
A second band power converter 105 for converting the frequency component obtained by T into band power, a voice power calculator 106 for averaging the obtained band power in the time direction, and a calculation based on the obtained noise power and voice power. A band weight calculation unit 107 for calculating a weight for each band, a weighting unit 108 for weighting a frequency spectrum obtained from the audio signal by the second FFT with a weight for each band, and performing inverse FFT on the weighted frequency spectrum to generate a voice. Output inverse FFT section 109
Consists of

The block length is, for example, 256 points, and the FFT
Match the score of. At the time of FFT, windowing is performed using, for example, a Hanning window, and the same processing is repeated while shifting by 128 points, which is half the block length. Finally, reverse F
The waveform of the processing result obtained by the FT is added while overlapping 128 points at a time to restore the deformation due to the windowing and output.

For conversion into band power, for example, as shown in Table 1, frequency components are divided into 16 bands, and the sum of squares of the frequency components is calculated for each band to obtain band power.
The calculation of the noise power and the voice power is, for example, 1 for each band.
The following regression filter is used to perform the following equation.

P _{k, n} = a · pp _k + (1-a) · p _{k, n−1} (5) v _{k, n} = a · vv _k + (1-a) · v _{k, n−1} (6) Here, k is the band number, n is the scent of the block, p is the band power of the averaged noise channel, pp is the band power of the current block of the noise channel, and v is the average of the voice channel. , Vv is the band power of this block of the voice channel, and a is a constant. As the value of a, for example, 0.5 is used.

Next, the band weight calculation unit calculates the weight w _{k, n} for each band by the following equation using the obtained noise and the band power of the voice. w _{k, n} = | v _{k, n} -p _{k, n} | / v _{k, n} (7) Next, using the weight for each band, the frequency component of the voice channel is weighted by the following equation, for example. Y _{i, n} = X _{i, n} · w _{k, n} (8) where Y _{i, n} is a weighted frequency component, X _{i, n} is a frequency component obtained by the second FFT of the voice channel, i is the frequency component number, and the weight w _{k, n} of the band k corresponding to the frequency component number i in Table 1 is used.

[0069]

[Table 1]

The flow of the processing of the voice emphasizing unit by 2chSS will be described with reference to FIG. First, make the initial settings,
For example, the block length = 256, the number of FFT points = 256, the number of shift points = 128, and the number of bands = 16 (step S30).
1). Next, in the first FFT, data of the noise channel is read, windowing and FFT are performed, and a frequency component of noise is obtained (step S302). Next, the second F
The voice channel data is read in the FT, windowing and FFT are performed, and a frequency component of the voice is obtained (step S303). Next, the first band power converter calculates the band power of the noise from the frequency components of the noise in accordance with the correspondence shown in Table 1 (step S304). Next, the second
The band power conversion unit calculates the band power of the sound from the frequency components of the sound according to the correspondence in Table 1 (step S305). Next, in the noise power calculation unit, an average noise power is obtained according to the equation (5) (step S3).
06). Next, the audio power calculation unit obtains an average audio power according to the equation (6) (step S307).
Next, the band weight calculation unit obtains the band weight according to the equation (7) (step S308). Next, the weighting unit processes the frequency component of the voice in step S308.
Is weighted according to equation (8) (step S309). Next, in the inverse FFT unit, the frequency component weighted in step S309 is inverse FFT to obtain a waveform, and the last one of the waveforms obtained up to the previous block is obtained.
The output is superimposed on 28 points (step S31)
0).

Steps S302 to S310 are repeated until there is no input. In this case, it is convenient to perform the block processing in synchronization with the entire processing including the processing of the beamformer. In this case, the block length of the beamformer is set to be equal to the shift length of 128 points of the voice emphasizing unit. .

(Fourth Embodiment) FIG. 11 shows an audio processing apparatus according to this embodiment. In the third embodiment, two beamformers are used to control their target directions to the noise source direction and the target sound source direction, respectively.
If the target sound source and the noise source are fixed and their directions are known, there is no need to control the target direction of the beamformer. Therefore, as in this embodiment, the target sound source direction estimating unit 93 of FIG. It is also possible to adopt a configuration in which the second control units 94 and 96 are omitted. In this case, the first beamformer 121 is directed toward the strongest noise source, and the second beamformer 122 is directed toward the target sound source. Since the processing in this case can be easily performed simply by omitting the sound source direction estimating unit and the target direction control unit of the beamformer in the second embodiment, detailed description will be omitted.

(Fifth Embodiment) FIG. 12 shows a configuration of a speech processing apparatus having a speech enhancement processing function according to the present embodiment. If there is no noise source stronger than the target voice, the second beamformer for suppressing noise as in the present embodiment can also be omitted. Also in this case, since the processing of the second beamformer is only omitted, it can be easily implemented and will not be described again.

(Sixth Embodiment) FIG. 15 shows the configuration of a speech processing apparatus having a speech section detection function according to this embodiment. In the second embodiment, a method for improving voice section detection performance in a noise environment by using a target sound source direction obtained from a filter of a first beamformer for suppressing a signal from a target sound source for voice section detection. As described above, the present embodiment can further improve the voice section detection performance by detecting the voice section using both the target sound source direction and the output of the voice enhancement processing described in the third embodiment. It was made.

As shown in FIG. 15, the present embodiment has a configuration in which the speech / non-speech determination unit 70 described in the second embodiment is added to the configuration of the third embodiment. The feature of the present embodiment is that the output of the voice enhancement unit 190 after the voice enhancement processing is used instead of the output of the second beamformer used in the second embodiment.

As described above, the output of the first beamformer for suppressing the signal from the target sound source is used as a noise signal for 2ch.
By performing voice enhancement processing by SS, the conventional 2c
Noise can be suppressed more accurately than hSS, and voice section detection based on voice enhancement output and target sound source direction can greatly improve voice section detection performance under non-stationary noise.

The parameters used for detection in the above speech section detection are not limited to the output power of the beamformer and the direction of the target sound source. For example, the number of zero crossings, the slope of the spectrum, the LPC cepstrum, the Δ-cepstrum,
It is also possible to use parameters such as Δ2-cepstrum, LPC residual, autocorrelation coefficient, reflection coefficient, logarithmic cross-sectional area ratio, pitch, etc., and a combination thereof.

[0078]

As described above, according to the present invention, S
In an environment where the N ratio is low and the direction of the noise source cannot be specified, accurate detection of the voice section of the target sound source and voice enhancement processing can be performed.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an audio processing device according to a first embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of an adaptive beamformer processing unit according to the embodiment;

FIG. 3 is a block diagram showing a configuration of a beamformer in which a delay unit is inserted on the input side of one channel.

FIG. 4 is a flowchart showing a procedure of a sound source direction estimation process in the embodiment;

FIG. 5 is an explanatory diagram of a time delay between signals from two microphones.

FIG. 6 is a state transition diagram showing a processing flow in a first method for determining voice / non-voice in the embodiment.

FIG. 7 is a state transition diagram showing a processing flow in a first method for determining voice / non-voice in the embodiment.

FIG. 8 is a block diagram showing a configuration of an audio processing device according to a second embodiment of the present invention.

FIG. 9 is a flowchart showing a flow between processes in the embodiment;

FIG. 10 is a block diagram showing a configuration of an audio processing device according to a third embodiment of the present invention.

FIG. 11 is a block diagram showing a configuration of an audio processing device according to a fourth embodiment of the present invention.

FIG. 12 is a block diagram showing a configuration of an audio processing device according to a fifth embodiment of the present invention.

FIG. 13 is a block diagram showing a configuration of a speech enhancement unit using two-channel spectral subtraction.

FIG. 14 is a flowchart showing a processing procedure of a speech enhancement unit based on two-channel spectral subtraction;

FIG. 15 is a block diagram showing a configuration of an audio processing device according to a sixth embodiment of the present invention.

[Explanation of symbols]

 10-1 to 10-n Audio signal input terminal 10 Audio input unit 20 Beamformer processing unit 21 Subtractor 22 Adder 23 Delay unit 24 Adaptive filter 25 Subtractor 26 Delay unit 27 Beam Former body 30 Target sound direction estimation unit 40 Voice / non-voice determination unit 50-1 to 50-n Voice signal input terminal 50 Voice input unit 61 First beamformer 62 Second beamformer 63 Target sound source direction estimating unit 64 first control unit 65 noise source direction estimating unit 66 second control unit 70 voice / non-voice determining unit 80-1 to 80-n voice signal input terminal 80 voice input Unit 91: First beamformer processing unit 92: Second beamformer processing unit 93: Target sound source direction estimating unit 94: First control unit 95: Noise source direction estimating unit 96: Second control unit 100: Sound Emphasis unit 101 FFT unit 102 Band power conversion unit 103 Noise power calculation unit 104 FFT unit 105 Band power conversion unit 106 Voice power calculation unit 107 Band weight calculation unit 108 Weighting unit 109 Inverse FFT unit 110 -1 to 110-n audio signal input terminal 110 audio input unit 121 first beamformer processing unit 122 second beamformer processing unit 130 audio enhancement unit 140-1 to 140-n audio signal input Terminal 140: Audio input unit 150: First beamformer processing unit 160: Audio enhancement unit 170-1 to 170-n Audio signal input terminal 170: Audio input unit 181: First beamformer processing unit 182: Second Beamformer processing unit 183 ... target sound source direction estimating unit 184 ... first control unit 185 ... noise source direction estimating unit 186 The second control unit 190 ... sound enhancement unit 200 ... voice / non-voice determining section

Claims (14)

[Claims] An audio input step of inputting an audio signal through a plurality of channels, and a beamformer process for suppressing a signal coming from a target sound source is performed on the audio signal input in the audio input step. A beam former processing step, a target sound source direction estimating step of estimating a direction of a target sound source from a filter coefficient obtained in the beam former processing step, and a direction of the target sound source estimated by the target sound source direction estimating step. A voice section determining step of determining a voice section of the voice signal. 2. An audio input step of inputting an audio signal through a plurality of channels, and performing a beamformer process on the audio signal input in the audio input step to suppress a signal coming from a target sound source. A first beamformer processing step, a target sound source direction estimating step of estimating a direction of a target sound source from a filter coefficient obtained in the first beamformer processing step, and a speech signal input in the speech input step. A second beamformer processing step of performing a beamformer processing for suppressing a signal coming from a noise source and outputting a signal from a target sound source, and a filter coefficient obtained in the second beamformer processing step. A noise source direction estimating step of estimating a direction of a noise source, and the target sound source direction estimating step A first control step of controlling the second beamformer processing step based on the estimated direction of the target sound source and an output power obtained by the first and second beamformer processing steps; A second beamformer processing step for controlling the first beamformer processing step based on the noise source direction estimated by the noise source direction estimation step and the output power obtained by the first and second beamformer processing steps; A voice processing method, comprising: a control step; and a voice section determining step of determining a voice section of the voice signal based on the direction of the target sound source estimated in the target sound source direction estimation step. 3. The voice section determining step determines the voice section of the voice signal based on the direction of the target sound source estimated in the target sound source direction estimating step and the power of the voice signal. Item 3. The audio processing method according to item 1 or 2. 4. An audio input unit for inputting an audio signal through a plurality of channels, and performing a beamformer process on the audio signal input by the audio input unit for suppressing a signal coming from a target sound source. A beamformer; a target sound source direction estimating means for obtaining a direction of a target sound source from a filter coefficient obtained by the beamformer; and a voice section of the voice signal based on the direction of the target sound source estimated by the target sound source direction estimating means. And a voice section determining means for determining the voice section. 5. An audio input unit for inputting audio via a plurality of channels, and a beamformer for applying a beamformer process to the audio signal input by the audio input unit for suppressing a signal coming from a target sound source. 1 beamformer, target sound source direction estimating means for estimating the direction of a target sound source from the filter coefficients obtained by the first beamformer, and a sound signal input by the sound input means coming from a noise source. Beamformer for performing a beamformer process for suppressing a signal to be output and outputting a signal from a target sound source, and a noise source for estimating a direction of the noise source from a filter coefficient obtained by the second beamformer Direction estimating means, the direction of the target sound source estimated by the target sound source direction estimating means, and the output power of the first and second beamformers. A first control unit for controlling the second beamformer based on the direction of the noise source estimated by the noise source direction estimating unit and output powers of the first and second beamformers. Second control means for controlling the first beamformer, and voice section determining means for determining a voice section of the voice signal based on the direction of the target sound source estimated by the target sound source direction estimating means. An audio processing device characterized by performing. 6. The voice section determining means determines the voice section of the voice signal based on the direction of the target sound source estimated by the target sound source direction estimating means and the power of the voice signal. Item 6. The audio processing device according to item 4 or 5. 7. A voice input step of inputting voice via a plurality of channels, and a beamformer process for suppressing a signal coming from a target sound source on the voice signal input in the voice input step. A beam source processing step, a target sound source direction estimating step of estimating a target sound source direction from a filter coefficient obtained by the first beam former processing, and a noise source for a voice signal input in the voice input step. A second beamformer processing step of suppressing a signal arriving from the above and performing a beamformer processing for outputting a signal from a target sound source; and determining a noise source direction from a filter coefficient obtained by the second beamformer processing. A noise source direction estimation step for estimating, and a target sound source estimated by the target sound source direction estimation step A first control step of controlling the second beamformer processing step based on the direction and output powers of the first and second beamformer processing; and a noise source direction estimated by the noise source direction estimation step. A second control step of controlling the first beamformer processing step based on the output power obtained by the first and second beamformer processing steps; and a first beamformer processing step And a voice emphasizing step of suppressing noise in the output obtained by the second beamformer processing step and enhancing voice based on at least one of the output obtained by the above and the target sound source direction. The audio processing method to do. 8. A voice section detection step of detecting a voice section of the voice signal based on the target sound source direction estimated by the target sound source direction estimation step and a voice signal in which voice is emphasized by the voice enhancement step. The audio processing method according to claim 7, further comprising: 9. An audio inputting step of inputting an audio signal through a plurality of channels, and performing a beamformer process on the audio signal input in the audio inputting step to suppress a signal coming from a target sound source. A second beamformer processing step of performing a first beamformer processing step and a beamformer processing for suppressing a signal coming from a noise source with respect to the audio signal input in the audio input step and outputting a signal from a target sound source. A beamformer processing step; and a speech enhancement step of suppressing noise in the output obtained by the second beamformer processing step based on the output obtained by the first beamformer processing step to enhance voice. A voice processing method comprising: 10. An audio input step of inputting an audio signal through a plurality of channels, and performing a beamformer process on the audio signal input in the audio input step to suppress a signal coming from a target sound source. A beamformer processing step, and voice enhancement for suppressing noise in a voice signal input via any one of the plurality of channels based on an output obtained in the beamformer processing step to enhance voice. And a voice processing method. 11. An audio input unit for inputting an audio signal through a plurality of channels, and performing a beamformer process on the audio signal input by the audio input unit to suppress a signal coming from a target sound source. A first beamformer; a target sound source direction estimating means for estimating a target sound source direction from a filter coefficient obtained by the first beamformer; a sound signal input by the sound input means coming from a noise source Beamformer for performing a beamformer process for suppressing a signal to be output and outputting a signal from a target sound source, and a noise source direction for estimating a noise source direction from a filter coefficient obtained by the second beamformer Estimating means, a target sound source direction estimated by the target sound source direction estimating means, and an output power of the first and second beamformers. First control means for controlling the processing of the second beamformer based on the above, and the noise source direction estimated by the noise source direction estimating means and the output power of the first and second beamformers. Second control means for controlling the processing of the first beamformer based on at least one of an output of the first beamformer and a target sound source direction estimated by the target sound source direction estimating means. A voice emphasis unit that suppresses noise in the output of the second beamformer and emphasizes voice. 12. A voice section detecting means for detecting a voice section of the voice signal based on a target sound source direction estimated by the target sound source direction estimating means and a signal in which voice is emphasized by the voice emphasizing means. The audio processing device according to claim 11, wherein: 13. An audio input means for inputting an audio signal via a plurality of channels, and performing a beamformer process on the audio signal input by the audio input means for suppressing a signal coming from a target sound source. A first beamformer, and a second beamformer for performing a beamformer process for suppressing a signal coming from a noise source with respect to the audio signal input by the audio input unit and outputting a signal from a target sound source And a voice emphasizing unit that suppresses noise in the output of the second beamformer based on the output of the first beamformer and emphasizes voice. 14. An audio input unit for inputting an audio signal via a plurality of channels, and performing a beamformer process on the audio signal input by the audio input unit for suppressing a signal coming from a target sound source. A beamformer; and a voice emphasis unit that suppresses noise in a signal input through one of the plurality of channels based on an output of the beamformer, and emphasizes and outputs voice. An audio processing device characterized by the above-mentioned.