JP4313728B2 - Voice recognition method, apparatus and program thereof, and recording medium thereof - Google Patents

Description

  The present invention is used in a device that performs voice recognition on a voice signal picked up by a microphone, such as a voice response device, and emits a voice synthesis signal corresponding to the recognition result from a speaker. Speech recognition method for recognizing input speech signal by obtaining likelihood of probability model modeled from feature vector for each recognition category for feature vector series of input speech signal, device and program thereof, and recording medium thereof About.

  In conventional speech recognition, modeling is performed using a stochastic model such as a Hidden Markov Model (hereinafter referred to as HMM) for each speech recognition category such as phonemes, syllables, and words constituting a recognition result candidate. The method has high recognition performance and has become the mainstream of current speech recognition technology. A conventional speech recognition apparatus using an HMM will be briefly described with reference to FIG. The audio signal input from the input terminal 11 is converted into a digital signal by the A / D converter 12. A feature vector extraction unit 13 extracts a speech feature vector from the digital signal. For each recognition category, the HMM created for the speech unit is read from the model memory 14 in advance, and the likelihood calculation unit 15 calculates the matching likelihood of each model with respect to the extracted speech feature vector. The speech unit (recognition category) represented by the model showing the largest matching likelihood is output from the output unit 16 as a recognition result. Corresponding portions in the specification and the drawings are denoted by the same reference numerals, and redundant description is omitted.

  Two methods for recognizing speech on which additive noise such as background noise is superimposed will be described. The first is a method of recognizing after suppressing the noise superimposed on the input speech. Various noise suppression methods have been proposed. Here, a spectral subtraction method (hereinafter referred to as SS method) will be described (for example, see Non-Patent Document 1). Since two signals that are additive in the time domain are also additive on the linear power spectrum, the SS method subtracts the estimated noise component on the linear power spectrum from the noise-superimposed speech signal to extract the speech component. .

A speech recognition apparatus using the SS method will be briefly described with reference to FIG. The voice / noise determination unit 21 determines whether the input voice signal that is a digital signal is noise or noise-superimposed voice. If the determination is noise, the determination unit 21 connects the voice / noise switch 22 to the noise terminal 22a side, and connects the output side of the A / D conversion unit 12 to the average noise power spectrum calculation unit 23. The average power spectrum in the noise interval in the input speech signal is calculated. When the determination unit 21 determines that it is a noise-superimposed speech section to be recognized, the speech / noise switch 22 is switched to the speech terminal 22b side, and the output side of the A / D conversion unit 12 is connected to the noise-superimposed speech power spectrum. It connects to the calculation part 24 and calculates the power spectrum of the noise superimposed voice in the input voice signal. In the suppression processing unit 25, the average noise power spectrum is subtracted from the power spectrum of the noise superimposed speech at each time. The power spectrum Y ^D (t, f) after noise suppression at the frequency f of the power spectrum at time t is calculated as follows.

D (Y (t, f)) = Y (t, f) −αN ^ (f)
Y ^D (t, f) = D (Y (t, f)): D (Y (t, f))> βY (t, f) Y ^D (t, f) = βY (t, f) Other cases (1)
Here, Y (t, f) is the time t of the input noise superimposed speech, the power spectrum of the frequency f,
N ^ (f) is the time average noise power spectrum of the estimated frequency f,
α is a subtraction coefficient and is usually larger than 1.
β is a flooring coefficient and is smaller than 1.

A feature parameter for speech recognition (for example, a 12-dimensional Mel-Frequency Cepstrum Coefficient (MFCC)) is calculated by the feature vector extraction unit 13 from the power spectrum output from the suppression processing unit 25. The subsequent processing is as described with reference to FIG.
As a second example, recognition of noise superimposed speech by the HMM synthesis method will be described. Noise data that is expected to be superimposed on the recognition target speech signal is superimposed on a clean speech learning data set that does not contain noise, an HMM is created, and the obtained HMM is used to generate a noise superimposed speech signal. On the other hand, if speech recognition is performed, high recognition performance can be obtained.

However, noise in the surrounding environment where voice recognition is used varies, and it is difficult to predict in advance. Furthermore, the amount of data of the clean speech learning data set for creating the HMM is enormous, and therefore, for example, 100 hours is used to create a noise superimposed speech model by superimposing noise data that seems to be superimposed. It takes a long calculation time. Therefore, it is not realistic to record the noise of the surrounding environment where voice recognition is used at the time of recognition and create and use the HMM because it takes a long processing time to create the HMM.
Therefore, for example, as shown in Patent Document 1, a clean speech HMM is created in advance based on a large amount of clean speech learning data set without noise, and a noise HMM is created by observing background noise during recognition. Synthesize with voice HMM. The obtained noise superimposed speech HMM is an approximation of a speech model including background noise at the time of recognition, and is recognized using this. The processing time required for noise model creation and model synthesis is several seconds to several tens of seconds. Since an HMM that is a probabilistic model is used, it is possible to take into account voice fluctuations and noise fluctuations.
Japanese Patent No. 3247746 Steven F. Boll: “Suppression of Acoustic Noise in Speech Using Spectral Subtraction,” IEEE Transactions on Acoustics, Speech and Signal Processing, Vol. ASSP-27, No. 2, pp. 113-120, April 1979

For example, in a voice response device between a human and a machine using a voice recognition method, a voice or sound for guidance to a user is often emitted from a speaker installed in the device. In such a device configuration, a voice recognition microphone installed in the device may circulate not only the surrounding background noise but also the guidance sound generated by the voice response device itself and be input as an echo. In many cases, this also becomes noise for the speech recognition apparatus as well as ambient noise. These noises cause deterioration in speech recognition performance.
The present invention has been made in view of the above, and an object of the present invention is not only ambient noise but also speech and sound wraparound echoes generated by a speech synthesizer used together with a speech recognition device such as a speech response device. To provide a speech recognition method, apparatus, program, and recording medium with high recognition performance regardless of existence.

The present invention is a speech recognition method for performing speech recognition using a probability model for a speech signal collected by the microphone in an apparatus that collects an acoustic signal by a microphone and emits the acoustic signal from a speaker,
The sound emission signal component from the speaker (hereinafter referred to as echo signal) in the input signal from the microphone is suppressed using the supply signal to the speaker, and the noise signal component in the input signal is suppressed. The suppressed signal is converted into a feature vector sequence, and it is determined whether or not the feature vector sequence is a speech section including a speech signal to be recognized. If the determination is not a speech section, noise is generated using the feature vector sequence. The model is learned, and the noise model and a clean speech model created in advance using clean speech data without noise are combined to generate a noise superimposed speech model. The likelihood for the recognition category is calculated using the vector sequence and the noise superimposed speech model, and the recognition result is output based on the calculated likelihood.

  According to the present invention, not only the noise signal component in the input signal from the microphone is suppressed, but also the echo signal is suppressed using the signal supplied to the speaker, and a noise model is generated from the signal subjected to both the suppressions. Since the speech recognition is performed using the synthesis model for the noise superimposed speech signal which is synthesized with the clean speech model, echo and noise are suppressed, and the S / N (signal-to-noise ratio) is improved, It is difficult to be affected by not only environmental noise but also echo, and a high recognition rate can be obtained. Moreover, since the noise model and the clean speech model are combined to form a noise superimposed speech model, the noise superimposed speech model can be created in a short time.

[First Embodiment]
An example of the functional configuration of the first embodiment of the present invention is shown in FIG. 3, and an example of the processing procedure is shown in FIG. The present invention is applied to voice recognition in a voice response device, for example. That is, a guidance sound such as a guidance voice for inducing speech to the user of the voice response device or a “beep” sound that prompts the user to speak is emitted from the speaker 31. In order to emit this guidance voice or guidance sound (hereinafter referred to as system voice), a digital system voice signal is synthesized by the output system voice generation unit 32, and this digital system voice signal is converted into an analog signal by the voice reproduction unit 33. It is converted into a system audio signal and supplied to the speaker 31.

The voice uttered by the user is collected by the microphone 34, and the collected voice signal is input to the A / D conversion unit 12 through the input terminal 11. Ambient noise is picked up by the microphone 34, and a wraparound echo of the system sound emitted from the speaker 31 is picked up. That is, the input signal supplied from the microphone 34 to the input terminal 11 is obtained by superimposing the ambient noise signal and the echo signal on the speech signal to be recognized by the user.
In the first embodiment, the digital input signal from the A / D converter 12 and the digital system voice signal from the output system voice generator 32 are input to the echo / noise suppressor 35, and the echo / noise suppressor 35 receives the input signal. The ambient noise signal and the echo signal superimposed on it are suppressed (step S1). In this example, the echo signal is first suppressed by the system voice signal in the echo unit 35a (step S1a). For this echo suppression, for example, a method of an echo canceller (echo canceller) used in a telephone conference system or a video conference system can be used. For example, the transfer characteristic from the speaker 31 through the microphone 34 to the echo unit 35a is adaptively estimated, the estimated transfer characteristic is convoluted with the system sound signal, a pseudo echo signal is generated, and the pseudo echo signal is subtracted from the input signal. To obtain an echo-suppressed input signal.

Next, the echo-suppressed input signal is input to the noise unit 35b, and the ambient (background) noise component superimposed on the input signal is suppressed (step S1b). In this noise suppression, for example, the average minimum level in the input signal is regarded as the background noise level, and signals below this level are removed.
Further, in this example, the echo-suppressed and noise-suppressed signal and the digital system voice signal are input to the residual echo unit 35c, and can be removed by the echo unit S1a affected by background noise such as an echo signal other than the background noise level. The residual echo signal that did not exist is removed from the echo and noise-suppressed input signal (step S1c). For the residual echo suppression, the same technique as that used in, for example, a video conference system can be used. For example, as shown in Japanese Patent No. 3420705, Japanese Patent No. 3507020, and Japanese Patent Application Laid-Open No. 2003-284183, an acoustic (echo path) coupling amount is obtained from an input signal and a system audio signal, and accordingly, In other words, it is only necessary to suppress, that is, to give a loss to the echo and noise-suppressed input signal.

  The input signal subjected to the echo and noise suppression processing from the echo / noise suppression unit 35 is input to the feature vector extraction unit 36, and the feature vector is converted into a probability model, in this example, a feature vector sequence necessary for HMM learning ( Step S2). This feature vector series is input to the section determination unit 37, and from the feature vector series, the current input signal is only a noise signal component, that is, a noise section of only an ambient noise signal or an echo signal, or a noise signal component. It is determined which one of the voice sections of the noise superimposed voice signal is superimposed with the voice signal (step S3).

If the determination result output is input to the voice / noise switch 38 from the section determination unit 37 and the determination result output is for the noise section, the switch 38 is switched to the terminal 38 _N side, and the feature vector from the feature vector extraction unit 36 is displayed. The sequence is input to the noise model learning unit 39. The noise model learning unit 39 learns a plurality of analysis frames of the input feature vector to generate a noise HMM (step S4). This noise HMM corresponds to the echo and noise-suppressed ambient noise signal or this and the echo signal.
The clean speech model memory 41 stores clean speech HMMs learned for each recognition category in units of speech to be recognized based on a large number of clean speech data free from noise. The clean speech HMM and noise HMM are input to the model synthesis unit 42, and these HMMs are synthesized and stored in the noise superimposed speech model memory 43 as noise superimposed speech HMM (step S5).

If the determination result output from the section determination unit 32 is for the voice section, the voice / noise switch 38 is switched to the terminal 38 _S side, and the noise-superimposed voice subjected to echo and noise suppression processing from the feature vector extraction unit 36. The feature vector sequence of the signal is input to the likelihood calculation unit 44. The likelihood calculating unit 44 calculates the likelihood of each noise superimposed speech model in the noise superimposed speech model memory 43 for the input feature vector series (step S6). The likelihood calculated for each recognition category is input to the output unit 16, and the recognition category of the maximum model in the input likelihood is output as a recognition result (step S7).

  The generation of the noise HMM may be performed by emitting the system sound during a preparation period (during the addition) for operating the voice response device, or may be performed in a section before the user utters. In the latter case, it may be always performed immediately before each utterance of the user. In this case, the noise superimposed speech in the noise superimposed speech model memory 43 is generated by the noise superimposed speech model synthesized by the model synthesis unit 43. The model is updated (step S5). In this way, even if the position of the user with respect to the voice response device changes, the influence of S / N (signal-to-noise ratio) is small, and it becomes better by estimating the echo path, and the recognition rate is improved.

  As described above, according to the first embodiment, the echo signal is suppressed by the echo / noise suppression unit 35, and the noise model is generated from the feature vector sequence of the input signal that has been echoed and suppressed in the noise period. In addition, since the likelihood of the noise-superimposed speech HMM is calculated for the feature vector sequence of the noise-superimposed speech signal that is echo- and noise-suppressed in the speech section and has an improved S / N (signal to noise ratio), the noise model is Since learning, it is possible to always generate a noise model corresponding to the ambient (background) noise in the place without predicting and generating the noise of the environment used in advance, and the state of the ambient noise changes In addition, a noise model corresponding to this is obtained, and the recognition rate is improved. Furthermore, since the noise superimposed speech model is generated by synthesizing the noise model and the clean speech model, the processing time is short.

[Second Embodiment]
An example of the functional configuration of the second embodiment of the present invention is shown in FIG. 5, and an example of the processing procedure is shown in FIG. Differences from the first embodiment will be described.
The feature vector series from the feature vector extraction unit 36 is input to the section determination unit 51. The section determination unit 51 includes an echo indicating whether the system sound is being emitted from the output system sound generation unit 32. A signal is also input. The section determination unit 51 determines whether the current input signal is a noise section including only the surrounding (background) noise signal or a noise / echo section including the surrounding (background) noise signal and the echo signal based on the input feature vector series and the echo presence / absence signal. It is determined whether the noise section or the voice section of the noise superimposed voice signal in which this and the echo signal are superimposed. For example, after step S2, it is determined whether the section determination result is a voice section (step S11), and if it is not a voice section, it is determined whether it is a noise section (step S12).

Switch 52 by the decision result output it is determined that the noise section is switched to the terminal 52 _S, a feature vector sequence of from the feature vector extraction unit 36 is input to the noise model learning unit 53, the noise model learning unit 53 is inputted, wherein A noise HMM corresponding to the ambient noise signal subjected to noise and echo suppression processing based on the vector sequence is learned (step S13).
Switch 52 is switched to the terminal 52 _E by the judgment result output which is determined as a noise-echo interval, the feature vector series from the feature vector extraction section 36 is inputted to the noise echo model learning unit 54, the noise echo learning section 54 Learns the noise / echo HMM corresponding to the superposed signal of the ambient noise signal and echo signal subjected to noise and echo suppression processing based on the input feature vector sequence (step S14).

The noise HMM from the noise model learning unit 53 and the noise / echo model from the noise / echo model learning unit 54 are input to the model synthesis unit 55, and these are combined with the clean speech HMM from the clean speech model memory 41, respectively. Thus, the noise superimposed speech HMM is generated and stored in the noise superimposed speech model memory 43, or the stored content is updated (step S15).
Switch 52 is switched to the terminal 52 _S by the judgment result output is determined that the speech segment, a feature vector sequence from the feature vector extraction unit 36 is input to the likelihood calculating unit 44. Others are the same as the first embodiment.

According to this configuration, when the user utters in a state where the system voice is not emitted, the likelihood using the noise superimposed voice HMM obtained by synthesizing the noise HMM and the clean voice HMM is increased, and the user can When the system voice is uttered, the likelihood of using the noise superimposed voice HMM synthesized from the noise / echo HMM and the clean voice HMM becomes high, and the input signal and the recognition model are more reliable. Since they match, a higher recognition rate can be obtained.
[Modification]
In the first and second embodiments, the input signal is subjected to echo suppression processing, noise suppression processing is performed, and residual echo suppression processing is then performed. As shown by broken lines in FIGS. Suppression may be omitted. In this case, as shown in parentheses in these drawings, noise suppression processing may be performed first, and then echo suppression processing may be performed.

  As a noise suppression method, as shown in, for example, Japanese Patent No. 3309895, Japanese Patent No. 3454402, Japanese Patent No. 3459363, etc., an input signal is converted into a frequency domain signal and divided into a plurality of frequency bands. Noise suppression may be performed on the signal in the corresponding frequency band of the input signal while estimating the noise component for each frequency band. In this way, the possibility that the recognition target speech signal is suppressed more than necessary for a certain band or that noise suppression is insufficient is reduced, S / N is improved, and a higher recognition rate is obtained. Will be.

The echo suppression method and the residual echo suppression method are also more effective when converted into the frequency domain. Further, the model is not limited to the HMM, and other probability models may be used.
The apparatus shown in FIGS. 3 and 5 may be operated by a computer. In this case, a voice recognition program for causing a computer to execute the steps of the processing procedure shown in FIG. 4 or FIG. 6 is installed in a computer from a recording medium such as a CD-ROM, a magnetic disk device, or a semiconductor storage device, or The program may be downloaded via a communication line and executed by a computer.

The block diagram which shows the function structure of the conventional speech recognition apparatus. The block diagram which shows the function structure of the speech recognition apparatus using the conventional spectrum subtraction method. The block diagram which shows the function structural example of 1st Embodiment of this invention apparatus. The flowchart which shows the process sequence example of 1st Embodiment of this invention method. The block diagram which shows the function structural example of 2nd Embodiment of this invention apparatus. The flowchart which shows the process sequence example of 2nd Embodiment of this invention method.

Claims (6)

A speech recognition method for performing speech recognition using a probability model for a sound signal collected by the microphone in a device that collects an acoustic signal by a microphone and emits the acoustic signal from a speaker,
Noise that suppresses a sound emission signal component (hereinafter referred to as echo signal) from the speaker in the input signal from the microphone by using a supply signal to the speaker and suppresses an ambient noise signal in the input signal.・ Echo suppression step,
A feature vector extracting step of converting the signal in which the echo signal and the ambient noise signal are suppressed into a feature vector sequence;
The feature vector sequence is either a speech section including a speech signal to be recognized, a noise section of only an ambient noise signal, or a noise / echo section in which the ambient noise signal and the echo signal exist. An interval determining step for determining whether or not
A noise model learning step of learning a noise model using the feature vector sequence determined to be a noise interval in the interval determination step;
A noise echo model learning step of learning a noise / echo model using the feature vector sequence determined to be a noise / echo interval in the interval determination step;
Noise superimposed speech model synthesis that generates a noise superimposed speech model by combining the noise model and the noise / echo model with a clean speech model created in advance using clean speech data that does not contain noise signals or echo signals Steps,
A likelihood calculating step of calculating a likelihood for a recognition category using the feature vector sequence determined as a speech section in the section determining step and the noise-superimposed speech model;
An output step for outputting a recognition result based on the calculated likelihood. The noise / echo suppression step includes a step of suppressing an echo signal in the input signal by a supply signal to the speaker, a step of suppressing an ambient noise signal in the input signal in which the echo signal is suppressed, the echo signal and The speech recognition method according to claim 1, further comprising a step of suppressing a remaining echo signal in the input signal in which the ambient noise signal is suppressed. A speech recognition device that is used for a device that collects an acoustic signal by a microphone and emits an acoustic signal from a speaker, and that recognizes the speech signal collected by the microphone using a probability model,
Noise that receives an input signal from the microphone and a supply signal to the speaker, suppresses the sound emission signal component (hereinafter referred to as echo signal) in the input signal, and suppresses an ambient noise signal in the input signal・ Echo suppression part,
A feature vector extraction unit that receives an input signal in which the echo signal and the ambient noise signal are suppressed, and converts the signal into a feature vector sequence;
The feature vector sequence and a signal indicating whether or not a sound emission signal is supplied to the speaker are input, and the feature vector sequence is of a speech section including a recognition target speech signal or of a noise section of only ambient noise signals. A section determination unit that determines whether the one is a noise / echo section including the echo signal and the ambient noise,
A switch that receives the feature vector series and the determination result output and separates and outputs the feature vector series into three series according to the determination result output;
A noise model learning unit that receives a feature vector sequence of the noise section separated by the switch and learns a noise model for the feature vector sequence;
A feature vector sequence of the noise / echo section separated by the switch is input, and for this feature vector, a noise / echo model learning unit for learning a noise / echo model,
A clean speech model memory for storing a clean speech model created based on clean speech data without noise, and
The noise model, the noise echo model, and the clean speech model are input, and the noise synthesis model that generates the noise superimposed speech model by synthesizing the noise model and the noise echo model and the clean speech model, and the noise superimposed speech model A noise superimposed speech model memory in which is stored,
A likelihood calculation unit that receives the feature vector sequence of the speech section separated by the switch and the noise superimposed speech model, and calculates the likelihood for each recognition category of the feature vector sequence based on the noise superimposed speech model;
A speech recognition apparatus, comprising: a recognition result output unit that receives a likelihood for each recognition category and outputs a recognition result. The noise / echo suppression unit receives an input signal and a supply signal to the speaker, and an echo unit that suppresses an echo signal in the input signal;
An output signal of the echo part is input, and a noise part for suppressing an ambient noise signal in the input signal in which the echo signal is suppressed;
An output signal of the noise unit and a supply signal to the speaker are input, and a residual echo unit that suppresses the remaining echo signal in the input signal in which the echo signal and the ambient noise signal are suppressed. The speech recognition apparatus according to claim 3, wherein Speech recognition program for executing the steps of the speech recognition method according to claim 1 or 2 in a computer. A computer-readable recording medium on which the voice recognition program according to claim 5 is recorded.

Images