KR20120102306A - Apparatus and method for processing speech in noise environment - Google Patents

Abstract

PURPOSE: A voice processing method in a noise environment and an apparatus thereof are provided to recognize a command by extracting a target voice signal from a mixing signal including a voice signal and a noise. CONSTITUTION: A voice processing apparatus receives a mixing signal generated from a plurality of sound sources through a microphone(210). The voice processing apparatus removes a signal generated from the received mixing signal(220). The voice processing apparatus recognizes a predetermined command from a voice section(230). [Reference numerals] (210) Receiving a mixing signal which is generated from a plurality of sound sources through a micro phone; (220) Reinforcing a target signal by removing a signal which generates a voice direction preset from the received mixing signal as a signal which is generated from the preset voice direction; (230) Recognizing a command which is preset from an existing voice section which existing a target signal of a user who orients a voice processing device; (AA) Start; (BB) End

Description

Apparatus and method for processing speech in noise environment

The present invention relates to an apparatus and method for processing a voice in a noisy environment, and in particular, a target device recognizes a user's command in an environment in which a user's voice and other noise are mixed, thereby allowing a user to operate the target device without a separate button operation. The present invention relates to a control apparatus, a method and a recording medium recording the method.

Speech recognition is a topic that has been studied for a long time with the development of speech processing technology. Various studies have been conducted to improve the recognition rate related to speech recognition, and continue to be studied. Speech recognition has a relatively high recognition rate in experimental environments where no noise is present. However, in an environment where speech recognition technology is actually applied, there is a problem that the speech recognition rate drops sharply due to various noises, unpredictable speaker states, and channel characteristics. Therefore, the speech recognition technology, which has been studied to reduce the inconvenience of manual operation of the user, is not widely used even though it is a more intuitive and easy to use method than the conventional manual operation interface.

Meanwhile, in order to process voice input using a voice input / output interface, voice recording and recognition are basically performed using start and end buttons for informing a corresponding input device to start and end of voice input. Since there is a troublesome problem such as manual button operation, there is a limit to the actual general user freely using the voice input.

The technical problem to be solved by the present invention overcomes the problem that the performance of speech recognition is degraded due to the unpredictable speaker's state and channel characteristics under the mixed environment of the user's voice and various noises, and as a result, in various devices It is intended to solve the inconvenience of not using speech recognition technology and still relying on manual operation. On the other hand, another technical problem to be solved by the present invention is to eliminate the hassle of manually operating the start and end of the voice input in the voice input process for speech recognition.

In order to solve the above technical problem, the voice processing method in a noise environment according to the present invention comprises the steps of receiving a mixed signal generated from a plurality of sound sources through the microphone; Reinforcing only a target signal by removing a signal generated from a remaining sound source direction except a preset sound source direction from the received mixed signal; And recognizing a preset command from a voice section existing among the enhanced target signals only for a user who is directed to a voice processing device.

Further, in the voice processing method in the noise environment, the step of reinforcing only the target signal may include: removing an echo signal generated from the voice processing device and input through the microphone; Separating the respective signals from the received mixed signals and searching for directions of the corresponding signals; And removing noise signals generated from sound source directions other than the preset sound source direction among the searched directions.

Furthermore, in the voice processing method in the noise environment, the step of recognizing the command may include: detecting a voice section using a vowel feature among the enhanced target signals; Selectively determining a user located in a sound source corresponding to the detected voice section as a voice recognition target; And detecting the preset command from the determined voice section of the voice recognition target.

On the other hand, the following provides a computer readable recording medium having recorded thereon a program for executing the voice processing method in the noise environment described above on a computer.

In order to solve the above technical problem, a voice processing apparatus in a noise environment according to the present invention includes a receiver for receiving a mixed signal generated from a plurality of sound sources through a microphone; A signal reinforcing unit for reinforcing only a target signal by removing a signal generated from a remaining sound source direction except a preset sound source direction from the received mixed signal; And a voice command recognition unit configured to recognize a preset command from a voice section existing among the enhanced target signals only for a user who is directed to a voice processing device.

The signal reinforcement unit may further include an echo canceller configured to remove an echo signal generated from the voice processor and input through the microphone; A direction search unit for searching for directions of the corresponding signals by separating the respective signals from the received mixed signals; And a noise removing unit for removing a noise signal generated from the remaining sound source directions except for a preset sound source direction among the searched directions.

Further, the voice command recognition unit in the speech processing apparatus in the noise environment, the voice detection unit for detecting a voice interval using the vowel feature among the enhanced target signal; A target determination unit to selectively determine a user located in a sound source corresponding to the detected voice section as a voice recognition target; And a command detector detecting the preset command from the determined voice section of the voice recognition target.

The present invention improves the performance of speech recognition by extracting the target speech signal from the mixed signal including the speech signal and the noise and recognizes a command, and can easily operate and control the target device without manual operation. In addition, the present invention detects a speech section using a vowel feature in a speech input process for speech recognition, thereby extracting and recognizing a command from a continuous speech signal without any troublesome manual operation.

1 is a diagram illustrating an exemplary situation and a basic idea in which embodiments of the present invention are implemented in a noisy environment.
2 is a flowchart illustrating a voice processing method in a noise environment according to an embodiment of the present invention.
3 is a flowchart illustrating a process of reinforcing a target signal in detail in the voice processing method of FIG. 2 according to an embodiment of the present invention.
4 is a flowchart illustrating a process of removing a noise signal in more detail in the voice processing method of FIG. 3 according to an embodiment of the present invention.
5 is a flowchart illustrating a process of recognizing a command in more detail in the voice processing method of FIG. 2 according to an exemplary embodiment of the present invention.
6 is a flowchart illustrating a process of detecting a voice section in more detail in the voice processing method of FIG. 5 according to an exemplary embodiment of the present invention.
7A and 7B are flowcharts illustrating two embodiments for selectively determining a user as a voice recognition target in the voice processing method of FIG. 5 according to an embodiment of the present invention, respectively.
8 is a flowchart illustrating a process of detecting a preset command in more detail in the voice processing method of FIG. 5 according to an embodiment of the present invention.
9 is a block diagram illustrating a speech processing apparatus in a noisy environment according to an embodiment of the present invention.
FIG. 10 is a block diagram illustrating the signal reinforcement unit in more detail in the voice processing apparatus of FIG. 9 according to an embodiment of the present invention.
FIG. 11 is a block diagram illustrating in detail the voice command recognition unit in the voice processing apparatus of FIG. 9 according to an embodiment of the present invention.

Before describing the embodiments of the present invention, the environment in which the embodiments are implemented will be briefly described, and the basic idea that the embodiments are commonly employed will be presented.

1 is a view for explaining an exemplary situation and the basic idea that the embodiments of the present invention are implemented in a noisy environment, a plurality of users are arranged on the left and right around the TV, the sound of people's conversation, Assume that there is a mixture of various noises, including sound.

According to various embodiments of the present disclosure, when a command is made through a voice to a target device (typically corresponding to a TV in FIG. 1), which is a target of speech recognition, the speaker may be a person who looks at the target device. Assume that a person exists in a certain area from the device. In addition, the command that can be utilized in the target device is assumed to be a specific command that is predetermined and stored. Accordingly, embodiments of the present invention target only the voice signal of a talker who exists in a specific area in the target device and looks at the target device under a noisy environment.

In Fig. 1, attention is given to four users 110, 120, 130, 140, which are numbered. Since the leftmost user 110 is located too far from the TV and the viewing angle is not appropriate, the user 110 may be excluded from speech recognition. Of course, depending on the nature of the target device may be included in the object of speech recognition. Alternatively, other users who are not visible on the screen will not be subject to speech recognition even if there are users speaking. The second user 120 is looking at the TV, which is the target device, and may be an object of speech recognition because it is located within a valid viewing angle. The third user 130 and the fourth user 140 also exist within an effective viewing angle with the TV, which is the target device, but may not be a target of speech recognition because they are not directed toward the TV.

Using this basic idea, it is possible to recognize voice commands aimed at a target device more accurately and quickly by removing signals that are not subject to speech recognition in a mixed voice and noise environment. That is, embodiments of the present invention are intended to improve the performance of the voice recognizer (meaning a voice recognition target device) by correcting distortion of voice signals generated from various objects in order to increase the performance of voice recognition. In addition, the user can replace this process with a button-like voice (which can be a keyword) without the user having to push (push to talk) the beginning and end of the voice, which is the most troublesome problem in using the voice interface. To increase the convenience and use of voice interface.

Hereinafter, embodiments of the present invention will be described in more detail with reference to the accompanying drawings. Like reference numerals in the drawings refer to like elements.

2 is a flowchart illustrating a voice processing method in a noise environment according to an embodiment of the present invention, and includes the following steps.

In operation 210, a mixed signal generated from a plurality of sound sources is received through a microphone. The microphone is a device that receives mixed signals, and may be implemented as a single single microphone, but is composed of a plurality of microphones to collect mixed signals generated from different sound sources and to easily process the collected signals. It would be more advantageous to be a microphone array.

Only the target signal is reinforced by removing signals generated from the sound source direction other than the preset sound source direction from the mixed signal received in operation 220 through 210. The step 220 corresponds to a process of generating a basic signal for improving the performance of speech recognition in an environment in which a large number of users and various noises exist, and the sound generated from a sound source other than a human after receiving a sound signal (that is, And other sound signals except for voice signals.), A desired signal is searched among signals existing in various directions, and a series of processes for reinforcing only the signals necessary for speech recognition are performed. Each process will be described in detail later with reference to FIGS. 3 and 4.

In operation 230, only the user who is directed to the speech processing apparatus recognizes a preset command from a voice section existing among the enhanced target signals in operation 220. This step 230 includes a method for automating a manual operation for notifying the beginning and the end of the voice processing used in the existing voice interface device. That is, in step 230, the target device (meaning the speech processing device for speech recognition) always waits for the user's response (meaning a voice command), and is designated in an environment in which various sounds and a plurality of users exist. Only speech that indicates a user's command is determined as a voice section.

As described above, in order to recognize the command from the command voice interval, the target device should always be able to guarantee the reliability of waiting for the user's voice command and not malfunctioning. To this end, the embodiment of the present invention performs a series of processes for detecting a voice section, recognizing a user's face, recognizing a talker, and recognizing a specific keyword through step 230. This series of steps may be added or subtracted in some steps to improve performance or speed up speech recognition in terms of implementation and implementation. As a result, the speech processing apparatus determines whether the input sound signal is a voice through the above process, determines whether a registered user is giving a command, determines whether an authorized user is speaking, and inputs the input voice. By determining whether a signal includes a preset command, a voice processing method capable of always responding to a user's command without manual manipulation of a button can be provided. Each process will be described in detail later with reference to FIGS. 5 to 8.

FIG. 3 is a flowchart illustrating a process (step 220) of reinforcing a target signal in detail in the voice processing method of FIG. 2 according to an embodiment of the present invention, and includes the following steps.

In step 221, the echo signal generated from the voice processing apparatus and input through the microphone is removed. In general, when the microphone is disposed in close proximity to the speaker, a problem occurs in that sound output through the speaker is input to the microphone. That is, in the situation where the two-way voice input and output are simultaneously performed, the output of the speaker is input to the device to generate an acoustic echo that is heard again by the output of the speaker of the device. These echo signals must be removed because they not only interfere with accurate speech recognition but also cause great inconvenience to the user. This is called acoustic echo cancellation (AEC). The process of AEC is briefly described as follows.

First, it is assumed that a mixed sound including an output sound radiated from a speaker in addition to a user's voice and interference noise is input through a microphone. In operation 221, the output signal applied to the speaker is input as a factor, and the output signal of the speaker is removed from the sound source signal input through the microphone using a specific filter. Such a filter may be configured as an adaptive filter capable of removing an acoustic echo included in a sound source signal by receiving feedback of an output signal continuously applied to a speaker over time. have.

In the AEC method, various algorithms such as least mean square (LMS), normalized least mean square (NLMS), recursive least square (RLS), and the like are introduced, and the method of implementing AEC using the above methods is described in the present invention. It can be easily understood by those of ordinary skill in the art, detailed description thereof will be omitted.

Now, the echo signal has been removed from the mixed signals. Hereinafter, steps for improving the recognition rate of a specific user's voice among signals generated in various directions through a method for reinforcing a sound at a specific location by finding a location where a sound signal is generated are presented.

In operation 222, each signal is separated from the received mixed signals to search for directions of the corresponding signals. Acoustic location tracking can be achieved using a steered beamformer such as steered response power (SRP), high resolution spectral estimation such as multiple signal classification (MUSIC), or GCC ( A method using a time delay of arrival such as generalized cross correlation may be used. Each method can be easily understood by those skilled in the art to which the present invention pertains, and a detailed description thereof will be omitted.

In step 223, the noise signal generated from the sound source direction other than the preset sound source direction among the directions found in step 222 is removed. Through this process, the target of the operation can be reduced by eliminating the sound generated at a location other than a specific direction (meaning the user's location) among the signals searched in step 222, and as a result, a quick voice recognition can be performed. . A specific processing method will be described below with reference to FIG. 4.

4 is a flowchart illustrating a process of removing a noise signal in more detail in the voice processing method of FIG. 3 according to an embodiment of the present invention.

In operation 223a, a cancellation signal is generated for side lobes except for a main lobe corresponding to a predetermined sound source direction among beam patterns for signals. The beam pattern refers to a graph obtained by measuring electric field strength of electromagnetic waves incident or radiated from signal input / output devices such as speakers and microphones. Thus, the further away from the reference point of the graph, the greater the field strength, which means that it has directivity in that direction. In the beam pattern, small and thin beam patterns appear to the left and right around the main main lobe. Small, thin beam patterns, except for the main lobe, which exhibits strong directivity, are called side lobes, which appear as non-uniform radiation patterns in the radiation pattern. As a result, side lobes or non-uniform radiation patterns become factors that prevent the convergence of sound field characteristics such as directivity in acoustic equipment.

That is, in 223a, the enhanced signal for the preset sound source direction is left, and other signals are considered to be noise and are removed. In detail, the directivity to the target signal may be improved by adaptively removing the side lobe signal present in the output signal of the beamforming technique independent of the signal. When the mixed signal including the voice signal and the noise is input, the signal is first amplified by fixed beamforming using a fixed filter. Meanwhile, only a noise signal is extracted from the mixed signal through a blocking matrix to be a removal signal.

In step 223b, the noise signal is removed by combining the mixed signal and the cancellation signal generated through step 223a. That is, the cancellation signal generated through step 223a uses an adaptive filter to remove the noise signal in the fixed beam-formed mixed signal, and the target signal (noise is removed to completely remove the residual noise from the mixed signal). The relationship between the output signal and the input signal can be expressed as a transfer function to generate an appropriate cancellation signal by changing the filter coefficients for the adaptive filter.

In order to utilize this adaptive filter, an open-loop feed-forward scheme may be utilized. The open feed-forward scheme is known in the field of signal processing as one of the methods of controlling a component that may have any displacement value through a relationship with a plurality of components constituting the system. Here, the control means that a state of a certain physical quantity is brought into a state suitable for a desired purpose. In other words, the control in the embodiment of the present invention refers to adjusting a variable that a specific configuration of the signal processing apparatus (which will be a cancellation signal generator) may have so that the signal processing apparatus may output a desired result. . This makes it possible to take advantage of predictive control in which a series of components of a system do not have a close-loop in which measurements and performances are repeated, and detects control values and changes the results.

Through a series of processes as described above, an embodiment of the present invention improves the performance of speech recognition by extracting a target voice signal from a mixed signal including a voice signal and a noise and recognizes a command, and facilitates the target device without manual operation. Operation and control are possible.

FIG. 5 is a flowchart illustrating a process of recognizing a command in more detail in the voice processing method of FIG. 2 according to an embodiment of the present invention, and includes the following steps.

In operation 231, a voice section is detected using a vowel feature among the target signals enhanced through the above processes. In general, in determining the speech section included in the sound signal, there is a problem that it is difficult to extract the features of the entire speech, and signal corruption and open-set problems. In order to solve the problem, the speech recognition process extracts and stores the spectral features from the pre-learned noise-free vowel signal and extracts the stored spectral features from the input acoustic signal having the noise distortion. It is used to perform matching between the vowel signal and the input acoustic signal in which the noise distortion occurs. The reason is that signal corruption and open-set problems of the prior art were caused by performing matching using features extracted from input acoustic signals distorted by noise. Therefore, the embodiment of the present invention can measure the similarity by extracting only the spectral features of the vowel and performing matching between the vowel signal and the input acoustic signal.

To this end, vowel characteristic information indicating a peak band where a characteristic peak is located in the frequency spectrum of the vowel is stored through a learning step in advance. In extracting feature peaks from a vowel spectrum, various types of extraction schemes can be applied. In this embodiment, a peak having an energy larger than a predetermined threshold among the spectral peaks of the vowel is simplified for simplicity of operation. Can be extracted as a feature peak. Human speech (voice or speech) is largely composed of consonants and vowels. The peak band where the characteristic peaks are present on the vowel spectrum has a significantly higher energy than the surrounding band so that it does not disappear easily due to noise distortion. Therefore, the vowel part of the human voice is easy to detect even in a real life environment in which various noises are generated. The entire vowel can be detected.

Step 231 will be described in more detail with reference to FIG. 6 is a flowchart illustrating a process of detecting a voice section in more detail in the voice processing method of FIG. 5 according to an exemplary embodiment of the present invention.

First, in step 231a, vowel feature information indicating a peak band where a feature peak is located in the spectrum of the vowel is stored in advance.

In the present embodiment, the entire spectral band of the vowel is divided into a certain number of unit bands, and the unit band corresponding to the peak band in the vowel spectrum is represented by 1, and the band corresponding to the valley band, which is a band other than the peak band, is represented. The vowel characteristic information is generated by indicating the unit band as 0. For example, the vowel feature information may be generated in the form of a peak signature vector. That is, when a feature peak or more than a threshold is extracted from an average spectrum of an N dimension obtained by applying a Discrete Fourier Transform (DFT) to training data about a vowel signal, it corresponds to a peak band where a feature peak is located. 1 may be assigned to each dimension, and 0 may be assigned to each dimension corresponding to a valley band, which is a band other than the peak band, to generate an N-dimensional peak signature vector.

Next, in step 231b, the target signals enhanced by using the average energy of the corresponding band corresponding to the peak band indicated by the vowel characteristic information stored in step 231a and the non-corresponding band except the corresponding band in the spectrum of the previously enhanced target signals. Detects a section corresponding to the voice. That is, by using average energy of a corresponding band corresponding to the peak band represented by the vowel characteristic information stored above on the spectrum of the enhanced target signals and an irrelevant band except the corresponding band, The voice section is detected by determining whether the target signals correspond to the voice. Specifically, the difference between the average energy of the corresponding band and the average energy of the non-compliant band is calculated and compared with the predetermined threshold value. When the calculated value is larger than the threshold value, the input sound is judged as a voice including a vowel. If it is regarded as a voice interval and the calculated value is less than or equal to the threshold value, the input sound may determine noise or non-voice sound.

Through the above process, the voice section is now detected. Returning to FIG. 5 again, the process proceeds to step 232.

In step 232, the user located in the sound source corresponding to the detected voice section in step 231 is selectively determined as a voice recognition target. This process is to determine the user's command to the product and to limit the object when the voice is present. Various methods may be used to limit such a user. Hereinafter, two embodiments will be described with reference to FIGS. 7A and 7B.

First, in selectively determining a user as a voice recognition target, facial recognition may be used to determine whether a user at a corresponding location issues a command.

In detail, in FIG. 7A, in step 232a, it is determined whether the user is directed to the voice processing apparatus using face recognition. In step 232b, it is determined whether the user is a pre-registered user based on the result of face recognition. In step 232c, the user who satisfies both the determination result of step 232a and the test result of step 232b is determined as the voice recognition target. Through the above-described series of operations, it is possible to determine whether the user is a previously authorized user and to increase the accuracy of the authority and product operation. Practical embodiments and descriptions of face recognition are beyond the scope of the present invention and will not be described here.

Second, in selectively determining a user as a speech recognition object, whether a user is a valid user may be determined using hidden Markov models (HMM).

In detail, in step 232d of FIG. 7B, it is checked whether the user is a registered user based on the speaker's voice recognition result using the user's voice signal and hidden Markov model. In step 232e, the user who satisfies the test result of step 232d is checked. Determined as a voice recognition target.

In particular, an embodiment of the present invention may use a Subspace Distribution Clustering Hidden Markov Model (SDCHMM). SDCHMM divides the entire multivariate Gaussian into feature spaces called subspaces, and then quantizes the distributions belonging to each subspace to quantize the subspace distribution prototype, or codeword. ) To represent the original full space distribution. That is, find a combination of subspace prototypes that most closely resembles the original overall spatial distribution and link them. In this way, the SDCHMM can express the original total spatial distribution using a small number of subspace codewords.

Specifically, the parameters of the subspace distribution clustering hidden Markov model (SDCHMM) are transformed into the hidden Markov model (HMM) for the entire space, the hidden Markov model (HMM) is transformed into a linear spectral domain, and then the maximum likelihood linear The spectral transform can be used to adapt the hidden Markov model (HMM) of the above linear spectral domain to the speaker to recognize the user at high speed.

Through the above process, the user is now determined. Returning to FIG. 5 again, the flow proceeds to step 233.

In step 233, a preset command is detected from the voice section of the voice recognition target determined in step 232. The keyword recognition process determines whether a language, that is, a command, is required for an operation process of a target device among various languages of a person. As a result, the voice processing device may reduce the probability of malfunction while maintaining a standby state for receiving a command among sound signals at all times, and the user may not operate the device without any specific action (manual input of a button, etc.). You can give appropriate commands.

8 is a flowchart illustrating a process of detecting a preset command in more detail in the voice processing method of FIG. 5 according to an embodiment of the present invention. This process is performed by using a recognized voice command to determine whether a matching command group exists by matching a sentence with a product command.

First, in step 233a, a syllable model and a command set including representative pronunciations of syllables that may be pronounced by a user are stored in advance. Here, the syllable model is composed of representative pronunciations of syllables that can be pronounced by the user. The syllable model may be configured to store only one syllable pronunciation for syllables with similar vowels and only one common syllable pronunciation for syllables with similar pronunciation. Like Hangul, phonetic letters can be written with one syllable. Therefore, the speech spoken in syllable units suppresses the occurrence of articulation in continuous pronunciation and enables relatively accurate pronunciation input. The Hangul complete code is 2350 characters long and can be generated with a small syllable model of around 1000 when grouped with distinguishable representative pronunciations. In addition, if the consecutively pronounced speech is searched for the syllable candidate list that is most similar to each other, and the words and sentences that can be generated through the language model are configured, accurate speech recognition is possible even through a relatively small syllable model.

In operation 233b, the feature of each syllable is extracted from the voice section of the voice recognition target determined previously. Next, in step 233c, at least one syllable candidate is generated by comparing the feature extracted in step 233b with the syllable model stored in step 233a. That is, the probability of words that can be combined into a plurality of syllable candidates is calculated using the syllable model generated in operation 233a, and a plurality of sentence candidates in which words are connected are generated according to the probability of the words. Then, in step 233d, the command included in the stored instruction set among the combinable words from the syllable candidates generated through step 233c is detected.

According to the embodiments of the present invention described above, a separate cumbersome manual operation (push to talk) is specified by detecting a voice section using a vowel feature in a voice input process for voice recognition. Commands can be extracted and recognized in continuous voice signals without In particular, the above embodiments can apply the voice interface used in the limited environment to various environments and products, and exhibit high recognition performance.

FIG. 9 is a block diagram illustrating a voice processing apparatus 900 in a noise environment according to an exemplary embodiment of the present invention, and includes a receiver 10, a signal enhancer 20, and a voice command recognizer 30. . Furthermore, the embodiment of the present invention may optionally include a recognition enhancer 40. Hereinafter, each component will be described in turn, but the description overlapping with the contents already described through the above embodiments will be described.

The receiver 10 receives a mixed signal generated from a plurality of sound sources through a microphone. The receiver 10 corresponds to step 210 of FIG. 2.

The signal reinforcement unit 20 reinforces only the target signal by removing a signal generated from the remaining sound source direction except the preset sound source direction from the mixed signal received through the receiver 10. The signal reinforcement unit 20 corresponds to step 220 of FIG. 2 and describes detailed components through FIG. 10.

The signal reinforcement unit 20 is an echo cancellation unit 21 which removes an echo signal generated from the voice processing device and input through the microphone, and separates the respective signals from the received mixed signals to search for the directions of the corresponding signals. The search unit 22 and a noise removing unit 23 for removing a noise signal generated from the sound source direction other than the preset sound source direction among the searched directions. The detailed configurations 21, 22, and 23 correspond to steps 221 to 223 of FIG. 3, respectively, and thus detailed description thereof will be omitted.

The voice command recognition unit 30 recognizes a preset command from a voice section existing among the target signals enhanced by the signal reinforcement unit 20, targeting only a user who is directed to the voice processing device. The voice command recognition unit 30 corresponds to step 230 of FIG. 2 and describes the detailed components through FIG. 11.

The voice command recognizer 30 detects a voice section using a vowel feature among the enhanced target signals, and selectively determines a user located in a sound source corresponding to the detected voice section as a voice recognition target. And a command detector 33 for detecting a preset command from the voice section of the determined voice recognition target. The detailed configurations 31, 32, and 33 correspond to steps 231 to 233 of FIG. 5, respectively, and a detailed description thereof will be omitted.

On the other hand, the recognition improving unit 40 corresponds to a configuration for improving the speech recognition of the information obtained through the above process. The recognition enhancer 40 may use various speech recognition models, but in the embodiment of the present invention, a speaker adaptation method using evolutionary learning may be used.

In the recognition mode in which the speech recognition system is performing speech recognition, it is possible to perform feature conversion for rapid environmental adaptation, and in the standby mode in which the speech recognition system is not performing speech recognition, a sufficient amount of pre-stored speech data is stored. The model transformation is performed using the adaptive data. For example, the process of transforming this feature may be performed by pre-determining an environmental parameter using a maximum likelihood method. In other words, this embodiment is an evolutionary learning process for speaker adaptation, which performs feature transformation and model transformation repeatedly to ensure stable and high recognition performance regardless of the amount of adaptive data accumulated.

Speaker adaptation consists of estimating the speaker-independent acoustic model and re-estimating the speaker-independent acoustic model estimated by the speaker-dependent acoustic model. That is, a multidimensional feature vector such as MFCC (Mel-Frequency Cepstral Coefficient) is extracted from a large amount of speech data obtained from an unspecified number of speakers, and an EM-Expectaion-Maximization (EM) technique is applied to the extracted feature vectors. Estimate the Hidden Markov Model (HMM). The estimated HMM has speaker-independent characteristics as an acoustic model for an unspecified number of speakers. In addition, the present embodiment may estimate the speaker and the environmental dependent acoustic model using the already estimated speaker and environment independent acoustic model and relatively small amount of voice data obtained from a specific speaker (user).

Meanwhile, the present invention can be embodied as computer readable codes on a computer readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored.

Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device and the like, and also a carrier wave (for example, transmission via the Internet) . In addition, the computer-readable recording medium may be distributed over network-connected computer systems so that computer readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the present invention can be easily deduced by programmers skilled in the art to which the present invention belongs.

The present invention has been described above with reference to various embodiments. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

110, 120, 130, 140: multiple users in noisy environment
900: speech processing unit
10: receiver
20: signal reinforcement unit 21: echo cancellation unit
22: direction search unit 23: noise removal unit
30: voice command recognition unit 31: voice detection unit
32: target determination unit 33: command detection unit
40: recognition improvement unit

Claims (19)

Receiving a mixed signal generated from a plurality of sound sources through a microphone;
Reinforcing only a target signal by removing a signal generated from a remaining sound source direction except a preset sound source direction from the received mixed signal; And
And recognizing a preset command from a voice section existing among the enhanced target signals only for a user who is directed to a voice processing device. The method of claim 1,
Recognizing the command,
Detecting a voice section using a vowel feature among the enhanced target signals;
Selectively determining a user located in a sound source corresponding to the detected voice section as a voice recognition target; And
And detecting the preset command from the determined voice section of the voice recognition target. The method of claim 2,
Detecting the voice section,
Pre-storing vowel feature information indicating a peak band at which a feature peak is located in the spectrum of the vowel; And
In the spectrum of the enhanced target signals, a section corresponding to a voice among the enhanced target signals using the average energy of the corresponding band corresponding to the peak band indicated by the stored vowel feature information and the non-corresponding band except the corresponding band is used. A voice processing method comprising the step of detecting. The method of claim 2,
Selectively determining the user as a voice recognition target,
Determining whether the user is facing the speech processing device using face recognition;
Checking whether the user is a registered user based on a result of the facial recognition; And
And determining a user who satisfies both the determination result and the test result as the speech recognition target. The method of claim 2,
Selectively determining the user as a voice recognition target,
Checking whether the user is a registered user based on a speaker recognition result using the user's voice signal and hidden Markov models (HMM); And
And determining a user who satisfies the test result as the speech recognition target. The method of claim 2,
The detecting of the preset command may include:
Pre-storing a syllable model and a command set consisting of representative pronunciations of syllables that may be pronounced by a user;
Extracting a feature of each syllable from the determined speech section of the speech recognition object;
Generating at least one syllable candidate by comparing the extracted feature with the stored syllable model; And
And detecting a command included in the stored instruction set from among the words that can be combined from the generated syllable candidates. The method of claim 1,
Enhancing only the target signal,
Removing an echo signal generated from the voice processing device and input through the microphone;
Separating the respective signals from the received mixed signals and searching for directions of the corresponding signals; And
And removing a noise signal generated from the sound source direction other than the preset sound source direction among the searched directions. The method of claim 7, wherein
The searching of the directions of the signals may be performed using at least one of a time delay of arrival (TDOA), a beam-forming method, and a high resolution spectral analysis method. Voice processing method. The method of claim 7, wherein
Removing the noise signal,
Generating a cancellation signal for a side lobe except a main lobe corresponding to the preset sound source direction among beam patterns for the signals; And
Synthesizing the mixed signal and the generated cancellation signal. A computer-readable recording medium having recorded thereon a program for executing the method of any one of claims 1 to 9. A receiver which receives a mixed signal generated from a plurality of sound sources through a microphone;
A signal reinforcing unit for reinforcing only a target signal by removing a signal generated from a remaining sound source direction except a preset sound source direction from the received mixed signal; And
And a voice command recognition unit configured to recognize a preset command from a voice section existing among the enhanced target signals only for a user who is directed to a voice processing device. The method of claim 11,
The voice command recognition unit,
A voice detector for detecting a voice section using the vowel feature among the enhanced target signals;
A target determination unit to selectively determine a user located in a sound source corresponding to the detected voice section as a voice recognition target; And
And a command detector to detect the preset command from the determined voice section of the voice recognition target. The method of claim 12,
The voice detector,
Pre-store vowel characteristic information indicating the peak band in which the characteristic peak is located in the spectrum of the vowel,
In the spectrum of the enhanced target signals, a section corresponding to a voice among the enhanced target signals using the average energy of the corresponding band corresponding to the peak band indicated by the stored vowel feature information and the non-corresponding band except the corresponding band is used. A voice processing device, characterized by detecting. The method of claim 12,
The target determination unit,
Using face recognition to determine whether the user is aimed at the speech processing device,
Checking whether the user is a pre-registered user based on the facial recognition result;
And a user who satisfies both the determination result and the test result as the voice recognition target. The method of claim 12,
The target determination unit,
Checking whether the user is a registered user based on a speaker recognition result using the user's voice signal and a hidden Markov model,
And a user who satisfies the test result as the voice recognition target. The method of claim 12,
The command detection unit,
Pre-save syllable model and command set consisting of representative pronunciations of syllables that can be pronounced by the user,
Extracts the characteristics of each syllable from the determined speech section of the speech recognition object,
Generating at least one syllable candidate by comparing the extracted feature with the stored syllable model,
And detecting a command included in the stored command set among words that can be combined from the generated syllable candidate. The method of claim 11,
The signal reinforcement unit,
An echo canceling unit for removing an echo signal generated from the speech processing apparatus and input through the microphone;
A direction search unit for searching for directions of the corresponding signals by separating the respective signals from the received mixed signals; And
And a noise removing unit configured to remove a noise signal generated from a sound source direction other than a preset sound source direction among the searched directions. The method of claim 17,
And the direction searching unit searches for the directions of the signals using at least one of an arrival time delay method, a beam forming method, and a high resolution spectrum estimation method. The method of claim 17,
The noise-
Generating a cancellation signal for the side lobe except for the main lobe corresponding to the preset sound source direction among the beam patterns for the signals;
And synthesized the mixed signal and the generated cancellation signal.

Images