KR20230001968A - Voice and gesture integrating device of vehicle - Google Patents

Abstract

The present invention relates to a voice and gesture recognition device for a vehicle to extract only voice commands and accurately recognize a user's hand gestures. According to one embodiment of the present invention, the voice and gesture recognition device comprises: a transmitting pad emitting an electric field to detect gestures of a user's hand; a receiving pad detecting distortion of the electric field emitted from the transmitting pad; a gesture recognizer recognizing the gesture by the distortion of the electric field in the receiving pad; a microphone unit detecting the user's voice signal; a voice preprocessor performing preprocessing including noise-canceling for the voice signal with respect to the voice signal detected by the microphone unit; a voice recognizer recognizing the voice signal preprocessed by the voice preprocessor; and a communicator transmitting the signals for the gestures recognized by the gesture recognizer and the voice signals detected by the voice recognizer to an electronic control unit of a vehicle.

Description

Vehicle voice and gesture recognition device {VOICE AND GESTURE INTEGRATING DEVICE OF VEHICLE}

The present invention relates to a voice and gesture recognition device for a vehicle, and more particularly, to a voice and gesture recognition device for a vehicle that allows a driver to control equipment of a vehicle while driving.

While the vehicle is operating, it is necessary for the driver to continuously look ahead. While the vehicle is operating in this way, the driver must operate a physical button or switch for the corresponding function to control other functions of the vehicle. Manipulating various functions of the vehicle while driving the vehicle may cause a problem in that the driver cannot concentrate on driving the vehicle.

Accordingly, in order to manipulate various functions of the vehicle, research is being conducted to manipulate the functions of the vehicle through gestures such as voice recognition or a driver's hand motion. Conventionally, voice recognition can generally be divided into two stages. The first step requires pre-processing technology to clearly separate the driver's voice regardless of surrounding noise. It is necessary to derive conservative results considering safety as much as possible.

However, conventional voice recognition technology has a fixed angle of view to recognize only the voice of a person (eg, a driver) in a specific location, making it difficult for a passenger or a passenger in the back seat to use voice recognition. In addition, in the case of a product that recognizes voice based on a server, there is a problem that an Internet network must be connected.

Furthermore, as described above, in the case of controlling vehicle functions limited to voice recognition, it is not easy to control various vehicle functions in detail.

Republic of Korea Patent No. 10-1650769 (2016.08.18.) Republic of Korea Patent No. 10-1260053 (2013.04.25.)

An object to be solved by the present invention is to provide a voice and gesture recognition device for a vehicle capable of extracting only accurate voice commands by separating noise generated from the user's surroundings from voice and accurately recognizing the user's hand gestures. .

A voice and gesture recognition apparatus according to an embodiment of the present invention includes a transmission pad emitting an electric field to detect a gesture of a user's hand; a receiving pad that detects distortion of the electric field emitted from the transmitting pad; a gesture recognizer recognizing the gesture by distortion of an electric field in the receiving pad; a microphone unit for detecting the user's voice signal; a voice pre-processor which performs pre-processing on the voice signal detected by the microphone unit, including removing noise of the voice signal; a voice recognizer for recognizing the voice signal preprocessed by the voice preprocessor; and a communicator transmitting signals and voice signals for gestures recognized by the gesture recognizer and the voice recognizer to an electronic control unit of the vehicle.

The gesture detector may further include a gesture detector detecting depth and direction of the gesture by distortion of an electric field at the receiving pad, and the gesture recognizer may recognize the gesture using the depth and direction recognized by the gesture detector.

The gesture detector detects a gesture pattern by detecting the depth and direction of the gesture, compares the detected gesture pattern with a preset gesture pattern, and the gesture recognizer detects the gesture pattern detected by the gesture detector and the preset gesture pattern. When the patterns match, the gesture may be recognized.

The microphone unit may include a plurality of microphones, and the plurality of microphones may be disposed at different positions inside the vehicle.

The voice preprocessor may include a noise removal unit removing noise from the voice signal detected by the microphone unit; a reverberation removal unit that removes reverberation from the voice signal from which the noise has been removed by the noise removal unit; a direction and position detector for detecting a direction and position of a voice signal from the voice signal from which the reverberation has been removed by the reverberation remover; and a beam forming unit configured to form a beam for the audio signal whose direction and position are detected by the direction and position detection unit.

The noise removal unit may remove noise by removing a signal corresponding to a high frequency higher than the frequency range of the user's voice signal and a signal corresponding to a low frequency lower than the frequency range of the user's voice signal.

A voice signal detection unit for detecting a voice signal considering the direction and position of the user's voice signal from the voice signal preprocessed by the voice preprocessor, wherein the voice recognition unit determines the direction and location of the voice signal in the voice signal detector. The considered audio signal can be recognized.

A voice pattern detection unit which compares the voice signal detected by the voice signal detector with a preset voice signal pattern and outputs the voice pattern, wherein the voice recognition unit outputs the detected voice signal corresponding to the voice signal pattern preset by the voice pattern detector. Recognizable.

According to the present invention, a voice signal can be detected and a user's voice can be accurately recognized by separating a recognized voice from noise through a pre-processing process to form a beam.

In addition, since a user's gesture can be recognized using an electromagnetic field, the user's gesture can be recognized in three dimensions, thereby providing an effect of accurately recognizing the gesture.

1 is a block diagram illustrating a voice and gesture recognition device according to an embodiment of the present invention.
2 is a diagram illustrating a receiving pad of a voice and gesture recognition device according to an embodiment of the present invention.
3 is a diagram illustrating a gesture detector of a voice and gesture recognition apparatus according to an embodiment of the present invention.
4 is a diagram illustrating a microphone unit of a voice and gesture recognition device according to an embodiment of the present invention.
5 is a diagram illustrating a voice preprocessor of a voice and gesture recognition apparatus according to an embodiment of the present invention.
6 is a diagram illustrating a voice pattern detection unit of a voice and gesture recognition apparatus according to an embodiment of the present invention.
7 is a diagram for explaining an example of a gesture operation of a voice and gesture recognition apparatus according to an embodiment of the present invention.
8 is a diagram for explaining an example of a gesture operation of a voice and gesture recognition apparatus according to an embodiment of the present invention.
9 is a diagram for explaining an example of a gesture operation of a voice and gesture recognition apparatus according to an embodiment of the present invention.
10 is a diagram for explaining an example of a gesture operation of a voice and gesture recognition apparatus according to an embodiment of the present invention.
11 is a diagram for explaining a gesture recognition method of a voice and gesture recognition apparatus according to an embodiment of the present invention.
12 is a diagram for explaining a voice recognition method of a voice and gesture recognition apparatus according to an embodiment of the present invention.

Hereinafter, specific embodiments for implementing the present invention will be described in detail with reference to the drawings.

In addition, in the description of the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description will be omitted.

In addition, when a component is referred to as 'connecting', 'supporting', 'connecting', 'supplying', 'transferring', or 'contacting' to another component, it is directly connected to, supported by, or connected to the other component. It may be supplied, delivered, or contacted, but it should be understood that other components may exist in the middle.

Terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise.

In addition, in this specification, expressions such as upper, lower, side, etc. are described based on the drawings, and it is made clear in advance that they may be expressed differently if the direction of the object is changed. For the same reason, some components in the accompanying drawings are exaggerated, omitted, or schematically illustrated, and the size of each component does not entirely reflect the actual size.

In addition, terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by these terms. These terms are only used to distinguish one component from another.

As used herein, the meaning of "comprising" specifies specific characteristics, regions, integers, steps, operations, elements, and/or components, and other specific characteristics, regions, integers, steps, operations, elements, elements, and/or groups. does not exclude the presence or addition of

1 is a block diagram illustrating a voice and gesture recognition device according to an embodiment of the present invention. 2 is a diagram showing a receiving pad of a voice and gesture recognition device according to an embodiment of the present invention, and FIG. 3 is a diagram showing a gesture detector of a voice and gesture recognition device according to an embodiment of the present invention. 4 is a diagram showing a microphone unit of a voice and gesture recognition device according to an embodiment of the present invention, and FIG. 5 is a diagram showing a voice preprocessor of a voice and gesture recognition device according to an embodiment of the present invention. 6 is a diagram illustrating a voice pattern detection unit of a voice and gesture recognition apparatus according to an embodiment of the present invention. 7 to 10 are diagrams for explaining an example of a gesture operation of a voice and gesture recognition apparatus according to an embodiment of the present invention.

A voice and gesture recognition apparatus 100 according to an embodiment of the present invention will be described with reference to FIGS. 1 to 10 . The voice and gesture recognition apparatus 100 according to an embodiment of the present invention is used by a user to manipulate various functions of a vehicle while the vehicle is running. The voice and gesture recognition apparatus 100 includes a transmitting pad 110, a receiving pad 120, a gesture detector 130, a gesture recognizer 140, a microphone unit 150, a voice preprocessor 160, and a voice signal. It includes a detection unit 170, a voice pattern detection unit 180, a voice recognizer 190 and a communication unit 200.

The transmission pad 110 recognizes a user's hand gesture. The transmission pad 110 emits a predetermined electric field, and the electric field emitted from the transmission pad 110 may be distributed over a predetermined area on the transmission pad 110 . Therefore, when a user's hand or the like is placed in the electric field emitted from the transmission pad 110 or a specific gesture is made using the hand, the electric field is distorted. The transmitting pad 110 transmits the signal generated as the electric field is distorted to the receiving pad 120 . To this end, the transmission pad 110 may be electrically connected to the reception pad 120 or a signal may be transmitted through wireless communication.

The receiving pad 120 receives the signal transmitted from the transmitting pad 110 and senses the user's hand motion through a signal in which the electric field of the received signal is distorted. To this end, the receiving pad 120 includes a central portion 121, an eastern portion 123, a western portion 125, a southern portion 127, and a northern portion 129.

The central portion 121 is disposed to have a predetermined width. The central part 121 may recognize whether the user's gesture is fixed at a predetermined position for a predetermined period of time or longer without moving.

The east part 123 is disposed on the right side of the central part 121 and is disposed to have a predetermined width. The east part 123 may recognize when the user's gesture moves in the right direction (x-axis direction) with respect to the central part 121 .

The western portion 125 is disposed on the left side of the central portion 121 and is disposed to have a predetermined width. The western portion 125 may recognize when the user's gesture moves in the left direction (-x-axis direction) with respect to the central portion 121 .

The southern portion 127 is disposed below the central portion 121 and has a predetermined width. The southern portion 127 may recognize when the user's gesture moves in a downward direction (-y direction) with respect to the central portion 121 .

The northern portion 129 is disposed above the central portion 121 and has a predetermined width. The northern part 129 may recognize when the user's gesture moves in an upward direction (y-axis direction) with respect to the central part 121 .

In addition, the receiving pad 120 recognizes that the user's gesture moves in a complex manner to the central portion 121, the eastern portion 123, the western portion 125, the southern portion 127, and the northern portion 129, and responds to the user's gesture operation. It can recognize complex motions (eg, motions of drawing a circle, etc.)

The gesture detector 130 detects depth and direction of the user's gesture motion recognized from the receiving pad 120 . To this end, the gesture detector 130 includes a depth detection unit 131 and a direction detection unit 133.

The depth detector 131 detects a depth of a gesture operated by the user by using a distance between the user's hand and the transmission pad 110 on the transmission pad 110 .

The direction detection unit 133 detects directions using gestures of users recognized in the center 121, east 123, west 125, south 127, and north 129 of the receiving pad 120, respectively. .

In this way, the depth detection unit 131 and the direction detection unit 133 respectively detect the depth and direction of the user's gesture at a predetermined time interval (eg, 0.1 ms), and the pattern in which the electric field is distorted by the detected gesture. It is compared with the distortion pattern of the preset electric field.

The preset distortion pattern of the electric field may be a distortion pattern of the electric field learned and stored using an artificial neural network. That is, the preset distortion pattern of the electric field learns that the electric field is distorted according to the user's gesture, and the learned result can be continuously updated.

Therefore, the gesture detector 130 compares the user's gesture detected through the depth detection unit 131 and the direction detection unit 133 with a preset distortion pattern of the electric field, and when the specific pattern matches, the result of the detection of the specific pattern is displayed. It is passed on to the gesture recognizer 140.

The gesture recognizer 140 recognizes the user's gesture when the user's gesture corresponds to a specific pattern from the gesture detector 130, and also determines whether the recognized user's gesture matches the vehicle control command. The gesture recognizer 140 transmits a corresponding signal to the communicator when the user's gesture and the vehicle control command match, and recognizes the user's gesture again without transmitting the corresponding signal to the communicator when they do not match.

For example, the user's gesture matching the vehicle control command turns on or off the driver's seat light when the palm moves to the left (-x-axis direction) when the palm moves in both directions as shown in FIG. When the palm moves to the right (x-axis direction), the passenger seat light can be turned on or off. In addition, when the palm is not moved and maintained for a predetermined time or more (eg, 5 seconds or more), an emergency light may be turned on.

In addition, as shown in FIG. 8, when the palm is moved to the north (y-axis direction) or south (-y-axis direction), the sunroof may be partially opened or partially closed (eg, 10%), , When the palm is moved to the north (y-axis direction) or the south (-y-axis direction) twice, the sunroof may be fully opened or completely closed.

And, as shown in FIG. 9, the gesture of rotating the finger clockwise may be an operation to open the sunroof partially (eg, 5%), but if it is continuously rotated, it may be an operation to open the sunroof little by little each time it is rotated. there is. In addition, as shown in FIG. 10, the gesture of rotating the finger in a counterclockwise direction operates to partially close the sunroof (eg, 5%), but if it is continuously rotated, the sunroof is closed little by little each time it is rotated. can be

As shown in FIG. 4 , the microphone unit 150 includes a first microphone 151 , a second microphone 153 and a third microphone 155 .

The first microphone 151, the second microphone 153, and the third microphone 155 are each installed inside the vehicle and may be installed in different locations. For example, the first microphone 151 is disposed adjacent to the driver's seat of the vehicle, the second microphone 153 is disposed adjacent to the passenger seat of the vehicle, and the third microphone 155 is disposed adjacent to the front and rear seats of the vehicle. can be placed in between.

The microphone unit 150 recognizes voice signals through the first microphone 151, the second microphone 153, and the third microphone 155, respectively. At this time, the microphone unit 150 detects a signal of a predetermined dB or more among the voices recognized by the first microphone 151 , the second microphone 153 , and the third microphone 155 . That is, the microphone unit 150 does not recognize noise such as a small sound corresponding to a certain dB or less among voice signals recognized by each of the first microphone 151, the second microphone 153, and the third microphone 155. It recognizes only voice signals corresponding to a certain dB level or higher.

The voice preprocessor 160 preprocesses the voice signal recognized by the microphone unit 150 . To this end, the voice preprocessor 160 includes a noise removal unit 161, a reverberation control unit, a direction and position detection unit 165, and a beam forming unit 167.

The noise removal unit 161 removes noise from the voice signal recognized by the microphone unit 150 . The noise removal unit 161 removes noise from the voice signal by removing high and low frequency signals from the voice signal. The noise removal unit 161 sets a frequency range corresponding to the user's voice and removes a high frequency signal having a higher frequency than the set frequency range and a low frequency signal having a lower frequency than the set frequency range.

The reverberation removal unit 163 removes reverberation, which is a sound that remains even after the sound stops ringing in the voice signal corresponding to the frequency range set by the noise removal unit 161. As the reverberation is removed by the reverberation removal unit 163, only the user's voice signal can be clearly left in the voice signal recognized by the microphone unit 150.

The direction and position detection unit 165 determines which one of the first microphone 151, the second microphone 153, and the third microphone 155 of the microphone unit 150 receives the voice signal from which the reverberation has been removed by the reverberation removal unit 163. The direction and position of the user's voice signal are detected through whether it is recognized as the largest among the ones.

For example, when the first microphone 151 recognizes the largest voice signal and the second microphone 153 and the third microphone 155 recognize the loudest voice signal in that order, the position at which the voice signal is recognized is The direction and location of the voice signal generated between the first microphone 151 and the second microphone 153 or adjacent to the first microphone 151 are detected. Accordingly, when the first microphone 151 is disposed close to the driver's seat of the vehicle, the direction and position detection unit 165 may detect that the voice signal is generated in the driver's seat of the vehicle.

The beam forming unit 167 forms a beam for the voice signal by using the direction and position of the voice signal detected by the direction and position detection unit 165 . The beam of the voice signal formed by the beam forming unit 167 forms directivity in the direction in which the user's voice is generated, and can selectively obtain a voice signal input from a direction of interest.

That is, when a voice signal is generated in the driver's seat, as it passes through the beam forming unit 167, the voice signal can be more clearly processed for the voice signal generated in the driver's seat.

As described above, as the voice signal is preprocessed by the voice preprocessor 160, the voice signal output through the voice preprocessor 160 can be processed so that an optimal voice signal can be detected.

The audio signal detector 170 may receive the audio signal preprocessed by the audio preprocessor 160 and detect an optimal audio signal for the input audio signal. For example, when the location where the voice signal is generated is the driver's seat, the voice signal detector 170 may be more optimized for the voice signal generated in the driver's seat to detect the voice signal.

The voice pattern detector 180 compares the voice signal detected by the voice signal detector 170 with a preset voice pattern, and determines whether the detected voice signal corresponds to the preset voice pattern. The voice pattern detector 180 includes a voice memory 181 for this purpose. Preset voice patterns may be stored in the voice memory 181 .

Preset voice patterns stored in the voice memory 181 are, for example, open sunroof, open sunroof, open sunroof, close sunroof, close sunroof, close sunroof, open second stage of sunroof, sunroof 2 Close the door, driver's seat reading light on, driver's seat light on, driver's seat light on, left light on, left light on, left light on, left light on, driver's seat reading light off, driver's seat light off, driver's seat light off, left light off, left light off, Left light off, Left light off, Passenger seat reading light on, Passenger light on, Passenger light on, Right light on, Right light on, Right light on, Right light on, Passenger seat reading light off, Passenger light off, Passenger light off, Right light off, Right light Turn it off, turn off the right light, turn off the right light, turn on the interior light, turn on the interior light, turn on all lights, turn on all lights, turn on all lights, turn all lights on, turn off the interior light, turn off the interior light, turn off all lights, turn off all lights, all lights Off, all lights off, door interlock on, door interlock off, send SOS, SOS off, send SOS Test, SOS Test Off, save seat position 1, save seat position 2, set seat position 1, seat position 1 Burn mode, change seat position 1, set seat position 2, seat position 2 mode, change seat position 2, seat relax mode, cooling seat strong, turn cooling seat hard, cooling seat medium, cooling seat weak , Cooling sheet soft, Sheet cool, Chair cool, Cooling sheet off, Heat sheet strong, Heat sheet hard, Heat sheet medium, Heat sheet weak, Heat sheet weak, Seat warm, Chair warm, It may be: turn off heat seat, turn on thermostat, set temperature xx, turn on air conditioner, turn off air conditioner, turn on heater, turn off heater, outside air intake mode, outside air blocking mode, etc.

The voice recognizer 190 compares the detected voice signal with a preset voice pattern to determine whether it matches the control command of the vehicle. In this way, when the voice signal detected by the voice signal detector in the voice recognizer 190 matches the vehicle control command, the corresponding signal is transmitted to the communicator, and when it does not match, the voice signal is not transmitted to the communicator and the voice signal is recognized again. do.

The communication unit 200 transmits signals and voice signals for gestures recognized by the gesture recognizer 140 and the voice recognizer 190 to the vehicle electronic control unit (ECU) 10 . Accordingly, the vehicle electronic control unit 10 controls vehicle functions to correspond to the gesture signal and the voice signal received from the communication unit 200 .

11 is a diagram for explaining a gesture recognition method of a voice and gesture recognition apparatus according to an embodiment of the present invention.

Referring to FIG. 11 , a gesture recognition method in the voice and gesture recognition apparatus 100 will be described.

An electric field is generated in the transmission pad 110 (S101).

A predetermined electric field is emitted from the transmission pad 110 to recognize a user's hand gesture. Accordingly, the transmitting pad 110 transmits an electrical signal for a user's hand gesture to the receiving pad 120 through distortion of the electric field.

The receiving pad 120 detects the sensed electrical signal (S103).

The receiving pad 120 sequentially detects electric signals sensed from the central part 121, the eastern part 123, the western part 125, the southern part 127, and the northern part 129. The order of sequentially detecting signals of the central part 121, the eastern part 123, the western part 125, the southern part 127, and the northern part 129 from the receiving pad 120 may be changed.

The movement position by time is recorded for the detected electrical signal (S105).

The gesture detector 130 detects the depth and direction of the user's gesture at a predetermined time interval (eg, 0.1 ms). The depth of the gesture is measured using the distance between the user's hand and the transmission pad 110, and the direction of the gesture is detected through the direction in which the user's hand moves on a plane.

The electric field distortion of the detected gesture is compared with a preset electric field pattern (S107).

A pattern in which the electric field is distorted by the detected gesture is compared with a preset distortion pattern of the electric field, and when the pattern matches the specific pattern, the result of the detection of the specific pattern is output.

It is checked whether the distortion pattern of the electric field for the detected gesture matches the preset distortion pattern of the electric field (S109).

The preset distortion pattern of the electric field may be a distortion pattern of the electric field learned and stored using an artificial neural network. Therefore, when the detected user's gesture is compared with a preset distortion pattern of the electric field and matches a specific pattern, a result of the detection of the specific pattern is output.

The hand gesture is recognized (S111).

Recognition of the hand motion is performed in the gesture recognizer 140, and when the user's gesture corresponds to a specific pattern from the gesture detector 130, the user's gesture is recognized.

It is checked whether the gesture for the recognized hand motion matches the vehicle control command (S113).

The gesture recognizer 140 determines whether the gesture of the recognized hand motion matches the vehicle control command, and determines whether the recognized user's gesture matches the vehicle control command.

A communication command is transmitted (S115).

Therefore, if the recognized user's gesture matches the vehicle control command, the gesture recognizer 140 transmits the corresponding command to the communicator.

12 is a diagram for explaining a voice recognition method of a voice and gesture recognition apparatus according to an embodiment of the present invention.

Voice signals are sequentially received (S201).

Voice signals are sequentially received from the first microphone 151, the second microphone 153, and the third microphone 155 installed inside the vehicle. In this case, the first microphone 151, the second microphone 153, and the third microphone 155 may be installed at different locations inside the vehicle.

Among the received voice signals, a voice signal having a specific dB or higher level is detected (S203).

The first microphone 151, the second microphone 153, and the third microphone 155 detect a voice signal corresponding to a sound corresponding to a predetermined or more dB in order to minimize noise reception.

Noise is removed from the detected voice signal (S205).

Noise is a voice signal other than a frequency within a predetermined range corresponding to the user's voice signal. Accordingly, the noise removal unit 161 may remove noise by removing high frequencies higher than a predetermined range or low frequencies lower than a predetermined range.

Reverberation is removed from the noise-removed voice signal (S207).

The reverberation remover 163 may remove the reverberation from the noise-removed voice signal, leaving only a signal for the user's voice in the voice signal.

The direction and position of the voice signal from which the reverberation has been removed is detected (S209).

The direction and position detector 165 detects the direction and position of the voice signal from which the reverberation has been removed. The direction and position detection unit 165 determines the direction of the user's voice signal through which one of the first microphone 151, the second microphone 153, and the third microphone 155 of the microphone unit 150 is recognized as the loudest. and detect location.

A beam is formed for the audio signal whose direction and position are detected (S211).

The beam forming unit 167 forms a beam for the voice signal by using the direction and position of the voice signal detected by the direction and position detection unit 165 .

An optimal audio signal for the audio signal on which the beam is formed is detected (S213).

The audio signal detector 170 may receive the audio signal preprocessed by the audio preprocessor 160 and detect an optimal audio signal for the input audio signal.

The detected voice signal and voice pattern are compared (S215).

The voice pattern detector 180 compares the voice signal detected by the voice signal detector 170 with a preset voice pattern, and determines whether the detected voice signal corresponds to the preset voice pattern.

It is checked whether the pattern is a command for vehicle control with respect to the confirmed voice signal (S217).

The voice recognizer 190 compares the detected voice signal with a preset voice pattern to determine whether it matches the control command of the vehicle.

The confirmed voice signal is transmitted (S219).

In the voice recognizer 190, if the voice signal detected by the voice signal detector matches the vehicle control command, the corresponding signal is transmitted to the communicator, and if it does not match, the voice signal is not transmitted to the communicator and the voice signal is recognized again. .

As described above, the detailed description of the present invention has been made by the embodiments with reference to the accompanying drawings, but since the above-described embodiments have only been described as preferred examples of the present invention, it is believed that the present invention is limited only to the above embodiments. Should not be understood, the scope of the present invention should be understood as the following claims and equivalent concepts.

100: voice and gesture recognition device
110: transmission pad
120: receiving pad
121: central part
123: East
125: West
127: South
129: North
130: gesture detector
131: depth detector
133: direction detection unit
140: gesture recognizer
150: microphone unit
151: first microphone
153: second microphone
155: third microphone
160: voice preprocessor
161: noise removal unit
163: reverberation removal unit
165: direction and position detection unit
167: beam forming unit
170: voice signal detection unit
180: voice pattern detection unit
181 Voice memory
190: voice recognizer
200: Ministry of Communication
10: vehicle electronic control unit

Claims (8)

a transmitting pad that emits an electric field to detect a user's hand gesture;
a receiving pad that detects distortion of the electric field emitted from the transmitting pad;
a gesture recognizer recognizing the gesture by distortion of an electric field in the receiving pad;
a microphone unit for detecting the user's voice signal;
a voice pre-processor which performs pre-processing on the voice signal detected by the microphone unit, including removing noise of the voice signal;
a voice recognition unit recognizing the voice signal preprocessed by the voice preprocessor; and
A communicator for transmitting a signal and a voice signal for the gesture recognized by the gesture recognizer and the voice recognition unit to an electronic control unit of the vehicle,
Voice and gesture recognition devices. The method of claim 1,
Further comprising a gesture detector for detecting the depth and direction of the gesture by distortion of the electric field at the receiving pad,
The gesture recognizer recognizes the gesture using the depth and direction recognized by the gesture detector.
Voice and gesture recognition devices. The method of claim 2,
The gesture detector detects a gesture pattern by detecting the depth and direction of the gesture, compares the detected gesture pattern with a preset gesture pattern,
The gesture recognizer recognizes the gesture when the gesture pattern detected by the gesture detector and the preset gesture pattern match,
Voice and gesture recognition devices. The method of claim 1,
The microphone unit includes a plurality of microphones,
The plurality of microphones are disposed at different positions inside the vehicle,
Voice and gesture recognition devices. The method of claim 4,
The voice preprocessor,
a noise removal unit removing noise from the voice signal detected by the microphone unit;
a reverberation removal unit that removes reverberation from the voice signal from which the noise has been removed by the noise removal unit;
a direction and position detector for detecting a direction and position of a voice signal from the voice signal from which the reverberation has been removed by the reverberation remover; and
A beam forming unit for forming a beam for the audio signal whose direction and position are detected by the direction and position detection unit,
Voice and gesture recognition devices. The method of claim 5,
The noise removal unit removes noise by removing a signal corresponding to a high frequency higher than the frequency range of the user's voice signal and a signal corresponding to a low frequency lower than the frequency range of the user's voice signal.
Voice and gesture recognition devices. The method of claim 5,
Further comprising a voice signal detector for detecting a voice signal considering the direction and position of the user's voice signal from the voice signal preprocessed by the voice preprocessor;
The voice recognition unit recognizes the voice signal considered for the direction and location in the voice signal detector,
Voice and gesture recognition devices. The method of claim 7,
Further comprising a voice pattern detector for comparing the voice signal detected by the voice signal detector with a preset voice signal pattern and outputting the result;
The voice recognition unit recognizes the detected voice signal corresponding to the voice signal pattern preset in the voice pattern detector,
Voice and gesture recognition devices.

Images