CN112927685B - Dynamic voice recognition method and device thereof - Google Patents

Abstract

The invention provides a dynamic voice recognition method and a device. The dynamic voice recognition method comprises the following steps of executing a first stage: the voice data is detected by the digital microphone and stored in the first memory, the voice is detected in the voice data to generate a voice detection signal, and the second stage or the third stage is selectively determined and executed by the first processing circuit according to the total effective data amount, the transmission bit rate of the digital microphone and the identification interval time. And executing the second stage, wherein the first processing circuit outputs a first instruction to the second processing circuit, and the second processing circuit enables the memory access circuit to transfer the sound data to the second memory and store the sound data as voice data according to the first instruction. Executing the third stage, the first processing circuit outputs a second instruction to the second processing circuit, the second processing circuit instructs the memory access circuit to transfer the sound data to the second memory and store the sound data as voice data according to the second instruction, and the second processing circuit confirms whether the voice data matches with the preset voice instruction.

Description

Dynamic speech recognition method and device

Technical Field

The present invention relates to a speech detection and recognition technology, and in particular to a dynamic speech recognition method and device thereof.

Background Art

In existing electronic devices, voice assistant technology is widely used in various fields and supports voice wake-up function. When the voice assistant is in standby mode, it still needs to listen to hot words and give corresponding responses when hot words appear. Therefore, the voice assistant must be woken up regularly. The processing system of the voice assistant will start in standby mode to use the voice activity detection circuit to detect whether there is a human voice, and only when a human voice appears will it further enter voice recognition to confirm whether there are hot words in the human voice, and then determine whether to execute the system startup of the electronic device or perform corresponding operations.

However, the sensitivity of waking up the voice assistant regularly for detection is poor. At the same time, the processing system of the voice assistant also needs to meet low-power operation to meet relevant energy requirements.

Summary of the invention

In view of this, the present invention proposes a dynamic speech recognition method, which includes executing a first stage: using a digital microphone to detect sound data and store it in a first memory; detecting a human voice in the sound data and generating a human voice detection signal; and selectively determining to execute a second stage or a third stage through a first processing circuit according to the total effective data volume, the transmission bit rate of the digital microphone, and the recognition interval time. In executing the second stage, the first processing circuit outputs a first instruction to the second processing circuit, and the second processing circuit causes the memory access circuit to transfer the sound data to the second memory and store it as voice data according to the first instruction. In executing the third stage, the first processing circuit outputs a second instruction, and the second processing circuit causes the memory access circuit to transfer the sound data to the second memory and store it as voice data according to the second instruction, and the second processing circuit confirms whether the voice data in the second memory matches a preset voice instruction.

The present invention further proposes a dynamic speech recognition device, comprising a digital microphone, a first memory, a voice activity detection circuit, a memory access circuit, a second memory, a first processing circuit and a second processing circuit. The digital microphone is used to detect a sound data. The first memory is electrically connected to the digital microphone to store the sound data. The voice activity detection circuit is electrically connected to the digital microphone to detect the sound data and generate a human voice detection signal. The memory access circuit is electrically connected to the first memory to transfer the sound data to the second memory according to a first instruction to store it as voice data. The first processing circuit is electrically connected to the voice activity detection circuit. The second processing circuit is electrically connected to the first processing circuit, the second memory and the memory access circuit. Among them, this dynamic speech recognition device is used to execute the aforementioned dynamic speech recognition method.

According to some embodiments, when the first processing circuit receives the human voice detection signal, the first processing circuit outputs the first instruction or the second instruction after a recognition interval time.

According to some embodiments, the identification interval time is determined by a budget relationship value, when the budget relationship value is less than or equal to the target average power consumption * previous cycle time * 1/3, the identification interval time is 2 seconds; when the budget relationship value is greater than the target average power consumption * previous cycle time * 1/3 and less than or equal to the target average power consumption * previous cycle time * 2/3, the identification interval time is 1.5 seconds; and when the budget relationship value is greater than the target average power consumption * previous cycle time * 2/3, the identification interval time is 1 second.

According to some embodiments, the budget relationship value is target average power consumption*previous cycle time-(first average power consumption of first stage*first time of first stage+second average power consumption of second stage*second time of second stage+third average power consumption of third stage*third time of third stage), wherein the previous cycle time is equal to the sum of the first time, the second time and the third time.

According to some embodiments, the third average power consumption is greater than the second average power consumption, and the second average power consumption is greater than the first average power consumption.

According to some embodiments, after generating the human voice detection signal, the first processing circuit determines whether the first memory is full of sound data, and proceeds to the next step when the first memory is full of sound data.

In summary, the present invention takes user experience into consideration when performing dynamic voice recognition and triggers the search for a preset voice command (hot word) in standby mode, thereby reducing average power consumption and providing a method with better sensitivity.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.

FIG. 1 is a block diagram of an electronic device according to an embodiment of the present invention.

FIG. 2 is a schematic flow chart of a dynamic speech recognition method according to an embodiment of the present invention.

FIG. 3 is a waveform diagram of a dynamic speech recognition device according to an embodiment of the present invention.

FIG. 4 is a flow chart of a dynamic speech recognition method according to another embodiment of the present invention.

Description of reference numerals:

10 Electronic devices

20 Dynamic speech recognition device

21 digital microphones

22 First Memory

23 Voice activity detection circuit

24 memory access circuit

25 first processing circuit

26 second processing circuit

27 Second Memory

30 Audio and video processing circuit

31~33 core processing circuit

34～36 Third memory

C1 First Instruction

C2 second instruction

SD1 sound data

SD2 Voice Data

SS human voice detection signal

ST1 Phase 1

ST2 Phase 2

ST3 Phase 3

T cycle time

T1～T2 time

Ti identification interval time

Steps S10 to S28

Steps S30 to S36

DETAILED DESCRIPTION

FIG. 1 is a block diagram of an electronic device according to an embodiment of the present invention. Referring to FIG. 1 , the electronic device 10 includes a dynamic voice recognition device 20, a video processing circuit 30, a plurality of core processing circuits 31-33 and a plurality of third memories 34-36, and the plurality of core processing circuits 31-33 are all electrically connected to the third memories 34-36. When the dynamic voice recognition device 20 recognizes a preset voice command in standby mode, the electronic device 10 executes a system boot procedure so that the video processing circuit 30, the plurality of core processing circuits 31-33 and the plurality of third memories 34-36 can cooperate with each other to play the video signal received by the electronic device 10. In one embodiment, the electronic device 10 can be a television, but is not limited thereto.

The dynamic speech recognition device 20 includes a digital microphone 21, a first memory 22, a voice activity detection circuit 23, a memory access circuit 24, a first processing circuit 25, a second processing circuit 26, and a second memory 27. The digital microphone 21 is used to detect a sound data SD1. The first memory 22 is electrically connected to the digital microphone 21 and is used to store the sound data SD1. In one embodiment, the first memory 22 can be, but is not limited to, a static random access memory (SRAM).

The voice activity detection circuit 23 is electrically connected to the digital microphone 21 to detect the sound data SD1 and generate a human voice detection signal SS. In one embodiment, the voice activity detection circuit 23 can be but is not limited to a voice recognition chip or a voice recognition processing circuit.

The memory access circuit 24 is electrically connected to the first memory 22 and the second memory 27, and is used to transfer the sound data SD1 to the second memory 27 according to a first instruction, so as to store the sound data SD1 as a voice data SD2. In one embodiment, the memory access circuit 24 can be but not limited to a direct memory access (DMA) circuit, and the second memory 27 can be but not limited to a dynamic random access memory (DRAM).

The first processing circuit 25 is electrically connected to the voice activity detection circuit 23, and is used to generate a first instruction C1 or a second instruction C2 according to the human voice detection signal SS. The second processing circuit 26 is electrically connected to the first processing circuit 25, the second memory 27 and the memory access circuit 24. The second processing circuit 26 causes the memory access circuit 24 to transfer the sound data SD1 to the second memory 27 and store it as the voice data SD2 according to the first instruction C1; or the second processing circuit 26 causes the memory access circuit 24 to transfer the sound data SD1 to the second memory 27 and store it as the voice data SD2 according to the second instruction C2, and confirms whether the voice data SD2 in the second memory 27 matches a preset voice instruction. In one embodiment, the first processing circuit 25 can use a microcontroller with low power consumption, such as an 8051 microcontroller, but the present invention is not limited thereto. The second processing circuit 26 can use various types of processing circuits such as a general microprocessor, a microcontroller, a central processing unit, etc., but the present invention is not limited thereto.

In one embodiment, the first instruction C1 or the second instruction C2 is an instruction for modifying a shared state.

FIG2 is a flow chart of a dynamic speech recognition method according to an embodiment of the present invention, and FIG3 is a waveform chart of a dynamic speech recognition device according to an embodiment of the present invention. Please refer to FIG1 , FIG2 and FIG3 simultaneously. The dynamic speech recognition method includes utilizing a dynamic speech recognition device 20 to execute a first stage ST1 (steps S10 to S18, step S22) and to execute a second stage ST2 (step S20) or a third stage ST3 (steps S24 to S26). The following is a detailed description of each stage.

In the first stage ST1 (pure standby stage), as shown in step S10, the digital microphone 21 is used to detect the sound data SD1, and the sound data SD1 is stored in the first memory 22. As shown in step S12, the voice activity detection circuit 23 detects whether there is a human voice in the sound data SD1, and is triggered to generate a human voice detection signal SS when a human voice is detected in the sound data SD1, and transmits the human voice detection signal SS to the first processing circuit 25. As shown in step S14, the first processing circuit 25 determines whether the first memory 22 is full of sound data SD1, and proceeds to the next step S16 when the sound data SD1 is full, so as to ensure that there is enough sound data SD1 for the subsequent steps. As shown in step S16, the first processing circuit 25 selectively determines to execute the second stage ST2 (DMA stage) or the third stage ST3 (voice recognition stage) according to a total effective data amount, the transmission bit rate of the digital microphone 21 and an identification interval time Ti.

In one embodiment, the target average power consumption, the first average power consumption of the first stage ST1, the second average power consumption of the second stage ST2, and the third average power consumption of the third stage ST3 are known, and the time occupied by each stage in the previous cycle time T has been obtained, including the first time Ta of the first stage ST1, the second time Tb of the second stage ST2, and the third time Tc of the third stage ST3, wherein the previous cycle time T is equal to the sum of the first time Ta, the second time Tb, and the third time Tc, that is, T = Ta + Tb + Tc. In one embodiment, this cycle time T can be but is not limited to 16 seconds. Therefore, a budget relationship value (Budget) related to power use can be obtained through the above parameters, and this budget relationship value is the target average power consumption * previous cycle time T-(the first average power consumption of the first stage ST1 * the first time Ta of the first stage ST1 + the second average power consumption of the second stage ST2 * the second time Tb of the second stage ST2 + the third average power consumption of the third stage ST3 * the third time Tc of the third stage ST3).

After obtaining the budget relationship value, the identification interval time Ti can be dynamically determined according to the budget relationship value. Specifically, when the budget relationship value is less than or equal to the target average power consumption * the previous cycle time T*1/3, the identification interval time Ti is determined to be 2 seconds. When the budget relationship value is greater than the target average power consumption * the previous cycle time T*1/3 and less than or equal to the target average power consumption * the previous cycle time T*2/3, the identification interval time Ti is determined to be 1.5 seconds. When the budget relationship value is greater than the target average power consumption * the previous cycle time T*2/3, the identification interval time Ti is determined to be 1 second. Next, it is known that the total effective data volume is the sum of the effective data volume of the first memory 22 and the effective data volume of the second memory 27, and the transmission bit rate of the digital microphone 21. Therefore, when the total effective data volume is less than the product of the transmission bit rate of the digital microphone 21 and the identification interval time, the first processing circuit 25 determines to execute the DMA stage of the second stage ST2. When the total effective data amount is greater than or equal to the product of the transmission bit rate of the digital microphone 21 and the recognition interval time, the first processing circuit 25 determines to execute the speech recognition stage of the third stage ST3 .

When the first processing circuit 25 decides to execute the second stage ST2, as shown in step S18, the first processing circuit 25 first wakes up the second processing circuit 26, and then enters the second stage ST2. In the second stage ST2, as shown in step S20, the first processing circuit 25 outputs the first instruction C1 to the second processing circuit 26, and the second processing circuit 26 causes the memory access circuit 24 to transfer the sound data SD1 in the first memory 22 to the second memory 27 according to the first instruction C1, so as to store it as voice data SD2. In the second stage ST2, the voice data SD2 is only converted to the second memory 27 through the memory access circuit 24, and voice recognition is not required.

When the first processing circuit 25 decides to execute the third stage ST3, as shown in step S22, the first processing circuit 25 first wakes up the second processing circuit 27, and then enters the third stage ST3. In the third stage ST3, as shown in step S24, the first processing circuit 25 outputs the second instruction C2 to the second processing circuit 26, and the second processing circuit 26 then causes the memory access circuit 24 to transfer the sound data SD1 in the first memory 22 to the second memory 27 according to the second instruction C2 to store as voice data SD2, and confirm whether the voice data SD2 in the second memory 27 matches the preset voice instruction. As shown in step S26, the second processing circuit 26 determines whether the voice data SD2 in the second memory 27 matches the preset voice instruction. If the voice data SD2 is confirmed to match the preset voice instruction, the system boot procedure is executed as shown in step S28 to wake up other circuits, including the audio and video processing circuit 30, the core processing circuits 31-33 and the third memories 34-36, etc. to boot the system.

FIG4 is a flow chart of a dynamic speech recognition method according to another embodiment of the present invention. Please refer to FIG1 , FIG3 and FIG4 simultaneously. The dynamic speech recognition method includes utilizing the dynamic speech recognition device 20 to execute a first stage ST1 (steps S10 to S16) and to execute a second stage ST2 (step S30) or a third stage ST3 (steps S32 to S34). The following is a detailed description of each stage.

In the first stage ST1 (pure standby stage), as shown in step S10, the digital microphone 21 is used to detect the sound data SD1, and the sound data SD1 is stored in the first memory 22. As shown in step S12, the voice activity detection circuit 23 detects whether there is a human voice in the sound data SD1, and when the human voice is detected, it will be triggered to generate a human voice detection signal SS and transmit it to the first processing circuit 25. As shown in step S14, the first processing circuit 25 determines whether the first memory 22 is full of sound data SD1, and proceeds to the next step S16 when the sound data SD1 is full, so as to ensure that there is enough sound data SD1 for the subsequent steps. As shown in step S16, the first processing circuit 25 selectively determines to execute the second stage ST2 (DMA stage) or the third stage ST3 (voice recognition stage) according to a total effective data amount, the transmission bit rate of the digital microphone 21 and an identification interval time Ti.

When the first processing circuit 25 decides to execute the second stage ST2, as shown in step S30, in the second stage ST2, the first processing circuit 25 outputs the first instruction C1 and wakes up the second processing circuit 26. The second processing circuit 26 causes the memory access circuit 24 to transfer the sound data SD1 in the first memory 22 to the second memory 27 according to the first instruction C1 to store it as voice data SD2.

When the first processing circuit 25 decides to execute the third stage ST3, as shown in step S32, in the third stage ST3, the first processing circuit 25 outputs the second instruction C2 and wakes up the second processing circuit 26. The second processing circuit 26 causes the memory access circuit 24 to transfer the sound data SD1 in the first memory 22 to the second memory 27 according to the second instruction C2 to store it as voice data SD2, and confirms whether the voice data SD2 in the second memory 27 matches the preset voice instruction. As shown in step S34, the second processing circuit 26 determines whether the voice data SD2 in the second memory 27 matches the preset voice instruction. If the voice data SD2 is confirmed to match the preset voice instruction, the system boot procedure is executed as shown in step S28 to wake up all circuits to boot up the system.

The multiple steps (S10-S26 and S30-S34) of the above-mentioned dynamic speech recognition method are only examples and are not limited to the order of execution of the above-mentioned examples. Without violating the spirit and scope of the present invention, various operations under the dynamic speech recognition method can be appropriately increased, replaced, omitted or executed in a different order.

In one embodiment, when the first processing circuit 25 receives the human voice detection signal SS, the first processing circuit 25 will output the first instruction C1 or the second instruction C2 after the identification interval time Ti. As shown in FIG1 and FIG3, when the first processing circuit 25 receives the human voice detection signal SS at time T1, the first processing circuit 25 will output the first instruction C1 or the second instruction C2 at time T2 after the identification interval time Ti, wherein the identification interval time Ti can be dynamically determined based on the aforementioned method to ensure that the second processing circuit 26 and the second memory 27 are enabled only after the received sound data SD1 is sufficient to reflect the preset voice instruction, so that low-power operation can be satisfied to meet the relevant specifications of energy requirements.

In one embodiment, if the keyword set by the preset voice command is "Hi, TV", please refer to FIG. 1 and FIG. 3. At time T1, the digital microphone 21 detects external sound and generates sound data SD1, and the first memory 22 stores the sound data SD1. For example, the digital microphone 21 detects that the user says the voice command "Hi, TV..." to the dynamic voice recognition device 20. At the same time, the voice activity detection circuit 23 determines that the sound data SD1 has a human voice and outputs a human voice detection signal SS. At time T2, the first processing circuit 25 outputs the first command C1 or the second command C2. The second processing circuit 26 and the second memory 27 are also enabled. At this time, the second processing circuit 26 enables the memory access circuit 24 according to the first command C1 or the second command C2 to transfer the sound data SD1 to the second memory 27 and store it as voice data SD2. Therefore, the second processing circuit 26 can analyze the voice data SD2 to confirm whether the voice data SD2 matches the preset voice command "Hi, TV", and the second processing circuit 26 confirms that the voice data SD2 matches the preset voice command to wake up other circuits to execute the system boot procedure.

In one embodiment, the first stage ST1 uses the digital microphone 21, the first memory 22, the voice activity detection circuit 23 and the first processing circuit 25 in the dynamic speech recognition device 20. The second stage ST2 uses the digital microphone 21, the first memory 22, the voice activity detection circuit 23, the memory access circuit 24, the first processing circuit 25, part of the second processing circuit 26 (only part of the function of the second memory is activated) and the second memory 27 in the dynamic speech recognition device 20. The third stage ST3 uses all the circuits of the digital microphone 21, the first memory 22, the voice activity detection circuit 23, the memory access circuit 24, the first processing circuit 25, the second processing circuit 26 and the second memory 27 in the dynamic speech recognition device 20. Therefore, the third average power consumption of the third stage ST3 is greater than the second average power consumption of the second stage ST2, and the second average power consumption is greater than the first average power consumption of the first stage ST1. For example, the power consumption corresponding to the first stage ST1 is about 0.5 watts, the power consumption corresponding to the third stage ST3 is 4 watts, and the power consumption corresponding to the second stage ST2 is between the two.

Therefore, the present invention can determine the budget relationship value according to the time occupied by each stage in the previous cycle time T (the first time, the second time and the third time) and the average power consumption of each stage, so as to dynamically determine the length of the recognition interval time Ti according to the budget relationship value, and then judge whether it is necessary to recognize the voice data (execute the second stage ST2 or the third stage ST3) accordingly, so that the voice recognition can be dynamically performed according to the power consumption of the actual operation. Therefore, the present invention can take the user experience into consideration when performing dynamic voice recognition, and can reduce the average power consumption when triggering the search for the preset voice command in the standby mode, so as to provide a method with better sensitivity.

The embodiments described above are only for illustrating the technical ideas and features of the present invention, and their purpose is to enable those familiar with the technology to understand the contents of the present invention and implement them accordingly. They cannot be used to limit the patent scope of the present invention. That is, all equivalent changes or modifications made according to the spirit disclosed by the present invention should still be included in the patent scope of the present invention.

Claims (9)

1. A dynamic speech recognition method, comprising:

A first phase is performed:

detecting sound data by a digital microphone and storing the sound data in a first memory;

Detecting voice in the voice data to generate a voice detection signal; and

Selectively determining to execute a second stage or a third stage by a first processing circuit according to a total effective data amount, a transmission bit rate of the digital microphone and an identification interval time;

the second phase is performed:

the first processing circuit outputs a first instruction to a second processing circuit, and the second processing circuit enables a memory access circuit to transfer the sound data to a second memory and store the sound data as voice data according to the first instruction; and

The third phase is performed:

The first processing circuit outputs a second instruction to the second processing circuit, the second processing circuit enables the memory access circuit to transfer the sound data to the second memory and store the sound data as the voice data according to the second instruction, and the second processing circuit confirms whether the voice data in the second memory matches a preset voice instruction, wherein the first processing circuit determines to execute the second stage when the total effective data amount is smaller than the product of the transmission bit rate of the digital microphone and the identification interval time; and when the total effective data amount is greater than or equal to the product of the transmission bit rate of the digital microphone and the identification interval time, the first processing circuit determines to execute the third stage, wherein the total effective data amount is the sum of the effective data amount of the first memory and the effective data amount of the second memory.

2. The method of claim 1, wherein the first processing circuit outputs the first command or the second command after the recognition interval time when the first processing circuit receives the voice detection signal.

3. The method of claim 2, wherein the recognition interval is determined by a budget relationship value, the recognition interval being 2 seconds when the budget relationship value is less than or equal to 1/3 of a period before the target average power consumption; the identification interval time is 1.5 seconds when the budget relation value is greater than the target average power consumption by 1/3 of the previous period time and less than or equal to the target average power consumption by 2/3 of the previous period time; and the identification interval is 1 second when the budget is greater than the target average power consumption by 2/3 of the previous period.

4. The method of claim 3, wherein the budget relationship value is the target average power consumption for the previous period of time- (the first average power consumption for the first phase of time + the first average power consumption for the second phase of time + the second average power consumption for the second phase of time + the third average power consumption for the third phase of time), wherein the previous period of time is equal to the sum of the first time, the second time and the third time.

5. The method of claim 4, wherein the third average power consumption is greater than the second average power consumption, and the second average power consumption is greater than the first average power consumption.

6. The method of claim 1, wherein after the step of generating the voice detection signal, further comprising: judging whether the first memory is full of the sound data, and continuing to proceed to the next step when the sound data is full.

7. The method of claim 1, wherein in the step of performing the first stage, after selectively determining to perform the second stage or the third stage, further comprising: the first processing circuit wakes up the second processing circuit.

8. The method of claim 1, wherein the first processing circuit wakes up the second processing circuit when the first processing circuit outputs the first instruction or the second instruction.

9. A dynamic speech recognition device, comprising:

a digital microphone for detecting a sound data;

A first memory electrically connected to the digital microphone for storing the sound data;

a voice activity detection circuit electrically connected to the digital microphone for detecting the voice data and generating a voice detection signal;

a memory access circuit electrically connected to the first memory, the memory access circuit transferring the audio data to a second memory for storing as a voice data;

A first processing circuit electrically connected to the voice activity detection circuit; and

The second processing circuit is electrically connected with the first processing circuit, the second memory and the memory access circuit;

the dynamic voice recognition device is used for executing the following steps:

A first phase is performed:

detecting the sound data by the digital microphone and storing the sound data in the first memory;

The voice activity detection circuit detects voice in the voice data to generate the voice detection signal; and

Selectively determining to execute a second stage or a third stage by the first processing circuit according to a total effective data amount, a transmission bit rate of the digital microphone and an identification interval time;

the second phase is performed:

The first processing circuit outputs a first instruction to the second processing circuit, and the second processing circuit enables the memory access circuit to transfer the sound data to the second memory and store the sound data as the voice data according to the first instruction; and

The third phase is performed:

The first processing circuit outputs a second instruction to the second processing circuit, the second processing circuit enables the memory access circuit to transfer the sound data to the second memory and store the sound data as the voice data according to the second instruction, and the second processing circuit confirms whether the voice data in the second memory matches a preset voice instruction, wherein the first processing circuit determines to execute the second stage when the total effective data amount is smaller than the product of the transmission bit rate of the digital microphone and the identification interval time; and when the total effective data amount is greater than or equal to the product of the transmission bit rate of the digital microphone and the identification interval time, the first processing circuit determines to execute the third stage, wherein the total effective data amount is the sum of the effective data amount of the first memory and the effective data amount of the second memory.