CN114420124B - Speech recognition method - Google Patents

Abstract

The application provides a voice recognition method, in the application, after recognizing the similar throwing devices of the environment in which the current moment is located, when carrying out voice recognition, two throwing devices are started simultaneously, one throwing device is used for collecting audio signals containing user audio, the other throwing device is used for collecting mixed audio of audio and noise in the environment, then audio separation is carried out on the audio signals containing the user audio through the mixed audio, so that the user audio which is less influenced by the environment is obtained, then voice recognition is carried out on the user audio, and therefore a voice instruction of a user is obtained.

Description

A method of speech recognition

technical field

The present application relates to the technical field of speech recognition, and in particular, to a speech recognition method.

Background technique

With the development of science and technology, speech recognition technology is applied to more and more fields to assist users in using equipment.

For medical testing equipment put on the market, in order to facilitate self-service use by patients, a voice recognition function is installed on the medical testing equipment. For example: the patient's voice can be "I want to detect blood pressure", then the medical detection device recognizes the text information through the voice recognition function, and then determines the function that the patient wants to enable through the semantics of the text information, and then starts the blood pressure detection function.

When performing speech recognition on medical testing equipment on the market, it will be affected by the surrounding audio and noise, such as: the sound of vehicles driving on the street, the sound of surrounding people, and the electromagnetic noise in the surrounding environment, etc. , these ambient audio and noise will affect speech recognition, resulting in inaccurate speech recognition results.

SUMMARY OF THE INVENTION

In view of this, the embodiments of the present application provide a speech recognition method to improve the accuracy of the speech recognition result.

The application provides a speech recognition method, the method includes:

Within the preset time period, when the delivery device enables the voice function, the voice acquisition module and the video data acquisition module are simultaneously activated to collect audio data and video data with the same time attribute;

The four-point labeling method is used to label the two corners of the lips and the lips of the upper and lower lips collected in the video data;

Measure the opening angle of the mouth corner, the distance between the upper and lower lips, and the distance between the two mouth corners when the mouth shape changes each time in the video data, so as to obtain multiple sets of measurement data, wherein the mouth corner opening angle Including a first included angle and a second included angle, the first included angle is the included angle between the connection line between the upper lip lip and the corner of the mouth and the connecting line between the two mouth corners, and the second included angle is the lower lip lip The angle between the line between the center and the corners of the mouth and the line between the two corners of the mouth;

The mouth opening angle, the spacing and the distance in each group of measurement data are input to the evaluation function as input parameters to obtain lip change data, wherein the speech evaluation function is: Pre=k*(p *Angle(a,b)+q*Line(LAF,LDE)), k, p, q are the weight coefficients of each cost function, Angle(a,b) is the first angle and the second angle The cost function of the angle, Line(LAF, LDE) is the cost function of the distance and the distance;

Using the lip change data and the lip shape database to compare and obtain a lip language recognition result, and carry out audio recognition to the audio data to obtain a speech recognition result;

According to the lip language recognition result, determine the accuracy rate of the speech recognition result, the proportion of incorrectly recognized characters in the speech recognition result in each division of the entire speech, and the percentage of consecutive typos in the speech recognition result. quantity;

Weighted summation is performed on the accuracy rate, the proportion and the quantity to obtain a speech recognition feature value representing the speech recognition feature of the delivery device;

After obtaining the speech recognition feature value of each of the delivery devices, for each of the delivery devices, calculate the first difference between the speech recognition feature values of other delivery devices and the speech recognition feature value of the delivery device;

Sorting the first difference values according to a specified order to obtain a similarity ranking indicating that the delivery device and other delivery devices are similarly affected by the surrounding environment and noise;

After the similarity ranking corresponding to the delivery device is obtained, when the voice function is enabled on the delivery device again, a target delivery device is selected from other delivery devices according to the similarity ranking corresponding to the delivery device, and the target delivery device is activated. The voice collection module collects the target audio signal, wherein the target delivery device is the delivery device with the highest similarity ranking among the delivery devices not used by the user when the delivery device enables the voice function again;

According to the audio feature of the target audio signal, audio separation is performed on the audio signal collected by the delivery device, so as to obtain the user audio collected when the delivery device enables the voice function again;

Perform voice recognition according to the user audio to obtain the user's voice command.

Optionally, after obtaining the lip language recognition result and the speech recognition result, the similarity ranking can also be obtained through the following steps:

According to the lip language recognition result, determine the accuracy rate of the speech recognition result, the proportion of incorrectly recognized characters in the speech recognition result in each division of the entire speech, and the percentage of consecutive typos in the speech recognition result. Quantity; for each delivery device, calculate the first similarity of the accuracy rate, the second similarity of the proportion, and the third similarity of the number of consecutive typos between the delivery device and other delivery devices; The corresponding first similarity, the second similarity and the third similarity are weighted and summed to obtain the similarity used to indicate that the delivery device and other delivery devices are affected by the environment and noise; according to the specified Ranking the similarity in order to obtain a similarity ranking indicating that the delivery device and other delivery devices are similarly affected by the surrounding environment and noise; or,

Perform text similarity calculation on the text information corresponding to the lip language recognition result and the text information corresponding to the speech recognition result to obtain the text similarity corresponding to the delivery device; for each delivery device, calculate the delivery device The second difference value between the text similarity of , and the text similarity of other delivery devices; sort the second difference according to the specified order to obtain a value used to represent that the delivery device and other delivery devices are affected by the surrounding environment and noise. Affects similarity ranking for similar situations.

Optionally, according to the audio characteristics of the target audio signal, audio separation is performed on the audio signal collected by the delivery device to obtain the user audio collected when the delivery device enables the voice function again, including:

Aligning the audio frame with the same acquisition moment in the audio signal collected by the target audio signal and the throwing device, to obtain multiple groups of audio frame groups with the same audio acquisition moment;

For each group of audio frames, use the audio feature of the audio frame corresponding to the target audio signal in the group of audio frames to perform audio separation on the audio frame corresponding to the audio signal collected by the delivery device to obtain the corresponding audio frame group. User audio frames collected when the delivery device enables the voice function again;

The user audio frames corresponding to the multiple audio frame groups are spliced to obtain the user audio collected when the delivery device enables the voice function again.

Since the audio and noise in the surrounding environment will change over time, for example, the audio and noise in the surrounding environment will be higher than other time periods during the morning peak and evening peak, so in this application, it is necessary to periodically collect audio In order to determine the delivery devices with similar audio and noise in the surrounding environment in the corresponding time period, when determining the delivery devices with similar audio and noise in the surrounding environment, for each device, first determine the lip in the current time period. Speech recognition results and speech recognition results. Since the lip recognition results are not affected by the audio and noise in the surrounding environment, the lip recognition results can be used to determine the accuracy of the speech recognition results, as well as the wrongly recognized words in the speech recognition results. The proportion in each division of the entire speech and the number of consecutive typos in the speech recognition result, where the accuracy of the speech recognition result is used to represent the influence of the audio and noise in the surrounding environment on the speech recognition result, and the speech recognition The proportion of the incorrectly recognized words in each division of the whole speech in the result is used to represent the distribution of the incorrectly recognized words in the whole speech. The occurrence of audio and noise, the above three parameters are used to determine the first difference between the speech recognition feature value of other delivery devices and the speech recognition feature value of the delivery device, and the first difference is used to represent the The similarity of the environment where the delivery device and other delivery devices are located, and then the first difference is sorted, and the obtained sorting result is used to indicate the similarity ranking of the delivery device and other delivery devices affected by the surrounding environment and noise. When using a delivery device to enable the voice function again, in order to reduce the impact of audio and noise in the environment on the delivery device, when enabling the voice function of the delivery device, the delivery device that is not used by the user is also activated at the same time. The voice acquisition module of the target delivery device with the highest similarity ranking. At this time, the delivery device can collect mixed audio, and the target delivery device can collect the mixed audio composed of audio and noise in the surrounding environment, and then pass the audio in the target delivery device. The audio characteristics of the mixed audio perform audio separation on the audio signal collected by the throwing device to obtain the user's audio collected by the throwing device. Since the obtained user's audio reduces the impact of the audio and noise in the surrounding environment on it. Therefore, it is beneficial to improve the accuracy of the speech recognition result, which in turn is beneficial to improve the accuracy of obtaining the speech command.

In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the preferred embodiments are exemplified below, and are described in detail as follows in conjunction with the accompanying drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present application more clearly, the following drawings will briefly introduce the drawings that need to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore do not It should be regarded as a limitation of the scope, and for those of ordinary skill in the art, other related drawings can also be obtained according to these drawings without any creative effort.

1 is a schematic flowchart of a speech recognition method provided by an embodiment of the present application;

FIG. 2 is a schematic flowchart of another speech recognition method provided by an embodiment of the present application.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only It is a part of the embodiments of the present application, but not all of the embodiments. The components of the embodiments of the present application generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Thus, the following detailed description of the embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the application as claimed, but is merely representative of selected embodiments of the application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

The following embodiments can be applied to human health management to assist people in a healthier life. The following is a detailed description of the present application.

FIG. 1 is a schematic flowchart of a speech recognition method provided by an embodiment of the present application. As shown in FIG. 1 , the speech recognition method includes the following steps:

Step 101: Within a preset time period, when the delivery device enables the voice function, the voice collection module and the video data collection module are simultaneously activated to collect audio data and video data with the same time attribute.

Specifically, the types of devices to be placed include blood pressure detection devices, heart rate detection devices, or multi-function integrated detection devices, etc. There are multiple placement devices, and they are placed in different locations to detect and cover a certain area. Due to the surrounding environment The audio and noise will change over time, for example: the audio and noise in the surrounding environment will be higher than other time periods during the morning peak and evening peak, so it is necessary to collect audio data periodically to determine the corresponding time. The influence of audio and noise in the surrounding environment in the segment on the speech recognition results. At this time, it is necessary to start the speech acquisition module and the video data acquisition module at the same time to collect audio data and video data with the same time attribute. Speech is not affected by the audio and noise in the surrounding environment, so the degree of influence of the audio and noise in the surrounding environment on the speech recognition result can be determined according to the video data.

Step 102: Use the four-point labeling method to label the two corners of the lips and the upper and lower lips collected in the video data.

Specifically, since different pronunciations correspond to different mouth shapes, the different mouth shapes here include: the size of the open mouth, the distance between the two corners of the mouth, and the line between the two corners of the mouth and the upper lip In order to obtain these data, it is necessary to mark the two mouth corners of the lips and the lips of the upper and lower lips collected in the video data, so that according to the The data obtained by marking the connection between the points is used to obtain the lip language recognition result.

It should be noted that other numbers of labels can also be used, for example: the six-point labeling method, the six points are points A and F at the corners of the mouth on both sides, and the midpoints of the left and right lips in the upper lip are points B and G. The midpoints of the left and right lips in the lower lip are points C and H. The specific labeling method can be set according to the actual required accuracy, and will not be described in detail here.

Step 103: Measure the opening angle of the corner of the mouth, the distance between the upper and lower lips, and the distance between the two corners of the mouth when each mouth shape changes in the video data, to obtain multiple sets of measurement data, wherein the corners of the mouth are The opening angle includes a first included angle and a second included angle, the first included angle is the included angle between the line connecting the middle of the upper lip and the corners of the mouth and the connecting line between the two corners of the mouth, and the second included angle is The angle between the line between the middle of the lower lip and the corners of the mouth and the line between the two corners of the mouth.

Step 104: Input the mouth opening angle, the spacing and the distance in each group of measurement data as input parameters to the evaluation function to obtain lip change data, wherein the speech evaluation function is: Pre=k *(p*Angle(a,b)+q*Line(LAF,LDE)), k, p, q are the weight coefficients of each cost function, Angle(a,b) is the first angle and the The cost function of the second included angle, Line(LAF, LDE) is the cost function of the distance and the distance.

It should be noted that LAF is the distance between the two corners of the mouth, and LDE is the distance between the upper and lower lips. Since different regions have different pronunciation methods, p, q, and k can be determined according to the placement area of the device. set up.

Step 105: Use the lip change data and the lip shape database to compare to obtain a lip language recognition result, and perform audio recognition on the audio data to obtain a speech recognition result.

Step 106: Determine the accuracy of the speech recognition result according to the lip language recognition result, the proportion of incorrectly recognized characters in the speech recognition result in each division of the entire speech, and the speech recognition result. The number of consecutive typos.

Specifically, the accuracy of the speech recognition result is used to represent the degree of influence of audio and noise in the surrounding environment on the overall speech recognition result, and the proportion of incorrectly recognized words in the speech recognition result in each division of the entire speech is used for Characterize the distribution of the wrongly recognized words in the whole speech. The number of consecutive wrong words in the speech recognition result is used to indicate the occurrence of sudden audio and noise in the surrounding environment. The above three parameters can be used to determine the surrounding environment. General information about audio and noise.

Step 107: Perform a weighted summation on the accuracy rate, the proportion and the number to obtain a speech recognition feature value used to represent the speech recognition feature of the delivery device.

Specifically, since different data have different influences on the accuracy of the speech recognition result, it is necessary to assign different weights to the above three parameters, and then perform a weighted summation, so as to use the obtained result as a result of the speech recognition characteristics of the delivery device. Speech recognition feature value.

It should be noted that the specific weight distribution of the above three parameters can be set according to actual needs, which is not specifically limited here.

Step 108: After obtaining the speech recognition feature values of each of the delivery devices, for each of the delivery devices, calculate the first difference between the speech recognition feature values of other delivery devices and the speech recognition feature value of the delivery device. .

Specifically, the first difference is used to represent the similarity of the environments where the two delivery devices are located, wherein the smaller the first difference is, the higher the similarity of the environments where the two delivery devices are located.

Step 109: Rank the first difference values according to a specified order to obtain a similarity ranking indicating that the delivery device and other delivery devices are similarly affected by the surrounding environment and noise.

Specifically, after sorting the first difference, it is possible to know the similarity of the environment where a certain delivery device and other delivery devices are located, so that it can be determined according to the ranking that the delivery device and other delivery devices are affected by the surrounding environment and noise Ranking of similar situations, for example, the placement devices include: placement device 1, placement device 2, placement device 3, and placement device 4. Taking placement device 1 as an example, after obtaining the difference between placement device 1 and placement device 2, placement After the difference between device 1 and delivery device 3, and the difference between delivery device 1 and delivery device 4, the above three differences are sorted, so that delivery device 2, delivery device 3 and delivery device 4 can be obtained. The similarity ranking with the environment where the delivery device 1 is located.

It should be noted that, for different delivery devices, the obtained similarity rankings may be different. For example, after ranking the environment where delivery device 1 is located, the delivery device with the highest similarity may be delivery device 2, but for After ranking the environments where the placement device 2 is located, the placement device with the highest similarity may be the placement device 3 .

Step 110: After obtaining the similarity ranking corresponding to the delivery device, when the delivery device enables the voice function again, select a target delivery device from other delivery devices according to the similarity ranking corresponding to the delivery device, and activate the target delivery device. The voice collection module of the delivery device collects the target audio signal, wherein the target delivery device is the delivery device with the highest similarity ranking among the delivery devices not used by the user when the delivery device enables the voice function again.

Specifically, after obtaining the similarity ranking corresponding to each delivery device, when a certain delivery device is used by the user again, when enabling the voice function of the delivery device, it is necessary to simultaneously activate other delivery devices that are not used, and are not in use with the delivery device. The delivery device with the highest similarity in the environment where the delivery device is located shall be the delivery device 2, delivery device 3 and delivery device 4 in descending order of the similarity with the delivery device 1 where the voice function is used again. , when the voice function of the delivery device 1 is used again, it is necessary to detect whether the delivery device 2 is used. If it is not used, when the voice function of the delivery device 1 is activated, the voice acquisition module of the delivery device 2 is activated at the same time, so that the delivery device 2. The audio and noise of the surrounding environment can be collected. If the delivery device 2 is currently in use, it is necessary to detect whether the delivery device 3 is used. If it is not used, when the voice function of the delivery device 1 is activated, simultaneously start the delivery device 3. The voice collection module of the device 3, so that the delivery device 3 can collect the audio and noise of the surrounding environment, and so on, until the target delivery device is obtained.

Step 111: Perform audio separation on the audio signal collected by the delivery device according to the audio feature of the target audio signal, so as to obtain the user audio collected when the delivery device enables the voice function again.

Specifically, after the voice collection module of the target delivery device collects the target audio signal composed of audio and noise in the environment, the audio signal collected by the delivery device can be separated by using the audio features of the target audio signal, thereby The user audio of the user collected by the delivery device is obtained. Since the target delivery device and the environment where the delivery device is currently located are highly similar, after audio separation of the audio signal collected by the delivery device, the obtained user audio can be reduced. Audio is affected by audio and noise in the surrounding environment, which is beneficial to improve the accuracy of subsequent speech recognition.

It should be noted that the specific separation method used can be set according to actual needs, which is not specifically limited here.

Step 112: Perform voice recognition according to the user audio to obtain the user's voice command.

Specifically, after the speech recognition is performed, the text information corresponding to the user's audio can be obtained, and then the user's user instruction can be determined through semantic analysis. For example, when the text information recognized by the speech is "I want to detect blood pressure", then the semantic After the analysis, it can be determined that the user wants to activate the blood pressure detection function at this time, and the blood pressure detection module can be activated at this time.

Of course, after the function that the user wants to activate is determined, a reminder message on whether to activate the corresponding function can also be displayed on the display interface, and the corresponding module can be activated directly after the user confirms. Difficulty of using the delivery device.

In the present application, since the influence of audio and noise in the surrounding environment on the user's audio is reduced before speech recognition is performed, it is beneficial to improve the accuracy of the speech recognition result, and further to improve the accuracy of obtaining the speech instruction.

At the same time, compared with the recognition method of recognizing the user's lip language through video data, speech recognition has the advantages of fast recognition speed and low recognition cost.

In a feasible embodiment, after obtaining the lip language recognition result and the speech recognition result, the above-mentioned similarity ranking can also be obtained in the following manner:

Mode 1: Determine the accuracy of the speech recognition result, the proportion of incorrectly recognized characters in the speech recognition result in each division of the entire speech, and the percentage of the speech recognition result according to the lip language recognition result. The number of consecutive typos; for each delivery device, calculate the first similarity of the accuracy of the delivery device and other delivery devices, the second similarity of the proportion, and the third similarity of the number of consecutive typos; The first similarity, the second similarity and the third similarity corresponding to the delivery device are weighted and summed to obtain a similarity that indicates that the delivery device and other delivery devices are affected by the environment and noise ; Sort the similarity according to a specified order, so as to obtain a similarity ranking for indicating that the delivery device and other delivery devices are similarly affected by the surrounding environment and noise.

Method 2: Perform text similarity calculation on the text information corresponding to the lip language recognition result and the text information corresponding to the speech recognition result to obtain the text similarity corresponding to the delivery device; for each delivery device, calculate the second difference between the text similarity of the delivery device and the text similarity of other delivery devices; sort the second difference according to the specified order, so as to obtain a value used to indicate that the delivery device and other delivery devices are affected by surrounding Similarity ranking for similar situations in which environment and noise influences are similar.

In a feasible implementation, FIG. 2 is a schematic flowchart of another speech recognition method provided by this embodiment of the present application. As shown in FIG. 2 , when step 111 is executed, the following steps can be performed:

Step 201 : Align the target audio signal and the audio frame with the same collection moment in the audio signal collected by the delivery device, so as to obtain multiple groups of audio frame groups with the same audio collection moment.

Step 202, for each group of audio frames, use the audio feature of the audio frame corresponding to the target audio signal in the group of audio frames, and perform audio separation on the audio frame corresponding to the audio signal collected by the throwing device to obtain the audio frame. User audio frames collected when the delivery device corresponding to the group enables the voice function again.

Step 203: Splicing the user audio frames corresponding to the multiple audio frame groups to obtain the user audio collected when the delivery device enables the voice function again.

Specifically, since the target audio signal and the audio signal collected by the delivery device have the same recording start time point, the audio collected by the two delivery devices can be aligned in time, so that the audio frames with the same time stamp can be Divided into the same group, for the audio frames in the same group, the current environments of the two corresponding delivery devices are relatively similar. After aligning the video frames, you can use the audio frame corresponding to the target audio signal to the delivery device. The audio frames in the collected audio signals are subjected to audio separation, and during the audio separation, each audio frame group can be processed in parallel, thereby helping to improve the data processing speed.

It should be noted that, the splicing method specifically used when performing audio splicing can be set according to actual needs, which is not specifically limited here.

It should be noted that like numerals and letters refer to like items in the following figures, so that once an item is defined in one figure, it does not require further definition and explanation in subsequent figures, Furthermore, the terms "first", "second", "third", etc. are only used to differentiate the description and should not be construed as indicating or implying relative importance.

Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present application, and are used to illustrate the technical solutions of the present application, rather than limit them. The embodiments describe the application in detail, and those of ordinary skill in the art should understand that: any person skilled in the art can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed in the application. Changes can be easily conceived, or equivalent replacements are made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions in the embodiments of the present application. All should be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims (3)

1. a speech recognition method, is characterized in that, described method comprises: Within the preset time period, when the delivery device enables the voice function, the voice acquisition module and the video data acquisition module are simultaneously activated to collect audio data and video data with the same time attribute; The four-point labeling method is used to label the two corners of the lips and the lips of the upper and lower lips collected in the video data; Measure the opening angle of the mouth corner, the distance between the upper and lower lips, and the distance between the two mouth corners when the mouth shape changes each time in the video data, so as to obtain multiple sets of measurement data, wherein the mouth corner opening angle Including a first included angle and a second included angle, the first included angle is the included angle between the connection line between the upper lip lip and the corner of the mouth and the connecting line between the two mouth corners, and the second included angle is the lower lip lip The angle between the line between the center and the corners of the mouth and the line between the two corners of the mouth; The mouth opening angle, the distance and the distance in each set of measurement data are input into the evaluation function as input parameters to obtain lip change data, wherein the evaluation function is: Pre=k*(p* Angle(a,b)+q*Line(LAF,LDE)), k, p, q are the weight coefficients of each cost function, Angle(a,b) is the first angle and the second angle The cost function of , Line(LAF, LDE) is the cost function of the distance and the distance; Using the lip change data and the lip shape database to compare and obtain a lip language recognition result, and carry out audio recognition to the audio data to obtain a speech recognition result; According to the lip language recognition result, determine the accuracy rate of the speech recognition result, the proportion of incorrectly recognized characters in the speech recognition result in each division of the entire speech, and the percentage of consecutive typos in the speech recognition result. quantity; Weighted summation is performed on the accuracy rate, the proportion and the quantity to obtain a speech recognition feature value representing the speech recognition feature of the delivery device; After obtaining the speech recognition feature value of each of the delivery devices, for each of the delivery devices, calculate the first difference between the speech recognition feature values of other delivery devices and the speech recognition feature value of the delivery device; Sorting the first difference values according to a specified order to obtain a similarity ranking indicating that the delivery device and other delivery devices are similarly affected by the surrounding environment and noise; After the similarity ranking corresponding to the delivery device is obtained, when the voice function is enabled on the delivery device again, a target delivery device is selected from other delivery devices according to the similarity ranking corresponding to the delivery device, and the target delivery device is activated. The voice collection module collects the target audio signal, wherein the target delivery device is the delivery device with the highest similarity ranking among the delivery devices not used by the user when the delivery device enables the voice function again; According to the audio feature of the target audio signal, audio separation is performed on the audio signal collected by the delivery device, so as to obtain the user audio collected when the delivery device enables the voice function again; Perform voice recognition according to the user audio to obtain the user's voice command. 2. The method of claim 1, wherein, after obtaining the lip language recognition result and the speech recognition result, the similarity ranking can also be obtained by the following steps: According to the lip language recognition result, determine the accuracy rate of the speech recognition result, the proportion of incorrectly recognized characters in the speech recognition result in each division of the entire speech, and the percentage of consecutive typos in the speech recognition result. Quantity; for each delivery device, calculate the first similarity of the accuracy rate, the second similarity of the proportion, and the third similarity of the number of consecutive typos between the delivery device and other delivery devices; The corresponding first similarity, the second similarity and the third similarity are weighted and summed to obtain the similarity used to indicate that the delivery device and other delivery devices are affected by the environment and noise; according to the specified Ranking the similarity in order to obtain a similarity ranking indicating that the delivery device and other delivery devices are similarly affected by the surrounding environment and noise; or, Perform text similarity calculation on the text information corresponding to the lip language recognition result and the text information corresponding to the speech recognition result to obtain the text similarity corresponding to the delivery device; for each delivery device, calculate the delivery device The second difference value between the text similarity of , and the text similarity of other delivery devices; sort the second difference according to the specified order to obtain a value used to represent that the delivery device and other delivery devices are affected by the surrounding environment and noise. Affects similarity ranking for similar situations. 3. The method of claim 1, wherein, according to the audio feature of the target audio signal, audio separation is performed on the audio signal collected by the throwing device, so as to obtain when the throwing device enables the voice function again. to the user audio, including: Aligning the audio frame with the same acquisition moment in the audio signal collected by the target audio signal and the throwing device, to obtain multiple groups of audio frame groups with the same audio acquisition moment; For each group of audio frames, use the audio feature of the audio frame corresponding to the target audio signal in the group of audio frames to perform audio separation on the audio frame corresponding to the audio signal collected by the delivery device to obtain the corresponding audio frame group. User audio frames collected when the delivery device enables the voice function again; The user audio frames corresponding to the multiple audio frame groups are spliced to obtain the user audio collected when the delivery device enables the voice function again.

Images