CN115331673B - Voiceprint recognition household appliance control method and device in complex sound scene - Google Patents

Abstract

The invention provides a voiceprint recognition household appliance control method and device in a complex sound scene, and relates to the field of household appliance control. The template audio fully considers various conditions in a complex sound scene, has better representativeness, and lays a foundation for improving the voiceprint recognition precision in the complex sound scene. And the similarity detection model based on the template audio, the voiceprint recognition decision model based on the SVM model and the voiceprint recognition model based on the convolutional neural network are used for sequentially judging, so that the voiceprint recognition precision is improved. The model is simple to complex, the audio frequency which is easy to judge can obtain the result by using the simple model, the audio frequency signal which is difficult to judge can obtain the result by using the complex model, and the consumption of computing resources is reduced.

Description

A voiceprint recognition home appliance control method and device in a complex sound scene

technical field

The present invention relates to the field of home appliance control, in particular to a voiceprint recognition home appliance control method and device in a complex sound scene.

Background technique

With the advancement of science and technology, more and more modern household appliances are widely used by consumers. As an important identification technology, voiceprint recognition can identify the identities of family members, so that home appliances can accept instructions from specific family members and prevent interference from instructions from unrelated people. Under normal circumstances, ordinary voiceprint recognition technology can already guarantee high recognition accuracy, so as to realize precise control of home appliances by specific family members.

However, in the process of using voiceprint recognition technology to control home appliances, it is often accompanied by complex sound scenes, which greatly reduces the recognition accuracy of voiceprint recognition technology. With the significant reduction in recognition accuracy, the application value of home appliances based on voiceprint recognition control methods has also declined significantly. Therefore, how to design a voiceprint recognition home appliance control method in a complex sound scene, which can also ensure the accuracy of voiceprint recognition in a complex sound scene, has very important application value.

Contents of the invention

In order to overcome the above-mentioned problems or at least partially solve the above-mentioned problems, embodiments of the present invention provide a voiceprint recognition home appliance control method and device in a complex sound scene.

Embodiments of the present invention are achieved like this:

In the first aspect, the embodiment of this law provides a voiceprint recognition home appliance control method in a complex sound scene, including:

In multiple sound scenes, record multiple audio segments of specific family members;

Encode multiple audio segments;

After encoding, calculate the similarity between the audios of each family member, retain a piece of audio whose similarity is greater than the preset value, and identify all retained audios as template audios;

Use all template audio as positive training samples, and collect audio from multiple non-specific family members as negative training samples, use machine learning models for training, and obtain a voiceprint recognition decision model;

When the home appliance user outputs a piece of audio, calculate the similarity between the piece of audio and the template audio. If the similarity between the piece of audio and any template audio is greater than the preset similarity, it will be directly recognized as the audio of a specific family member; If the similarity of any template audio is less than the preset similarity, proceed to the next step;

The voiceprint recognition decision model is used to judge whether the output audio of the home appliance user is the audio of a specific family member.

Based on the first aspect, in some embodiments of the present invention, the above machine learning model is an SVM model.

Based on the first aspect, in some embodiments of the present invention, the above step of using the voiceprint recognition decision-making model to determine whether the output audio of the household appliance user is the audio of a specific family member includes:

If the score of the voiceprint recognition decision result based on the SVM model is greater than the first preset score, it is directly recognized as the audio of a specific family member; if the score of the voiceprint recognition decision result based on the SVM model is less than the second preset score, it is directly recognized as non-specific The audio of the family member, if the score of the voiceprint recognition decision result based on the SVM model is between the first preset score and the second preset score, then proceed to the next step;

The voiceprint recognition model based on the convolutional neural network is used to make a final judgment on the output audio of the home appliance user, and judge whether it is the audio of a specific family member.

Based on the first aspect, in some embodiments of the present invention, when the above-mentioned home appliance user outputs a piece of audio, the step of calculating the similarity between the piece of audio and the template audio includes:

Carry out the following audio and template audio: audio filtering, calculating the short-term energy of the audio signal, and intercepting the effective data of the audio signal;

Calculate the cosine distance between the audio and template audio.

Based on the first aspect, in some embodiments of the present invention, the above-mentioned steps of separately recording multiple pieces of audio of specific family members in multiple sound scenes include:

Record multiple audio clips of specific family members in one or more situations of high noise, multiple people talking, and low voice;

When recording, control the duration of each audio segment within 5 seconds.

Based on the first aspect, in some embodiments of the present invention, the above step of encoding multiple segments of audio includes:

Use the I-Vector calculation method to encode multi-segment audio.

Based on the first aspect, in some embodiments of the present invention, the step of collecting audios of a plurality of non-specific family members as negative training samples includes:

More than 50 audios of unspecified family members are collected as negative training samples.

In the second aspect, an embodiment of the present invention provides a voiceprint recognition home appliance control system in a complex sound scene, including:

Recording module: in multiple sound scenes, record multiple pieces of audio of specific family members;

Encoding module: encode multi-segment audio;

Calculation similarity module: After encoding, calculate the similarity between each family member's audio pairwise, retain a piece of audio whose similarity is greater than the preset value, and identify all retained audio as template audio;

Training module: use all template audio as positive training samples, and collect audio from multiple non-specific family members as negative training samples, use machine learning model for training, and obtain voiceprint recognition decision-making model;

Recognition module: when the home appliance user outputs a piece of audio, calculate the similarity between the piece of audio and the template audio, and if the similarity between the piece of audio and any template audio is greater than the preset similarity, it is directly recognized as the audio of a specific family member;

Judgment module: If the similarity between this segment of audio and any template audio is less than the preset similarity, use the voiceprint recognition decision-making model to judge whether the output audio of the home appliance user is the audio of a specific family member.

In a third aspect, an embodiment of the present invention provides an electronic device, including:

at least one processor, at least one memory, and a data bus; wherein:

The processor and the memory communicate with each other through the data bus; the memory stores program instructions executable by the processor, and the processor invokes the program instructions to execute the above method.

In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium stores a computer program, and the computer program enables a computer to execute the above-mentioned method.

Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

(1) Template audio fully considers a variety of situations in complex sound scenes, and has good representativeness, laying a foundation for improving the accuracy of voiceprint recognition in complex sound scenes.

(2) The similarity detection model based on template audio, the voiceprint recognition decision model based on SVM model, and the voiceprint recognition model based on convolutional neural network are used to make judgments in sequence, which improves the accuracy of voiceprint recognition.

(3) Use the similarity detection model based on template audio, the voiceprint recognition decision model based on SVM model, and the voiceprint recognition model based on convolutional neural network to make judgments in sequence. The models range from simple to complex, and the audio that is easy to judge uses a simple model The result can be obtained, and the audio signal that is difficult to judge can be obtained by using a complex model, which reduces the consumption of computing resources.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention, and thus It should be regarded as a limitation on the scope, and those skilled in the art can also obtain other related drawings based on these drawings without creative work.

Fig. 1 is a flow chart of an embodiment of a voiceprint recognition home appliance control method in a complex sound scene according to the present invention;

Fig. 2 is a flowchart of an embodiment of a control method for voiceprint recognition household appliances in a complex sound scene according to the present invention;

3 is a structural block diagram of an embodiment of a voiceprint recognition home appliance control device in a complex sound scene according to the present invention;

FIG. 4 is a structural block diagram of an embodiment of an electronic device according to the present invention.

Icons: 1. Input module; 2. Coding module; 3. Calculation similarity module; 4. Training module; 5. Recognition module; 6. Judgment module; 7. Processor; 8. Memory; 9. Data bus.

detailed description

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. The components of the embodiments of the invention generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations.

Accordingly, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

In the embodiments provided in this application, it should be understood that the disclosed methods and devices may also be implemented in other ways. The system embodiments are illustrative only, for example, the block diagrams in the figures show the architecture, functionality and operation of possible implementations of systems and computer program products according to various embodiments of the present application. In this regard, each block in the block diagram may represent a module, program segment, or part of code that includes one or more executable instructions for implementing specified logical functions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified function or action , or may be implemented by a combination of dedicated hardware and computer instructions.

In addition, each functional module in each embodiment of the present application may be integrated to form an independent part, each module may exist independently, or two or more modules may be integrated to form an independent part.

If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device, which may be a personal computer, a server, or a network device, etc., execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and other media that can store program codes. .

In the description of the embodiments of the present invention, "multiple" means at least 2.

In the description of the embodiments of the present invention, it should also be noted that, unless otherwise specified and limited, the terms "setting" and "connection" should be interpreted in a broad sense, for example, it can be a fixed connection or an optional connection. Detachable connection, or integral connection; it can be mechanical connection or electrical connection; it can be direct connection or indirect connection through an intermediary, and it can be the internal communication of two components. Those of ordinary skill in the art can understand the specific meanings of the above terms in the present invention according to specific situations.

Example

Please refer to Figure 1. In the first aspect, the embodiment of this law provides a voiceprint recognition home appliance control method in a complex sound scene, including:

S1: In multiple sound scenes, record multiple audio segments of specific family members;

In this step, multiple sound scenes are duplication scenes, including high noise, many people talking, low voice, etc., and cover all the situations of voiceprint recognition in the use of home appliances as comprehensively as possible. The audio duration can be set according to the actual situation. Generally, the time for voice control of home appliances is relatively short, and each audio segment can be within 5 seconds. Template audio fully considers a variety of situations in complex sound scenes, and is representative, laying the foundation for improving the accuracy of voiceprint recognition in complex sound scenes.

S2: encode multiple segments of audio;

In this step, the multi-segment audio is encoded using the I-Vector calculation method. In practical applications, because the speaker information and various interference information are mixed together in the speaker's voice, the channels of different acquisition devices also have differences, which will make the collected voice doped with channel interference information. This noise information will cause the disturbance of speaker information. The traditional GMM-UBM method has no way to overcome this problem, resulting in unstable system performance. In the GMM-UBM model, each target speaker can be described by the GMM model. Since the UBM model is adapted to the GMM model of each speaker, only the mean value is changed, and no adjustments are made to the weight and covariance, so most of the speaker information is contained in the mean value of the GMM. In addition to most of the speaker information, the GMM mean vector also contains channel information. Joint Factor Analysis (JFA) can model speaker differences and channel differences separately, so that channel differences can be well compensated and system performance can be improved. However, since JFA requires a large amount of training corpus with different channels, it is difficult to obtain and the calculation is complex, so it is difficult to put it into practical use. Proposed by Dehak, based on I-Vector factor analysis technology, a new solution is proposed. The JFA method is to model the speaker difference space separately from the channel difference space, while the I-Vector-based method is to model the global difference and model the two as a whole, which relaxes the training corpus The limit, and the calculation is simple, the performance is also comparable.

S3: After encoding, calculate the similarity between the audios of each family member, keep a piece of audio whose similarity is greater than the preset value, and identify all the retained audios as template audios;

In this step, calculating the similarity between the audios of each family member includes performing audio filtering on the two pieces of audio, calculating the short-term energy of the audio signal, intercepting the effective data of the audio signal, and calculating the cosine distance of the two pieces of audio. Reserve a piece of audio whose similarity is greater than the preset value, and identify all the retained audio as template audio. The preset value can be set reasonably according to actual needs.

S4: Use all template audio as positive training samples, and collect audio from multiple non-specific family members as negative training samples, use the machine learning model for training, and obtain a voiceprint recognition decision model;

In this step, the step of collecting audios of multiple non-specific family members as negative training samples includes: collecting audios of more than 50 non-specific family members as negative training samples. The above machine learning model may be an SVM model.

S5: When the home appliance user outputs a piece of audio, calculate the similarity between the piece of audio and the template audio, if the similarity between the piece of audio and any template audio is greater than the preset similarity, it is directly recognized as the audio of a specific family member; If the similarity between the audio and any template audio is less than the preset similarity, proceed to the next step;

In this step, a template audio-based similarity detection model can be used to calculate the similarity between the segment of audio and the template audio. When the above-mentioned home appliance user outputs a piece of audio, the steps of calculating the similarity between the piece of audio and the template audio include: performing audio filtering on the piece of audio and the template audio, calculating the short-term energy of the audio signal, and intercepting the effective data of the audio signal; Cosine distance between segment audio and template audio.

S6: Use the voiceprint recognition decision-making model to judge whether the output audio of the household appliance user is the audio of a specific family member.

The similarity detection model based on the template audio and the voiceprint recognition decision model based on the SVM model are used to make judgments in sequence, which improves the accuracy of voiceprint recognition; the model is from simple to complex, and the audio that is easy to judge can use a simple model to get the result, but it is difficult The judged audio signal is then used to obtain the result with a complex model, which reduces the consumption of computing resources.

Based on the first aspect, in some embodiments of the present invention, the above step of using the voiceprint recognition decision-making model to determine whether the output audio of the household appliance user is the audio of a specific family member includes:

Please refer to Figure 2, S61: If the score of the voiceprint recognition decision result based on the SVM model is greater than the first preset score, directly recognize the audio of a specific family member; if the score of the voiceprint recognition decision result based on the SVM model is less than the second preset score Score, directly identified as the audio of a non-specific family member, if the score of the voiceprint recognition decision result based on the SVM model is between the first preset score and the second preset score, proceed to the next step;

S62: Use the voiceprint recognition model based on the convolutional neural network to make a final judgment on the output audio of the household appliance user, and judge whether it is the audio of a specific family member.

The similarity detection model based on template audio, the voiceprint recognition decision model based on SVM model, and the voiceprint recognition model based on convolutional neural network are used to make judgments in sequence, which improves the accuracy of voiceprint recognition; using similarity detection based on template audio The model, the voiceprint recognition decision model based on the SVM model, and the voiceprint recognition model based on the convolutional neural network are judged in turn. The models range from simple to complex. The audio that is easy to judge can use the simple model to get the result, and the audio signal that is difficult to judge can be obtained again. Results are obtained with complex models, reducing computational resource consumption.

Please refer to Figure 3. In the second aspect, the embodiment of the present invention provides a voiceprint recognition home appliance control system in a complex sound scene, including:

Input module 1: In multiple sound scenes, record multiple audio segments of specific family members;

Encoding module 2: Encoding multi-segment audio;

Computational similarity module 3: After encoding, calculate the similarity between the audios of each family member, retain a piece of audio whose similarity is greater than the preset value, and identify all retained audios as template audios;

Training module 4: Use all template audio as positive training samples, and collect audio from multiple non-specific family members as negative training samples, use machine learning model for training, and obtain voiceprint recognition decision-making model;

Recognition module 5: When the home appliance user outputs a piece of audio, calculate the similarity between the piece of audio and the template audio, and if the similarity between the piece of audio and any template audio is greater than the preset similarity, directly identify it as the audio of a specific family member;

Judgment module 6: If the similarity between the segment of audio and any template audio is less than the preset similarity, use the voiceprint recognition decision model to judge whether the output audio of the home appliance user is the audio of a specific family member.

For the specific implementation manner of the device, please refer to the implementation manner of the above-mentioned method, and details will not be repeated here.

Please refer to FIG. 4. In a third aspect, an embodiment of the present invention provides an electronic device, including:

at least one processor 7, at least one memory 8 and a data bus 9; wherein:

The processor 7 and the memory 8 communicate with each other through the data bus 9; the memory 8 stores program instructions executable by the processor 7, and the processor 7 invokes the program instructions to execute the above method. For example, the above steps S1-S6 are performed.

In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium stores a computer program, and the computer program enables a computer to execute the above-mentioned method. For example, the above steps S1-S6 are performed.

To sum up, the present invention provides a voiceprint recognition home appliance control method in a complex sound scene. The template audio fully considers various situations in a complex sound scene, and is relatively representative. Recognition accuracy lays the foundation. The similarity detection model based on template audio, the voiceprint recognition decision model based on SVM model, and the voiceprint recognition model based on convolutional neural network are used to make judgments in sequence, which improves the accuracy of voiceprint recognition. The model ranges from simple to complex. Audio that is easy to judge can be obtained by using a simple model, and audio signals that are difficult to judge can be obtained by using a complex model, which reduces the consumption of computing resources.

The above are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, but that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be regarded as exemplary and not restrictive in all points of view, and the scope of the application is defined by the appended claims rather than the foregoing description, and it is intended that the scope of the present application be defined by the appended claims rather than by the foregoing description. All changes within the meaning and range of equivalents of the elements are embraced in this application. Any reference sign in a claim should not be construed as limiting the claim concerned.

Claims (8)

1. A voiceprint recognition home appliance control method in a complex sound scene, characterized in that it comprises: In multiple sound scenes, record multiple audio segments of specific family members; Encode multiple audio segments; After encoding, calculate the similarity between the audios of each family member, retain a piece of audio whose similarity is greater than the preset value, and identify all retained audios as template audios; All template audios are used as positive training samples, and the audios of multiple non-specific family members are collected as negative training samples, and a machine learning model is used for training to obtain a voiceprint recognition decision model, and the machine learning model is an SVM model; When the home appliance user outputs a piece of audio, calculate the similarity between the piece of audio and the template audio. If the similarity between the piece of audio and any template audio is greater than the preset similarity, it will be directly recognized as the audio of a specific family member; If the similarity of any template audio is less than the preset similarity, proceed to the next step; Use the voiceprint recognition decision-making model to judge whether the output audio of the home appliance user is the audio of a specific family member; The step of using the voiceprint recognition decision-making model to determine whether the output audio of the household appliance user is the audio of a specific family member includes: If the score of the voiceprint recognition decision result based on the SVM model is greater than the first preset score, it is directly recognized as the audio of a specific family member; if the score of the voiceprint recognition decision result based on the SVM model is less than the second preset score, it is directly recognized as non-specific The audio of the family member, if the score of the voiceprint recognition decision result based on the SVM model is between the first preset score and the second preset score, then proceed to the next step; The voiceprint recognition model based on the convolutional neural network is used to make a final judgment on the output audio of the home appliance user, and judge whether it is the audio of a specific family member. 2. The voiceprint recognition home appliance control method in a complex sound scene according to claim 1, wherein when the user of the home appliance outputs a piece of audio, the step of calculating the similarity between the piece of audio and the template audio includes : Carry out the following audio and template audio: audio filtering, calculating the short-term energy of the audio signal, and intercepting the effective data of the audio signal; Calculate the cosine distance between the audio and template audio. 3. The method for controlling home appliances based on voiceprint recognition in a complex sound scene according to claim 1, wherein the step of respectively recording multiple audio segments of specific family members in multiple sound scenes comprises: Record multiple audio clips of specific family members in one or more situations of high noise, multiple people talking, and low voice; When recording, control the duration of each audio segment within 5 seconds. 4. The voiceprint recognition home appliance control method in a complex sound scene according to claim 1, wherein the step of encoding multiple segments of audio comprises: Use the I-Vector calculation method to encode multi-segment audio. 5. The voiceprint recognition home appliance control method in a complex sound scene according to claim 1, wherein the step of collecting the audio of a plurality of non-specific family members as negative training samples comprises: More than 50 audios of unspecified family members are collected as negative training samples. 6. A voiceprint recognition home appliance control device in a complex sound scene, characterized in that it comprises: Recording module: in multiple sound scenes, record multiple pieces of audio of specific family members; Encoding module: encode multi-segment audio; Calculation similarity module: After encoding, calculate the similarity between each family member's audio pairwise, retain a piece of audio whose similarity is greater than the preset value, and identify all retained audio as template audio; Training module: all template audios are used as positive training samples, and the audios of multiple non-specific family members are collected as negative training samples, and a machine learning model is used for training to obtain a voiceprint recognition decision model, and the machine learning model is an SVM model; Recognition module: when the home appliance user outputs a piece of audio, calculate the similarity between the piece of audio and the template audio, and if the similarity between the piece of audio and any template audio is greater than the preset similarity, it is directly recognized as the audio of a specific family member; Judgment module: if the similarity between the audio and any template audio is less than the preset similarity, use the voiceprint recognition decision-making model to judge whether the output audio of the home appliance user is the audio of a specific family member; The judgment module includes: Recognition sub-module: if the score of the voiceprint recognition decision result based on the SVM model is greater than the first preset score, directly recognize the audio of a specific family member; if the score of the voiceprint recognition decision result based on the SVM model is less than the second preset score, directly If the audio is identified as a non-specific family member, if the score of the voiceprint recognition decision result based on the SVM model is between the first preset score and the second preset score, proceed to the next step; Final judgment sub-module: Use the convolutional neural network-based voiceprint recognition model to make a final judgment on the output audio of the home appliance user, and judge whether it is the audio of a specific family member. 7. An electronic device, characterized in that it comprises: at least one processor, at least one memory, and a data bus; wherein: The processor and the memory communicate with each other through the data bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to perform the procedure described in claim 1. to any of the methods described in 5. 8. A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores a computer program, and the computer program causes the computer to execute the method according to any one of claims 1 to 5 .

Images