CN116705046A - Sound pickup method, device, non-volatile storage medium and terminal equipment - Google Patents

Abstract

The application discloses a pickup method, a pickup device, a nonvolatile storage medium and terminal equipment. Wherein the method comprises the following steps: collecting sound signals using a microphone array comprising a plurality of microphones; detecting a target azimuth angle corresponding to the sound signal acquired by the microphone array; and under the condition that the target azimuth angle is in the preset angle interval, performing signal enhancement processing on the sound signal to obtain a target sound signal. The application solves the technical problem that the sound processing result is not ideal because the sound pickup equipment cannot accurately distinguish the sound needing to be enhanced.

Description

Sound pickup method, device, non-volatile storage medium and terminal equipment

technical field

The present invention relates to the field of sound equipment, in particular to a sound pickup method, device, non-volatile storage medium and terminal equipment.

Background technique

How to pick up sound clearly by the microphone and how to distinguish whether the sound collected by the microphone is valuable sound or noise and other types of sound that need to be suppressed is a problem faced by the industry.

When the sound collection device collects sound, the microphone of the sound collection device will collect sounds of different types and objects. However, the user of the sound collection device may only need the microphone to collect his own voice, and suppress other sounds. In the existing methods and systems for using microphones to pick up sound, in the presence of noise and reverberation, the performance of the algorithm drops significantly and the distortion of the speech is relatively large. At the same time, when there are two or more speakers speaking at the same time, the effect is not ideal .

For the above problems, no effective solution has been proposed yet.

Contents of the invention

Embodiments of the present invention provide a sound pickup method, device, non-volatile storage medium, and terminal equipment to at least solve the technical problem that the sound pickup equipment cannot accurately distinguish the sound that needs to be enhanced, resulting in unsatisfactory sound processing results.

According to an aspect of an embodiment of the present invention, there is provided a sound pickup method, including: collecting sound signals by using a microphone array including a plurality of microphones; detecting the corresponding target azimuth angle when the sound signals are collected by the microphone array; When the target azimuth angle is within the preset angle interval, signal enhancement processing is performed on the sound signal to obtain the target sound signal.

Optionally, the detection of the corresponding target azimuth angle when the sound signal is collected by the microphone array includes: converting the sound signal into a digital signal; inputting the digital signal into a convolutional neural network (CNN) model, The target azimuth angle corresponding to the digital signal is obtained, wherein the CNN model is used to locate the sound direction.

Optionally, the inputting the digital signal into the convolutional neural network (CNN) model to obtain the target azimuth angle corresponding to the digital signal includes: using the feature extraction module of the CNN model to extract generalized Cross-correlation features and filter bank features; using the positioning module of the CNN model to determine the target azimuth angle corresponding to the digital signal based on the generalized cross-correlation features and the filter bank features.

Optionally, the positioning module using the CNN model determines the target azimuth angle corresponding to the digital signal based on the generalized cross-correlation feature and the filter bank feature, including: using the CNN model The positioning module, based on the generalized cross-correlation feature and the filter bank feature, determines the probability value that the microphone array collects the sound signal from any azimuth angle; using the positioning module, based on the probability value , to determine the target azimuth angle corresponding to the digital signal.

Optionally, the multiple microphones included in the microphone array are multiple directional microphones, and the multiple directional microphones are sequentially located on multiple vertices of a regular polygon.

Optionally, the method further includes: outputting the target sound signal to a target device, wherein the target device is used to play or store the target sound signal.

Optionally, the method further includes: when the target azimuth is outside the preset angle range, suppressing the sound signal to obtain a noise signal; outputting the noise signal to the target device .

Optionally, before adopting a microphone array including a plurality of microphones to collect sound signals, the method further includes: presenting first prompt information, wherein the first prompt information is used to prompt the preset angle interval; When the target azimuth is within the preset angle interval, the method further includes: displaying second prompt information, wherein the second prompt information is used to prompt the target azimuth corresponding to the sound signal located within the preset angle interval; when the target azimuth is outside the preset angle interval, the method further includes: displaying third prompt information, wherein the third prompt information is used to prompt The target azimuth angle corresponding to the sound signal is outside the preset angle range.

According to another aspect of the embodiments of the present invention, there is also provided a sound pickup method, including: receiving a sound signal, wherein the sound signal is collected by a microphone array including a plurality of microphones; detecting that the sound signal is collected by the When the microphone array collects the corresponding target azimuth angle; when the target azimuth angle is within the preset angle interval, perform signal enhancement processing on the sound signal to obtain the target sound signal; output the target sound signal to the target device, wherein the target device is used to play or store the target sound signal.

According to another aspect of the embodiments of the present invention, there is also provided a sound pickup device, including: a collection module, configured to collect sound signals using a microphone array including a plurality of microphones; a first detection module, used to detect the sound signals The corresponding target azimuth angle when collected by the microphone array; the first enhancement module is used to perform signal enhancement processing on the sound signal to obtain the target sound when the target azimuth angle is within a preset angle interval Signal.

According to another aspect of the embodiments of the present invention, there is also provided a sound pickup device, including: a receiving module for receiving a sound signal, wherein the sound signal is collected by a microphone array including a plurality of microphones; the second detection A module for detecting the corresponding target azimuth angle when the sound signal is collected by the microphone array; a second enhancement module for detecting the sound signal when the target azimuth angle is within a preset angle interval The signal is subjected to signal enhancement processing to obtain a target sound signal; an output module is configured to output the target sound signal to a target device, wherein the target device is used to play or store the target sound signal.

According to still another aspect of the embodiments of the present invention, there is also provided a non-volatile storage medium, the non-volatile storage medium includes a stored program, wherein the non-volatile memory is controlled when the program is running. The device where the storage medium is located executes any one of the sound pickup methods described above.

According to still another aspect of the embodiments of the present invention, there is also provided a terminal device, the terminal device includes a processor, and the processor is used to run a program, wherein, when the program is running, it executes any one of the above audio pickup method.

In the embodiment of the present invention, the sound signal is collected by using a microphone array including a plurality of microphones; the corresponding target azimuth angle is detected when the sound signal is collected by the microphone array; The sound signal is subjected to signal enhancement processing to obtain the target sound signal, which achieves the purpose of enhancing the sound from the preset angle range, thereby achieving the technical effect of improving the sound pickup effect of the sound pickup device, and further solving the problem that the sound pickup device cannot be accurately distinguished Technical issues that require enhanced sound resulting in suboptimal sound processing results.

Description of drawings

The accompanying drawings described here are used to provide a further understanding of the present invention and constitute a part of the application. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations to the present invention. In the attached picture:

Fig. 1 shows a kind of block diagram of the hardware structure of the computer terminal that is used to realize sound pickup method;

Fig. 2 is a schematic flow chart of sound pickup method 1 provided according to an embodiment of the present invention;

3 is a schematic diagram of a triangular microphone array provided according to an optional embodiment of the present invention;

Fig. 4 is a schematic structural diagram of a CNN model provided according to an optional embodiment of the present invention;

FIG. 5 is a schematic diagram of a sound pickup process of a terminal device provided according to an optional embodiment of the present invention;

FIG. 6 is a schematic flow diagram of a sound pickup method 2 provided according to an embodiment of the present invention;

Fig. 7 is a structural block diagram of a sound pickup device 1 provided according to an embodiment of the present invention;

Fig. 8 is a structural block diagram of a second sound pickup device provided according to an embodiment of the present invention.

Detailed ways

In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only It is an embodiment of a part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

It should be noted that the terms "first" and "second" in the description and claims of the present invention and the above drawings are used to distinguish similar objects, but not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a sequence of steps or elements is not necessarily limited to the expressly listed instead, may include other steps or elements not explicitly listed or inherent to the process, method, product or apparatus.

According to an embodiment of the present invention, an embodiment of a sound pickup method is provided. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and, although In the flowcharts, a logical order is shown, but in some cases the steps shown or described may be performed in an order different from that shown or described herein.

The method embodiment provided in Embodiment 1 of the present application may be executed in a mobile terminal, a computer terminal, or a similar computing device. Fig. 1 shows a block diagram of the hardware structure of a computer terminal for realizing the sound pickup method. As shown in Figure 1, the computer terminal 10 may include one or more (shown by 102a, 102b, ..., 102n in the figure) processors (processors may include but not limited to microprocessors MCU or programmable logic devices Processing device such as FPGA), memory 104 for storing data. In addition, it can also include: a display, an input/output interface (I/O interface), a universal serial bus (USB) port (which can be included as one of the ports of the BUS bus), a network interface, a power supply, and/or or camera. Those of ordinary skill in the art can understand that the structure shown in FIG. 1 is only a schematic diagram, and it does not limit the structure of the above-mentioned electronic device. For example, computer terminal 10 may also include more or fewer components than shown in FIG. 1 , or have a different configuration than that shown in FIG. 1 .

It should be noted that the one or more processors and/or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuit may be implemented in whole or in part as software, hardware, firmware or other arbitrary combinations. In addition, the data processing circuit can be a single independent processing module, or be fully or partially integrated into any of the other components in the computer terminal 10 . As mentioned in the embodiment of the present application, the data processing circuit is used as a processor control (for example, the selection of the terminal path of the variable resistor connected to the interface).

The memory 104 can be used to store software programs and modules of application software, such as the program instruction/data storage device corresponding to the sound pickup method in the embodiment of the present invention, and the processor executes each program by running the software programs and modules stored in the memory 104. A functional application and data processing, that is, to realize the sound pickup method of the above-mentioned application program. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory that is remotely located relative to the processor, and these remote memories may be connected to the computer terminal 10 through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The display can be, for example, a touch-screen liquid crystal display (LCD), which enables a user to interact with the user interface of the computer terminal 10 .

Fig. 2 is a schematic flow chart of a sound pickup method 1 provided according to an embodiment of the present invention. As shown in Fig. 2 , the method includes the following steps:

Step S202, using a microphone array including a plurality of microphones to collect sound signals. It should be noted that the microphone array including multiple microphones can be located in the sound collection device at the sound collection site, and the sound collection device can be a hand-held microphone for users, or a conference microphone fixed on a table.

In a possible application scenario, the sound collection device including the microphone array may be located in a conference room, and the user of the device wishes to mainly collect the voice of the user in front of the microphone, while ignoring or suppressing other sounds in the conference room. However, the sound signal collected by the microphone array may come from the user in front of the microphone or from other directions in the meeting room, so it needs to be screened.

Step S204, detecting the corresponding target azimuth angle when the sound signal is collected by the microphone array. In this step, the target azimuth indicates the direction from which the sound signal is transmitted, that is, the sound signal is transmitted to the microphone array along the target azimuth and collected by the microphone array. Optionally, the target azimuth can be expressed with the sound collection device where the microphone array is located as the center, and the horizontal plane as the plane where the coordinate system is located, to establish a plane Cartesian or polar coordinate system, and then express the target azimuth in the above coordinate system middle. Optionally, the value range of the target azimuth angle may vary from 0° to 360°.

Step S206, when the target azimuth angle is within the preset angle interval, perform signal enhancement processing on the sound signal to obtain the target sound signal.

The microphone array can realize sound pickup in a full range of angles. This step can further realize the screening of the sound signals collected by the microphone array to determine whether the sound signal corresponding to the target azimuth angle is the sound that the user wants to collect and use. Optionally, when the sound signal is within the preset angle interval, the sound can be considered as the sound to be collected, and at this time, the sound signal is enhanced to maximize the sound effect.

Through the above steps, the sound signal is collected by using a microphone array comprising a plurality of microphones; the corresponding target azimuth angle is detected when the sound signal is collected by the microphone array; when the target azimuth angle is within a preset angle interval, the sound signal is performed Signal enhancement processing, to obtain the target sound signal, to achieve the purpose of enhancing the sound from the preset angle range, thereby achieving the technical effect of improving the sound pickup effect of the sound pickup device, and then solving the problem that the sound pickup device cannot accurately distinguish the sound that needs to be enhanced Sound Technical issues that cause unsatisfactory sound processing results.

The foregoing method may be applicable to the following scenarios. It should be noted that the following scenarios are only illustrative examples, and the sound pickup method mentioned in this embodiment may also be applied to other application scenarios.

In an exemplary application scenario, the sound collection device may be a microphone in a conference room, and the device includes a microphone array. The user hopes that the microphone will focus on collecting the voice of the speaker in front of the microphone, and avoid collecting the voices of other people or objects in the conference room. At this time, the microphone can set the preset angle interval as the angle interval where the speaker is located, and then collect the sound signal transmitted into the microphone array, and judge whether the sound comes from the preset angle interval according to the sound signal. When the target azimuth angle corresponding to the sound is within the preset angle interval, the sound can be considered to be the sound of the speaker, so the sound signal is enhanced to obtain the target sound signal.

As an optional embodiment, when the target azimuth is outside the preset angle range, you can also choose to suppress the sound signal to obtain a noise signal, that is, to suppress or shield the sound from other directions in the meeting room, To ensure that the final sound signal collected and output to the loudspeaker is mainly the sound from the speaker. Through this optional embodiment, the selection of the sound that needs to be retained can be realized. Combining with the above-mentioned embodiment of enhancing the sound signal to obtain the target sound signal, it can be realized that the voice of the speaker can be enhanced and the sound in other directions can be suppressed. The voice of the speaker in the angular range is highlighted, improving the rendering effect of his voice.

Optionally, after the sound signal is enhanced or suppressed, the target sound signal or the noise signal may also be output to a target device for playing or storing the target sound signal. Wherein, the target device may be a live speaker, or a remote server or other remote terminal devices.

For example, when the target device includes a loudspeaker, and the on-site sound collection device in the meeting room and the loudspeaker are integrated on a conference terminal device, the conference terminal device includes a processor, and the processor can obtain the audio data collected by the microphone array from the sound collection device. sound signal, and then perform data processing on the sound signal to calculate the target azimuth angle corresponding to the sound signal, and then compare it with the preset angle range of the conference terminal device to determine whether the target azimuth angle is within the preset angle range; In the case of the preset angle interval, the processor enhances the sound signal to obtain the target sound signal, and outputs the target sound signal to the speaker; when the sound is outside the preset angle interval, the processor suppresses the sound signal to obtain The noise signal is then also output to the speaker; finally, the enhanced target sound signal and the suppressed noise signal can be played simultaneously by the speaker to highlight the sound from the preset angle range and improve its sound presentation effect.

For another example, the target device can be a remote server. After the sound collection device collects the sound signal, the sound signal can be processed by a local processor to obtain an enhanced or suppressed signal, and then the signal can be transmitted in the form of a video stream or an audio stream. Send to the remote server for forwarding or saving; or the sound collection device directly uploads the sound signal to the server or sends it to other remote terminal devices, and the other terminal devices process and save the sound signal.

As an optional embodiment, before adopting a microphone array including a plurality of microphones to collect sound signals, the following steps may also be included: displaying first prompt information, wherein the first prompt information is used to prompt a preset angle interval; When the target azimuth is within the preset angle interval, the method further includes: displaying second prompt information, wherein the second prompt information is used to prompt that the target azimuth corresponding to the sound signal is within the preset angle interval; If it is outside the preset angle interval, the method further includes: displaying third prompt information, wherein the third prompt information is used to prompt that the target azimuth angle corresponding to the sound signal is outside the preset angle interval.

In this optional embodiment, a device for displaying prompt information can be set on the sound collection device, such as a ring-shaped prompt light, and the sound collection device can control the prompt light to only light up a part of the area on the ring, which can be Corresponding to the preset angle range, in this case, it can be considered that the device is displaying the first prompt information, which is used to instruct the terminal device to enhance the angle range of the sound, and remind the speaker to speak at the sound collection device along this angle. Further, other indicator lights can also be set on the sound collection device to remind the speaker whether it is aimed at the sound collection device within the range of the preset angle interval when the speaker is performing a sound test, for example, when the indicator light flashes a green signal , it is displaying the second prompt information, and when the indicator light flashes red signal, it is displaying the third prompt information.

As an optional embodiment, the ring-shaped prompt light on the sound collection device can be set to respond to the user's gesture interaction, change the preset angle interval according to the gesture interaction action, and modify the preset angle interval corresponding to the sound collection device to Target angle range. The method provided in this optional embodiment can improve the functional flexibility of the sound collection device. When the user finds that his voice has not been enhanced, the user can touch the area of the ring warning light to lighten the corresponding preset angle range on the ring warning light. The position of the sound pickup device can be moved to the position corresponding to its position by dragging gestures. According to the change, the sound collection device can adjust the preset angle range to the target angle range, and then let the entire sound pickup device enhance the sound from the target angle range, while suppressing the sound from the target angle range. Sounds from other angles.

As an optional embodiment, the multiple microphones included in the microphone array may be multiple directional microphones, and the multiple directional microphones are sequentially located on multiple vertices of a regular polygon. A directional microphone is a microphone with different sensitivities to sounds coming from different directions. For example, a cardioid directional microphone can be used. Side and rear sounds. Further, multiple directional microphones can be arranged on multiple vertices of the regular polygon, and their pickup angle ranges are staggered in turn, so that the pickup ranges of each directional microphone do not overlap and complement each other, realizing 360° The range of omnidirectional sound pickup, using this setting method, can improve the sound pickup effect of the overall microphone array, and the sound pickup effect is clearer. 3 is a schematic diagram of a triangular microphone array provided according to an optional embodiment of the present invention. As shown in FIG. On the three vertices of an equilateral triangle, and make its positive direction point to different directions with an angle difference of 120° in turn, the pickup range of each cardioid directional microphone is a positive and negative deviation of 60° with the front as the axis, Each cardioid microphone picks up a total of 120°, and three cardioid microphones achieve 360° omnidirectional pickup. Those skilled in the art can understand that when the shape formed by a plurality of cardioid directional microphones is set to other regular polygons, the number of cardioid directional microphones can be increased and the parameters can be adjusted to change the sound pickup of each cardioid directional microphone. Angle range to avoid overlap of pickup range.

As an optional embodiment, detecting the corresponding target azimuth angle when the sound signal is collected by the microphone array may include the following steps: converting the sound signal into a digital signal; inputting the digital signal into the convolutional neural network CNN model to obtain a digital The target azimuth angle corresponding to the signal, where the CNN model is used to locate the direction of the sound. In this embodiment, after the sound signal is collected by the microphone array, the analog signal can be converted into a multi-channel digital signal through an ADC analog-to-digital converter, which can be used as the input of the CNN model.

The existing conventional sound source localization method only supports a single sound source, and the localization is inaccurate when there is noise and reverberation. In order to solve this problem, in this optional embodiment, a multi-sound source detection and localization algorithm based on the convolutional neural network (CNN) model is adopted, and multi-sound source detection and localization are performed based on the CNN model, which can support two or more sound sources. Source localization improves the performance of sound source localization in the presence of noise and reverberation. Among them, the CNN model can adopt a model structure based on CNN's four-layer convolution. Further, according to product characteristics or application scenarios, certain adjustments can be made to the number of layers, structure or parameters of the CNN model, for example, the number of convolution layers can be increased. , using a five-layer convolutional model structure.

As an optional embodiment, the digital signal is input into the CNN model of the convolutional neural network to obtain the target azimuth angle corresponding to the digital signal, and the following method can be adopted: the feature extraction module of the CNN model is used to extract the generalized cross-correlation from the digital signal characteristics and filter bank features; the positioning module of the CNN model is used to determine the target azimuth angle corresponding to the digital signal based on the generalized cross-correlation features and filter bank features. Among them, the generalized cross correlation feature is Generalized Cross Correlation, referred to as GCC; the filter bank feature is filter bank, referred to as FB, which reflects the amplitude-frequency characteristics of the signal, that is, the amplitude corresponding to different frequency bands. It should be noted that the digital signal input to the CNN model comes from at least two microphones in the microphone array, that is, after at least two microphones in the microphone array respectively collect the sound signal, the sound signal is converted into a digital signal and input to the CNN model, Based on this, the CNN model extracts the generalized cross-correlation features and filter bank features in the digital signal, and inputs the features into the positioning module to determine the corresponding target azimuth. Optionally, the localization module may adopt a multi-layer convolutional structure.

As an optional embodiment, the positioning module of the CNN model is used to determine the target azimuth angle corresponding to the digital signal based on the generalized cross-correlation feature and the filter bank feature. and filter bank features to determine the probability value of the sound signal collected by the microphone array from any azimuth angle; the positioning module is used to determine the target azimuth angle corresponding to the digital signal based on the probability value.

Fig. 4 is a schematic structural diagram of the CNN model provided according to an optional embodiment of the present invention. As shown in Fig. 4, the CNN model first extracts GCC and FB from the digital signal, wherein the feature dimension of GCCFB is 51 * 40 * 6 dimensions; further Specifically, the GCCFB is sequentially input into five convolutional layers, and then outputs the possibility that the azimuth angle of the digital signal is in the range of 0° to 360°, where DOA means that the maximum likelihood method is used for probability estimation.

FIG. 5 is a schematic diagram of a sound pickup process of a terminal device according to an optional embodiment of the present invention. Specifically, the sound pickup process of a terminal device may include the following steps:

S51. Configure a preset angle interval of the terminal device, where the terminal device may include a sound collection device, a processor, and a speaker.

S52. The microphone array including multiple microphones in the sound collection device collects the sound signal and inputs it to the processor, and the processor processes the sound signal into a digital signal.

S53, the CNN model of the processor extracts the feature of the digital signal and inputs it into the convolution layer to detect and locate the source direction of the sound signal.

S54, judging whether the azimuth angle corresponding to the sound signal is within the preset angle range, if yes, proceed to step S55, and if not within the range, proceed to step S56.

S55. Strengthen the digital signal to obtain the target sound signal.

S56, suppressing the digital signal to obtain a noise signal.

S57. Output the processed audio signal to the target device, where the target device may be a speaker in the terminal device, or a remote server connected to the terminal device, or other remote terminal devices for playing or storing audio data.

Fig. 6 is a schematic flow chart of the sound pickup method 2 provided according to an embodiment of the present invention. As shown in Fig. 6, the method includes the following steps:

Step S602, receiving a sound signal, wherein the sound signal is collected by a microphone array including a plurality of microphones.

Step S604, detecting the corresponding target azimuth angle when the sound signal is collected by the microphone array.

Step S606, if the target azimuth angle is within the preset angle range, perform signal enhancement processing on the sound signal to obtain the target sound signal.

Step S608, outputting the target sound signal to a target device, wherein the target device is used to play or store the target sound signal.

Through the above steps, the purpose of enhancing the sound from the preset angle range is achieved, thereby achieving the technical effect of improving the sound pickup effect of the sound pickup device, and further solving the sound processing results caused by the sound pickup device being unable to accurately distinguish the sound that needs to be enhanced Not ideal technical issues.

According to an embodiment of the present invention, a sound pickup device 1 for implementing the above sound pickup method 1 is also provided. FIG. 7 is a structural block diagram of a sound pickup device 1 provided according to an embodiment of the present invention. As shown in FIG. 7 , the The first sound pickup device 70 includes: an acquisition module 72 , a first detection module 74 and a first enhancement module 76 , and the first sound pickup device will be described below.

Acquisition module 72, for adopting the microphone array that comprises a plurality of microphones to collect sound signal;

The first detection module 74 is connected to the above-mentioned acquisition module 72, and is used to detect the corresponding target azimuth angle when the sound signal is collected by the microphone array;

The first enhancement module 76 is connected to the above-mentioned first detection module 74, and is used for performing signal enhancement processing on the sound signal to obtain the target sound signal when the target azimuth angle is within the preset angle interval.

It should be noted here that the acquisition module 72, the first detection module 74 and the first enhancement module 76 correspond to steps S202 to S206 in the embodiment, and the examples and application scenarios implemented by the three modules are the same as those of the corresponding steps , but not limited to the content disclosed in the above embodiments. It should be noted that, as a part of the device, the above modules can run in the computer terminal 10 provided in the embodiment.

According to an embodiment of the present invention, there is also provided a sound pickup device 2 for implementing the above sound pickup method 2. FIG. 8 is a structural block diagram of a sound pickup device 2 provided according to an embodiment of the present invention. As shown in FIG. 8 , the The second sound pickup device 80 includes: a receiving module 82 , a second detection module 84 , a second enhancement module 86 and an output module 88 , and the second sound pickup device 80 will be described below.

The receiving module 82 is configured to receive a sound signal, wherein the sound signal is collected by a microphone array comprising a plurality of microphones;

The second detection module 84 is connected to the above-mentioned receiving module 82, and is used to detect the corresponding target azimuth angle when the sound signal is collected by the microphone array;

The second enhancement module 86 is connected to the above-mentioned second detection module 84, and is used to perform signal enhancement processing on the sound signal to obtain the target sound signal when the target azimuth angle is within the preset angle interval;

The output module 88 is connected to the above-mentioned second enhancement module 86, and is used to output the target sound signal to the target device, wherein the target device is used to play or store the target sound signal.

It should be noted here that the above receiving module 82, the second detection module 84, the second enhancement module 86 and the output module 88 correspond to steps S602 to S608 in the embodiment, and the examples realized by multiple modules and corresponding steps It is the same as the application scenario, but not limited to the content disclosed in the above embodiments. It should be noted that, as a part of the device, the above modules can run in the computer terminal 10 provided in the embodiment.

An embodiment of the present invention may provide a terminal device. Optionally, in this embodiment, the terminal device may be located in at least one network device among multiple network devices in a computer network. The terminal device includes a memory and a processor.

Wherein, the memory can be used to store software programs and modules, such as the program instructions/modules corresponding to the sound pickup method and device in the embodiment of the present invention, and the processor executes various functional applications by running the software programs and modules stored in the memory. And data processing, promptly realize above-mentioned pickup method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory located remotely from the processor, and these remote memories may be connected to the computer terminal through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The processor can call the information stored in the memory and the application program through the transmission device to perform the following steps: adopt a microphone array including a plurality of microphones to collect sound signals; detect the corresponding target azimuth angle when the sound signal is collected by the microphone array; When the azimuth is within the preset angle range, signal enhancement processing is performed on the sound signal to obtain the target sound signal.

The processor can also call the information stored in the memory and the application program through the transmission device to perform the following steps: receive the sound signal, wherein the sound signal is collected by a microphone array including a plurality of microphones; detect when the sound signal is collected by the microphone array Corresponding target azimuth; when the target azimuth is within the preset angle range, perform signal enhancement processing on the sound signal to obtain the target sound signal; output the target sound signal to the target device, wherein the target device is used for playback or storage Target sound signal.

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a non-volatile storage medium, the storage medium It may include: a flash disk, a read-only memory (Read-Only Memory, ROM), a random access device (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like.

The embodiment of the present invention also provides a non-volatile storage medium. Optionally, in this embodiment, the above-mentioned non-volatile storage medium may be used to store program codes executed by the sound pickup method provided in the above-mentioned embodiment.

Optionally, in this embodiment, the above-mentioned non-volatile storage medium may be located in any computer terminal in the group of computer terminals in the computer network, or in any mobile terminal in the group of mobile terminals.

Optionally, in this embodiment, the non-volatile storage medium is configured to store program codes for performing the following steps: collecting sound signals by using a microphone array including a plurality of microphones; detecting when the sound signal is collected by the microphone array Corresponding target azimuth; when the target azimuth is within the preset angle range, perform signal enhancement processing on the sound signal to obtain the target sound signal.

Optionally, in this embodiment, the non-volatile storage medium is configured to store program codes for performing the following steps: receiving a sound signal, wherein the sound signal is collected by a microphone array including a plurality of microphones; detecting the sound The corresponding target azimuth angle when the signal is collected by the microphone array; when the target azimuth angle is within the preset angle range, perform signal enhancement processing on the sound signal to obtain the target sound signal; output the target sound signal to the target device, wherein, The target device is used to play or store the target sound signal.

The serial numbers of the above embodiments of the present invention are for description only, and do not represent the advantages and disadvantages of the embodiments.

In the above-mentioned embodiments of the present invention, the descriptions of each embodiment have their own emphases, and for parts not described in detail in a certain embodiment, reference may be made to relevant descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed technical content can be realized in other ways. Wherein, the device embodiments described above are only illustrative. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into Another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of units or modules may be in electrical or other forms.

A unit described as a separate component may or may not be physically separated, and a component shown as a unit may or may not be a physical unit, that is, it may be located in one place, or may be distributed over multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If an integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on such an understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes. .

The above is only a preferred embodiment of the present invention, it should be pointed out that, for those of ordinary skill in the art, without departing from the principle of the present invention, some improvements and modifications can also be made, and these improvements and modifications can also be made. It should be regarded as the protection scope of the present invention.

Claims (13)

1. A sound pickup method, comprising:

collecting sound signals using a microphone array comprising a plurality of microphones;

detecting a target azimuth angle corresponding to the sound signal acquired by the microphone array;

and under the condition that the target azimuth angle is in a preset angle interval, performing signal enhancement processing on the sound signal to obtain a target sound signal.

2. The method of claim 1, wherein detecting a target azimuth angle corresponding to when the sound signal is acquired by the microphone array comprises:

converting the sound signal into a digital signal;

and inputting the digital signal into a convolutional neural network CNN model to obtain a target azimuth angle corresponding to the digital signal, wherein the CNN model is used for positioning the sound direction.

3. The method of claim 2, wherein inputting the digital signal into a convolutional neural network CNN model to obtain a target azimuth corresponding to the digital signal comprises:

extracting generalized cross-correlation features and filter bank features from the digital signals by adopting a feature extraction module of the CNN model;

and determining a target azimuth angle corresponding to the digital signal based on the generalized cross-correlation characteristic and the filter bank characteristic by adopting a positioning module of the CNN model.

4. The method of claim 3, wherein the determining, by the positioning module employing the CNN model, a target azimuth corresponding to the digital signal based on the generalized cross-correlation feature and the filter bank feature comprises:

determining a probability value of the microphone array for acquiring the sound signal from any azimuth angle based on the generalized cross-correlation feature and the filter bank feature by adopting the positioning module of the CNN model;

and determining a target azimuth angle corresponding to the digital signal based on the probability value by adopting the positioning module.

5. The method of claim 1, wherein the plurality of microphones included in the microphone array are a plurality of directional microphones, and wherein the plurality of directional microphones are sequentially positioned on a plurality of vertices of a regular polygon.

6. The method according to claim 1, wherein the method further comprises:

and outputting the target sound signal to target equipment, wherein the target equipment is used for playing or storing the target sound signal.

7. The method of claim 6, wherein the method further comprises:

under the condition that the target azimuth angle is outside the preset angle interval, the sound signal is restrained to obtain a noise signal;

and outputting the noise signal to the target equipment.

8. The method of claim 1, wherein the step of determining the position of the substrate comprises,

before collecting sound signals with a microphone array comprising a plurality of microphones, the method further comprises:

displaying first prompt information, wherein the first prompt information is used for prompting the preset angle interval;

in the case that the target azimuth is within the preset angle interval, the method further includes: displaying second prompt information, wherein the second prompt information is used for prompting that the target azimuth corresponding to the sound signal is located in the preset angle interval;

in the case that the target azimuth angle is outside the preset angle interval, the method further includes: and displaying third prompt information, wherein the third prompt information is used for prompting that the target azimuth corresponding to the sound signal is located outside the preset angle interval.

9. A sound pickup method, comprising:

receiving a sound signal, wherein the sound signal is acquired by a microphone array comprising a plurality of microphones;

detecting a target azimuth angle corresponding to the sound signal acquired by the microphone array;

under the condition that the target azimuth angle is in a preset angle interval, carrying out signal enhancement processing on the sound signal to obtain a target sound signal;

and outputting the target sound signal to target equipment, wherein the target equipment is used for playing or storing the target sound signal.

10. A sound pickup apparatus, comprising:

the system comprises an acquisition module, a sound signal acquisition module and a sound signal acquisition module, wherein the acquisition module is used for acquiring sound signals by adopting a microphone array comprising a plurality of microphones;

the first detection module is used for detecting a target azimuth angle corresponding to the sound signal acquired by the microphone array;

the first enhancement module is used for carrying out signal enhancement processing on the sound signal under the condition that the target azimuth angle is in a preset angle interval to obtain a target sound signal.

11. A sound pickup apparatus, comprising:

the receiving module is used for receiving sound signals, wherein the sound signals are acquired by a microphone array comprising a plurality of microphones;

the second detection module is used for detecting a target azimuth angle corresponding to the sound signal acquired by the microphone array;

the second enhancement module is used for carrying out signal enhancement processing on the sound signal to obtain a target sound signal under the condition that the target azimuth angle is in a preset angle interval;

and the output module is used for outputting the target sound signal to target equipment, wherein the target equipment is used for playing or storing the target sound signal.

12. A non-volatile storage medium, characterized in that the non-volatile storage medium comprises a stored program, wherein the device in which the non-volatile storage medium is located is controlled to perform the pick-up method according to any one of claims 1 to 9 when the program is run.

13. A terminal device, characterized in that the terminal device comprises a processor for running a program, wherein the program is run to perform the sound pick-up method according to any one of claims 1 to 9.