JP5321171B2 - Sound processing apparatus and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To extract only non-objective stationary sound in an environment where the non-objective stationary sound and non-objective variation sound exist. <P>SOLUTION: A sound source separation section 30 specifies the intensity XB(k) of each non-objective sound frequency FB where the non-objective sound of a direction different from the objective sound in K pieces of frequency f1 to FK, for each unit section from a sound signal S1 and a sound signal S2. A noise estimation section 42 creates a noise spectrum N for each unit section. That is, in the noise estimation section 42, when the intensity XB(k) of a frequency fk (non-objective sound frequency FB) in the n-th unit section is lower than a threshold XTH, the intensity &mu;n of the frequency fk in a noise spectrum N of the unit section is set according to the intensity XB(k) and the intensity &mu;n-1(k) of the noise spectrum N of the (n-1)-th unit section. When the intensity XB(k) exceeds the threshold XTH, the intensity &mu;n(k) is set according to the intensity &mu;n-1(k), and the intensity XB(k) is not reflected. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  In the present invention, a non-target sound (especially a stationary component) is obtained from a mixed sound of sound arriving from a predetermined direction (hereinafter referred to as “target sound”) and sound other than the target sound (hereinafter referred to as “non-target sound”). It relates to estimation technology.

  Conventionally, a technology for selecting each of a plurality of frequencies (frequency bands) in a plurality of sound signals generated by a plurality of sound collecting devices into a target sound frequency in which the target sound is dominant and a non-target sound frequency in which the non-target sound is dominant. Proposed by For example, Non-Patent Document 1 discloses a technique (SAFIA) for selecting a frequency having a high intensity of a sound signal generated by a sound collecting device close to a target sound source among a plurality of sound signals as a target sound frequency. Further, in Patent Document 1, a target sound dominance signal in which a target sound is emphasized and a target sound inferior signal in which the target sound is suppressed are generated by delaying and adding a plurality of sound signals (that is, beam formation), and the target sound dominance is generated. A technique is disclosed in which a frequency whose signal intensity exceeds the intensity of the target sound inferior signal is selected as the target sound frequency.

Mariko Aoki, et al., "Sound source segregation based on controlling incident angle of each frequency component of input signals acquired by multiple microphones", Acoustical Science and Technology, Vol.22, No.2 p.149-p.157, 2001

JP 2006-197552 A

  By the way, under the techniques of Non-Patent Document 1 and Patent Document 1, the target sound and the non-target sound are distinguished on the basis of whether or not the sound is an incoming sound from a predetermined direction. Therefore, there is noise that is stationary in time (hereinafter referred to as “non-target steady sound”), such as operating noise of air conditioning equipment and crowded noise in crowds, and acoustic characteristics (for example, volume and pitch) ) Is changing every moment, in an environment where sounds such as sounds and musical sounds (hereinafter referred to as “non-target fluctuation sounds”) arrive from a different direction from the target sound, both non-target stationary sounds and non-target fluctuation sounds are not distinguished. Extracted as the target sound. That is, it is difficult to extract only non-target steady sounds as non-target sounds. In view of the above circumstances, an object of the present invention is to extract only non-target stationary sound with high accuracy in an environment where both non-target stationary sound and non-target fluctuation sound exist.

In order to solve the above-described problems, the sound processing apparatus according to the present invention generates a non-target sound coming from a different direction from a target sound among a plurality of frequencies from a plurality of sound signals generated by a plurality of sound collecting devices. Comprising a sound source separation means for specifying the strength (amplitude or power) of each dominant non-target sound frequency component for each unit section, and a noise estimation means for generating a noise spectrum for each unit section, The intensity of one non-target sound frequency component in the first unit section (for example, the intensity XB (k) in FIG. 4) is one non-target sound in the noise spectrum of the second unit section before the start of the first unit section. If the intensity at a frequency (for example, the intensity μn-1 (k) in FIG. 4) exceeds a threshold (for example, the threshold XTH in FIG. 4), the intensity at one non-target sound frequency in the noise spectrum of the first unit interval ( For example, the intensity μn (k)) in FIG. One non-target sound frequency component in the first unit section is set according to the intensity of the one non-target sound frequency component and the intensity at the one non-target sound frequency in the noise spectrum of the second unit section. If the intensity exceeds the threshold, the intensity at one non-target sound frequency in the noise spectrum of the first unit section is reflected in the second unit without reflecting the intensity of the component of one non-target sound frequency in the first unit section. Set to a value that exceeds the intensity at one non-target sound frequency in the noise spectrum of the unit interval. In the above configuration, when the intensity of one non-target sound frequency component in the first unit section exceeds a threshold (for example, when a non-target fluctuation sound is generated at one non-target sound frequency), the first unit section Since the intensity of the noise spectrum is set without reflecting the intensity of the component of the non-target sound frequency in, only the non-target stationary sound is extracted with high accuracy (that is, the non-target fluctuation sound is effectively suppressed). Can be generated. In addition, since a numerical value exceeding the noise spectrum intensity of the second unit section is applied as the noise spectrum intensity of the first unit section, the non-target stationary sound newly generated during the operation of the sound processing apparatus is appropriately treated as the noise spectrum. Can be included.

  The intensity of the noise spectrum of the first unit section is the intensity of the noise spectrum of one unit section (second unit section) before the start of the first unit section (for example, immediately before), or before the start of the first unit section. It is set according to the intensity of each noise spectrum of a plurality of unit intervals (second unit intervals). According to a preferred aspect of the present invention, when the intensity of one non-target sound frequency component in the first unit section is below a threshold, the noise estimation means The weighted sum (for example, Equation (2)) with the intensity at one non-target sound frequency in the noise spectrum of the second unit section is set as the intensity at one non-target sound frequency in the noise spectrum of the first unit section. To do. In the above aspect, since the weighted sum of the intensity of the non-target sound frequency in the first unit interval and the intensity of the noise spectrum in the second unit interval is calculated as the intensity of the noise spectrum in the first unit interval, There is an advantage that it is not necessary to hold the noise spectrum over a plurality of past unit intervals when viewed from the interval. The “frequency” in the present invention is a concept including a frequency band having a spread on the frequency axis in addition to a single frequency on the frequency axis.

  In a preferred aspect of the present invention, the sound source separation means generates a target sound spectrum composed of components of each target sound frequency where the target sound is dominant among a plurality of frequencies, and subtracts the noise spectrum from the target sound spectrum. Suppressing means is provided. In the above aspect, since the noise spectrum of the non-target stationary sound is subtracted from the target sound spectrum composed of the components selected for the target sound frequency, both the non-target fluctuation sound and the non-target stationary sound are effectively suppressed. Is possible.

The sound processing apparatus according to each aspect described above is realized by hardware (electronic circuit) such as a DSP (Digital Signal Processor) dedicated to sound processing, and a general-purpose arithmetic processing apparatus such as a CPU (Central Processing Unit). This is also realized through collaboration with programs. The program according to the present invention is based on a plurality of sound signals generated by a plurality of sound collecting devices, and the intensity of each non-target sound frequency component in which a non-target sound arriving from a direction different from the target sound is dominant among a plurality of frequencies. Sound source separation processing for identifying each unit section and processing for generating a noise spectrum for each unit section, where the intensity of one non-target sound frequency component in the first unit section is the start of the first unit section If the noise spectrum of the second unit interval is less than the threshold value exceeding the intensity at one non-target sound frequency, the intensity at one non-target sound frequency in the noise spectrum of the first unit interval is It is set according to the intensity of one non-target sound frequency component and the intensity of one non-target sound frequency in the noise spectrum of the second unit section, and the one non-target sound frequency component of the first unit section Strength is threshold If the value exceeds the value, the intensity at one non-target sound frequency in the noise spectrum of the first unit section is not reflected in the intensity of the component of one non-target sound frequency in the first unit section. The computer executes noise estimation processing for setting a numerical value exceeding the intensity at one non-target sound frequency in the noise spectrum. According to the above program, the same operation and effect as the signal processing apparatus according to the present invention are exhibited. The program of the present invention is provided to a user in a form stored in a computer-readable recording medium and installed in the computer, or provided from a server device in a form of distribution via a communication network and installed in the computer. Is done.

1 is a block diagram of a sound processing apparatus according to a first embodiment of the present invention. It is a block diagram of a sound source separation part. It is a graph for demonstrating the process by a signal processing part. It is a flowchart of operation | movement of a noise estimation part. It is a spectrogram of the noise spectrum in a 1st embodiment. It is the spectrogram of the noise spectrum in contrast.

<A: First Embodiment>
FIG. 1 is a block diagram of a sound processing apparatus according to the first embodiment of the present invention. As shown in FIG. 1, a sound collection device M1 and a sound collection device M2 are connected to the sound processing apparatus 100. The sound collection device M1 and the sound collection device M2 are omnidirectional (substantially omnidirectional) microphones that generate a signal representing a surrounding acoustic waveform. The mixed sound of the target sound and the non-target sound reaches the sound collecting device M1 and the sound collecting device M2 from the surroundings. Each of the sound collecting device M1 and the sound collecting device M2 generates an electrical signal representing a waveform of a mixed sound of the target sound and the non-target sound. The sound collecting device M1 generates a sound signal S1, and the sound collecting device M2 generates a sound signal S2.

  The target sound is sound that arrives at the sound collecting device M1 and the sound collecting device M2 from the known direction D0. For example, when the sound processing apparatus 100 is mounted on an electronic device (for example, a mobile phone) to which a user's speech sound is input, the speech sound arrives as a target sound from the front direction D0 with respect to the main body of the electronic device. The sound collection device M1 and the sound collection device M2 are arranged apart from each other along a direction perpendicular to the direction D0 in which the target sound arrives. On the other hand, the non-target sound is sound coming from a direction (DR, DL) different from the direction D0 of the target sound. The non-target sound arrives at the sound collecting device M1 and the sound collecting device M2 from the direction DR of 45 ° clockwise with respect to the direction D0 and the direction DL of 45 ° counterclockwise with respect to the direction D0.

  The sound processing apparatus 100 generates a sound signal SOUT that suppresses the non-target sound of the mixed sound of the target sound and the non-target sound from the sound signal S1 and the sound signal S2. The sound signal SOUT is reproduced as sound by being supplied to a sound emitting device (for example, a speaker or headphones). Note that an A / D converter that converts the sound signal S1 and the sound signal S2 into a digital signal and a D / A converter that converts the sound signal SOUT into an analog signal are omitted for convenience.

  As shown in FIG. 1, the sound processing device 100 is realized by a computer system including an arithmetic processing device 12 and a storage device 14. The storage device 14 stores a program and various data for generating the sound signal SOUT from the sound signal S1 and the sound signal S2. A known recording medium such as a magnetic recording medium or a semiconductor recording medium is arbitrarily employed as the storage device 14. The arithmetic processing unit 12 functions as a plurality of elements (frequency analysis unit 20, sound source separation unit 30, noise estimation unit 42, noise suppression unit 44, signal synthesis unit 50) by executing a program stored in the storage device 14. To do. A configuration in which an electronic circuit (DSP) dedicated to sound processing realizes each element of the arithmetic processing device 12 and a configuration in which each element of the arithmetic processing device 12 is mounted in a plurality of integrated circuits are also adopted. .

  The frequency analysis unit 20 calculates a frequency spectrum P1 for each of a plurality of unit sections (frames) obtained by dividing the sound signal S1 on the time axis. For specifying the frequency spectrum P1, known frequency analysis such as FFT (Fast Fourier Transform) processing is arbitrarily employed. Further, the frequency analysis unit 20 specifies the frequency spectrum P2 for each unit section of the sound signal S2 by the same method as the specification of the frequency spectrum P1.

  The sound source separation unit 30 of FIG. 1 uses the target sound frequency FA and the non-target sound for each of the K (K is a natural number) frequencies (frequency bands) f1 to fK discretely set on the frequency axis. By selecting the frequency FB, the target sound spectrum QA and the non-target sound spectrum QB are generated for each unit section. The target sound frequency FA is a frequency where the target sound is dominant (typically the frequency where the target sound volume exceeds the volume of the non-target sound), and the non-target sound frequency FB is a frequency where the non-target sound is dominant (typically Specifically, the non-target sound volume is higher than the target sound volume). The target sound spectrum QA1 is composed of components of the target sound frequency FA, and the non-target sound spectrum QB is composed of components of the non-target sound frequency FB. In order to select the target sound frequency FA and the non-target sound frequency FB, as illustrated below, a method using the difference between the direction D0 where the target sound arrives and the direction (DR, DL) where the non-target sound arrives (Patent Document 1) is preferably employed.

  FIG. 2 is a block diagram of the sound source separation unit 30. As shown in FIG. 2, the sound source separation unit 30 includes a signal processing unit 32, a frequency selection unit 34, and an intensity specifying unit 36. The signal processing unit 32 compares a plurality of frequency spectra (P0, PR, PL) in which the incoming sound from each of the plurality of directions (D0, DR, DL) is suppressed by comparing with the incoming sound from the other direction. And the frequency spectrum P2. FIG. 3 is a graph for explaining the contents of processing by the signal processing unit 32. The horizontal axis in FIG. 3 means the angle θ with the target sound direction D0 as the reference (0 °), and the vertical axis in FIG. 3 means the signal intensity (amplitude or power).

  As shown in FIG. 2, the signal processing unit 32 includes a first processing unit 321, a second processing unit 322, and a third processing unit 323. The first processing unit 321 generates the frequency spectrum P0 by subtracting the frequency spectrum P2 from the frequency spectrum P1. Since the target sound arriving from the direction D0 reaches the sound collecting device M1 and the sound collecting device M2 with substantially the same phase, the frequency spectrum P0 is the purpose arriving from the direction D0, as indicated by the symbol B0 (solid line) in FIG. This corresponds to a spectrum in which sound is suppressed with respect to incoming sound from another direction. In other words, the first processing unit 321 is a blind spot control type (null) beamformer that forms a blind spot on sound collection in the direction D0.

  The second processing unit 322 generates the frequency spectrum PR by subtracting the frequency spectrum D (P1) of the signal obtained by delaying the sound signal S1 by the delay amount D from the frequency spectrum P2. The delay amount D is set to the time difference between the time when the incoming sound from the direction DR reaches the sound collecting device M1 and the time when it reaches the sound collecting device M2. Therefore, the frequency spectrum PR corresponds to a spectrum in which the non-target sound arriving from the direction DR is suppressed with respect to the incoming sound from another direction, as indicated by a symbol BR (broken line) in FIG. In other words, the second processing unit 322 is a blind spot control type beam former that forms a blind spot on sound collection in the direction DR. Similarly, the third processing unit 323 subtracts the frequency spectrum D (P2) of the signal obtained by delaying the sound signal S2 by the delay amount D from the frequency spectrum P1, as indicated by reference sign BL in FIG. This is a blind spot control type beam former that generates a frequency spectrum PL in which non-target sounds from DL are suppressed.

  2 compares the intensities of the three types of frequency spectra (P0, PR, and PL) generated by the signal processing unit 32 for each of the K frequencies f1 to fK. Each of f1 to fK is sorted into a target sound frequency FA and a non-target sound frequency FB. As shown in FIG. 2, the frequency selection unit 34 includes a first comparison unit 341 and a second comparison unit 342.

  The first comparison unit 341 generates the frequency spectrum PLR by comparing the intensities at each of the K frequencies f1 to fK between the frequency spectrum PR and the frequency spectrum PL. The intensity of the frequency spectrum PLR at the frequency fk is set to the lower one of the intensity at the frequency fk of the frequency spectrum PR and the intensity at the frequency fk of the frequency spectrum PL. Since the frequency spectrum PR is a spectrum in which the non-target sound from the direction DR is suppressed, and the frequency spectrum PL is a spectrum in which the non-target sound from the direction DL is suppressed, the frequency spectrum PLR is the non-purpose of the direction DR and the direction DL. This corresponds to a spectrum in which the sound is suppressed (that is, a spectrum in which the target sound from the direction D0 is emphasized).

  The second comparison unit 342 compares the intensities of the K frequencies f1 to fK between the frequency spectrum P0 and the frequency spectrum PLR. The frequency spectrum P0 is a spectrum that emphasizes the non-target sound, and the frequency spectrum PLR is a spectrum that emphasizes the target sound. Accordingly, the second comparison unit 342 selects, as the target sound frequency FA, the frequency fk in which the intensity of the frequency spectrum PLR exceeds the intensity of the frequency spectrum P0 among the K frequencies f1 to fK, and the K frequencies f1 to fK. The frequency fk in which the intensity of the frequency spectrum P0 exceeds the intensity of the frequency spectrum PLR is selected as the non-target sound frequency FB.

  The intensity specifying unit 36 in FIG. 2 generates a target sound spectrum QA and a non-target sound spectrum QB for each unit section using the result of selection by the frequency selection unit 34. The target sound spectrum QA is a series (XA (1) to XA (K)) of intensity XA (k) set for each frequency fk according to the intensity of the target sound, and the non-target sound spectrum QB is a non-target sound spectrum QB. This is a series (XB (1) to XB (K)) of intensity XB (k) set for each frequency fk according to the intensity of sound. The setting of the intensity XA (k) and the intensity XB (k) will be described in detail below.

  As shown in FIG. 3, the non-target sound is emphasized in the frequency spectrum P0 (symbol B0), and the target sound is emphasized in the frequency spectrum PLR. Therefore, the intensity specifying unit 36 uses the intensity XA (k) of each frequency fk selected as the target sound frequency FA in the target sound spectrum QA as the intensity (mainly derived from the target sound) of the frequency spectrum PLR. The intensity at the frequency fk of the frequency spectrum P0 (mainly derived from the non-target sound) is subtracted from the intensity of the frequency spectrum P0. As described above, the intensity XA (k) of each target sound frequency FA is calculated by subtracting the frequency spectrum P0 from the frequency spectrum PLR (spectral subtraction), so that the non-included frequency included in the target sound frequency FA of the frequency spectrum PLR. It is possible to generate a target sound spectrum QA in which the influence of the target sound is effectively reduced. However, a configuration in which the intensity of the frequency spectrum PLR in which the target sound is emphasized is set as the intensity XA (k) of the target sound spectrum QA is also suitable. The intensity XA (k) of each frequency fk selected as the non-target sound frequency FB in the target sound spectrum QA is set to zero.

  Further, the intensity specifying unit 36 uses the intensity XB (k) at each frequency fk selected as the non-target sound frequency FB in the non-target sound spectrum QB at the frequency fk of the frequency spectrum P1 generated by the frequency analysis unit 20. Set to strength. It should be noted that the intensity XB of the non-target sound spectrum QB at the non-target sound frequency FB is set to the intensity at the frequency fk of the frequency spectrum P2, or the intensity at the frequency fk of the frequency spectrum P0 (mainly derived from the non-target sound). A configuration in which the intensity at the frequency fk of the frequency spectrum PLR (mainly the intensity derived from the target sound) is subtracted from the intensity of the frequency spectrum PLR is also employed. The intensity XB (k) of each frequency fk selected as the target sound frequency FA in the non-target sound spectrum QB is set to zero.

  By the way, the component of the non-target sound frequency FB (non-target sound) includes the target sound in addition to the non-target stationary sound that is temporally steady (small change in acoustic characteristics such as volume and pitch). Non-target fluctuation sounds coming from other directions are included. Non-objective steady sound is noise such as operating noise of air conditioning equipment and crowded noise in crowds, and non-objective sound is voice or musical sound whose acoustic characteristics such as volume and pitch change every moment. It is a disturbance sound. The noise estimation unit 42 in FIG. 1 generates a noise spectrum N for each unit section by suppressing (ideally removing) non-target fluctuation sound in the non-target sound spectrum QB. The noise spectrum N of the nth unit section is a series of intensities μn (1) to μn (K) at each of the K frequencies f1 to fK.

  FIG. 4 is a flowchart of an operation in which the noise estimation unit 42 generates the noise spectrum N of the nth unit section. The process of FIG. 4 is sequentially executed for each unit section. When the processing of FIG. 4 is started, the noise estimation unit 42 initializes the variable k to 1 (step S1). The variable k is a number that specifies any of the K frequencies f1 to fK.

  The noise estimation unit 42 determines whether or not the frequency fk is the non-target sound frequency FB (step S2). When the frequency fk is the non-target sound frequency FB, the noise estimation unit 42 determines that the intensity XB (k) at the frequency fk (non-target sound frequency FB) in the non-target sound spectrum QB of the nth unit section is the threshold value XTH. It is determined whether or not (step S3).

As defined by the following formula (1), the threshold value XTH is the intensity μn−1 () at the frequency fk of the noise spectrum N generated by the noise estimation unit 42 for the immediately preceding ((n−1) th) unit interval. k) multiplied by a coefficient τ. The coefficient τ is set to a predetermined value (for example, 2) exceeding 1. Therefore, the threshold value XTH is set to a numerical value (variable value corresponding to the intensity μn-1 (k)) exceeding the intensity μn-1 (k). A predetermined initial value is applied as the intensity μn-1 (k) for the first unit section.
XTH ＝ τ ・ μn-1 (k) (1)

  Since the intensity of the non-target fluctuation sound is likely to change compared to the non-target stationary sound, the intensity XB (k) of the frequency fk at which the non-target fluctuation sound is generated in the non-target sound spectrum QB varies greatly with time. Therefore, the comparison between the intensity XB (k) and the threshold value XTH in step S3 corresponds to a process of determining whether or not a non-target fluctuation sound is generated at the frequency fk in the non-target sound spectrum QB. That is, when the intensity XB (k) exceeds the threshold value XTH, the frequency fk component of the non-target sound spectrum QB is estimated to correspond to the non-target fluctuation sound, and when the intensity XB (k) is lower than the threshold value XTH, the non-target sound It is estimated that the component of the frequency fk of the spectrum QB does not correspond to the non-target fluctuation sound (corresponds to the non-target stationary sound). The coefficient τ in Equation (1) is statistical so that the intensity XB (k) exceeds the threshold value XTH when a non-target fluctuation sound is generated, and the intensity XB (k) falls below the threshold value XTH when only non-target stationary sound exists. Selected experimentally or experimentally.

When the intensity XB (k) of the non-target sound spectrum QB is lower than the threshold value XTH (that is, when no non-target fluctuation sound is generated at the frequency fk), the noise estimation unit 42 performs the non-target of the nth unit section. From the intensity XB (k) at the frequency fk of the sound spectrum QB and the intensity μn-1 (k) at the frequency fk of the noise spectrum N of the (n-1) th unit interval, the nth noise spectrum N The intensity μn (k) at the frequency fk is calculated (step S4). The intensity μn (k) is defined by the following formula (2), for example, the intensity XB (k) in the non-target sound spectrum QB of the nth unit section and the (n−1) th unit It is calculated as a weighted sum (weighted average) with the intensity μn-1 (k) in the noise spectrum N of the section. The coefficient α in Expression (2) is set to a positive number (for example, 0.9) less than 1. As understood from Equation (2), the larger the coefficient α, the smaller the influence of the intensity XB (k) on the intensity μn (k) (the influence of the intensity XB (k) in each past unit interval increases. ).
μn (k) = α ・ μn-1 (k) + (1-α) ・ XB (k) (2)

On the other hand, when the intensity XB (k) of the non-target sound spectrum QB exceeds the threshold value XTH (S3: YES), the noise estimation unit 42 calculates the (n-1) th noise spectrum as shown in Equation (3). The intensity μn−1 (k) at the frequency fk of N is set as the intensity μn (k) at the frequency fk (non-target sound frequency FB) of the nth noise spectrum N (step S5). That is, when the intensity XB (k) exceeds the threshold value XTH (when the intensity XB (k) increases due to the generation of the non-target fluctuation sound of the frequency fk), the intensity XB (k) of the non-target sound spectrum QB is It is not reflected in the intensity μn (k). Therefore, in the noise spectrum N, the non-target fluctuation sound in the non-target sound spectrum QB is suppressed (removed).
μn (k) = μn-1 (k) (3)

  When the frequency fk is the target sound frequency FA (S2: NO), the noise estimation unit 42 calculates the intensity μn (k) of the (n−1) th noise spectrum N in the same way as the equation (3). The intensity μn (k) at the frequency fk (target sound frequency FA) of the nth noise spectrum N is set (step S6).

  As understood from the equations (2) and (3), the intensity μn (k) of the noise spectrum N in the nth unit interval is a plurality of unit intervals in the past (before the (n-1) th) unit interval. It is a numerical value that cumulatively reflects the intensity of the noise spectrum N calculated for. That is, the intensity μn (k) of the noise spectrum N is equal to the intensity XB (k) of the non-target sound spectrum QB over a plurality of unit intervals where the intensity XB (k) of the frequency fk selected as the non-target sound frequency FB is lower than the threshold value XTH. ) Is a smoothed (averaged) value.

  As described above, when the intensity μn (k) is set in each step (S4, S5, S6), the noise estimation unit 42 determines whether or not the variable k has reached the predetermined value K (step S7). If the variable k has not reached the predetermined value K, the noise estimation unit 42 adds 1 to the variable k (step S8), and then proceeds to step S2. That is, the intensity μn (k) is sequentially calculated for each of the K frequencies f1 to fK. When the variable k reaches the numerical value K (that is, when the calculation of the intensity μn (1) to μn (K) is completed), the noise estimation unit 42 ends the process of FIG. 4 (S7: YES). A series of intensities μn (1) to μn (K) for K frequencies f1 to fK corresponds to the noise spectrum N of the nth unit interval.

  The noise suppression unit 44 in FIG. 1 generates a noise suppression spectrum QC by subtracting the noise spectrum N generated by the noise estimation unit 42 from the target sound spectrum QA generated by the sound source separation unit 30 (spectral subtraction). Specifically, the noise suppression unit 44 calculates the intensity μn (at the frequency fk of the noise spectrum N generated for the unit section from the intensity XA (k) of the frequency fk at the target sound spectrum QA of the nth unit section. A noise suppression spectrum QC is generated by subtracting k).

That is, the intensity XC (k) at the frequency fk of the noise suppression spectrum QC for the nth unit interval is expressed by the equation (4a). However, the intensity XC (k) of the frequency fk at which the right side (XA (k) −μn (k)) of the formula (4a) is a negative number is set to zero. In addition, the noise suppression spectrum QC is expressed by Equation (4b). The symbol e ^{jθ (k) in the} equation (4b) is a phase component of the target sound spectrum QA. As can be understood from the equations (4a) and (4b), the noise suppression spectrum QC is a sound obtained by suppressing the non-target stationary sound (noise spectrum N) from the incoming sound (target sound spectrum QA) from the direction D0 (ie, the noise spectrum N). This corresponds to the spectrum of the target sound.
XC (k) = XA (k) -μn (k) (4a)
QC = {XA (k) −μn (k)} e ^{jθ (k)} (4b)

  The signal synthesis unit 50 generates a time-domain sound signal SOUT from the noise suppression spectrum QC generated by the noise suppression unit 44. As shown in FIG. 1, the signal synthesis unit 50 includes an adjustment unit 52, a synthesis unit 54, and an inverse conversion unit 56. The adjustment unit 52 multiplies each intensity XB (1) to XB (K) of the non-target sound spectrum QB generated by the sound source separation unit 30 by a coefficient p. The coefficient p is set to a predetermined positive number (for example, 0.01).

  The synthesizer 54 generates an output spectrum R for each unit section by synthesizing the noise suppression spectrum QC generated by the noise suppression unit 44 and the non-target sound spectrum QB processed by the adjustment unit 52. The output spectrum R includes the intensity XC (k) of each frequency fk selected as the target sound frequency FA in the noise suppression spectrum QC and the intensity of each frequency fk selected as the non-target sound frequency FB out of the non-target sound spectrum QB. XB (k) is a series arranged along the frequency axis. That is, the intensity of each frequency fk selected for the target sound frequency FA in the output spectrum R is set to the intensity XC (k) of the noise suppression spectrum QC, and selected for the non-target sound frequency FB in the output spectrum R. The intensity of each frequency fk is set to a multiplication value of the intensity XB (k) of the non-target spectrum QB and the coefficient p. As described above, since the non-target sound spectrum QB adjusted by the adjusting unit 52 is synthesized with the noise suppression spectrum QC, the noise suppression spectrum QC is output as the output spectrum R (non-target sound frequency FB of the reproduced sound). Compared with a configuration in which the intensity of the sound is set to zero, it is possible to generate a audibly natural reproduced sound.

  The inverse conversion unit 56 converts the output spectrum R of each unit section into a time domain signal by inverse FFT processing, and generates a sound signal SOUT by connecting the converted signals of each unit section to each other on the time axis. To do. By supplying the sound signal SOUT to the sound emitting device (not shown), the reproduction sound in which the non-target sound is suppressed and the target sound is emphasized is emitted.

  In the above embodiment, the target sound frequency FA and the non-target sound frequency FB are selected using the difference between the direction D0 of the target sound and the direction (DR, DL) of the non-target sound. Even if the target sound has similar acoustic characteristics, the target sound spectrum QA and the non-target sound spectrum QB can be separated with high accuracy. Further, since the noise spectrum N generated from the non-target sound spectrum QB is subtracted from the target sound spectrum QA, the noise suppression spectrum QC (and the output spectrum R and reproduced sound) that effectively reduces the non-target steady sound is obtained. There is an advantage that it can be generated.

  Further, in the above form, the intensity XB (k) is not reflected in the intensity μn (k) of the noise spectrum N for the frequency fk where the intensity XB (k) of the non-target sound spectrum QB exceeds the threshold value XTH. Even in an environment where both stationary sound and non-target fluctuation sound exist, it is possible to generate a noise spectrum N obtained by extracting only non-target stationary sound with high accuracy. The effect of this form is explained in full detail below.

  5 and 6 are time series (spectrogram) of the noise spectrum N of each unit section. FIG. 5 is a time series of the noise spectrum N in the first embodiment, and FIG. 6 is a time series of the noise spectrum N in comparison with the first embodiment. In contrast, regardless of the intensity XB (k) of the non-target sound frequency FB, the intensity μn (k) of the noise spectrum N is calculated by the equation (2) (that is, steps S3 and S5 in FIG. (Omitted form).

  5 and 6, a straight line in which the frequency fk (peak frequency) having a high intensity in the noise spectrum N is connected along the time axis is illustrated. A thicker straight line means higher strength. In the illustrations of FIGS. 5 and 6, the non-target stationary sound that does not change with time exists on the low frequency side of the noise spectrum N (non-target sound spectrum QB). 5 and 6 show the time when the non-target fluctuation sound is generated.

  In contrast, the intensity μn (k) of the noise spectrum N is calculated by the equation (2) regardless of the intensity XB (k) of the non-target sound spectrum QB (that is, regardless of the presence or absence of non-target fluctuation sound). . Therefore, the noise spectrum N includes both non-target stationary sounds and non-target fluctuation sounds. Since the past intensity μn-1 (k) is cumulatively reflected in the intensity μn (k) calculated by Equation (2), a non-target fluctuation sound is generated at a specific time in the noise spectrum N. As shown in FIG. 6, the intensity μn (k) of the frequency fk is maintained at a high value over a plurality of subsequent unit intervals even when the non-target sound fluctuation sound is stopped. Therefore, the intensity of the target sound spectrum QA at the frequency fk where the target sound fluctuation sound is generated is excessively reduced by the processing by the noise suppressing unit 44, which may cause annoying musical noise.

  In contrast to the proportionality, in the first embodiment, the intensity Xn (k) of the frequency fk at which the intensity XB (k) exceeds the threshold value XTH has the intensity XB (k) (that is, the non-target fluctuation sound of the frequency fk). (Intensity) is not reflected, and as shown in FIG. 5, a noise spectrum N in which non-target fluctuation sound is suppressed is generated. Therefore, there is an advantage that the intensity of the frequency fk where the non-target fluctuation sound is generated in the target sound spectrum QA is prevented from being excessively reduced, and the generation of musical noise is suppressed. Since the non-target fluctuation sound is suppressed in the noise spectrum N, the effect of reducing the non-target fluctuation sound from the target sound spectrum QA by the processing by the noise suppression unit 44 is small. However, since the non-target fluctuation sound arriving from the direction DR or the direction DL is excluded from the target sound spectrum QA by the selection by the sound source separation unit 30, the noise suppression unit 44 does not reduce the non-target fluctuation sound. In addition, it is possible to generate a reproduced sound in which both the non-target steady sound and the non-target fluctuation sound are suppressed with high accuracy.

  Incidentally, the intensity specifying unit 36 of the first embodiment generates the target sound spectrum QA by subtracting the frequency spectrum P0 in which the non-target sound is emphasized from the frequency spectrum PLR in which the target sound is emphasized. That is, the non-target sound can be suppressed only by the processing by the intensity specifying unit 36. However, for example, when the non-target stationary sound is included in the incoming sound from the direction D0, the non-target stationary sound is not sufficiently suppressed even if the frequency spectrum P0 is subtracted from the frequency spectrum PLR. According to the first embodiment in which the noise spectrum N of the non-target stationary sound is subtracted from the target sound spectrum QA, the configuration that suppresses the non-target sound only by the processing by the intensity specifying unit 36 (that is, the configuration in which the noise suppression unit 44 is omitted). ) Has an advantage that non-target stationary sound is effectively suppressed.

<B: Second Embodiment>
Next, a second embodiment of the present invention will be described. In addition, about the element in which an effect | action and a function are equivalent to 1st Embodiment in each following aspect, the same code | symbol as the above is attached | subjected and each detailed description is abbreviate | omitted suitably.

  In the first embodiment, when the intensity XB (k) of the non-target sound spectrum QB exceeds the threshold value XTH, the intensity μn−1 (k) of the past noise spectrum N is changed to the intensity μn of the nth noise spectrum N. Set to (k). According to the above configuration, the influence of the non-target fluctuation sound can be removed from the noise spectrum N, while the non-target steady state starts to be newly generated during the operation of the sound processing apparatus 100 with the intensity XB (k) exceeding the threshold value XTH. Sound (hereinafter particularly referred to as “new stationary sound”) is also removed from the noise spectrum N. Therefore, there is a possibility that the suppression of the new stationary sound is insufficient. The second embodiment is configured to solve the above problems.

In the second embodiment, the process of step S5 in FIG. 4 is different from that of the first embodiment. When the intensity XB (k) of the non-target sound spectrum QB exceeds the threshold value XTH (that is, when a non-target fluctuation sound or a new stationary sound is generated), the noise estimation unit 42 calculates Formula (3) of the first embodiment. Instead, the following equation (5) is calculated. That is, the noise estimation unit 42 sets the product of the intensity μn−1 (k) of the (n−1) th noise spectrum N and the coefficient β as the intensity μn (k) of the nth noise spectrum N. Set (step S5).
μn (k) ＝ β ・ μn-1 (k) (5)

  The coefficient β is set to a predetermined value (for example, 1.01) exceeding 1. Therefore, in a plurality of unit intervals in which the state where the intensity XB (k) exceeds the threshold value XTH continues, the intensity μn (k) of the noise spectrum N increases with time, and the non-target sound (non-target fluctuation sound or new steady sound) ) The intensity μn (k) approaches the intensity of the non-target steady sound more rapidly as the coefficient β increases.

  In the above embodiment, since the intensity μn (k) of the noise spectrum N approaches the intensity of the new stationary sound over time, the noise spectrum N reflecting the characteristics of the new stationary sound is generated. Therefore, it is possible to effectively suppress non-target sounds including new stationary sounds from the target sound spectrum QA.

  Note that the intensity μn (k) of the noise spectrum N increases with time according to the calculation of Equation (5) not only when a new stationary sound is generated but also when a non-target fluctuation sound is generated. That is, in the second embodiment, the occurrence of non-target fluctuation sound is reflected in the intensity μn (k) of the noise spectrum N. However, since the non-target fluctuation sound is likely to change with time, the possibility of being maintained at a high intensity for a long time is sufficiently low as compared with the new stationary sound. That is, even when a non-target fluctuation sound is generated, before the intensity μn (k) of the noise spectrum N sufficiently approaches the non-target fluctuation sound, the non-target fluctuation sound decreases to an intensity below the threshold value XTH. (Equation (2) is applied to the calculation of the intensity μn (k)), thereby suppressing the increase in the intensity μn (k). Therefore, although the intensity μn (k) of the noise spectrum N increases with time when the intensity XB (k) exceeds the threshold value XTH, the intensity μn (k) increases when the non-target fluctuation sound is generated. Is small enough. That is, according to the second embodiment, there is an advantage that the noise spectrum N reflecting the new stationary sound can be generated while sufficiently suppressing the influence of the non-target fluctuation sound.

<C: Modification>
Various modifications can be made to the embodiments exemplified above. Specific modifications are exemplified below. Two or more aspects may be arbitrarily selected from the following examples and combined.

(1) Modification 1
The content of processing by the noise suppression unit 44 is changed as appropriate. For example, the following formula (6) is used instead of the formula (4a) for the calculation of the intensity XC (k) of the noise suppression spectrum QC. However, when the right side of formula (6) (XA (k) −γ · μn (k)) is below a predetermined value δ · μn (k), the intensity XC (k) is set to δ · μn (k). . The coefficient γ is set to a predetermined value of 1 or more (for example, 3 to 6), and the coefficient δ is set to a positive number that is sufficiently smaller than 1 (for example, 0.01).
XC (k) = XA (k) −γ ・ μn (k) (6)

  As understood from the equation (6), the intensity μn (k) of the noise spectrum N is excessively subtracted from the intensity XA (k) (oversubtraction), so that the non-target sound (non-target steady sound) is sufficient. It is possible to generate a suppressed high-quality reproduced sound. On the other hand, the intensity XC (k) of the noise suppression spectrum QX is the predetermined value δ for the frequency fk where the right side (XA (k) −γ · μn (k)) of the equation (6) is lower than the predetermined value δ · μn (k). Since it is set to μn (k), it is possible to prevent the intensity XC (k) from decreasing to zero and generate a natural reproduced sound.

(2) Modification 2
As exemplified below, a configuration in which the noise estimation unit 42 variably controls the coefficient α in Expression (2) is also suitable.
Since the noise suppression spectrum QC is calculated by subtracting the noise spectrum N from the target sound spectrum QA, if the noise spectrum N changes with changes in the characteristics (for example, volume) of the non-target stationary sound, the characteristics of the noise suppression spectrum QC Also changes. On the other hand, as understood from the equation (2), the influence of the intensity XB (k) of the non-target sound frequency FB in the nth unit interval is larger as the coefficient α is larger (the coefficient (1-α) is smaller). It is suppressed. Therefore, the change in the volume of the target sound in the reproduced sound when the volume of the non-target sound is changed is reduced as the coefficient α is increased.

  If the target sound volume fluctuates significantly during a period in which the target sound is dominant (a period in which the target sound frequency FA is large), an unnatural impression will occur, so that the target sound frequency FA in the nth unit section A configuration in which the noise estimation unit 42 variably controls the coefficient α is suitable so that the coefficient α increases as the number increases (the number of non-target sound frequencies FB decreases). According to the above configuration, since the change in the volume of the target sound is suppressed even if the volume of the non-target stationary sound changes during the period in which the target sound is dominant, a natural reproduction sound can be generated for hearing. Is possible.

(3) Modification 3
The method of selecting the K frequencies f1 to fK into the target sound frequency FA and the non-target sound frequency FB is appropriately changed. Specifically, the technique (SAFIA) disclosed in Non-Patent Document 1 and Japanese Patent Laid-Open No. 10-313497 is used for selecting the target sound frequency FA and the non-target sound frequency FB. For example, it is assumed that the sound collection device M1 is closer to the target sound source than the sound collection device M2, and the sound collection device M2 is closer to the non-target sound source than the sound collection device M1. The sound source separation unit 30 compares the intensities of the K frequencies f1 to fK between the frequency spectrum P1 and the frequency spectrum P2, and selects the frequency fk having a high intensity of the frequency spectrum P1 as the target sound frequency FA. A frequency fk having a high intensity of the frequency spectrum P2 is selected as a non-target sound frequency FB. According to the above configuration, since the signal processing unit 32 of FIG. 2 is not required, there is an advantage that the processing and configuration of the sound processing device 100 are simplified.

  Instead of the blind spot control type beamformer, the following configuration in which a delay addition type beamformer is adopted in the signal processing unit 32 (first processing unit 321, second processing unit 322, third processing unit 323) is also suitable. . The first processing unit 321 generates the frequency spectrum P0 in which the target sound in the direction D0 is emphasized by adding the frequency spectrum P1 and the frequency spectrum P2. The second processing unit 322 generates the frequency spectrum PR in which the non-target sound in the direction DR is emphasized by adding the frequency spectrum P2 and the frequency spectrum P1 to which the delay amount D is added. Similarly, the third processing unit 323 generates a frequency spectrum PL in which the non-target sound in the direction DL is emphasized. The first comparison unit 341 sets the intensity at the frequency fk of the frequency spectrum PLR to the higher intensity of the intensity at the frequency fk of the frequency spectrum PR and the intensity at the frequency fk of the frequency spectrum PL. Therefore, the frequency spectrum PLR is a spectrum in which the non-target sound in the direction DR and the direction DL is emphasized. Then, the second comparison unit 342 selects, as the non-target sound frequency FB, the frequency of which the intensity of the frequency spectrum PLR exceeds the intensity of the frequency spectrum P0 among the K frequencies, and the frequency spectrum P0 of the K frequencies. A frequency whose intensity exceeds the intensity of the frequency spectrum PLR is selected as the target sound frequency FA.

  A configuration in which the signal processing unit 32 processes the sound signal S1 and the sound signal S2 in the time domain is also employed. That is, the signal processing unit 32 subtracts the sound signal S2 from the sound signal S1, the signal SR obtained by subtracting the sound signal S1 provided with the delay amount D from the sound signal S2, and the sound signal provided with the delay amount D. A signal SL obtained by subtracting S2 from the sound signal S1 is generated. The frequency analysis unit 20 arranged at the subsequent stage of the signal processing unit 32 converts the signal S0 into the frequency spectrum P0, converts the signal SR into the frequency spectrum PR, and converts the signal SL into the frequency spectrum PL.

(4) Modification 4
The method of calculating the intensity μn (k) when the intensity XB (k) of the non-target sound spectrum QB is lower than the threshold value XTH (S3: NO) is not limited to Expression (2). For example, the average (moving average) of the intensities XB (k) over a predetermined number of unit sections including the nth unit section is calculated as the intensity μn (k). That is, the number of noise spectra N (number of unit sections) used for calculating the intensity μn (k) is arbitrarily changed.

  In the second embodiment, when the intensity XB (k) exceeds the threshold value XTH (S3: YES), the method of calculating the intensity μn (k) is based on the past intensity μn-1 (k) and the coefficient β. It is not limited to multiplication (Formula (5)). For example, a configuration in which an addition value of the past intensity μn−1 (k) and a predetermined positive number is calculated as the intensity μn (k) is also employed. That is, when the intensity XB (k) exceeds the threshold value XTH, a configuration in which a numerical value exceeding the intensity μn−1 (k) of the past noise spectrum N is set as the intensity μn (k) is preferable.

(5) Modification 5
A configuration in which the noise suppression spectrum QC generated by the noise suppression unit 44 is output as an output spectrum R to the inverse conversion unit 56 (a configuration in which the adjustment unit 52 and the synthesis unit 54 are omitted) is also employed. However, since the intensity of the non-target sound frequency FB in the noise suppression spectrum QC becomes zero, the reproduced sound generated from the noise suppression spectrum QC may have an unnatural impression on hearing. Therefore, from the viewpoint of natural reproduction sound generation, the configuration of FIG. 1 is preferable in which the non-target sound spectrum QB processed by the adjustment unit 52 is combined with the noise suppression spectrum QC.

(6) Modification 6
In each of the above embodiments, the noise spectrum N is used for suppression of the non-target stationary sound (generation of the noise suppression spectrum QC). However, the use of the sound processing apparatus according to the present invention (use of the noise spectrum N) is non-target stationary sound. It is not limited to repression. For example, the sound processing apparatus of the present invention is preferably used as an apparatus for extracting non-target stationary sound from a mixed sound of target sound, non-target stationary sound, and non-target fluctuation sound.

DESCRIPTION OF SYMBOLS 100 ... Sound processing device, 12 ... Arithmetic processing device, 14 ... Memory | storage device, 20 ... Frequency analysis part, 30 ... Sound source separation part, 32 ... Signal processing part, 34 ... Frequency selection part, 36 ... ... intensity specifying part, 42 ... noise estimating part, 44 ... noise suppressing part, 50 ... signal combining part, 52 ... adjusting part, 54 ... combining part, 56 ... inverse converting part.

Claims (4)

From the multiple sound signals generated by multiple sound collection devices, specify the intensity of each non-target sound frequency component in which the non-target sound dominates from a different direction from the target sound among multiple frequencies for each unit section Sound source separation means to perform,
Noise estimation means for generating a noise spectrum for each unit section,
The noise estimation means includes
The intensity of the component of one non-target sound frequency in the first unit section is below a threshold value exceeding the intensity at the one non-target sound frequency in the noise spectrum of the second unit section before the start of the first unit section. , The intensity at the one non-target sound frequency in the noise spectrum of the first unit section, the intensity of the component of the one non-target sound frequency in the first unit section, and the noise spectrum of the second unit section. Set according to the intensity at the one non-target sound frequency,
When the intensity of the component of the one non-target sound frequency in the first unit section exceeds the threshold, the intensity at the one non-target sound frequency in the noise spectrum of the first unit section is determined as the first unit section. The sound processing device is set to a numerical value exceeding the intensity at the one non-target sound frequency in the noise spectrum of the second unit section without reflecting the intensity of the component of the one non-target sound frequency in. When the intensity of one non-target sound frequency component in the first unit section exceeds the threshold value, the noise estimation means determines the intensity at the one non-target sound frequency in the noise spectrum of the second unit section as 1 Is set as the intensity at the one non-target sound frequency in the noise spectrum of the first unit section.
The sound processing apparatus according to claim 1 . The sound source separation means generates a target sound spectrum composed of components of each target sound frequency in which the target sound is dominant among the plurality of frequencies,
The sound processing apparatus according to claim 1 or claim 2 comprising a noise suppression means for subtracting the noise spectrum from the target sound spectrum. From the multiple sound signals generated by multiple sound collection devices, specify the intensity of each non-target sound frequency component in which the non-target sound dominates from a different direction from the target sound among multiple frequencies for each unit section Sound source separation processing,
A process of generating a noise spectrum for each unit section,
The intensity of the component of one non-target sound frequency in the first unit section is below a threshold value exceeding the intensity at the one non-target sound frequency in the noise spectrum of the second unit section before the start of the first unit section. , The intensity at the one non-target sound frequency in the noise spectrum of the first unit section, the intensity of the component of the one non-target sound frequency in the first unit section, and the noise spectrum of the second unit section. Set according to the intensity at the one non-target sound frequency,
When the intensity of the component of the one non-target sound frequency in the first unit section exceeds the threshold, the intensity at the one non-target sound frequency in the noise spectrum of the first unit section is determined as the first unit section. Noise estimation processing for setting a numerical value exceeding the intensity at the one non-target sound frequency in the noise spectrum of the second unit interval without reflecting the intensity of the component at the one non-target sound frequency in the computer. The program to be executed.

Images