JP4854032B2 - Acoustic likelihood parallel computing device and program for speech recognition - Google Patents

Description

  The present invention relates to an acoustic likelihood parallel calculation apparatus and its program in speech recognition, and in particular, acoustic likelihood that can shorten the processing time of likelihood calculation by parallelizing product-sum operation parts of acoustic features in speech recognition. The present invention relates to a computing device and its program.

  In speech recognition, speech digitally sampled from a microphone is converted into time-series acoustic features, and pattern matching is performed with acoustic models for each recognition unit such as vowels and consonants. On the other hand, an acoustic model expressed by a Hidden Markov Model (HMM) is composed of hundreds of states (State), and each state has a likelihood (38-dimensional) for the input of acoustic features (38 dimensions). It has a multidimensional normal distribution that outputs logarithmic probabilities.

  This multidimensional normal distribution is expressed by the following arithmetic expression (1).

  Here, μ is an average, σ is variance, and χ is a 38-dimensional (38) parameter independent of each other.

  When the equation (1) is logarithmized to be a likelihood, it is represented by the sum of quadratic functions as the following equation (2).

  Here, since the first term in parentheses {} of the arithmetic expression (2) is a constant term and can be calculated in advance, the arithmetic expression (2) is the 38th product sum of the second term in the parentheses {}. It becomes an operation. This sum-of-products calculation of acoustic features is called acoustic likelihood calculation.

  Now, the following Patent Publication 1 discloses a method for performing high-speed execution by decomposing the likelihood calculation expression (operation expression (2)) into a simple product-sum operation expression and reducing the calculation cost of each product-sum operation Is disclosed. Further, Patent Document 2 below discloses a method of performing a high speed by providing a threshold for the amount of change of the acoustic feature value and narrowing down the number of executions of the acoustic likelihood calculation. Furthermore, the following Patent Document 3 discloses a method of performing high-speed execution by making a constant part of a known likelihood calculation formula into a table and reducing the amount of calculation.

On the other hand, the following Non-Patent Document 1 reduces the useless acoustic likelihood calculation by pre-selecting the state of the acoustic model, and the other acoustic model in advance for the state of the unselected acoustic model. A method of executing at high speed by applying an approximate value calculated from the above is disclosed.
JP 2005-031151 A JP 2000-250580 A Japanese Unexamined Patent Publication No. 2000-322081 Information Processing Society of Japan IPSJ-JNL4307023, Lee Sung-nobu and two others, "Reduction of acoustic likelihood calculation by selecting Gaussian mixture using phoneme environment independent HMM"

  In order to execute the acoustic likelihood calculation at high speed, if the conventional techniques described in Patent Documents 1 to 3 and Non-Patent Document 1 described above are applied, the following problems exist.

  (a) In the method described in Patent Document 1, if the number of states of the acoustic model and the number of dimensions of the acoustic feature amount increase, the degree of accumulation of the individual product-sum operations that are decomposed increases, so that parallelization is difficult. .

  (b) In the method described in Patent Literature 2 or Patent Literature 3, when the number of states of the acoustic model or the dimension number of the acoustic feature amount increases, the data storage area for storing the acoustic feature amount or constant increases, so that data storage is performed. It is difficult to implement with equipment with limited area.

  (c) In the method described in Non-Patent Document 1, when the number of states of the acoustic model and the dimension number of the acoustic feature amount increase, the data storage area for storing the approximate value calculated from the state of the non-selected acoustic model increases. For this reason, it is difficult to implement the device with a limited data storage area.

  The present invention has been made in view of the above-described problems of the prior art, and an object thereof is to provide an acoustic likelihood parallel computing device in speech recognition and a program thereof that can shorten the processing time of acoustic likelihood calculation. is there.

ここに、μは平均、σは分散、χは互いに独立するｎ次元（ｎ個）のパラメータである。 In order to achieve the above-described object, the present invention includes an acoustic feature amount conversion unit that analyzes input speech and converts it into an acoustic feature amount, and an acoustic likelihood parallel execution procedure registration unit that registers a parallel execution procedure of acoustic likelihood calculation. And an acoustic feature quantity holding unit that holds the acoustic feature quantity of the input speech and the state of the acoustic model, and an acoustic feature quantity of the input speech according to the acoustic likelihood parallel execution procedure registered by the acoustic likelihood parallel execution procedure registration unit And an acoustic likelihood parallel calculation unit that performs likelihood calculation using the state of the acoustic model, an acoustic likelihood parallel calculation control unit that outputs a likelihood calculation result from the acoustic likelihood parallel calculation unit, and the acoustic likelihood An acoustic likelihood calculation result acquisition unit that acquires an acoustic likelihood calculation result from a parallel calculation unit, and the acoustic likelihood parallel calculation unit generally includes a GPGPU (General Purpose Graphic Processing Unit) that performs graphic processing Use The acoustic likelihood calculation is performed by cascading parallel operations of the pixel shaders of the PGPU, and the pixel shaders are each of n of n dimensions (n is a positive integer larger than 1) reserved in the data storage area, The calculation of the second term (χ−μ) of the following equation (2) is performed on χ to calculate the n-dimensional result z, and then the equation (2 ) Of the second term (z × z) / (σ × σ) is collectively executed to calculate the n-dimensional result P ′ .

Here, μ is an average, σ is a variance, and χ is an n-dimensional (n) parameter independent of each other.

  The acoustic likelihood parallel calculation unit generally uses a GPGPU (General Purpose Graphic Processing Unit) that performs graphic processing, and cascades the parallel computations of the pixel shaders of the GPGPU. There is another feature in that the degree calculation is executed.

ここに、μは平均、σは分散、χは互いに独立するｎ次元（ｎ個）のパラメータである。 In addition, it is an acoustic likelihood parallel calculation program for executing acoustic likelihood calculation by cascading parallel operations of GPGPU pixel shaders, and the computer stores n dimensions (n is 1 from 1). A function of calculating the n- dimensional result z by executing the operation of (χ−μ) in the second term of the following equation (2) for each μ and χ of a large positive integer) , z The pixel shader realizes the function of collectively executing the second term (z × z) / (σ × σ) of the arithmetic expression (2) for σ and calculating the n- dimensional result P ′. There is still another feature in that a program is provided.

Here, μ is an average, σ is a variance, and χ is an n- dimensional ( n ) parameter independent of each other.

  According to the acoustic likelihood parallel calculation apparatus of the present invention, the conventional likelihood calculation method can be executed in parallel as it is by using a variable data storage area that does not depend on the number of states of the acoustic model or the dimension of the acoustic feature. . As a result, the arithmetic expression (2) can be executed in a short time.

  Further, by holding the number of states of the acoustic model and the number of dimensions of the acoustic feature quantity in the variable data storage area, the frequency of use of the data storage area before and after the acoustic likelihood parallel calculation and the total amount of data storage can be kept low. It becomes like this.

  First, the principle of the present invention will be described. In general PCs and mobile phone terminals, the processing power of the CPU is limited. Therefore, GPGPU (General Purpose Graphic Processing Unit), which normally performs graphic processing, is used to calculate acoustic likelihood, which requires a large amount of processing in speech recognition processing. ) Can be used to perform enormous arithmetic processing with high efficiency and high accuracy, and to perform speech recognition processing in a large vocabulary dictionary in real time. More specifically, a matrix parallel processing of vertex processing (vertex shader) and texture processing (pixel shader) used in 3D graphic processing, etc. is regarded as vector processing, and a cascaded processing form of multiple parallel processing is configured. Speedup is achieved by efficiently mapping parallel / sequential processing in acoustic likelihood calculation. According to the present invention, a huge amount of memory that has been necessary on the CPU side can be absorbed on the GPGPU side.

  Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a hardware configuration of the acoustic likelihood parallel computing apparatus.

  As shown in the drawing, the acoustic likelihood parallel computing device includes a CPU 1, a memory 2, a GPGPU (General Purpose Graphic Processing Unit) 3, a display 4, and a bus 5 that connects them as hardware configurations. The GPGPU 3 includes a GPU 31 and a GPU memory 32 having a variable data storage area. As can be seen from the figure, the GPU memory 32 can be directly accessed from the GPU 31 to read / write data, but cannot be accessed directly from the CPU 1. For this reason, the CPU 1 accesses the GPU memory 32 via the GPU 31.

  FIG. 2 is a functional block diagram of the acoustic likelihood parallel calculation device, which analyzes an input speech and converts it into an acoustic feature amount, and an acoustic function for registering a parallel execution procedure of acoustic likelihood calculation. Likelihood parallel execution procedure registration unit 12, acoustic feature quantity holding unit 14 that holds the acoustic feature quantity of the input voice and the state of the acoustic model, and the acoustic feature quantity of the input voice held by the acoustic feature quantity holding unit 14 State of acoustic model and acoustic likelihood parallel execution procedure registered in advance in acoustic likelihood parallel execution procedure registration unit 12, acoustic likelihood parallel calculation unit 15 for performing likelihood calculation, and acoustic likelihood parallel calculation unit 15 includes an acoustic likelihood parallel calculation control unit 13 that outputs a likelihood calculation result from 15.

  Here, the acoustic feature quantity conversion unit 11 and the acoustic likelihood parallel calculation control unit 13 are functions performed by the CPU 1 in FIG. 1, and the acoustic likelihood parallel execution procedure registration unit 12 corresponds to the memory 2. The acoustic feature quantity holding unit 14 corresponds to the GPU memory 32, and the acoustic likelihood parallel calculation unit 15 is a function performed by the GPU 31.

  Next, the operation of the acoustic likelihood parallel computing device will be described. Since GPGPU3 is originally intended to perform graphic processing, for example, processing of rotating pixels of an image or changing the color thereof at high speed, when this is used as an accelerator for CPU computation, It is necessary to set or load the size of the data area for the calculation, the definition of the variable for the calculation, and the processing procedure (program) of the calculation into the GPU 31 from the CPU 1 side.

  Therefore, as shown in FIG. 3A, the CPU 1 prepares a storage area (texture) for n independent parameters χ, variance σ of voice feature amount, and average μ in the GPU memory 32 via the GPU 31. The size of each storage area is, for example, 512 × 512 (pixels). In addition, the CPU 1 loads the GPU 31 with the definition of calculation variables and the calculation processing procedure as shown in FIG.

That is, in FIG. 2, the acoustic likelihood parallel execution procedure registration unit 12 performs the following initial setting as an initial setting phase for the acoustic likelihood parallel calculation unit 15.
<Initial setting phase>

  The voice feature amount is n-dimensional (n).

  ・ The number of states of the acoustic model is m.

  The mean of n is μ, the variance of n is σ, and n independent parameters are χ.

  The likelihood calculation result (likelihood probability of each state of the acoustic model) obtained from μ, σ, and χ is p.

  The size of the data storage area for holding μ, σ, χ, and p is set to l = m × n, and the data storage area is secured.

  When the initial setting phase is finished, the next step is a parallel operation phase. In the parallel operation phase, the operation of the second term on the right side of the equation (2), that is, 38 product-sum operations are performed. Specifically, the acoustic feature quantity conversion unit 11 in FIG. 2 converts the input voice into 38 time-series acoustic feature quantities (digital), and the 38 acoustic feature quantities are converted into the acoustic likelihood parallel calculation unit 15. Is stored in the acoustic feature quantity holding unit 14. Preferably, when 25 msec of speech is one frame, the number of acoustic features 38 × the number of states of the acoustic model 546 × the number of combinations of the states of the acoustic model × one frame, for example 38 × 546 X If the acoustic feature quantities for any number of combinations of the states of the acoustic model are stored in the acoustic feature quantity holding unit 14 collectively, the processing speed can be further improved.

As the acoustic feature amount, the 38-dimensional (generally n-dimensional) speech feature amount average μ _n , variance σ _n , and independent parameter χ _n are obtained. Using these feature amounts, the following parallel operation phase processing is performed.
<Parallel processing phase>

A calculation result of the second term (χ _n −μ _n ) is collectively executed for each μ _n , χ _n secured in the data storage area (acoustic feature amount holding unit 14), and the result Is z _n .

- Subsequently, _{z n,} with respect to sigma _n, arithmetic expression (2) the calculation of the second term _{(z n × z n) /} (σ n × σ n) collectively perform the operation result as a _{P n,} the It is set in the data storage area P ′ (acoustic feature quantity holding unit 14) for holding _Pn .

  FIG. 4 is a conceptual diagram of the parallel operation processing phase. This phase is performed by cascading parallel operations of the GPGPU pixel shaders. In the GPGPU pixel shader, the 38-dimensional result z is obtained by collectively executing the operation of the second expression (χ−μ) of the equation (2) for each of 38-dimensional μ and χ secured in the data storage area. In the pixel shader, the operation of the expression (2) second term (z × z) / (σ × σ) is collectively executed for z and σ, and the 38-dimensional result P ′ Is calculated.

When the parallel operation processing phase ends, the process proceeds to the end phase. In the end phase, the acoustic likelihood parallel calculation control unit 13 accesses the acoustic feature quantity holding unit 14 via the acoustic likelihood parallel calculation unit 15, reads n pieces of p, and performs the following end phase processing. .
<End phase>

A sigma calculation is performed on n-dimensional (n) P ′ _n on the data storage area (acoustic feature amount holding unit 14), and the total likelihood of the acoustic feature amount is calculated.

  That is, the acoustic likelihood parallel calculation control unit 13 reads out 38 values of P ′ from the storage area of P ′, and performs calculation (acoustic likelihood calculation) of the equation shown in FIG. Here, C is a constant part.

As described above, according to the present embodiment, n or n × 1 frame feature quantities of input speech are collectively stored in the GPU memory 32, and the parallel arithmetic processing described in the parallel arithmetic processing phase is performed. As a result, the calculation of the second term (z _n × z _n ) / (σ _n × σ _n ) of the calculation formula (2) can be executed collectively. Therefore, the conventional likelihood calculation method can be executed in parallel as it is, and the arithmetic expression (2) can be executed in a short time.

  Note that the present invention is not limited to the above-described embodiment, and the number of simultaneous vector operations of the pixel shader is increased to calculate a plurality of acoustic models (acoustic feature amount 38 × acoustic model state number 546 × acoustic model state). The speed can be improved by simultaneously executing the processing of any number of superpositions × M) (M is a positive integer greater than 1). For example, when audio is processed in parallel one frame at a time, M is, for example, the maximum number of parallel processing executions of audio frames in the embodiment.

It is a block diagram which shows the hardware constitutions of one Embodiment of this invention. It is a block diagram which shows the function of one Embodiment of this invention. It is a figure explaining the concept of an initial setting phase. It is a figure explaining the concept of a parallel processing phase. It is a figure explaining the concept of an end phase.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... CPU, 2 ... Memory, 3 ... GPGPU, 4 ... Display, 5 ... Bus, 31 ... GPU, 32 ... GPU memory, 11 ... Acoustic feature-value Conversion part, 12 ... Acoustic likelihood parallel execution procedure registration part, 13 ... Acoustic likelihood parallel calculation control part, 14 ... Acoustic feature-value holding part, 15 ... Acoustic likelihood parallel calculation part.

Claims (6)

An acoustic feature conversion unit that analyzes input speech and converts it into acoustic features;
An acoustic likelihood parallel execution procedure registration unit for registering parallel execution procedures of acoustic likelihood calculation;
An acoustic feature holding unit that holds the acoustic feature of the input speech and the state of the acoustic model;
In accordance with the acoustic likelihood parallel execution procedure registered by the acoustic likelihood parallel execution procedure registration unit, an acoustic likelihood parallel calculation unit that performs likelihood calculation using the acoustic feature quantity of the input speech and the state of the acoustic model;
An acoustic likelihood parallel calculation control unit that outputs a likelihood calculation result from the acoustic likelihood parallel calculation unit;
An acoustic likelihood calculation result acquisition unit for acquiring an acoustic likelihood calculation result from the acoustic likelihood parallel calculation unit;
Equipped with,
The acoustic likelihood parallel calculation unit generally uses a GPGPU (General Purpose Graphic Processing Unit) for performing graphic processing, and performs acoustic likelihood calculation by cascading parallel operations of pixel shaders of the GPGPU. Let
The pixel shader uses (χ−μ) in the second term of the following equation (2) for each of μ and χ of n dimensions (n is a positive integer greater than 1) reserved in the data storage area. To calculate the n-dimensional result z, and subsequently to the second term (z × z) / (σ × σ) of the arithmetic expression (2) for z and σ. And an acoustic likelihood parallel calculation device that calculates an n-dimensional result P ′.

Here, μ is an average, σ is a variance, and χ is an n-dimensional (n) parameter independent of each other. In the acoustic likelihood parallel computing device according to claim 1,
The number of simultaneous vector operations of the pixel shader is set to (n dimensions x M) (M is a positive integer greater than 1) , and the processing speed is improved by simultaneously executing operations for multiple (M) acoustic models. An acoustic likelihood parallel computing device characterized by that. In the acoustic likelihood parallel computing device according to claim 1 or 2,
An acoustic likelihood parallel computing device characterized in that processing (Σ operation) for adding the calculation processing results of each dimension is executed by a CPU. In the acoustic likelihood parallel computing device according to any one of claims 1 to 3,
The n-dimensionality is 38 dimensions, and the acoustic likelihood parallel computing device. An acoustic likelihood parallel calculation program for executing acoustic likelihood calculation by cascading parallel operations of pixel shaders of GPGPU,
On the computer,
For the n- dimensional (n is a positive integer greater than 1) μ and χ secured in the data storage area, the operation of (χ−μ) in the second term of the following equation (2) is executed in a batch. And a function for calculating the n- dimensional result z,
The pixel shader has the function of collectively executing the calculation of the second term (z × z) / (σ × σ) of the calculation formula (2) for z and σ and calculating the n- dimensional result P ′. A program to make it happen.

Here, μ is an average, σ is a variance, and χ is an n- dimensional ( n ) parameter independent of each other. The acoustic likelihood parallel calculation program according to claim 5,
The n-dimensional program is 38-dimensional .

Images