JP2017520016A5 - Excitation signal formation method of glottal pulse model based on parametric speech synthesis system - Google Patents

Description

The present invention relates generally to telecommunications systems and methods as well as speech synthesis. More particularly, the present invention relates to the formation of the excitation signal in the hidden Markov model based on a statistical parametric speech synthesis system.

A method of forming a excitation signal of the glottal pulse model based on parametric speech synthesis system is provided. In one embodiment, the fundamental frequency values are used to form the excitation signal. Excitation is modeled by using the sound source pulse which is selected from the database of a given speaker. Source signals, to identify glottal pulses used to form the excitation signal is segmented into glottal segments used in vector representation. To preserve the original signal extracted from the use and the speaker of the voice sample of the new distance metric will help the uptake of low-frequency information of the excitation signal. In addition, segment edge artifacts are eliminated by applying a unique segment combining method to accurately represent the speech quality of the speaker and at the same time improve the quality of speech synthesis.

In yet another embodiment, a) transforming the input text into context-dependent phoneme labels; b) a parametric model learned to predict fundamental frequency values, synthesized speech duration, and spectral characteristics of phoneme labels. Using the processing of the phoneme label created in step (a); c) one of the eigenglottal pulses and the predicted fundamental frequency value, the spectral characteristics of the phoneme label and the synthesized speech duration. or using one or more, and creating a excitation signal, using a filter to create the output of d) synthesizing speech, the step of combining the excitation signal and the spectral characteristics of the phoneme label A method for synthesizing speech using input text is presented.

FIG. 1 is a diagram illustrating an embodiment of a text-based hidden Markov model for a speech system. FIG. 2 is a diagram illustrating an embodiment of a signal. Figure 3 is a diagram showing embodiments of the excitation signal created. 4 is a diagram showing embodiments of the excitation signal created. FIG. 5 is a diagram illustrating an embodiment with overlapping boundaries. 6 is a diagram showing embodiments of the excitation signal created. FIG. 7 is a diagram illustrating an embodiment of glottal pulse identification. FIG. 8 is a diagram illustrating an embodiment of creating a glottal pulse database.

Excitation is generally estimated to be quasi-periodic sequence of impulses voiced region. Each column is separated from the previous column at a fixed time, such as T ₀ = 1 / F ₀ , where T ₀ represents the pitch period and F ₀ represents the fundamental frequency. In unvoiced regions, excitation is modeled as white noise. In voiced region, excitation is not an impulse train in practice. Rather excitation is a sequence of sound pulses generated by the vibration caused by the folding of the voice. The shape of the pulse may vary depending on various factors such as the speaker, the speaker's mood, linguistic context, and emotion.

As described in the European patent EP22442045 (acquired June 27, 2012, inventor Thomas Drugman et al.), Source pulses are mathematically processed as vectors by length normalization and impulse matching (through sampling). Has been. The final length of the normalized source pulse signal is resampled to fit the target pitch. The source pulse is not selected from the database, but is obtained through a series of calculations that process the pulse characteristics in the frequency domain. In addition, as not present pre-filtering ended while determining the linear prediction (LP) coefficients, the approximate excitation signal used to pulse database creation not taken the content of low-frequency source, the linear prediction coefficient inverse Used for filtering.

In statistical parametric speech synthesis, speech unit signals are represented by a set of parameters that can be used to synthesize speech. The parameter may be learned by a statistical model such as an HMM. In certain embodiments, the source / excitation is a signal passing through the appropriate filter to generate a given sound, speech may be represented as a source filter model. FIG. 1 is a diagram illustrating an embodiment of a text-based hidden Markov model (HMM) to a speech (TTS) system. An embodiment of an exemplary system may include two phases, for example, a learning phase and a synthesis phase.

The speech database 105 can include the amount of speech data used for speech synthesis. During the learning phase, the audio signal 106 is converted into parameters. Parameters may include excitation parameters and spectral parameters. Excitation parameter extraction 110 and the spectral parameter extraction 115 is generated from the audio signal 106 to be transmitted from the speech database 105. Hidden Markov model 120 may be learned using labels 107 from these extracted parameters and speech database 105. Any number of HMM models may arise from learning, and these context-dependent HMMs are stored in the database 125.

The synthesis phase begins as a context sensitive HMM 125 and is used to generate the parameter 140. The parameter generation 140 may utilize input from a corpus of text 130 that is synthesized with speech. Text 130 may go through analysis 135 and the extracted label 136 is used in generating parameter 140. In one embodiment, excitation parameters and spectral parameters may be generated at 140.

Excitation parameters may be used to generate an excitation signal 145, excitation signal 145 is input to synthesis filter 150 with the spectral parameters. The filter parameters are generally Mel Frequency Cepstrum Coefficients (MFCC) and are often modeled by statistical time series using HMM. The fundamental frequency as the predicted value and the time-series value of the filter may be used to synthesize filter, MFCC values are used to form the filter by creating a excitation signal from the basic frequency value .

Synthesized speech 155, excitation signal is generated when passing through the filter. Formation of excitation signal 145 is essential to the quality or synthesized speech 155 of the output. Low-frequency information of the excitation is not taken. Accordingly, it will be appreciated that there is a need for a method for improving uptake, the quality of the synthesized speech the contents of the low-frequency source of the excitation signal.

FIG. 2 is a graphical illustration of one embodiment of the signal region of an audio segment, indicated generally at 200. The signals are classified into segments based on the types of fundamental frequency values such as voiced segments, unvoiced segments and pause segments. The vertical axis 205 represents the fundamental frequency in hertz (Hz), while the horizontal axis 210 represents the passage of milliseconds (ms). Time series F ₀ 215 represents the fundamental frequency. The voiced sound region 220 has a series of peaks and can be regarded as a non-zero segment. As further detail described below, the non-zero segments 220 may be connected to form a excitation signal of the entire sound. The unvoiced sound region 225 does not have a peak in the graph 200 and can be regarded as a zero segment. A zero segment can represent an unvoiced segment given by a pause or phoneme label.

Figure 3 is a diagram showing an embodiment of the excitation signal created, represented by 300 as a whole. Figure 3 shows the excitation signal created both unvoiced segments and pause segments. The fundamental frequency time series value represented as F ₀ represents a signal region 305 that is classified into a voiced segment, an unvoiced segment, and a pause segment based on the F ₀ value.

Excitation signal 320 is generated for unvoiced segments and pause segments. If the pause occurs, the zero (0) is placed excitation signal. In unvoiced region, (in one embodiment, this can be determined empirically by listening test) white noise suitable energy is used as the excitation signal.

Signal area 305 is used to excitation generator 315 with glottal pulse 310 is subsequently used to generate the excitation signal 320. The glottal pulse 310 includes eigenglottic pulses identified from the glottal pulse database, and further details of its creation are described in FIG. 8 below.

Figure 4 is a diagram showing an embodiment of the excitation signal generated in the voiced segments, it indicated generally 400. Eigenglottic pulses are presumed to have been identified from the glottal pulse database (which is described in further detail in FIG. 7 below). The signal region 405 includes F ₀ values that can be predicted by the model from voiced sound segments. The length of the good F ₀ segment be represented by N _f is used to determine the length of the excitation signal using a mathematical equation.

Glottis boundary from F ₀ value 410 are created, 410 denotes a pitch boundary excitation signals voiced segment in the signal region 405. The pitch period array can be calculated using the following mathematical equation.

The glottal pulse 415 is used with the glottal boundary 410 identified in the superposition addition 420 of glottal pulses starting from each glottic boundary. Then in order to avoid boundary effects that are further described in FIGS. 5 and 6, excitation signal 425 is generated through the processing of the "stitching" or segments bonded.

FIG. 5 is a diagram illustrating an embodiment with overlapping boundaries, indicated generally at 500. Diagram 500 represents a series of glottal pulses 515 and overlapping glottal pulses 520 in a segment. The vertical axis 505 represents the amplitude of the excitation. The horizontal axis 510 may represent frame numbers.

Figure 6 is a diagram showing an embodiment of the excitation signal generated in the voiced segment, represented by 600 as a whole. "Stitching" is no ideal boundary effects (FIGS. 4) may be used to form the final excitation signal voiced segments. In certain embodiments, any different excitation signal number may be formed through overlap-add method shown in FIGS. 4 and 500 (FIG. 5). Different excitation signal may have a left circular shift 630 of the same amount to the shift amount and the glottal pulse signal increases to a constant in the glottis boundary 605. In one embodiment, if the glottal pulse signal 615 is less than the corresponding pitch period, the glottal pulse may be zero stretched 625 up to the length of the pitch period before the cyclic shift left 630 is performed. Arrangements with different pitch boundaries (P ^m (i), represented as m = 1, 2,... M−1) are each of the same length as P ⁰ . The array is calculated using the following mathematical equation:

  For each set of frame boundaries, it is excited by initializing glottal pulses to zero (0). StartA signal 635 is formed. Array P^m(I) Starting with each pitch boundary value of i = 1, 2,... K, the superposition addition 610 excites the glottal pulse 620.StartUsed to add to the first N samples. The formed signal is a stitched single excitation.StartCorresponds to the shift m.

In certain embodiments, the calculated arithmetic mean of all stitched, single excitation signal, 640 calculated represents the final excitation signal 645 of the voiced segment.

FIG. 8 is a diagram illustrating one embodiment of glottal pulse database creation, indicated generally at 800. The audio signal 805 undergoes pre-filtering such as pre-emphasis 810. Linear prediction (LP) analysis 815 is performed using the prefiltered signal to obtain LP coefficients. Therefore, it is possible to low-frequency information of the excitation is incorporated. Once the coefficients are determined, the coefficients are used to invert at 820 the filter of the unprefiltered original speech signal 805 to calculate an integrated linear prediction residual (ILPR) signal 825. ILPR signal 825 can be used as an approximation to the excitation signal or excitation signal. ILPR signal 825 is segmented 835 into glottal pulses using glottal segment / cycle boundaries determined from speech signal 805. Segmentation 835 may be performed using a zero frequency filtering technique (ZFF). The resulting glottal pulse can then be energy normalized. All speech pulses of all speech learning data are combined to form a speech pulse database 840.

Claims (28)

A method of forming a parametric model, comprising:
a. Calculating a glottal pulse distance metric between multiple glottal pulses;
b. Clustering a plurality of glottal pulses stored in a glottal pulse database into a number of clusters to determine the glottal pulse centroid;
c. Forming a corresponding vector database by associating a vector with each glottal pulse in the glottal pulse database, wherein the glottal pulse centroid and the distance metric are mathematically defined to determine an association;
d. Determining eigenvectors of the vector database;
e. Forming a parametric model by associating glottal pulses with each determined eigenvector from the glottal pulse database. The method of claim 1 , wherein the number of glottal pulses is two. The step (a) of claim 1 comprises
a. Decomposing the number of glottal pulses into corresponding subband components;
b. Calculating a subband distance metric between the corresponding subband components of each glottal pulse;
c. Further comprising the method of claim 1 comprising the steps of mathematically calculating the glottal pulse distance metrics using the subband distance metric. The step (c) of claim 3 is calculated by mathematical equations

Wherein d (x _i , y _i ) represents the distance metric and d _s ² (x _i ⁽ⁿ⁾ , y _i ⁽ⁿ⁾ ) represents the subband distance metric, Item 4. The method according to Item 3 . The method of claim 1 , wherein the number of clusters is 256. Clustering step of claim. 1 (b) is carried out using a modified k-means calculation using the glottal pulse distance metric method according to claim 1. The method of claim 6 , wherein the modified k-means calculation further comprises updating a cluster centroid with an element of the cluster that has a minimum distance sum of squares from all other elements of the cluster. 8. The method of claim 7 , further comprising terminating the clustering iteration if there is no shift in any of the centroids from the cluster. Determination of eigenvectors of the steps of claim 1 (d) is performed using a principal component analysis method of claim 1. The step (e) of claim 1 comprises
a. Determining the eigenvector;
b. Determining a vector best matching the eigenvector from the vector database; c. Determining the most suitable glottal pulse from the glottal pulse database;
d. Further comprising the method of claim 1 and a step of designating the glottal pulse from best fits the glottal pulse database to the eigenvector as a unique glottal pulse associated with the eigenvectors. Further comprising the step of learning the parametric model the formed for use in speech synthesis method according to claim 1. The learning is
a. Defining a learning text corpus;
b. Obtaining voice data by recording the learning text spoken by the voice talent;
c. Converting the learning text into a context-dependent phoneme label;
d. Determining a plurality of spectral characteristics of the speech data using the phoneme labels;
e. Predicting the fundamental frequency of the audio data;
f. 12. The method of claim 11 , further comprising performing parameter prediction on the audio stream using the spectral characteristics, the fundamental frequency, and a duration of the audio stream. A method of synthesizing speech using input text,
a. Converting the input text into a context-dependent phoneme label;
b. A step of fundamental frequency values, using the parametric model trained to predict the spectral characteristics of the synthesized speech duration and the phoneme label, processes the phoneme label created the in step (a),
c. And creating a excitation signal using the one or one or more of the specific glottal pulses and the predicted fundamental frequency values, spectral characteristics and the synthesized speech duration of the phoneme label,
d. Using filters to create an output of the synthesized speech, it viewed including the step of combining said spectral characteristics of said excitation signal and the phoneme label,
The step of generating the excitation signal,
e . A step of classifying the signal region of excitation of the type of segment,
f . Further seen including the step of creating the excitation signal of each type,
Type of the segment looking contains voiced, one of unvoiced and pauses or one or more,
g . Using the predicted fundamental frequency value from the model to create a glottal boundary indicating the pitch boundary of the excitation signal ;
h . Adding glottal pulses starting from each glottal boundary using a superposition addition method;
i . i. When the glottal pulse has a length less than the corresponding pitch period, the glottal pulse is zero-extended to the length of the pitch period before the left shift, and the amount of shift that increases constantly at the glottic boundary and the glottal pulse and creating a number of different excitation formed through the overlap-add method in the same amount of cyclic left shift for,
ii. Determining an arithmetic mean of the different excitation signal number,
iii. Further comprising a said arithmetic mean declaring the step of final excitation signal voiced segments, how the excitation signal in a voiced device signals is created and a step to avoid boundary effects at the excitation signal . A method of synthesizing speech using input text,
a. Converting the input text into a context-dependent phoneme label;
b. Processing the phoneme label created in step (a) using a parametric model learned to predict a fundamental frequency value, the synthesized speech duration, and a spectral characteristic of the phoneme label; ,
c. A step to create a unique glottal pulse and the predicted fundamental frequency values, the excitation signal using the one or more than one of said phoneme label spectral properties and the synthesized speech duration,
d. Using filters to create an output of the synthesized speech, comprising the steps of combining the spectral properties of the excitation signal and the phoneme label
Including
The proper glottal pulse is identified from a glottal pulse database, and the identification is
e . Calculating a glottal pulse distance metric between multiple glottal pulses;
f . Clustering the glottal pulse database into multiple clusters to determine the centroid of glottal pulses;
g . Forming a corresponding vector database by associating a vector with each glottal pulse in the glottal pulse database, wherein the glottal pulse centroid and the distance metric are mathematically defined to determine an association;
h . Determining eigenvectors of the vector database;
i . Forming a parametric model by associating glottal pulses with each determined eigenvector from the glottal pulse database. The method of claim 14 , wherein the number of glottal pulses is two. The step ( e ) of claim 14 comprises :
a. Decomposing the number of glottal pulses into corresponding subband components;
b. Calculating a subband distance metric between the corresponding subband components of each glottal pulse;
c. 15. The method of claim 14 , further comprising mathematically calculating the distance metric using the subband distance metric. The calculation of step (c) of claim 16 comprises the mathematical equation

Wherein d (x _i , y _i ) represents the distance metric and d _s ² (x _i ⁽ⁿ⁾ , y _i ⁽ⁿ⁾ ) represents the subband distance metric, Item 17. The method according to Item 16 . The method of claim 14 , wherein the number of clusters is 256. The clustering step (f) of claim 14 is carried out using a modified k-means calculation using the glottal pulse distance metric method according to claim 14. 20. The method of claim 19 , wherein the modified k-means calculation further comprises updating a cluster centroid with an element of the cluster that has a minimum sum of squares of distances from all other elements of the cluster. 21. The method of claim 20 , further comprising terminating the clustering iteration if it does not shift in any of the centroids from the cluster. The method of claim 14, wherein the determination of the eigenvectors of the step ( h ) of claim 14 is performed using principal component analysis. The step ( i ) of claim 14 comprises :
a. Determining the eigenvector;
b. Determining a vector best matching the eigenvector from the vector database; c. Determining the most suitable glottal pulse from the glottal pulse database;
d. 15. The method of claim 14 , further comprising: specifying the glottal pulse from the glottal pulse database that best matches the eigenvector as the eigenglottic pulse associated with the eigenvector. Further comprising constructing the glottal pulse database from speech signals, the configuration comprising: a. Performing pre-filtering on the audio signal to obtain a pre-filtered signal;
b. Analyzing the prefiltered signal to obtain inverse filtering parameters;
c. Performing inverse filtering of the audio signal using the inverse filtering parameters;
d. Calculating an integrated linear prediction residual signal using the inverse filtered speech signal;
e. Identifying glottal segment boundaries in the speech signal;
f. Segmenting the integrated linear prediction residual signal into glottal pulses using the identified glottic segment boundaries from the speech signal;
g. Performing normalization of the glottal pulses;
h. By collecting all the normalized glottal pulse obtained to the audio signal, and forming the glottal pulse database The method of claim 14. 25. The method of claim 24, wherein the analysis of step (b) of claim 24 is performed using linear prediction. 25. The method of claim 24, wherein the inverse filtering parameters in step (b) of claim 24 include linear prediction coefficients. 25. The method of claim 24, wherein the identification of step (e) of claim 24 is performed using a zero frequency filtering technique. 25. The method of claim 24, wherein the pre-filtering of step (a) of claim 24 includes pre-emphasis.